NHANES Survey Weights: We Ran the Same Analysis Three Ways and the Answer Moved 45%

Every analysis of the National Health and Nutrition Examination Survey rests on a choice most authors make without noticing. NHANES survey weights are either applied correctly, applied partially, or ignored, and the resulting number can differ by more than a third. We ran the same question three ways on the same 5,018 people to measure exactly how much that choice matters. The unweighted answer was 14.85 percent. The correct answer was 10.23 percent. Nothing about the data changed between those two figures, only the analysis.

What NHANES Survey Weights Actually Represent

NHANES uses a stratified, multistage probability sample. Certain groups, including older adults, some racial and ethnic populations, and low-income participants, are sampled at higher rates so that estimates for those groups are stable enough to publish. That design is a feature, and it is the reason NHANES can say anything reliable about smaller populations at all.

The consequence is that one participant in the data file does not stand for one American. A participant might represent 8,000 people or 80,000, depending on how likely someone like them was to be selected, whether they responded, and how the sample was adjusted to census population totals. Each weight variable carries that number. Analysing the file without them answers a question nobody asked, which is what the sample looks like rather than what the country looks like.

Three weight variables cover most work. WTINT2YR applies to anything collected in the household interview. WTMEC2YR applies to anything measured at the mobile examination centre, and it is the one most analyses need because laboratory and physical measurements come from there. Fasting subsample biomarkers and second-day dietary recalls carry their own weights again. Choosing the wrong one does not throw an error. It silently biases every estimate in the paper.

The Test We Ran

We took NHANES 2017 to 2018, the most recent complete standalone cycle, and asked a deliberately simple question. What share of US adults aged 20 and over have an HbA1c of 6.5 percent or higher, the laboratory threshold used to identify diabetes? Simple questions are the right test here because any error that appears cannot be blamed on modelling complexity.

After merging the demographic file with the glycohaemoglobin file and restricting to adults with a valid measurement and a positive examination weight, 5,018 participants remained. We then estimated the same quantity three ways.

The naive analysis treated every participant as one American and returned 14.85 percent, with a confidence interval of 13.86 to 15.83. Applying WTMEC2YR but declaring no strata or clusters returned 10.23 percent, interval 9.08 to 11.38. The correct specification, with the examination weight, the strata variable SDMVSTRA, the cluster variable SDMVPSU, and a subpopulation restriction rather than a filter, returned 10.23 percent, interval 9.17 to 11.29.

Weighting moved the point estimate by 4.62 percentage points, a relative overstatement of 45 percent. That is not a rounding difference. In a manuscript it is the difference between a defensible national estimate and a number a reviewer can dismiss in one sentence.

Why the Unweighted Number Was So Wrong

The direction of the error is predictable once you know the design. NHANES 2017 to 2018 oversampled older adults and several groups with higher diabetes burden. Counting each participant equally therefore over-represents exactly the people most likely to cross the HbA1c threshold. The sample is not broken, it was built that way on purpose, and the weights exist to undo it at the analysis stage.

Our weighted estimate corresponds to a population of 229,445,508 US adults, which is consistent with the census-based adult population for that period. The unweighted figure corresponds to nothing at all. It describes 5,018 individuals who happened to be sampled, and generalising from them is unjustified.

One caveat belongs here rather than buried at the end. Our outcome is HbA1c at or above 6.5 percent, which is narrower than total diabetes prevalence. It misses people whose diagnosed diabetes is well controlled below that threshold, and published total-diabetes figures that combine self-report with fasting glucose run higher. Precision about what was measured is part of the point.

What Strata and Clusters Do That Weights Alone Cannot

Notice that methods two and three produced an identical point estimate. Weights alone fixed the bias. What changed between them was the confidence interval, and that difference is the part authors most often miss.

Standard errors depend on how the sample was drawn. NHANES selects counties first, then segments, then households, so two participants from the same primary sampling unit resemble each other more than two strangers drawn at random. Ignoring that structure treats the sample as more informative than it is. Our design effect was 1.594, meaning the variance under the real design is roughly 59 percent larger than a simple random sample of the same size would give.

Degrees of freedom follow the same logic. With 15 strata and 30 primary sampling units, this design supplies 15 degrees of freedom, not 5,017. Software that has not been told about the design will happily compute an interval on thousands of degrees of freedom and report a precision the survey cannot support.

What the Correct Method Shows Across Subgroups

Applying the full design to subgroups makes the precision cost of small cells visible in a way the naive analysis hides. Running the same estimate by race and ethnicity on our 5,018 adults produced the results below, all with weights, strata, clusters and a subpopulation restriction applied.

Table 1. Weighted prevalence of HbA1c at or above 6.5 percent by race and ethnicity, NHANES 2017-2018 adults aged 20 and over. All estimates use WTMEC2YR with SDMVSTRA, SDMVPSU and a subpopulation restriction. CI width is in percentage points.

Read the last column rather than the third. The smallest group carries an interval more than twice as wide as the largest, which means an apparent difference between those two categories could be almost entirely noise. An unweighted analysis would have produced tighter intervals across the board and invited exactly the over-interpretation the design is meant to prevent.

This is also where authors get into difficulty with a reviewer. Reporting a subgroup estimate to two decimal places while omitting the interval implies a precision of roughly 0.01 points when the true uncertainty spans 8 points. Reviewers who work with survey data check for that pattern first.

Five NHANES Survey Weights Mistakes That Get Papers Rejected

1. Filtering the data instead of using a subpopulation

Restricting to adults by deleting rows before building the design object is the most common error we see. Variance estimation needs the full sample structure, including the strata and clusters that contain the dropped observations. Deleting them first produces standard errors that are wrong in a direction nobody can predict in advance. Every major package supports a subpopulation option, and it exists for this reason.

2. Pooling cycles without dividing the weights

Combining two-year cycles requires dividing the weights by the number of cycles pooled. Concatenating four cycles without rescaling produces a weighted population roughly four times the size of the United States, and any reviewer who checks the population total will catch it immediately.

3. Pooling variables that were not measured the same way

Laboratory methods change between cycles. NCHS publishes regression equations to bridge specific changes, and where no bridge exists the affected cycles have to be excluded. Pooling across a methodology change without addressing it turns a measurement artefact into a spurious time trend.

4. Reporting subgroups too small to support an estimate

In our own subgroup breakdown, the confidence interval widened from 3.5 percentage points in the largest group to 8.1 points in the smallest. Wide intervals are honest. Presenting a small-cell subgroup estimate without them, or without noting instability, is not.

5. Using the interview weight for examination data

Anything measured at the mobile examination centre requires the examination weight. Using the interview weight for a laboratory outcome is a silent error, because the code runs and the output looks reasonable.

Reproduce This Yourself

NHANES is public. There is no application, no data use agreement, and no fee, so every number above can be checked independently. The full specification in R is short.

library(haven); library(survey); library(dplyr)

demo <- read_xpt("DEMO_J.xpt")   # NHANES 2017-2018 demographics
ghb  <- read_xpt("GHB_J.xpt")    # glycohaemoglobin

d <- demo %>%
  select(SEQN, RIDAGEYR, SDMVSTRA, SDMVPSU, WTMEC2YR) %>%
  left_join(select(ghb, SEQN, LBXGH), by = "SEQN") %>%
  mutate(
    inpop = RIDAGEYR >= 20 & !is.na(LBXGH) & WTMEC2YR > 0,
    dm    = as.integer(LBXGH >= 6.5)
  )

# Build the design on the FULL file, then subset. Never filter first.
des <- svydesign(ids = ~SDMVPSU, strata = ~SDMVSTRA,
                 weights = ~WTMEC2YR, nest = TRUE, data = d)

svymean(~dm, subset(des, inpop), na.rm = TRUE)   # 10.23%

We cross-checked every figure in this article twice, once in R and once independently in Python, confirming the file hash, row count, weighted population total, mean HbA1c, and both prevalence estimates agreed exactly before anything was published. Verifying an analysis in a second engine catches parsing errors that no single pipeline will reveal on its own.

What Weighting Cannot Fix

Correct weighting solves one specific problem, which is unequal probability of selection. Several other threats survive it untouched, and a methods section that implies otherwise invites scepticism.

Non-response is only partially addressed. NHANES weights include a non-response adjustment, but that adjustment relies on observed characteristics. If people decline to participate for reasons connected to the outcome and not captured by the adjustment variables, residual bias remains regardless of how carefully the weights are applied.

The cross-sectional structure is unchanged. Every participant is measured once, so exposure and outcome are recorded simultaneously and temporality cannot be established. Weighting a cross-sectional association perfectly still leaves a cross-sectional association, and describing it in causal language is the second most common objection this literature attracts.

Measurement error persists too. Self-reported dietary intake, physical activity and medication use carry known reporting bias, and applying survey weights propagates that bias faithfully to the national level rather than removing it. Weighting improves generalisability, not accuracy of the underlying measurement.

What This Means for Your NHANES Paper

Editors have become noticeably sharper about this dataset, and there is now evidence for why. Maupin and colleagues, writing in BMC Medicine in 2026, searched publications from 2021 to July 2025 and defined redundancy as analysing the same exposures against the same outcomes in the same national population. They found substantial duplication built on NHANES specifically, and examined whether generative tools make formulaic submissions easier to produce and harder to screen. Reviewers assigned an NHANES manuscript now approach it expecting a template.

Two things follow. A study that repeats a published exposure-outcome pair will struggle regardless of execution quality, so novelty has to be established before the analysis begins rather than argued afterwards. And because the reviewer is already primed, the methods section carries more weight than usual. Naming the weight variable, the strata and cluster variables, the subpopulation approach, and the degrees of freedom signals competence in four lines, and their absence signals the opposite just as quickly.

Reporting standards help here. The STROBE checklist for observational studies is what most reviewers work from, and our STROBE tutorial covers the items that attract objections. Registration is worth considering too, and we set out the reasoning in why a retrospective study should still be registered. If your work involves registry rather than survey data, the same discipline applies to cleaning, as covered in six techniques for cleaning clinical registry data, and to the limits of what a convenience sample supports, discussed in our review of whether NEISS injury statistics are reliable. Where a question is genuinely causal rather than descriptive, Mendelian randomization and other instrumental variable approaches may suit better than a cross-sectional survey. For where any of these sit on the evidence hierarchy, see our guide to levels of evidence in clinical research.

Tools and Resources

Professional NHANES Analysis Support

If you would rather the survey design were handled correctly the first time, we build NHANES studies to the specification described above and hand over the code so the analysis is reproducible by you or by a reviewer.

Everything in this category is listed under Registry and Real-World Data Studies.

Conclusion

Applying NHANES survey weights is not a formatting requirement bolted on at the end. It determines whether a number describes the United States or describes a spreadsheet. Our test moved a headline prevalence figure by 4.6 percentage points, and the design variables that had no effect on the point estimate still changed the confidence interval and the degrees of freedom available to defend it.

Get the specification right and state it explicitly. A reviewer who can see the weight variable, the strata, the clusters and the subpopulation approach in your methods section has no reason to look for the error, because you have already shown it is not there.

References

1. Maupin D, Suchak T, Barnett A, Spick M. Dramatic increases in redundant publications in the Generative AI era. BMC Medicine. 2026;24(1):29. doi:10.1186/s12916-025-04569-y
2. Smirnova E, Cui E, Tabacu L, Leroux A. Scalar-on-function regression: Estimation and inference under complex survey designs. Statistics in Medicine. 2024;43(20). doi:10.1002/sim.10194
3. National Center for Health Statistics. NHANES Survey Methods and Analytic Guidelines. Centers for Disease Control and Prevention. wwwn.cdc.gov/nchs/nhanes/analyticguidelines.aspx
4. National Center for Health Statistics. NHANES 2017-2018 Demographic Variables and Sample Weights (DEMO_J). wwwn.cdc.gov/Nchs/Nhanes/2017-2018/DEMO_J.htm
5. National Center for Health Statistics. NHANES 2017-2018 Glycohemoglobin Laboratory Data (GHB_J). wwwn.cdc.gov/Nchs/Nhanes/2017-2018/GHB_J.htm
6. Lumley T. survey: analysis of complex survey samples. R package. cran.r-project.org/package=survey
7. von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement. The Lancet. 2007;370(9596):1453-1457. doi:10.1016/S0140-6736(07)61602-X

Analysis performed on the NHANES 2017-2018 public release by Editverse. All estimates independently reproduced in R and Python before publication. Data source: Centers for Disease Control and Prevention, National Center for Health Statistics.