How to Choose the Right Statistical Test: Decision Grid and a Real Worked Example

Published August 10, 2024  |  Last updated October 29, 2026

Use the tool above for a recommendation in under a minute, then read the section for that test. If you prefer a standalone version, the free test selector does the same job on its own page.

Choosing the test is only part of getting the statistics right. A 2026 review of 119 clinical prosthodontic studies found that 88% used an appropriate test, and all 19 studies that involved a statistician chose correctly. What went wrong more often was everything around the test: only 32% reported a sample-size calculation, 47% checked assumptions, and 83% reported no confidence intervals. In all, 35% of the studies contained at least one major statistical error.³ Earlier reviews of medical journals reached the same conclusion about planning, analysis and reporting.⁴ This guide therefore covers the choice and the checks that go with it, and it shows both on a real open dataset.

Key Takeaways

• Four questions pick the test. Goal, outcome type, number and relation of groups, then assumptions. The two lookup grids below turn the answers into a test name.
• Look at the data before you test. In the worked example the practice score takes only three values and 65 of 104 respondents share the top value, so a Shapiro-Wilk W of 0.68 is no surprise.
• Rank-based tests fit scores like that. Mann-Whitney U gave p = 0.014 for practice by sex, matching the published value; the Welch t-test gave p = 0.028.
• The same data can answer differently under two tests. Knowledge against practice gave Spearman rho = 0.18 (p = 0.066) but Pearson r = 0.23 (p = 0.018). Name the test in your protocol before you see the result.
• Check the denominator. Table 1 of the source paper lists 13.25% for ages 5–18, which is that group’s positives as a share of all 249 people tested; the infection rate inside the group is 34% (33/97).
• Do not run all pairwise t-tests. In 10,000 simulated null datasets, three pairwise t-tests produced at least one false positive 12.0% of the time, not 5%.

The Four Questions Behind Every Test Choice

1. What is your goal? Compare groups, describe a relationship, or predict an outcome from other variables.
2. What type is the outcome? Continuous (blood pressure), ordinal (a Likert or bounded score), categorical (yes or no), count, or time to event.
3. How many groups, and how are they related? Two or three or more, and independent (different people) or paired and repeated (the same people measured again).
4. Do the data meet the assumptions? Roughly normal continuous data, independent observations, and expected counts of at least 5 for chi-square. Judge these from plots and sample size, not from a single significance test.⁵

Test Lookup Grids: From Your Data to a Test Name

Figure 1 covers the most common question, comparing groups. Find the row that matches your outcome and the column that matches your design. The navy bar marks parametric tests that use means; the orange bar marks rank-based and count tests.

Figure 2 covers relationships and prediction, with what to report and what to watch for. Correlation describes an association between two variables; only regression can adjust for others.

Quick Reference Table

Parametric or Rank-Based: Which Family?

Parametric tests (t-test, ANOVA, Pearson r, linear regression) compare means and assume roughly normal data. They have more power when those assumptions hold. Rank-based tests (Mann-Whitney U, Wilcoxon, Kruskal-Wallis, Spearman) work on ranks, so they suit ordinal data, small skewed samples, outliers and bounded scores.

Two defaults help. First, prefer the Welch t-test to Student’s: it does not assume equal variances and gives up very little when variances are equal.⁶ Second, give an effect size for rank-based tests too. The rank-biserial correlation, r = 2U/(n₁n₂) − 1, is the difference between the proportion of pairs favouring each group.⁷

Worked Example: One Malaria Survey, Four Tests

Real data show the choices better than invented numbers. Joko and colleagues published a malaria knowledge, attitudes and practices (KAP) survey from Emana village in Cameroon in 2026, openly licensed under CC BY 4.0, with the datasets attached.⁸ Blood slides were read for 249 people, and 104 households also completed the KAP questionnaire. We recounted both datasets in R and reproduced most of the published p-values, which makes the paper a good test bed: one study, one dataset, four different test families.

Step 1: Look before you test

Figure 3 plots the three KAP scores. Knowledge and attitude are roughly symmetric, yet Shapiro-Wilk still rejects normality with 104 respondents, because the test flags even small departures in a sample this size. The plot is the better evidence.⁵ The practice score is the clear case: it takes only three values (2, 3 and 4), and 65 of 104 respondents share the top one. No transformation turns three values into a bell curve, so a rank-based test is the honest choice, and the authors used one.

Step 2: Match each question to a test

Each question below maps to one cell of the lookup grids. The last column shows what the obvious alternative test would have given.

Practice by sex. Women scored higher: 74% of women reached the top score against 49% of men (median 4 against 3). Mann-Whitney U gives p = 0.014 and a rank-biserial correlation of 0.24, which means that in 62% of woman-man pairs the woman scored higher (ties count as half). The paper reports p = 0.014, and we recount the same value. The Welch t-test reaches the same direction with p = 0.028, so the conclusion survives here, but a mean of a three-value score is a weaker summary than the proportions in panel A.

Knowledge by education. The four groups are very unequal (3, 39, 52 and 10 people), and the medians run from 4 to 6. Kruskal-Wallis gives p = 0.109 and eta-squared (H) of 0.03, a small effect that is not distinguishable from chance with these numbers. One-way ANOVA agrees (p = 0.128). Because the omnibus test is not significant, no post hoc comparisons are needed.

Positivity by age. Slides were positive for 12% of ages 0–5 (5 of 41), 34% of ages 6–18 (33 of 97) and 22% of those 19 and older (24 of 109). Every expected count is above 5 (the smallest is 8.5), so chi-square is valid and the exact test agrees. Sex was not associated with positivity: chi-square p = 0.129 without continuity correction, which matches the 0.13 in the paper (Yates 0.171, Fisher exact 0.139).

When two tests disagree

Pearson says knowledge and practice are associated (r = 0.23, p = 0.018); Spearman says the evidence is not quite there (rho = 0.18, p = 0.066). Neither is a mistake. Pearson asks about a straight-line relationship between the raw scores, and a handful of respondents with a score of 2 or 3 can carry it. Spearman asks whether higher ranks go with higher ranks, which suits scores with three values and many ties. The paper treated KAP scores as ordinal and used rank-based tests, and it reports no significant association. The lesson is procedural: write the test in your protocol before you see the data, and if you run two, report both. Picking whichever p-value is smaller is a form of p-hacking.

Step 3: Check the denominator

Choosing the right test does not rescue a misread table. Figure 5 shows Table 1 of the source paper with four things to check before you cite or reuse numbers like these.

1. 1Same percentage twice. Both sex rows read 31 (24.9%). The value 24.9% is 62 of 249, the overall rate. The rates by sex are 21.4% for women (31 of 145) and 29.8% for men (31 of 104).
2. 2A share of everyone, not a rate. The 13.25% beside the 5–18 group is 33 of all 249 people. Inside the group, 33 of 97 slides are positive: 34.0% (95% CI 24.7% to 44.3%). The paper calls the table “prevalence”, so check which denominator any percentage uses.
3. 3The p-values reproduce. Uncorrected chi-square gives p = 0.129 for sex (printed 0.13) and p = 0.016 for age (printed 0.02).
4. 4Rows that do not add up. The age rows list 5 + 33 + 24 + 6 = 68 positive slides, but the sex rows and the dataset both give 62. The dataset has two slides with no recorded age, and neither is positive.

What Reproduced and What Did Not

Why Not Run Every Pairwise t-Test?

It is often said that three pairwise t-tests at alpha 0.05 raise the false-positive rate to 14%. That figure, 1 − 0.95³, assumes the tests are independent, which pairwise comparisons are not, because they share groups. We simulated it instead: 10,000 null datasets per row, 30 observations per group, all drawn from the same normal distribution, with uncorrected pairwise Welch t-tests.

The formula overstates the inflation, but the simulated rate is still two to six times the 5% you intended. One-way ANOVA on the same three-group data held it at 4.9%. Test all the groups with one omnibus test first, then use a post hoc procedure that controls the family-wise error: Tukey or Games-Howell after ANOVA, Dunn with a Holm adjustment after Kruskal-Wallis.

Checking Assumptions Without Over-Trusting Shapiro-Wilk

• Normality. Plot a histogram and a Q-Q plot. Shapiro-Wilk is oversensitive in large samples and underpowered in small ones, so use it as one input, not a gate.⁵ With about 30 or more per group the t-test tolerates moderate departures.
• Equal variances. Use Welch’s t-test or Welch’s ANOVA instead of testing for equal variances first.
• Independence. Repeated measures or clustered data (patients within hospitals) need paired tests, mixed-effects models or GEE.
• Outliers. Inspect box plots; use Cook’s distance for regression, and prefer rank-based tests if a few extreme values drive the result.
• Counts. Check expected cell counts before chi-square; switch to an exact test below 5.

Regression: When Two Variables Are Not Enough

Correlation cannot adjust for confounders. When other variables matter, move to a model that matches the outcome (Figure 2): linear regression for a continuous outcome, logistic for a binary one, Cox proportional hazards for time to event, and Poisson or negative binomial for counts. Report the coefficient, odds ratio, hazard ratio or incidence-rate ratio with its 95% confidence interval, and check the model’s own assumptions: residual plots for linear models, proportional hazards for Cox, and overdispersion for count models. When the outcome is common (above about 10%), odds ratios overstate risk ratios, so consider a modified Poisson model with robust standard errors. Our guide to regression versus correlation goes further.

What a Complete Statistical Result Includes

1. The test and the reason for it. One sentence tying the test to the outcome type and design.
2. The statistic and degrees of freedom, for example H(3) = 6.06.
3. The exact p-value, not “p < 0.05”.
4. An effect size with a 95% confidence interval, and the direction and size of the effect in the units of the study. Epidemiology reviews recommend pairing every p-value with an effect size and avoiding “significant versus not significant” language.⁹
5. Assumption checks and any deviation. Reporting checklists such as SAMBR link each study design to the analysis and reporting steps it needs.¹⁰

Need the Analysis Run or Checked? Consult Editverse

Editverse subject-matter experts work to COPE, ICMJE and GPP 2022 guidance, with institutional and country-specific ethics followed. Choose the stage you are at.

Editverse

Statistical Analysis

Not sure which test fits your data? Have the choice made, run and documented properly.

You send: your data, research questions and preferred software (SPSS, R or STATA).

You get: annotated code or syntax, complete output files, an interpretation guide and publication-ready tables.

When: standard turnaround is 4 to 6 business days.

SPSS/R/STATA Statistical Analysis · $400

Analyse my data →

Editverse

Sample Size

Planning a study? Fix the test and the sample size before you collect data.

You send: your design, expected effect size, outcome measure and alpha level.

You get: a power analysis with several scenarios and a methods-section paragraph you can paste into your protocol.

When: before data collection, so the planned test has enough power.

Sample Size Calculation · From $150

Calculate my sample size →

Editverse

Manuscript Writing

Results in hand? Turn them into a journal-ready paper with the statistics written up.

You send: your outline, data, figures, tables and key references, plus the target journal.

You get: a full manuscript written by a subject-matter expert, with statistical review, journal formatting, cover letter and revision support until acceptance.

When: standard delivery is 15 days; express delivery is available.

Manuscript Writing Services · $1,500

Write my paper →

Frequently Asked Questions

How do I choose the right statistical test?

Answer four questions in order: your goal (compare groups, relate variables or predict an outcome), the type of outcome (continuous, ordinal, categorical, count or time to event), the number of groups and whether they are independent or paired, and whether the assumptions hold. Then read the test from the lookup grids on this page or use the interactive tool at the top. Write the choice in your protocol before you look at the results.

Should I use a parametric or non-parametric test?

Use a parametric test (t-test, ANOVA, Pearson r) when the outcome is continuous and roughly normal, or when each group has about 30 or more observations. Use a rank-based test (Mann-Whitney U, Wilcoxon, Kruskal-Wallis, Spearman) for ordinal data, bounded scores with many ties, small skewed samples or influential outliers. In a Cameroon malaria survey, a practice score with only three values (65 of 104 respondents at the top value) called for Mann-Whitney U, which gave p = 0.014 against p = 0.028 for a Welch t-test.

Should I use Student’s t-test or Welch’s t-test?

Use Welch’s t-test by default. It does not assume equal variances and loses very little power when the variances are equal, so there is little reason to pre-test for equal variances and then switch tests.

When should I use ANOVA instead of multiple t-tests?

Use ANOVA, or Kruskal-Wallis for ranks, whenever you compare three or more groups. In 10,000 simulated null datasets, three uncorrected pairwise t-tests at alpha 0.05 produced at least one false positive 12.0% of the time, four groups 20.2% and five groups 29.8%, while one-way ANOVA held the rate at 4.9%. If the omnibus test is significant, follow with Tukey or Games-Howell after ANOVA and Dunn’s test after Kruskal-Wallis.

What is the difference between chi-square and Fisher’s exact test?

Both test whether two categorical variables are associated. Use chi-square when every expected cell count is 5 or more, and Fisher’s exact test (Fisher-Freeman-Halton for larger tables) when any expected count is under 5 or the sample is small. In the malaria example the smallest expected count was 8.5, so chi-square (p = 0.016) and the exact test (p = 0.017) agreed.

How do I test for normality before choosing a test?

Look at a histogram and a Q-Q plot, and treat Shapiro-Wilk as one input rather than a pass or fail gate: it flags trivial departures in large samples and misses real ones in small samples. For t-tests the assumption concerns the group values or the paired differences, and with about 30 or more per group the t-test tolerates moderate non-normality.

Which effect size should I report with each test?

Cohen’s d or Hedges’ g for t-tests, eta-squared or omega-squared for ANOVA, rank-biserial r for Mann-Whitney and Wilcoxon, eta-squared (H) for Kruskal-Wallis, Cramer’s V or an odds ratio for chi-square, r or rho for correlation, and odds, hazard or incidence-rate ratios for logistic, Cox and Poisson models. Give the 95% confidence interval wherever one exists.

Can I run Pearson and Spearman and report the one that is significant?

No. Choose the test before you see the result and say why. In the malaria survey, knowledge against practice gave Pearson r = 0.23 (p = 0.018) but Spearman rho = 0.18 (p = 0.066); the scores had three to nine values with many ties, so the rank-based test matched the data. If you run both as a sensitivity analysis, report both.

References

1. Nayak BK, Hazra A. How to choose the right statistical test? Indian J Ophthalmol. 2011;59(2):85–86. https://doi.org/10.4103/0301-4738.77005
2. Gonzalez-Chica DA, Bastos JL, Duquia RP, Bonamigo RR, Martínez-Mesa J. Test of association: which one is the most appropriate for my study? An Bras Dermatol. 2015;90(4):523–528. https://doi.org/10.1590/abd1806-4841.20154289
3. Ben Suleiman A, Shuler CF, Hieawy A, von Bergmann H. Statistical errors and reporting deficiencies in clinical prosthodontic publications, 2019–2024. J Prosthodont. 2026;35(4):460–469. https://doi.org/10.1111/jopr.70086
4. Strasak AM, Zaman Q, Pfeiffer KP, Göbel G, Ulmer H. Statistical errors in medical research: a review of common pitfalls. Swiss Med Wkly. 2007;137(3–4):44–49. https://doi.org/10.4414/smw.2007.11587
5. Shatz I. Assumption-checking rather than (just) testing: the importance of visualization and effect size in statistical diagnostics. Behav Res Methods. 2023;56(2):826–845. https://doi.org/10.3758/s13428-023-02072-x
6. Delacre M, Lakens D, Leys C. Why psychologists should by default use Welch’s t-test instead of Student’s t-test. Int Rev Soc Psychol. 2017;30(1):92–101. https://doi.org/10.5334/irsp.82
7. Kerby DS. The simple difference formula: an approach to teaching nonparametric correlation. Compr Psychol. 2014;3:11.IT.3.1. https://doi.org/10.2466/11.IT.3.1
8. Joko S, Poumachu Y, Ouko R, Kala-Chouakeu NA, Bamou R, Elanga-Ndillé E, et al. Knowledge, attitudes and practices towards malaria in a peri-urban Cameroonian village: persistent silent transmission despite satisfactory awareness. PLoS One. 2026;21(5):e0348617. Open access, CC BY 4.0; supplementary datasets S1 and S3 recounted by Editverse in R. https://doi.org/10.1371/journal.pone.0348617
9. Hunter MK, Russell AJ, Maldonado G, Burstyn I. Exploring the proper use of p-values and confidence intervals in leading epidemiology journals. Glob Epidemiol. 2026;11:100247. https://doi.org/10.1016/j.gloepi.2026.100247
10. Dwivedi AK, Shukla R. Evidence-based statistical analysis and methods in biomedical research (SAMBR) checklists according to design features. Cancer Rep. 2020;3(4):e1211. https://doi.org/10.1002/cnr2.1211

Editverse Research Team | Updated October 2026