A study that finds nothing is not automatically a failure, and a study that finds something is not automatically true. What separates a trustworthy null result from a wasted one, and a real discovery from a lucky p-value, usually comes down to a calculation most researchers run before they collect a single data point. Power analysis answers one specific question. Given the smallest effect worth caring about, how many participants, samples, or observations does a study need to have a good chance of detecting it? Skip that calculation and a study can end up too small to find an effect that genuinely exists, or, less often, larger than it needs to be, burning through budget and participants’ time for no real gain.
Ethics boards, journal reviewers, and grant panels all ask for this number now, and for good reason. Decades of published trials went ahead with sample sizes chosen by habit, budget, or guesswork rather than by calculation, and the research literature on that pattern is not flattering. This guide walks through the four numbers a power analysis depends on, works through a full calculation with real arithmetic, covers power analysis for the four designs researchers use most often, and points out where studies most often get the math wrong.
In Simple Words
A power analysis is arithmetic you do before a study starts, not after it ends. It answers one practical question. How many people, animals, or samples do you need so that if a real effect exists, the study actually has a good chance of catching it? The calculation needs four ingredients working together. How big an effect matters to you, how much risk of a false alarm you will accept, how reliably you want to catch a true effect, and the sample size that makes the other three numbers consistent with each other. Get this right and a null result means something. Skip it and a null result might just mean the study never had a fair chance.
What a Power Analysis Actually Decides
Every hypothesis test carries two ways to get it wrong. A study can reject a true null hypothesis, a false alarm statisticians call a Type I error, or it can fail to reject a false null hypothesis, a miss called a Type II error. Alpha is the risk of a Type I error that a researcher agrees to accept, almost always set at 0.05. Power looks at the same test from the opposite angle. Power equals one minus beta, where beta is the Type II error rate, and it tells you the probability of correctly detecting an effect that is really there.
| Test Decision | Null Hypothesis Is Actually True | Null Hypothesis Is Actually False |
|---|---|---|
| Reject the null | Type I error, rate = α (usually 0.05) | Correct detection, rate = power = 1 − β |
| Fail to reject the null | Correct non-detection, rate = 1 − α | Type II error, rate = β |
Sample size is the lever connecting all of it. Holding effect size and alpha constant, a bigger sample pushes power up, because more data narrows the noise around an estimate and makes a real difference easier to separate from chance. A power analysis works that relationship backward, starting from the power and effect size a researcher wants and solving for the sample size that gets them there.
The Four Numbers Behind a Power Analysis
Four quantities determine each other in a power analysis, and fixing any three pins down the fourth. Effect size describes how large a difference or association counts as meaningful, expressed on a standardized scale so it can be compared across outcomes and studies. Alpha sets the tolerance for a false positive, almost always 0.05 in applied research, sometimes tightened to 0.01 when a false alarm would be expensive or dangerous. Power sets the tolerance for missing a real effect, conventionally 0.80 and increasingly 0.90 for confirmatory or high-stakes work. Sample size falls out once the other three are fixed.
Eighty percent became the conventional power target largely because of Jacob Cohen’s influential work on effect size and sample size, and it still shows up as the default in most journal and funder guidance. A growing number of methodologists now argue for 90% instead, given how often a real effect turns out smaller than researchers initially hoped.
Cohen also proposed benchmark values for small, medium, and large effects, and researchers across psychology, medicine, and epidemiology still reach for them when no better estimate exists from prior data. These are conventions, not laws of nature. A pilot study or a published effect size from closely related research beats a generic guess every time.
| Test | Effect Size Measure | Small | Medium | Large |
|---|---|---|---|---|
| Two-group comparison | Cohen’s d | 0.2 | 0.5 | 0.8 |
| Association between two continuous variables | Correlation r | 0.1 | 0.3 | 0.5 |
| Comparing three or more group means | Cohen’s f | 0.10 | 0.25 | 0.40 |
| Multiple regression | Cohen’s f² | 0.02 | 0.15 | 0.35 |
| Categorical association | Cohen’s w | 0.10 | 0.30 | 0.50 |
Picking the wrong row on that table is one of the more common ways a power analysis goes wrong. A medium effect in one field can be optimistic or pessimistic in another, and defaulting to “medium” without checking whether that matches what similar studies actually found is a guess dressed up as a calculation. Understanding how alpha and statistical significance relate to each other is worth reviewing before locking in these numbers, since the two decisions interact.
Underpowered Studies Are Not a Hypothetical Problem
The consequences of skipping this step are documented, not theoretical. A re-examination of 71 clinical trials that had been published as negative found that 67 of them, 94% of the sample, carried more than a 10% risk of having missed a genuine 25% treatment improvement, simply because they never enrolled enough patients to see it. Fifty of the same 71 trials, 70%, could have missed an improvement as large as 50% (Freiman et al., 1978, New England Journal of Medicine).
A follow-up analysis by Moher, Dulberg, and Wells looked at 383 trials published in JAMA, The Lancet, and the New England Journal of Medicine across 1975, 1980, 1985, and 1990. Among the 70 negative two-group trials with usable outcome data, only 16% had the statistical power to detect a 25% relative difference, and only 36% could detect a 50% relative difference, figures that barely moved across fifteen years of publishing. Reporting did improve over that period. Sample size calculations went from appearing in 0% of these papers in 1975 to 43% by 1990, real progress, though 43% still means most trials with negative results were not showing their work (Moher et al., 1994, JAMA).
Across four decades of meta-research, the same failure keeps resurfacing. Studies get built around whatever sample size a budget or a recruitment window allows, then get analyzed as though that number had been chosen to detect a specific, pre-stated effect. Usually it was not. Randomized controlled trial design and analysis depends on getting this piece right before a single participant is enrolled, since a trial cannot be rescued statistically once it is already underpowered and underway.
Choosing How to Run Your Power Analysis
Three practical routes lead to a sample size, and the right one depends on how standard the design is. For textbook designs, a two-group comparison, a one-way ANOVA, a correlation, a standard regression, closed-form formulas exist and free software applies them instantly. G*Power, built by a team at Heinrich Heine University Düsseldorf and documented in a widely cited methods paper, covers most of these tests through a point-and-click interface and remains the most commonly recommended free option in methods courses and journal guidance (Faul et al., 2007, Behavior Research Methods). R users get equivalent calculations through the pwr package, which mirrors G*Power’s output and fits naturally into a reproducible analysis script.
Formulas and G*Power both assume a fairly standard version of a design. Once a study involves clustering, patients nested within clinics for example, repeated measurements on the same person, planned missing data, or a non-standard model such as a mixed-effects or Bayesian analysis, closed-form formulas stop being reliable and simulation becomes the safer choice. A simulation-based power analysis generates thousands of synthetic datasets under an assumed effect size and analysis model, runs the actual planned statistical test on each one, and reports the proportion of runs that reach significance. Setting it up takes longer than typing numbers into G*Power, but it handles complexity closed-form formulas cannot.
Power Analysis for the Four Designs You Will Actually Run
Below are the four designs that cover most applied research, with the formula logic behind each and a real calculated example at a medium effect size, an alpha of 0.05, and 80% power. Every figure below came from the same statistical software researchers use for their own protocols, not from a rule of thumb.
Two Independent Groups (t-test)
Comparing a mean between two independent groups, a treatment arm against control, or an exposed group against an unexposed one, is the design most sample size discussions default to. Power here depends on Cohen’s d, the difference between group means expressed in standard deviation units. Logistic regression handles a similar two-group comparison when the outcome is binary rather than continuous, using a related but distinct calculation covered further down.
One-Way ANOVA (Three or More Group Means)
Once a comparison grows past two groups, an ANOVA replaces the t-test and the effect size shifts to Cohen’s f, which summarizes how spread out the group means are relative to the variability within each group. Designs that add clustering on top of multiple groups, such as a trial that randomizes clinics rather than individual patients, need a further adjustment called the design effect, which inflates the required sample size to account for patients in the same cluster resembling each other more than patients in different clusters.
Correlation (Association Between Two Continuous Variables)
Testing whether two continuous variables move together, blood pressure and age, or a questionnaire score and an outcome measure, uses Pearson’s r directly as the effect size. Choosing the right test in the first place matters here more than it might seem, since a relationship that looks linear in theory but is not in the actual data needs a different, often less powerful, non-parametric approach instead.
Multiple Regression (Testing Predictors of an Outcome)
Regression models testing several predictors at once, or a single predictor while adjusting for covariates, use Cohen’s f², a ratio comparing variance explained by the predictors of interest against variance left over. Multivariable regression in clinical research commonly needs more participants than a simple two-group comparison once several covariates enter the model, and logistic regression for a binary outcome carries its own additional rule of thumb on top of the standard power calculation, often stated as roughly ten observed events per predictor variable.
| Design | Medium Effect Size | Required Sample Size |
|---|---|---|
| Two independent groups (t-test) | d = 0.5 | 64 per group (N = 128) |
| One-way ANOVA, 3 groups | f = 0.25 | 53 per group (N = 159) |
| Correlation (Pearson r) | r = 0.3 | N = 85 |
| Multiple regression, 5 predictors | f² = 0.15 | N = 92 |
| Chi-square, 2×2 table | w = 0.3 | N = 88 |
All figures calculated at α = 0.05, two-sided, and 80% power, using R’s stats and pwr packages, which match G*Power’s output for the same inputs.
A Worked Example You Can Reuse
Here is the full calculation behind the first row of that table, worked by hand so the logic is visible rather than hidden inside software.
Picture a two-arm trial comparing a new intervention to standard care on a continuous outcome. Based on prior literature, a standardized difference of Cohen’s d = 0.5 is the smallest effect worth detecting, a conventional medium effect. Alpha is set at 0.05 two-sided, the standard threshold, with 80% power, an 80% chance of detecting that effect if it truly exists.
The normal-approximation formula for a two-group comparison is n per group equals 2 times the square of (z for alpha over 2, plus z for power), divided by d squared. The standard values are 1.96 for z at 0.025 and 0.84 for z at 80% power.
= 2 × (2.80)² ÷ 0.25
= 2 × 7.84 ÷ 0.25
= 15.68 ÷ 0.25
= 62.8, rounded up to 63 participants per group
That normal approximation lands close, but it is not exactly what G*Power or R compute. Both use the exact non-central t-distribution instead of the shortcut above, which is slightly more conservative. Running the identical inputs, d = 0.5, alpha = 0.05 two-sided, 80% power, through R’s power.t.test function returns 63.8 per group, rounding up to 64 participants per group, 128 total, the number a methods section would actually report and the same figure G*Power produces for this design.
The gap between the approximate and exact answer matters for one reason. Trusting a rough shortcut instead of the exact method software actually uses is the kind of small shortfall that compounds. Recruit only 40 participants per group instead of 64, a shortfall that can look minor on a budget spreadsheet, and achieved power for the same d = 0.5 effect drops to just under 60%, not the 80% the study was designed around. That is close to a coin flip for missing a real, moderate effect entirely, and it is precisely the shortfall pattern Freiman and Moher documented across decades of published trials.
| Sample Size Per Group | Achieved Power (d = 0.5, α = 0.05) |
|---|---|
| 40 | 59.8% |
| 64 | 80.2% |
| 86 | 90.0% |
For a design matching one of the rows in the earlier table, Editverse’s free sample size calculator runs this exact math without requiring R or G*Power installed locally, and the step-by-step guide to using it walks through every input field.
Mistakes That Quietly Break a Power Analysis
- Computing post-hoc power from your own result. Power calculated after the fact from a study’s own p-value is mathematically just a rescaled version of that p-value. It adds no real information and most methodologists advise against reporting it.
- Guessing an effect size with no pilot data or literature basis. Defaulting to “medium” because it sounds reasonable turns a calculation into a guess wearing a lab coat.
- Ignoring expected dropout or missing data. A power analysis gives the analyzed sample size needed, not the enrolled one. Expected attrition has to be added on top of the calculated number, not folded into it after the fact.
- Not adjusting for clustering, repeated measures, or multiple comparisons. Each of these changes the effective sample size or the alpha actually being spent, and skipping the adjustment quietly erodes the power a study believes it has.
- Treating power analysis as a box to check once for an ethics committee. Adding an arm, changing the primary outcome, or revising the analysis plan mid-study should trigger a fresh calculation, and that step gets skipped more often than it should.
- Confusing statistical significance with practical significance. A very large sample can be well powered to detect a difference too small to matter clinically or practically, which is its own kind of design failure.
Writing Up Your Power Analysis for a Manuscript or Grant
A methods section needs enough information for a reviewer or a grant panel’s statistician to actually check the power analysis, not just trust that one happened. That means spelling out where the assumed effect size came from, a pilot study, a published meta-analysis, or a stated minimal clinically important difference, alongside the alpha and power targets and the exact software or formula used.
- State the effect size and its source. “We assumed a medium effect, d = 0.5, based on [prior study]” is checkable. “We assumed a reasonable effect” is not.
- Report alpha and power explicitly. Do not make a reader infer these from the final sample size number alone.
- Name the software and test. G*Power, R’s
pwrpackage, PASS, and nQuery can produce slightly different numbers at the edges, so naming the tool lets someone else reproduce the figure. - Report the final number honestly, including any inflation for dropout or clustering. If fewer participants were enrolled than the calculation called for, say so and discuss the resulting power directly rather than leaving it for a reviewer to notice.
Grant reviewers read this section closely for a reason. A statistical approach section that does not hold together is one of the more common, avoidable reasons a proposal comes back with a weak score on rigor, and a compliance-focused grant proposal review catches that kind of gap long before submission rather than leaving it to an unlucky reviewer draw.
Getting Hands-On Help With Your Power Analysis
Some power analyses are routine. Others involve clustering, missing data, or a model complex enough that a template calculation will not hold up under review. Editverse’s statisticians handle both ends of that range.
Dissertation & Manuscript Statistical Analysis
$600
PhD statisticians handle the power analysis, sample size justification, and full analysis plan in SPSS, R, or STATA.
Grant Proposal Editing & Compliance Review
$1,190
A check that your stated power analysis actually matches your aims, your analysis plan, and your funder’s requirements before you submit.
Frequently Asked Questions About Power Analysis
What sample size do I need for a medium effect size?
It depends on the design, but as a reference point, detecting a medium effect (Cohen’s d = 0.5) between two independent groups at the conventional 80% power and 0.05 alpha needs 64 participants per group, 128 total. A one-way ANOVA across three groups at a comparable medium effect needs about 53 per group. Larger effects need far fewer participants, and smaller effects need dramatically more, which is why guessing the wrong effect size is such a costly mistake.
Should I use 80% or 90% power?
Eighty percent is still the default in most published research and most granting agency templates, but it accepts a one-in-five chance of missing a real effect. Confirmatory trials and registration studies increasingly specify 90% power instead, which raises the required sample size substantially, roughly 30 to 40% higher than the equivalent 80% calculation for a two-group t-test.
What is the difference between a priori and post-hoc power?
A priori power analysis happens before data collection and answers the question a study design should answer. How large a sample is needed to detect a specified effect. Post-hoc, or observed, power is calculated after the fact from a study’s own result and its own p-value, and most methodologists consider it close to uninformative, since it is mathematically just a transformation of the p-value already in hand rather than new evidence about anything.
Can I run a power analysis for a study I have already completed?
Only in a limited, backward-looking way. It is possible to calculate what sample size would have been needed for a specific effect size, which is useful for planning a follow-up or replication study, but calculating power from an already-observed effect and p-value does not tell a reviewer anything persuasive about the finished study.
Does power analysis apply to qualitative research?
No. Power analysis is a frequentist statistical concept built around hypothesis tests, p-values, and sampling distributions, none of which apply to qualitative methods. Qualitative researchers justify sample size differently, commonly through thematic saturation, the point at which additional interviews or data sources stop producing new codes or themes.
What effect size should I use if I have no pilot data?
A published effect size from a closely related study or a meta-analysis on the same outcome is always the better choice over a generic guess. When neither exists, Cohen’s conventional medium effect is a defensible fallback, but the methods section should state plainly that it is a convention rather than an estimate grounded in the study’s specific population, since a reviewer will likely ask.
How does dropout or attrition change the calculation?
A power analysis gives the analyzed sample size needed, not the enrolled one. Expecting 15% of participants to drop out or be lost to follow-up means dividing the target sample size by 0.85 to get the enrollment target. A study needing 128 analyzed participants with 15% expected attrition should enroll roughly 151.
Is G*Power free?
Yes. G*Power is free, developed and maintained by researchers at Heinrich Heine University Düsseldorf, and covers t-tests, ANOVA, correlation, regression, chi-square, and several less common designs through a downloadable desktop application for Windows and Mac.
What if my design does not match a standard formula?
Simulation-based power analysis is the standard answer for anything a closed-form formula cannot handle cleanly, including mixed-effects models, clustered or multilevel designs, and Bayesian analyses. It takes longer to set up than G*Power but works for essentially any design that can be coded and analyzed in R or a similar environment.
Key Takeaways
- ✓ Fix three of effect size, alpha, power, and sample size, and a power analysis solves for the fourth
- ✓ 0.05 alpha and 80% power are conventional starting points, not fixed rules
- ✓ Decades of published trials, from a 1978 audit to a 2021 re-analysis, keep finding the same underpowering problem
- ✓ G*Power and R’s pwr package cover standard designs for free; simulation covers the rest
- ✓ A worked calculation should show its arithmetic, not just its final number
- ✓ Inflate the enrollment target for expected dropout before locking in a recruitment plan
- ✓ Report the effect size, its source, alpha, power, and software used, every time
Need Your Power Analysis Checked Before You Submit?
Editverse’s statisticians review sample size justifications, grant approach sections, and full analysis plans for researchers across every field.
Related Editverse Resources
→ Grasp the Basics of Understanding Statistical Significance
→ Statistical Tests Guide: Making the Right Choice
→ Randomized Controlled Trials (RCTs): Design and Analysis
→ Master Cluster Randomized Trials & Multi-level Modeling
→ Logistic Regression in Medical Diagnostics
→ Multivariable Regression in Clinical Studies
→ Chi-Square Tests Simplified
→ Avoiding Common Statistical Errors in Research Papers
→ How to Use the Editverse Free Sample Size Calculator
References
- Freiman JA, Chalmers TC, Smith H Jr, Kuebler RR. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 “negative” trials. N Engl J Med. 1978;299(13):690-694. https://pubmed.ncbi.nlm.nih.gov/355881/
- Moher D, Dulberg CS, Wells GA. Statistical power, sample size, and their reporting in randomized controlled trials. JAMA. 1994;272(2):122-124. https://pubmed.ncbi.nlm.nih.gov/8015121/
- Button KS, Ioannidis JPA, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14(5):365-376. https://pubmed.ncbi.nlm.nih.gov/23571845/
- Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39(2):175-191. https://pubmed.ncbi.nlm.nih.gov/17695343/
- Reito A, Raittio L, Helminen O. Revisiting the Sample Size and Statistical Power of Randomized Controlled Trials in Orthopaedics After 2 Decades. JBJS Rev. 2021;9(3). https://pubmed.ncbi.nlm.nih.gov/32224628/
- Heinrich Heine University Düsseldorf. G*Power: statistical power analysis software. https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower
- Champely S, et al. pwr: Basic Functions for Power Analysis. The Comprehensive R Archive Network. https://cran.r-project.org/package=pwr
- Boston University School of Public Health. Power and Sample Size Determination. https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_power/bs704_power_print.html