{p_r}
# z: 2 * pnorm(-abs(z)) t: 2 * pt(-abs(t), df)
# chi-square: pchisq(x, df, lower.tail = FALSE) F: pf(x, df1, df2, lower.tail = FALSE)User guide
When to use. Convert a test statistic (z, t, χ² or F) into a p-value under the null hypothesis.
Assumptions. You already have a correctly computed statistic and the right degrees of freedom.
Common mistakes. A p-value is not the probability the null is true, nor the size of an effect.
Worked example
Click Load example data above, then Run Analysis to see a fully worked result with figure, formula and interpretation for this test.
References
- Wasserstein RL, Lazar NA. The ASA statement on p-values. Am Stat. 2016;70:129-133.
In Simple Words: What a P-Value Actually Tells You
A p-value answers one narrow question: if there were truly no effect (the null hypothesis were exactly true), how often would you see a result at least as extreme as the one you observed, by chance alone? A small p-value means your data would be surprising in a world with no effect — so the data are inconsistent with that no-effect assumption.
What it is not: a p-value is not the probability that the null hypothesis is true, it is not the probability your result happened by chance, and it does not measure how big or important an effect is. Those three confusions are the most common errors in the published literature, and reviewers are trained to catch them.
The Complete Guide to P-Values
The p-value is the most frequently reported statistic in biomedical research and, by a wide margin, the most frequently misunderstood. It was introduced by R.A. Fisher as an informal measure of evidence against a null hypothesis, then later fused with the Neyman–Pearson decision framework into the “reject if p < 0.05” rule that dominates journals today. That historical splice is the root of much of the confusion: the number is treated as a yes/no decision rule while being interpreted as a continuous measure of evidence, and the two readings do not always agree.
The formal definition
Given a test statistic computed from your data, the p-value is the probability — assuming the null hypothesis and all other model assumptions are correct — of obtaining a result at least as extreme as the one observed. Formally, for an observed statistic, p = P(|X| ≥ |stat| | H₀) for a two-tailed z or t test. For chi-square and F tests the p-value is the upper tail, p = P(X ≥ stat | H₀). The calculator above converts a z, t, χ² or F statistic (with the correct degrees of freedom) into this probability, and shows you the matching R command so the result is fully reproducible.
The single most important caveat: the p-value is conditional on the entire statistical model being correct — not just the null effect, but also the assumptions about distribution, independence, no selection bias, and no measurement error. A small p-value signals that something in that set of assumptions is implausible, and the null effect is only one candidate. This is the central message of the most widely cited modern guide to the topic (Greenland et al., 2016).
How to interpret your result
- p < 0.001 — very strong evidence against the null under the assumed model. Report the exact value (e.g.,
p = 0.0004), not “p < 0.05”. - 0.001 ≤ p < 0.05 — evidence against the null at the conventional threshold. Interpret in light of effect size and confidence interval.
- 0.05 ≤ p < 0.10 — weak/ambiguous. Do not describe as “trending toward significance” or “approaching significance”; this phrasing is criticised in the methodological literature.
- p ≥ 0.10 — data are reasonably compatible with the null. This is not proof of “no effect” — absence of evidence is not evidence of absence, especially in an underpowered study.
The five rules reviewers enforce
1. Report exact p-values
Write p = 0.03, not p < 0.05. Exact values let readers apply their own evidential threshold. Use p < 0.001 only below that floor.
2. Always pair with an effect size and 95% CI
A p-value alone is uninterpretable for clinical decisions. The confidence interval shows the range of effects compatible with your data; the effect size shows whether the difference matters.
3. Pre-specify one- vs two-tailed
Two-tailed is the default. A one-tailed test is only defensible when a directional hypothesis was registered before seeing the data.
4. Correct for multiple comparisons
Testing 20 outcomes at α = 0.05 yields roughly one false positive by chance. Use Bonferroni, Holm, or false-discovery-rate control and state which.
5. Never interpret “non-significant” as “no difference”
A large p-value in a small study usually means low power, not a true null. Report the CI so readers can judge what effects remain plausible.
Worked Examples
Real clinical scenarios. Enter the statistic into the calculator above to reproduce each p-value.
1. Two-group trial (z / normal)
A trial compares a new antihypertensive to placebo. The standardised test statistic for the difference in mean systolic BP is z = 2.5.
Reading: under the null of no difference, a z this extreme occurs ~1.2% of the time. Report alongside the mean difference and its 95% CI.
2. Small paired study (t)
A before/after study of a rehab protocol on 15 patients gives a paired t = 2.20 with df = 14.
Reading: just under 0.05, but with n = 15 the CI will be wide. This is exactly the case where reporting the interval (not just “significant”) matters.
3. Contingency table (χ²)
A 2×3 table of treatment vs response category yields χ² = 9.5 with df = 2.
Reading: strong evidence of association. Follow up with standardised residuals to see which cells drive it — the p-value alone does not localise the effect.
4. ANOVA omnibus test (F)
Three diets compared on weight change; one-way ANOVA gives F = 4.10 with df1 = 2, df2 = 57.
Reading: at least one group mean differs. The omnibus p does not say which pair — run a post-hoc test (Tukey HSD) with its own correction.
Frequently Asked Questions
Does a p-value tell me the probability my hypothesis is true?
No. This is the most common misinterpretation in the literature. The p-value is computed assuming the null hypothesis is true; it cannot also tell you the probability that the null is true. To get the probability of a hypothesis given the data, you need a Bayesian framework, not a p-value.
Is p = 0.04 “more significant” than p = 0.06?
The 0.05 cutoff is a convention, and the difference between 0.04 and 0.06 is far smaller than it feels. Treating one as a discovery and the other as a null finding is a known error — report exact values and interpret them continuously alongside the effect size and confidence interval.
What’s the difference between statistical and clinical significance?
Statistical significance asks whether an effect is distinguishable from zero given the data. Clinical significance asks whether the effect is large enough to matter to patients. In a large sample, a clinically trivial difference can be highly statistically significant; in a small sample, a clinically important difference can fail to reach significance. Always report both.
Should I use a one-tailed or two-tailed test?
Default to two-tailed. A one-tailed test doubles your apparent power but is only justified when a directional hypothesis was specified before data collection — ideally pre-registered. Reviewers are sceptical of one-tailed tests chosen after seeing the data.
Does this calculator match R and SPSS?
Yes. The calculator uses the same distribution functions as R 4.3.3 (pnorm, pt, pchisq, pf) and outputs the exact R code so you can reproduce and verify every result. Outputs are equivalent to SPSS and Stata for identical inputs and degrees of freedom.
Important Reads & References
Peer-reviewed, PubMed-indexed sources for your methods section and your own understanding.
The essential modern reference — a catalogue of the most common errors in interpreting p-values, confidence intervals, and power. Read this first.
A current, accessible review tracing the p-value from Fisher through the Neyman–Pearson framework, with practical reporting guidance.
Practical, reporting-focused guidance on the “P < 0.05” threshold and how to present p-values responsibly.
Translates the American Statistical Association’s warnings into clinician-friendly language; clarifies the significance-vs-relevance distinction.
On the danger of dichotomising results into “significant / not significant” and the uncertainty this obscures.
A concise primer built around the 2016 ASA statement — good for teaching and quick reference.
Need a Statistician to Verify Your Analysis?
Our biostatisticians provide publication-ready statistical analysis, methods text, and reviewer-response support — with R-based reproducible output.
Get Expert Statistical Help