Published  |  Last updated

Quick Answer: A t-test compares two means. The p-value shows how compatible the data are with no difference, and the effect size shows how large the difference is.1 Report both with a 95% confidence interval. In a published randomised trial of virtual reality (VR) glasses during chest tube removal (41 patients per group), Editverse recomputed pain immediately after removal from the published means and SDs: mean difference −1.68 VAS points (95% CI −2.56 to −0.80), Welch t = −3.81, df = 70.2, p = 0.0003, Hedges’ g = −0.83 (95% CI −1.28 to −0.38).2

T-Test, P-Value and Effect Size: What Each One Tells You

Three numbers are usually reported together when two groups are compared, and each answers a different question. Mixing them up is the most common error in Results sections.

Swipe sideways to see every column.

NumberWhat it answersWhat it does not tell you
t statisticHow far apart the two group means are, in units of their standard error.Whether the difference matters clinically.
p-valueHow incompatible the data are with the statistical model, including the hypothesis of no difference.The probability that the hypothesis is true, or the size or importance of the effect.1
Mean difference with 95% CIThe size of the difference in the original units, and the range of differences most compatible with the data.Whether the result will repeat; the 95% describes a long run of intervals, not one interval.3
Effect size (Cohen’s d, Hedges’ g)The difference in standard deviation units, so studies with different scales can be compared.Clinical importance by itself; the 0.2, 0.5 and 0.8 benchmarks are arbitrary.4

Which T-Test Should You Use?

The choice follows the design, not the software menu.

  1. 1One-sample. Compares one group’s mean with a fixed value.
  2. 2Independent samples. Compares the means of two separate groups, for example VR versus routine care, which is the case below.
  3. 3Paired. Compares two measurements on the same people, for example pain before and after a treatment. Effect sizes for paired designs should take the correlation between the measures into account.4

Student’s t-test assumes normality and equal variances. Delacre, Lakens and Leys argue that equal variances will seldom hold in psychological research and that Welch’s t-test, which does not assume them, should be the default.5 A 2022 correction to that paper states that its errors do not affect the main message, that Welch’s t-test should be privileged over Student’s t-test when comparing group means.6 A z-test treats the standard deviation as known, while a t-test estimates it from the sample, so its critical value is larger: 1.99 with 80 degrees of freedom against 1.96 for the normal distribution (computed in R).

A Real Case: Virtual Reality Glasses During Chest Tube Removal

Yanik and colleagues randomised 82 patients after open-heart surgery to wear VR glasses with forest-walk content from five minutes before to five minutes after chest tube removal (CTR), or to receive routine care, 41 per group. Pain was rated on a 10-cm visual analogue scale (VAS).2 The paper is open access under CC BY 4.0,7 which we confirmed in the licence field of the PubMed Central record.8 Because of that licence, we can reuse its numbers and redraw them. The table below gives the published means and standard deviations that Editverse used.

MeasureControl, mean (SD)VR group, mean (SD)
Pain before CTR3.76 (2.49)2.61 (1.51)
Pain immediately after CTR6.80 (2.34)5.12 (1.58)
Pain 5 min after4.24 (1.85)3.41 (1.28)
Pain 15 min after2.66 (1.57)1.95 (0.97)
Pain 30 min after2.19 (1.38)1.59 (0.74)
SpO₂ immediately after CTR91.54 (3.81)93.24 (3.58)

The published table also reports each group’s SD, which is all a t-test needs: two means, two SDs and two sample sizes. The authors themselves fitted a mixed-effects model adjusted for baseline, with Bonferroni-adjusted post hoc comparisons, and reported adjusted differences.2 What follows is a simpler, unadjusted teaching recomputation from the rounded published values. It is not a re-analysis of the trial and it should not replace the authors’ model.

Worked T-Test: Pain Immediately After Chest Tube Removal

For the Welch t-test, the standard error of the difference is the square root of (2.34²/41 + 1.58²/41) = 0.441. The difference in means is 5.12 − 6.80 = −1.68 VAS points, so t = −1.68 / 0.441 = −3.81. The Welch-Satterthwaite formula gives 70.2 degrees of freedom, and the two-sided p-value is 0.0003. The 95% confidence interval for the difference is −2.56 to −0.80 VAS points.

Forest plot of the mean difference in VAS pain score, VR minus control, with 95% confidence intervals. Editverse unadjusted estimates: immediately after chest tube removal -1.68 (-2.56 to -0.80), 5 min -0.83 (-1.53 to -0.13), 15 min -0.71 (-1.29 to -0.13), 30 min -0.60 (-1.09 to -0.11). Authors' adjusted estimates: -1.26 (-1.85 to -0.68), -0.41 (-0.99 to 0.18), -0.28 (-0.87 to 0.30), -0.19 (-0.77 to 0.40).
Mean difference in VAS pain, VR minus control, with 95% CI: Editverse unadjusted Welch estimates (navy) beside the authors’ published adjusted estimates (orange). Computed in R from the published summary data.

The two sets of estimates agree on the direction and on the biggest effect, immediately after CTR, but not on size or on the later time points. The authors’ adjusted difference is smaller (−1.26, 95% CI −1.85 to −0.68) than our unadjusted −1.68, and their intervals for 5, 15 and 30 minutes include zero while ours do not. The next sections explain why.

Cohen’s d and Hedges’ g With Confidence Intervals

Cohen’s d divides the mean difference by the pooled SD: d = −1.68 / 2.00 = −0.84. Sample-based estimates of the standardised difference overestimate the population effect, and the bias-corrected version is Hedges’ g.4 The correction factor with 80 degrees of freedom is 1 − 3/(4 × 80 − 1) = 0.991, giving g = −0.83. We obtained the 95% confidence interval from the noncentral t distribution: −1.28 to −0.38. Effect sizes have their own confidence intervals, which Lakens notes are often very large in experimental psychology.4

Forest plot of Hedges' g with 95% confidence intervals, VR minus control, 41 per group: baseline pain g = -0.55 (-0.99 to -0.11), pain immediately after chest tube removal g = -0.83 (-1.28 to -0.38), oxygen saturation immediately after g = 0.46 (0.02 to 0.89).
Hedges’ g with 95% CI for baseline pain, pain immediately after chest tube removal and oxygen saturation immediately after. Computed in R from the published summary data.

By Cohen’s rough benchmarks the pain effect is large (g about 0.8), but the interval runs from a small-to-medium effect (−0.38) to a very large one (−1.28), so a single number overstates our certainty. Lakens cautions that the benchmarks are arbitrary and should not be interpreted rigidly.4

All Six Comparisons in One Table

All values are Editverse’s unadjusted Welch recomputation from the published means and SDs with 41 per group, assuming every patient contributed at each time point (the published table gives no per-time sample sizes).

Swipe sideways to see every column.

MeasureMean difference, VR − control (95% CI)Welch t (df)pHedges’ g (95% CI)
Pain before CTR−1.15 (−2.06 to −0.24)−2.53 (65.9)0.014−0.55 (−0.99 to −0.11)
Pain immediately after CTR−1.68 (−2.56 to −0.80)−3.81 (70.2)0.0003−0.83 (−1.28 to −0.38)
Pain 5 min after−0.83 (−1.53 to −0.13)−2.36 (71.2)0.021−0.52 (−0.95 to −0.08)
Pain 15 min after−0.71 (−1.29 to −0.13)−2.46 (66.7)0.016−0.54 (−0.97 to −0.10)
Pain 30 min after−0.60 (−1.09 to −0.11)−2.45 (61.2)0.017−0.54 (−0.97 to −0.10)
SpO₂ immediately after CTR1.70 (0.08 to 3.32)2.08 (79.7)0.0410.46 (0.02 to 0.89)

When the P-Value and the Effect Size Tell Different Stories

Baseline imbalance. Before chest tube removal, the VR group already reported less pain (2.61 against 3.76; g = −0.55, Welch p = 0.014). An unadjusted comparison after the procedure therefore mixes the effect of VR with that head start. The authors’ model adjusts for baseline, and their adjusted difference (−1.26) is smaller than the unadjusted one (−1.68).2

Several time points. The trial compared groups at four times after removal. Unadjusted Welch p-values at 5, 15 and 30 minutes are 0.021, 0.016 and 0.017. Multiplying the unrounded p-values by four, as a simple Bonferroni illustration, gives 0.084, 0.065, 0.068, none below 0.05. The authors report Bonferroni-adjusted comparisons in which only the first time point after CTR differed.2 Greenland and colleagues warn that selecting which analyses to present by their p-values can produce small p-values even when the test hypothesis is correct.3

A small p-value is not a large effect. Oxygen saturation immediately after CTR gives Welch p = 0.041 with a mean difference of 1.70 percentage points, but g = 0.46 and a confidence interval of 0.02 to 0.89, which comes close to zero. The authors’ own adjusted difference at that time point is larger, 2.70 percentage points (95% CI 1.88 to 3.53; adjusted p < 0.001), so this unadjusted example understates the trial's SpO₂ result rather than weakening it.2 The ASA lists this distinction among its principles: a p-value does not measure the size of an effect or the importance of a result.1

What This Case Shows and What It Does Not

  1. 1It shows how the numbers are built. Two means, two SDs and two sample sizes are enough to reproduce t, df, p, the confidence interval, d and g, and to see which assumption changes the answer.
  2. 2It does not show that VR works. Our recomputation is unadjusted and uses rounded published values. The trial was run in one university hospital; its authors report that participants and the researcher collecting outcomes could not be blinded and that there was no placebo or active comparison group.
  3. 3It does not replace the authors’ model. Use their adjusted estimates for any statement about the trial’s findings. We computed the Editverse numbers in R 4.3.3.

How to Report a T-Test

  1. 1State the design and the test. Say independent or paired, and say Welch or Student.
  2. 2Give the group sizes and summary statistics. Report n, mean and SD for each group.
  3. 3Give the difference with its interval. Write the mean difference in the original units with a 95% CI.
  4. 4Give t, df and the exact p-value. For example t(70.2) = −3.81, p = 0.0003.
  5. 5Add an effect size with its interval. Name the measure, Cohen’s d or Hedges’ g, and give the 95% CI.
  6. 6Say what was adjusted. If you adjusted for baseline or for several comparisons, say so, and say whether the test was planned.

Reproduce the Calculation in R

This base R code reproduces the values above from summary statistics. We ran it in R 4.3.3.9 It uses only base R functions. Replace the values with your own means, SDs and sample sizes.

m1 <- 6.80; s1 <- 2.34; n1 <- 41   # control
m2 <- 5.12; s2 <- 1.58; n2 <- 41   # VR
se <- sqrt(s1^2/n1 + s2^2/n2)
df <- se^4 / ((s1^2/n1)^2/(n1-1) + (s2^2/n2)^2/(n2-1))
t  <- (m2 - m1)/se
p  <- 2*pt(-abs(t), df)
ci <- (m2 - m1) + c(-1, 1)*qt(0.975, df)*se
sp <- sqrt(((n1-1)*s1^2 + (n2-1)*s2^2)/(n1+n2-2))
d  <- (m2 - m1)/sp
J  <- 1 - 3/(4*(n1+n2-2) - 1)
g  <- d*J
ncp <- d/sqrt(1/n1 + 1/n2)
lim <- sapply(c(0.975, 0.025), function(q)
  uniroot(function(x) pt(ncp, n1+n2-2, ncp = x) - q, ncp + c(-10, 10))$root)
d_ci <- lim*sqrt(1/n1 + 1/n2); g_ci <- d_ci*J

Need the Analysis Run or Checked? Consult Editverse

Editverse subject-matter experts work to COPE, ICMJE and GPP 2022 guidance, with institutional and country-specific ethics followed.

Choose the stage you are at.

Editverse logo

Editverse

Basic Statistical Analysis

Need the right test run on your trial or study data?

You send: your dataset, research question and software preference.

You get: descriptive statistics and tests such as the independent or paired t-test, with publication-ready tables and a results narrative.

When: You have data and want the analysis done and reported. Standard turnaround is 3-5 days for small datasets (up to 100 subjects and 10 variables).

Basic Statistical Analysis · From $350

Get my analysis run →

Editverse logo

Editverse

Statistical Results Interpretation

Have the output but not the write-up for your Results section?

p = 0.03

You send: your existing statistical output and a short description of the study.

You get: an interpretation of the output, a Results narrative, table and figure recommendations and reviewer-ready explanations.

When: The analysis is run and needs explaining. Standard turnaround is 3-4 business days.

Statistical Results Interpretation · $250

Interpret my results →

Editverse logo

Editverse

Manuscript Writing

Ready to turn the whole project into a submitted paper?

You send: your data, notes and target journal.

You get: a manuscript written by subject-matter experts; the service description lists statistical review and verification among its quality-assurance steps.

When: The project is finished and the paper is not.

Manuscript Writing Services · $1,500

Write my paper →

Free Research Tools from Editverse

Frequently Asked Questions

What is the difference between a p-value and an effect size?

A p-value indicates how incompatible the data are with a specified statistical model, and it does not measure the size of an effect or its importance.1 An effect size such as Hedges’ g measures how large the difference is. In the virtual reality trial that Editverse recomputed, pain immediately after chest tube removal gave p = 0.0003 with g = −0.83, while oxygen saturation gave p = 0.041 with g = 0.46, so a smaller p-value came with a larger effect.

Should I use Student’s t-test or Welch’s t-test?

Delacre, Lakens and Leys argue that Welch’s t-test should be the default: it controls the Type 1 error rate better when variances are unequal and loses little when the assumptions of Student’s test are met.5 In the recomputed trial the groups were the same size, so both versions give t = −3.81 and p = 0.0003; Welch uses 70.2 degrees of freedom and Student’s pooled test uses 80.

Do I need to test for normality before a t-test?

Opinions differ and the evidence is mixed. A simulation by Rochon, Gondan and Kieser found that a preliminary Shapiro-Wilk test altered the Type I error rates of the test that followed, although the overall error rate and power of the two-stage procedure stayed acceptable in the cases they simulated.10 Delacre and colleagues also advise against choosing between Student’s and Welch’s test by a test of equal variances.5 Look at your data, consider the design and ask a statistician if the sample is small or skewed.

What counts as a small, medium or large Cohen’s d?

Cohen’s benchmarks are d = 0.2 small, 0.5 medium and 0.8 large, but Lakens notes these values are arbitrary and should not be interpreted rigidly, and that small effects can have large consequences. Judge size against related findings and the clinical meaning of the difference, for example VAS points.

What is the difference between Cohen’s d and Hedges’ g?

Both are standardised mean differences. Sample-based estimates overestimate the population effect, and the bias-corrected version of Cohen’s d is called Hedges’ g.4 With 82 participants the correction factor is 0.991 (computed in R), so d = −0.84 becomes g = −0.83 for pain immediately after chest tube removal.

What does a 95% confidence interval for the mean difference tell me?

Greenland and colleagues explain that the 95% level is a coverage property of a long sequence of intervals from valid models, not a property of any single interval.3 In practice the interval shows the range of differences that are most compatible with the data under the model. For pain immediately after chest tube removal, Editverse computed −1.68 VAS points with an interval of −2.56 to −0.80, which excludes zero.

Does p below 0.05 prove the treatment works?

No. The ASA states that p-values do not measure the probability that the studied hypothesis is true, and that conclusions should not rest only on whether a p-value passes a threshold.1 In the recomputed trial, the groups already differed in pain before chest tube removal (Welch p = 0.014), and the authors state that participants and the researcher could not be blinded, so the simple unadjusted t-test overstates what the adjusted analysis supports.

How many participants did the trial need for a t-test?

The authors planned for Cohen’s d = 0.80 with a two-sided significance level of 0.05 and 90% power, which gave 34 per group, and added about 20% to reach 41 per group. Editverse reproduced the first step in R with power.t.test, which returns 33.8 per group, rounded up to 34. With 41 per group and 80% power, the smallest standardised difference that test could detect is d = 0.63 (computed in R).

References

  1. American Statistical Association. ASA releases statement on statistical significance and p-values (6 principles; 7 March 2016). www.amstat.org/asa/files/pdfs/p-valuestatement.pdf
  2. Yanik TÇ, Gülşen M, Uğraş GA, Arslan S, Özdemir FE. The effect of virtual reality glasses used during chest tube removal following open-heart surgery on pain and hemodynamic parameters: a randomized controlled trial. BMC Cardiovasc Disord. 2026;26:993. Open access, CC BY 4.0; summary data used, t-tests and effect sizes recomputed and figures drawn by Editverse in R. doi.org/10.1186/s12872-026-06778-6
  3. Greenland S, Senn SJ, Rothman KJ, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31:337-350. doi.org/10.1007/s10654-016-0149-3
  4. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front Psychol. 2013;4:863. doi.org/10.3389/fpsyg.2013.00863
  5. Delacre M, Lakens D, Leys C. Why psychologists should by default use Welch’s t-test instead of Student’s t-test. Int Rev Soc Psychol. 2017;30(1):92-101. doi.org/10.5334/irsp.82
  6. Delacre M, Lakens D, Leys C. Correction: Why psychologists should by default use Welch’s t-test instead of Student’s t-test. Int Rev Soc Psychol. 2022;35(1):21. doi.org/10.5334/irsp.661
  7. Creative Commons. Attribution 4.0 International (CC BY 4.0) licence summary. creativecommons.org/licenses/by/4.0/
  8. National Center for Biotechnology Information. Entrez Programming Utilities Help: EFetch (used to read the licence and tables of the trial in PubMed Central, 31 October 2026). www.ncbi.nlm.nih.gov/books/NBK25499/
  9. R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. Version 4.3.3 used for every computed value in this post. www.R-project.org/
  10. Rochon J, Gondan M, Kieser M. To test or not to test: preliminary assessment of normality when comparing two independent samples. BMC Med Res Methodol. 2012;12:81. doi.org/10.1186/1471-2288-12-81

Editverse Research Team | Updated October 2026