Conducting Umbrella Reviews: What, How & Step-wise Tips

Published June 20, 2026  |  Last updated November 3, 2026

What Is an Umbrella Review?

An umbrella review is a systematic collection and assessment of multiple systematic reviews and meta-analyses done on a specific research topic.¹ The PRIOR statement lists other names for the same design, including overview of reviews, review of reviews and review of systematic reviews.³ The unit of analysis is a review, not a trial. Belbasis and colleagues describe two common uses: umbrella reviews of interventions, which assess the treatment effects of interventions for a disease, and umbrella reviews of epidemiological associations, which assess many risk factors for one disease (exposure-wide) or many diseases for one risk factor (phenome-wide).¹

When to choose one

Swipe sideways to see every column.

Advantages listed by the same authors: a bird-eye view of many interventions or associations, savings from not searching primary studies from scratch, and a clear picture of gaps. Disadvantages: validity depends on the quality of the included reviews, they hold no information on interventions or associations that no review has examined, and biases in primary studies can be compounded.¹

How to Conduct an Umbrella Review: 10 Steps

The ten steps below combine the sources cited in each step. The numbering is ours; the sources do not all use the same sequence.

1. 1Confirm that an umbrella review is needed. It is informative when several systematic reviews and meta-analyses exist on the topic, and Fusar-Poli and Radua list this check as the first of their ten points.^1,5
2. 2Prespecify and register the protocol. Prespecifying the protocol is their second point.⁵ PRIOR asks authors to give registration details or to state that the overview was not registered.³ The case below was registered in PROSPERO (CRD420261294673).⁴
3. 3Define the question. For interventions, state the population, intervention, comparison and outcomes (PICO); for associations, state the population, risk factor and outcome. The criteria are broader than in a single review, so define the breadth carefully.¹ JBI also frames the question in PICO format.⁶
4. 4Search for reviews. Belbasis and colleagues describe a search in two parts joined with AND: a filter for reviews and meta-analyses (for example, systematic review* OR meta-analys*) and the keywords for the topic.¹ The case searched PubMed, Embase and the Cochrane Library to 25 February 2026 and found 3,581 records.⁴
5. 5Screen and select. Use independent double screening. When reviews overlap, apply a stated rule: there are no set criteria, but options include the most recent meta-analysis or the one with the most studies, and quality should also be considered.¹ The case went from 3,581 records to 50 full texts to 14 reviews.⁴
6. 6Appraise the reviews. AMSTAR 2 has 16 items and rates overall confidence from high to critically low; its authors strongly recommend against adding the item ratings into one score.² JBI offers a 10-question appraisal tool as an alternative.⁶
7. 7Extract the data. Two researchers extract independently with a standardised form, taking the number of studies, sample size, events, study-specific effect estimates and 95% confidence intervals from each meta-analysis.¹
8. 8Measure overlap. Count how often the same trials recur across reviews with the corrected covered area (CCA).⁷ The case authors read 0 to 5% as slight, 6 to 10% as moderate, 11 to 15% as high and above 15% as very high overlap.⁴
9. 9Re-analyse and grade the evidence. Belbasis and colleagues advise repeating each meta-analysis from the study-specific data and then grading the strength of evidence: GRADE for interventions, and for associations the amount of evidence, level of significance, heterogeneity and hints of bias.¹ Fusar-Poli and Radua add reporting heterogeneity and potential biases, stratifying the evidence and running sensitivity analyses.⁵
10. 10Report and interpret. PRIOR has 27 main items with 19 sub-items for overviews of healthcare interventions.³ Show a flow chart, a table of every meta-analysis with its key features and the grading, and plots such as a forest plot or a Manhattan plot of p values.¹

Appraise the included reviews with AMSTAR 2

AMSTAR 2 keeps 10 of the original AMSTAR domains, has 16 items and rates overall confidence in the results of a review.²

Swipe sideways to see every column.

The seven critical domains are: protocol registered before the review began (item 2), adequacy of the literature search (item 4), justification for excluding individual studies (item 7), risk of bias from individual studies (item 9), appropriateness of the meta-analytical methods (item 11), consideration of risk of bias when interpreting results (item 13), and assessment of publication bias (item 15).

What to record for every meta-analysis

Belbasis and colleagues list the features to tabulate for each meta-analysis, including:¹

• number of cases or events, total sample size and number of studies;
• effect size metric and the meta-analysis method used (fixed or random effects);
• summary effect estimate, its 95% confidence interval and 95% prediction interval, and the p value;
• heterogeneity (Cochran Q p value, I² or between-study variance);
• the effect estimate of the largest study with its 95% confidence interval;
• signs of bias, such as small-study effects and excess significance.

A Real Umbrella Review Redrawn in R: Vitamin D in Children With Asthma

We chose an open access umbrella review whose licence is CC BY 4.0, read in the PMC record on 4 November 2026.⁴ Liu, Long and Yang searched PubMed, Embase and the Cochrane Library to 25 February 2026 for meta-analyses of randomised trials of vitamin D supplementation in children aged 0 to 18 years with asthma. They screened 3,581 records, assessed 50 full texts and included 14 systematic reviews, which rested on 26 unique randomised trials. They followed PRIOR, registered the protocol, rated the reviews with AMSTAR 2 and graded each estimate with GRADE and five credibility classes.⁴

Step 6 in practice: how good were the reviews?

One review was rated high, six low and seven critically low on AMSTAR 2.⁴ The paper reports that 36% of reviews did not report protocol registration, 86% gave no list of excluded studies and 29% did not account for risk of bias when interpreting results. With 14 reviews those percentages equal 5, 12 and 4 reviews (35.7%, 85.7% and 28.6%), which we computed in R as a check on the denominator.⁴

Step 9 in practice: which estimates hold up?

The table lists the 12 pooled estimates that the Results text quotes, with the evidence class the authors gave each. The last column is ours: we recomputed each p value from the published 95% confidence interval (standard error from the interval width, on the log scale for risk and odds ratios). We did not re-run the meta-analyses, because that needs the study-level data held in the original reviews and the supplement.⁴

Swipe sideways to see every column.

Class IV matches a confidence interval that excludes the null value in all 12 rows, and class V matches one that includes it, so the classes agree with the intervals we recomputed. 3 of the 12 estimates (25%) are nominally significant: serum 25(OH)D (MD 10.68 ng/mL), asthma recurrence (RR 0.53) and FVC % predicted. The FVC estimate is borderline, with p = 0.049 and an interval whose upper limit is −0.02, from two trials.⁴

Nominal significance is not the same as strong evidence. In the type 2 diabetes working example used by Belbasis and colleagues, 116 of 142 associations (82%) were significant at p < 0.05, but only 11 (8%) showed strong evidence on criteria that also consider heterogeneity, prediction intervals, small-study effects and excess significance.¹ The two reviews cover different topics, so the shares are not comparable; the contrast is the point. In the vitamin D case, GRADE certainty and credibility class also pointed in different directions: the null estimate for binary asthma control was graded high certainty (RR 1.01, 4 trials), while the significant rise in serum 25(OH)D was graded very low (class IV).⁴

Step 8 in practice: how much did the reviews overlap?

The 14 reviews drew on only 26 unique trials, and the authors report a CCA of 28.4%, which they class as very high overlap.⁴ The paper does not give the total number of trial occurrences in the main text. Rearranging the CCA formula, (N − r) / (r × c − r) with r = 26 trials and c = 14 reviews, implies N = 122: with N = 122 the formula returns 28.4%, and each trial would then appear in about 4.7 of the 14 reviews on average. This is our inference, computed in R, and not a number the authors state. For scale, Pieper and colleagues found a median CCA of 4.0 in 60 overviews published from 2009 to 2011.⁷

What the case shows and what it does not

It shows how the steps fit together: a registered protocol, a two-part search, double screening, appraisal, overlap, grading and reporting, with 14 reviews that collapse to 26 trials and 12 quoted estimates, none stronger than class IV in the Results text. The authors conclude that current evidence does not support routine vitamin D supplementation as an effective strategy to improve asthma control, reduce exacerbation or enhance lung function in children with asthma, despite its effect on increasing serum 25-hydroxyvitamin D levels and a possible benefit for asthma recurrence.⁴ It does not show that vitamin D is useless for every child: the authors point to one review whose subgroup with baseline serum 25(OH)D below 10 ng/mL had RR 0.48 (95% CI 0.28 to 0.83) for exacerbations, and note that children with such low levels were underrepresented in many of the included reviews and trials.⁴ Our own work was limited to recomputing p values, review counts and overlap from numbers in the paper. The paper text also mentions one outcome rated class III that is not among the 12 quoted estimates; it sits in a supplementary table that we did not read.

Need the Analysis Run or Checked? Consult Editverse

Editverse subject-matter experts work to COPE, ICMJE and GPP 2022 guidance, with institutional and country-specific ethics followed.

Choose the stage you are at.

Editverse

Literature Review

Need the search strategy and evidence table for your review?

You send: your research topic and question.

You get: a search strategy document with all databases and terms, a literature matrix (Excel), a gap analysis report and an annotated bibliography.

When: Turnaround is 7-10 business days.

Literature Review Support Package · $500

Get my search support →

Editverse

Statistical Analysis

Need your data analysed and reported?

You send: your data, research questions and preferred software.

You get: annotated code or syntax, complete output files, an output interpretation guide and publication-ready tables.

When: Standard turnaround is 4-6 business days.

SPSS/R/STATA Statistical Analysis · $400

Analyse my data →

Editverse

Manuscript Writing

Ready to turn your review into a submitted paper?

You send: your research outline or draft structure, raw data, key references and target journal preferences.

You get: a publication-ready manuscript written by PhD experts, with editing, statistical review, journal selection and submission support.

When: Standard 15-day delivery, or express 1-3 day turnaround.

Manuscript Writing Services · $1,500

Write my paper →

Frequently Asked Questions

What is an umbrella review?

An umbrella review is a systematic collection and assessment of multiple systematic reviews and meta-analyses on one research topic. The PRIOR statement notes that the same design is also called an overview of reviews, a review of reviews or a review of systematic reviews.

How is an umbrella review different from a systematic review?

A systematic review collects primary studies, while an umbrella review collects systematic reviews and meta-analyses. Belbasis and colleagues advise an umbrella review when several reviews already exist on a topic, a new systematic review when only a trivial number exist, and an update when several are outdated.

What are the main steps to conduct an umbrella review?

Belbasis and colleagues name four key steps: systematic literature search and study selection, data extraction, statistical analysis and grading of evidence, and interpretation of findings. This page splits them into ten steps, from checking that the review is needed to reporting it.

Which reporting guideline applies to an umbrella review?

For overviews of reviews of healthcare interventions, PRIOR is the dedicated guideline, with 27 main items and 19 sub-items. The case review on this page reported to PRIOR. Belbasis and colleagues also point to MOOSE and PRISMA, with a review as the building block instead of a primary study. Check which guideline your target journal names.

How do I appraise the quality of the included reviews?

AMSTAR 2 has 16 items and seven critical domains, and it gives four overall ratings: high, moderate, low and critically low. Its authors strongly recommend against combining the item ratings into an overall score. JBI also published a critical appraisal tool with 10 questions for systematic reviews and meta-analyses.

What is overlap and how is it measured?

Overlap means that the same primary studies appear in several included reviews. The corrected covered area (CCA) divides the repeated occurrences of studies by the number of unique studies times the number of reviews, less the number of unique studies. The case review reported a CCA of 28.4%, which its authors classed as very high overlap.

Should I re-run the meta-analyses in an umbrella review?

Belbasis and colleagues recommend repeating each meta-analysis from study-specific data instead of copying the published result, because published reviews often use inappropriate models or skip heterogeneity and small-study checks. This needs study-level data, so some reviews cannot be re-run, and that should be noted.

When is an umbrella review not the right choice?

When only a trivial number of relevant reviews exist, a new systematic review is more informative, and when several reviews are outdated, updating them matters more. An umbrella review also cannot cover interventions or associations that no review has examined, and its validity depends on the quality of the reviews it includes.

References

1. Belbasis L, Bellou V, Ioannidis JPA. Conducting umbrella reviews. BMJ Med. 2022;1(1):e000071. Open access, CC BY-NC 4.0 (licence read in the PMC record); process and numbers cited only, no figure reused. doi.org/10.1136/bmjmed-2021-000071
2. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. Open access, CC BY 4.0 (licence read in the PMC record). doi.org/10.1136/bmj.j4008
3. Gates M, Gates A, Pieper D, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ. 2022;378:e070849. Items and counts cited only. doi.org/10.1136/bmj-2022-070849
4. Liu J, Long Y, Yang Z. Vitamin D supplementation in children with asthma: an umbrella review. Nutrients. 2026;18(10):1560. Open access, CC BY 4.0 (licence read in the PMC record PMC13210331). Estimates, classes and counts are from this paper; figures on this page were redrawn and recomputed by Editverse. doi.org/10.3390/nu18101560
5. Fusar-Poli P, Radua J. Ten simple rules for conducting umbrella reviews. Evid Based Ment Health. 2018;21(3):95-100. Abstract read on PubMed. doi.org/10.1136/ebmental-2018-300014
6. Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc. 2015;13(3):132-140. Abstract read on PubMed. doi.org/10.1097/XEB.0000000000000055
7. Pieper D, Antoine SL, Mathes T, Neugebauer EAM, Eikermann M. Systematic review finds overlapping reviews were not mentioned in every other overview. J Clin Epidemiol. 2014;67(4):368-375. Abstract read on PubMed. doi.org/10.1016/j.jclinepi.2013.11.007

Editverse Research Team | Updated November 2026