VN: Umbrella Review Methods (2023)

  • Increase Font Size
  • Decrease Font Size
  • View as PDF

VN: Umbrella Review Methods (2023)

Methods for Umbrella Review on Vegetarian Diets for Disease Prevention

An overview of systematic reviews (SRs), or an umbrella review, was conducted due to high availability of existing SRs examining the relationships between the diets and outcomes of interest. This umbrella review followed methods described by the Cochrane Collaboration (Higgins JPT 2022) and the Academy of Nutrition and Dietetics (Handu, Moloney et al. 2016) and was reported using the preferred reporting items for overviews of reviews (PRIOR) checklist (Gates, Gates et al. 2022). Methods were specified a priori and registered at PROSPERO (CRD42023396453).(Rozga M 2023).

Eligibility Criteria

Eligibility criteria are described in full in the Search Plan and Results section for each research question. Briefly, participants were required to be presumably healthy adults (≥18 years) in the general population. The exposure of interest was following a vegetarian or vegan diet and the primary comparison of interest was a non-vegetarian diet. Primary outcomes of interest included disease incidence (type 2 diabetes mellitus, cardiovascular disease, hypertension, cancer, fracture, overweight/obesity), cardiovascular events and mortality. Coronary heart disease (CHD) and ischemic heart disease were both described as CHD. Secondary outcomes included nutrient status outcomes and other biomarkers of disease. Publication date was limited because SRs >5 years since publication may miss recent research. The included articles were limited to peer-reviewed articles published in the English language due to resource constraints. No supplemental primary studies were included. 

Information Sources

The database search was designed and conducted by an information specialist who searched MEDLINE, CINAHL, Cochrane Databases of Systematic Reviews, Food Science Source and SportsDiscus databases. The search terms included “vegetarian”, “vegan”, and “plant-based”, with limits on language (published in the English language), publication date (January 1, 2018-January 29, 2023) and study design (systematic reviews). The full search strategies can be found in the Search Plan for Vegetarian Umbrella Review. Relevant overviews of reviews were hand searched to identify any eligible SRs that may have been missed in the database search.

Selection Process

Following de-duplication, titles and abstracts of identified articles were independently screened by two reviewers using Rayyan screening software.(Ouzzani, Hammady et al. 2016) Each included title/abstract was considered in full-text review. Each full-text SR was assessed by two independent reviewers. Any discrepancies in title/abstract or full-text screening were settled through consensus or by review from expert panel members. Overlap of primary studies included in SRs was addressed during data extraction and evidence summary. 

Data Collection

A standardized data extraction sheet was created to extract study characteristics for each SR. Data was extracted by trained evidence analysts and checked by a second reviewer. Data included bibliographic information, participant health status and ages, exposure/intervention details [e.g, vegetarian, vegan or non-vegetarian diet, diet duration], study designs included in the SR, date of published studies in SRs, whether meta-analysis was conducted, if risk of bias (RoB) was assessed and with which tool, if certainty of evidence (COE) was assessed and with which tool, and outcomes of interest reported.

Quality Assessment

The quality of SRs was assessed by two independent reviewers with the updated measurement tool to assess systematic reviews (AMSTAR2) (Shea, Reeves et al. 2017, AMSTAR2 2021). AMSTAR2 has 16 questions, including seven critical domains assessing protocol registration, adequacy of literature search, justification for excluded studies, RoB and incorporation of RoB into interpretation, appropriateness of meta-analytic methods and assessment of publication bias (Shea, Reeves et al. 2017, AMSTAR2 2021). Discrepancies in AMSTAR2 ratings were settled by consensus or a third reviewer. RoB of primary studies in the included SRs were used for COE assessment in the umbrella review. The authors of this umbrella review did not assess RoB for primary studies within the included SRs. 

Synthesis Methods

Study characteristics of included SRs were described in a study characteristics table. When more than one SR examined the relationship between vegetarian diets and a specific outcome of interest, primary studies from each SR were plotted to determine overlap. Several of the SRs relied on previously conducted meta-analyses. Umbrella review authors analyzed study overlap according to primary studies, even if data from secondary studies was used. SRs with higher quality as assessed by the AMSTAR2 tool, those that conducted meta-analysis and/or assessed RoB and COE, and those that were more recent and comprehensive were prioritized when interpreting results. If only one SR conducted meta-analysis, umbrella review authors discussed if results from the other included SR(s) agreed. If more than one SR conducted a meta-analysis, umbrella review authors determined the appropriateness of combining the data from both meta-analyses to increase coverage and statistical power. When a novel meta-analysis could be conducted, data was extracted from the SRs, not the primary studies themselves (Higgins JPT 2022). If data could not be combined, the effect estimate from the higher quality study was prioritized. A summary of findings table was created using gradePRO (McMaster University and Evidence Prime 2020) and reports the number of studies and participants, effect size, factors contributing to evidence certainty and overall conclusion for each outcome.

When the meta-analysis was conducted for this umbrella review, the authors estimated effect size and 95% CI using a random-effects model (DerSimonian-Laird) due to the high heterogeneity expected when investigating dietary patterns in the general population. Results were reported as mean difference (95% CI). Results of the meta-analysis were visualized using forest plots. Publication bias was assessed using funnel plots and Begg’s and Egger’s statistics for outcomes with at least ten studies included. Heterogeneity was measured using the I2 measure. Sub-group analyses were conducted for each research question regarding the type of vegetarian diet and the study quality. Statistical analyses were conducted using OpenMeta (Wallace, Dahabreh et al. 2012)  and RStudio.(RStudio Team 2020) A p-value of <0.05 was considered statistically significant. 

Certainty of Evidence Assessment

If SR authors used the GRADE method to grade COE, this rating was used in the umbrella review and any deviations were justified. If SR authors use a different method to grade COE, it was translated to the GRADE method using a summary of findings table. The GRADE method considers study design and sample sizes, RoB of included studies, indirectness, inconsistency and imprecision of results (Guyatt, Oxman et al. 2013). If the SR authors did not assess COE, COE was determined using the GRADE method.

References:

  1. AMSTAR2. (2021). "AMSTAR 2 – The new and improved AMSTAR."   Retrieved June 2, 2023, from https://amstar.ca/Amstar-2.php.
  2. Gates M, Gates A, Pieper D, Fernandes RM, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ 2022 Aug 9:378:e070849. PMID: 35944924
  3. Guyatt G, Oxman AD, Sultan S, Brozek J, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol 2013 Feb; 66(2):151-157. PMID: 22542023
  4. Handu D, Moloney L, Wolfram T, Ziegler P, et al. Academy of Nutrition and Dietetics Methodology for Conducting Systematic Reviews for the Evidence Analysis Library. J Acad Nutr Diet. 2016 116(2):311-318. PMID: 26822985
  5. Higgins J, Chandler TJ, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 , February 2022; www.training.cochrane.org/handbook.
  6. McMaster University and Evidence Prime, I. (2020). GRADEpro GDT: GRADEpro Guideline Development Tool [Software].
  7. Ouzzani M, Hammady H, Fedorowicz Z,  et al. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016 Dec5; 5(1):210. PMID: 27919275
  8. Rozga M. (2023). "CRD42023396453."   Retrieved June 2, 2023, from https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=396453.
  9. RStudio Team (2020). RStudio: Integrated Development for R. Boston, MA, USA,, RStudio, PBC.
  10. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017 21:358:j4008. PMID: 28935701
  11. Wallace BC, Dahabreh IJ, Trikalinos TA, et al. Closing the Gap between Methodologists and End-Users: R as a Computational Back-End. J Statistical Software 2012 49(5):1-15. doi: 10.18638/jss.v049.i05