Methodological note 4. Key steps for conducting a systematic review

2024-12-31

Key steps for conducting a systematic review

Malak Kouitia,b,* ,Tania Fernández-Villac,d,  Amparo Gamero Llunae, Macarena Lozano-Lorcaf,g, Manuel Reig García-Galbish, Edna J Nava-Gonzálesi, Patricio Pérez-Armijoh.

aLaboratory of Health Sciences and Technologies, Higher Institute of Health Sciences, Hassan First University of Settat, Settat, Morocco. bDepartamento de Medicina Preventiva y Salud Pública, Universidad de Granada, Granada, Spain. cGrupo de Investigación en Interacciones Gen-Ambiente y Salud (GIIGAS) / Instituto de Biomedicina (IBIOMED), Universidad de León, León, España. dCentro de Investigación Biomédica en Red de Epidemiología y Salud Pública (CIBERESP), Madrid, España eDepartamento de Medicina Preventiva y Salud Pública, Ciencias de la Alimentación, Toxicología y Medicina Legal, Facultad de Farmacia, Universitat de València, Valencia, España. fDepartamento de Medicina Preventiva y Salud Pública, Facultad de Ciencias de la Salud de Melilla, Universidad de Granada, Melilla, España. gInstituto de Investigación Biosanitaria ibs.GRANADA, Granada, España. hFacultad de Ciencias de la Salud, Universidad de Isabel I, Burgos, España. iFacultad de Salud Pública y Nutrición, Universidad Autónoma de Nuevo León, Monterrey, Nuevo León, México

*m.kouiti@uhp.ac.ma

---

Given the growing body of scientific evidence, various types of reviews can be conducted to synthesise the available data. Up to 14 distinct types of reviews have been identified, including narrative reviews, scoping reviews, umbrella reviews and systematic reviews (1,2). Systematic reviews are comprehensive reviews that critically synthesise original studies using a rigorous, transparent and reproducible methodology. This process involves assessing the methodological validity and interpreting qualitative results or combining quantitative data through meta-analysis.

Protocol development and registration. The process for conducting a systematic review should be outlined in advance through a detailed protocol. To enhance transparency and avoid duplications, it is recommended to prospectively register the review in a specialised database, such as PROSPERO (https://www.crd.york.ac.uk/prospero/#aboutpage) or OSF (https://osf.io/).

Systematic review reporting. To ensure a structured and complete report, various tools are available to assist researchers, such as PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and MOOSE (Meta-analyses Of Observational Studies in Epidemiology) (3,4).

Research question. The first step in conducting a systematic review is to formulate a clear and relevant research question. This can be structured according to the PICO (population, intervention-exposure, comparison and outcomes) or SPIDER (sample, phenomenon of interest, design, evaluation, research type) format.

Search strategy. The next step is to conduct a comprehensive and systematic search. This involves creating a search equation using boolean operators (AND, OR and NOT) and keywords relevant to the topic, which can be combined as free-terms or, in databases like PubMed, as MeSH terms. The same search equation must be applied across two or more information sources (e.g., Medline, Scopus, Web of Science, Cochrane Library, etc.). It is recommended to use the advanced search features offered by each database, considering their specific properties (e.g., use of quotation marks, different filters, etc.). The use of search restrictions must be justified.

Additionally, search can be completed by consulting the reference lists of the included studies. It is also important to keep the search up to date throughout the review process.

Study selection. The retrieved articles undergo a selection process, starting with the removal of duplicates, followed by an initial screening based on the title and abstract. Articles that appear to meet the predefined selection criteria outlined in the protocol are then evaluated in full text for possible final inclusion. This process should be carried out independently by two reviewers.

Data extraction. In the fourth step, data from the selected studies are extracted into a pre-designed data sheet containing relevant information (e.g., authors and year of publication, selection criteria, participant characteristics, exposure, exposure measure, type of intervention, methods of analysis and outcomes, etc.). This process should also be conducted independently by two reviewers.

Methodological quality assessment. The methodological quality and potential risks of bias in the included studies should be evaluated using a validated tool.  As with the previous steps, this should be performed by two reviewers independently. Various tools are available depending on the study design, such as, the JADAD scale and the Cochrane risk of bias 2 (ROB 2) tool for randomised controlled trials (5,6), Newcastle-Ottawa for cohort and case-control studies (7), or the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) scale, which can be applied to different study designs (8).

Data synthesis and meta-analysis.  Depending on the review’s objective and the available data, quantitative results can be combined to estimate an overall effect through a meta-analysis.  When it is not feasible, a narrative synthesis can be conducted to describe the finding.

For dichotomous data, the pooled effect size can be estimated using measures of association, such as odds ratio or relative risk. While for continuous data, such as means and standard deviations, standardised mean differences, are used. The pooled effect is estimated typically using a random effects model and presented graphically using a forest plot. The degree of heterogeneity between studies is assessed using the chi-squared test or the Cochrane Q test. Publication bias can be visualised with a funnel plot and assessed with Egger's or Begg's test. Additional analyses, such as subgroup analyses or sensitivity analyses, can be performed to explore possible sources of heterogeneity. Statistical software such as STATA or R can be used for this purpose. 

Chapter 10 of the Cochrane Handbook for Systematic Reviews of Interventions offers comprehensive guidance on conducting a meta-analysis, including any necessary data conversions (9).

Discussion. The first paragraph of discussion should provide a brief summary of the main findings. Results should be interpreted in the context of previous studies emphasising possible differences. Additionally, it is also important to highlight the strengths of the review as well as potential limitations. Authors may also discuss the implication of their findings for clinical practice and future research.

Supplementary information. Several software programs have been developed to facilitate the processes of data selection and analysis, including:

- Covidence: https://www.covidence.org/,

- Rayyan: https://www.rayyan.ai/,  

- Revman: https://revman.cochrane.org/info,

- SR-Accelerator: https://sr-accelerator.com/.  

Authors can evaluate the quality of their systematic review and identify possible biases using tools such as the following scales:

- Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR 2): (https://www.evidencia.org/index.php/Evidencia/article/view/6834/4506)

- Risk of Bias in Systematic Reviews (ROBIS): (https://www.bristol.ac.uk/media-library/sites/social-community-medicine/robis/ROBIS%201.2%20Clean.pdf)

Referencias:

  1. A typology of literature reviews. Information & Management. 2015; 52(2): 183-199.
  2. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108.
  3. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., ... & Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71. doi: 10.1136/bmj.n71 
  4. 4)Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., ... & Thacker, S. B. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA, 283(15), 2008-2012. doi: 10.1001/jama.283.15.2008
  5. Jadad AR, Moore RA, Carrol D, Jenkinson C, Reynolds DJM, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled Clinical Trials. 1996;17(1):1-12. doi:10.1016/0197-2456(95)00134-4.
  6. Sterne JAC, Savović J, Page MJ, et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366. doi:10.1136/bmj.l4898.
  7. Wells GA, Shea BJ, O'Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Disponible en: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
  8. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. doi:10.1136/bmj.39489.470347.AD.
  9. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane handbook for systematic reviews of interventions. 2nd ed. Chichester (UK): John Wiley & Sons; 2019.