Defining ranges for certainty ratings of diagnostic accuracy: a GRADE concept paper

Monica Hultcrantz; Reem A Mustafa; Mariska M G Leeflang; Valéry Lavergne; Kelly Estrada-Orozco; Mohammed T Ansari; Ariel Izcovich; Jasvinder Singh; Lee Yee Chong; Anne Rutjes; Karen Steingart; Airton Stein; Nigar Sekercioglu; Ingrid Arevalo-Rodriguez; Rebecca L Morgan; Gordon Guyatt; Patrick Bossuyt; Miranda W Langendam; Holger J Schünemann

doi:10.1016/j.jclinepi.2019.05.002

Defining ranges for certainty ratings of diagnostic accuracy: a GRADE concept paper

J Clin Epidemiol. 2020 Jan:117:138-148. doi: 10.1016/j.jclinepi.2019.05.002. Epub 2019 May 18.

Authors

Monica Hultcrantz¹, Reem A Mustafa², Mariska M G Leeflang³, Valéry Lavergne⁴, Kelly Estrada-Orozco⁵, Mohammed T Ansari⁶, Ariel Izcovich⁷, Jasvinder Singh⁸, Lee Yee Chong⁹, Anne Rutjes¹⁰, Karen Steingart¹¹, Airton Stein¹², Nigar Sekercioglu¹³, Ingrid Arevalo-Rodriguez¹⁴, Rebecca L Morgan¹⁵, Gordon Guyatt¹⁵, Patrick Bossuyt³, Miranda W Langendam³, Holger J Schünemann¹⁶

Affiliations

¹ Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU), S:t Eriksgatan 117, SE-102 33, Stockholm, Sweden. Electronic address: monica.hultcrantz@sbu.se.
² Division of Nephrology and Hypertension, Department of Medicine, University of Kansas Medical Center, 3901 Rainbow Blvd, MS3002, Kansas City, KS 66160, USA; Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
³ Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam Public Health research institute, Amsterdam University Medical Centers, Meibergdreef 9, Amsterdam, The Netherlands.
⁴ Department of Medicine, Sacré-Coeur Hospital, University of Montreal, Montreal, Canada; Department of Clinical Affairs & Practice Guidelines, Infectious Disease Society of America, Arlington, VA, USA.
⁵ Clinical Research Institute, Faculty of Medicine, Universidad Nacional de Colombia, Bogotá, Colombia; Research Unit, Faculty of Medicine, Fundación Universitaria Sanitas, Bogotá, Colombia.
⁶ School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, 600 Peter Morand Crescent, Ottawa, Ontario K1G 5Z3, Canada.
⁷ Internal Medicine Service, German Hospital, Pueyrredón 1640, Buenos Aires C1118AAT, Argentina.
⁸ Medicine Service, VA Medical Center, 510, 20th street South, FOT 805B, Birmingham, AL, USA; Department of Medicine at the School of Medicine, University of Alabama at Birmingham (UAB), 1720 Second Ave South, Birmingham, AL 35294-0022, USA; Department of Epidemiology at the UAB School of Public Health, 1665 University Blvd., Ryals Public Health Building, Room 220, Birmingham, AL 35294-0022, USA.
⁹ Ateimed Consulting Ltd., 3rd Floor 166 College Road, Harrow, Middlesex HA1 1BH, UK.
¹⁰ Institute of Social and Preventive Medicine (ISPM), University of Bern, Mittelstrasse 43, 3012 Bern, Switzerland.
¹¹ Department of Clinical Sciences, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK.
¹² Programa de Ciências da Saúde - Universidade Federal de Ciências da Saúde de Porto Alegre (Ufcspa) Rua Sarmento Leite, 245 - CEP 90050-170, Porto Alegre, Brazil; Teaching and Research Unit - Grupo Hospitalar Conceição (GHC) Rua Francisco Trein, 596 - CEP - 91.350-200, Porto Alegre, Brazil.
¹³ Division of Nephrology, Department of Medicine, University of Toronto, 585 University Avenue, Toronto, Ontario M5G 2N2, Canada; Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
¹⁴ Clinical Biostatistics Unit, Hospital Universitario Ramon y Cajal, CIBER Epidemiology and Public Health, Ctra. Colmenar Km. 9,100, 28034 Madrid, Spain.
¹⁵ Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
¹⁶ Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; Department of Medicine, Michael G DeGroote Cochrane Canada Centre and GRADE centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.

PMID: 31112801
DOI: 10.1016/j.jclinepi.2019.05.002

Abstract

Objective: The objective of the study was to clarify how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) concept of certainty of evidence applies to certainty ratings of test accuracy.

Study design and setting: After initial brainstorming with GRADE Working Group members, we iteratively refined and clarified the approaches for defining ranges when assessing the certainty of evidence for test accuracy within a systematic review, health technology assessment, or guideline.

Results: Ranges can be defined both for single test accuracy and for comparative accuracy of multiple tests. For systematic reviews and health technology assessments, approaches for defining ranges include some that do not require value judgments regarding downstream health outcomes. Key challenges arise in the context of a guideline that requires ranges for sensitivity and specificity that are set considering possible effects on all critical outcomes. We illustrate possible approaches and provide an example from a systematic review of a direct comparison between two test strategies.

Conclusions: This GRADE concept paper provides a framework for assessing, presenting, and making decisions based on the certainty of evidence for test accuracy. More empirical research is needed to support future GRADE guidance on how to best operationalize the candidate approaches.

Keywords: Certainty of evidence; GRADE; Guidelines; Health technology assessments; Systematic reviews; Test accuracy.

MeSH terms

Decision Making
Diagnostic Techniques and Procedures / standards*
Humans
Sensitivity and Specificity
Systematic Reviews as Topic
Technology Assessment, Biomedical / methods*