The Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE): Construct and Content Validation Using a Modified Delphi Method

Abstract

Objectives: In order to scale-up successful innovations, more evidence is needed to evaluate programs that attempt to address the rising prevalence of diabetes and the associated burdens on patients and the healthcare system. This study aimed to assess the construct and content validity of the Diabetes Evaluation Framework for Innovative National Evaluations (DEFINE), a tool developed to guide the evaluation, design and implementation with built-in knowledge translation principles.

Methods: A modified Delphi method, including 3 individual rounds (questionnaire with 7-point agreement/importance Likert scales and/or open-ended questions) and 1 group round (open discussion) were conducted. Twelve experts in diabetes, research, knowledge translation, evaluation and policy from Canada (Ontario, Quebec and British Columbia) and Australia participated. Quantitative consensus criteria were an interquartile range of ≤1. Qualitative data were analyzed thematically and confirmed by participants. An importance scale was used to determine a priority multi-level indicator set. Items rated very or extremely important by 80% or more of the experts were reviewed in the final group round to build the final set.

Results: Participants reached consensus on the content and construct validity of DEFINE, including its title, overall goal, 5-step evaluation approach, medical and nonmedical determinants of health schematics, full list of indicators and associated measurement tools, priority multi-level indicator set and next steps in DEFINE's development.

Conclusions: Validated by experts, DEFINE has the right theoretic components to evaluate comprehensively diabetes prevention and management programs and to support acquisition of evidence that could influence the knowledge translation of innovations to reduce the burden of diabetes.

Keywords: delphi method; diabète de type 2; evaluation; health policy; health services; healthcare; méthode Delphi; politiques en matière de santé; services de santé; soins de santé; type 2 diabetes; évaluation.