Multisource assessment (MSA) uses multiple assessors to provide feedback. Little is known about the validity of using MSA feedback for improving students' ability to self-assess in a preclinical environment. Therefore, the aim of this study was to measure the validity of using a defined reflective process involving an MSA tool for building skill in dental students' self-evaluation of caries excavation on extracted teeth. As part of this process, 104 first-year students at one U.S. dental school used a self-generated study plan (SGSP) for structured reflection on MSA feedback during the 2013-14 academic year. Interrater agreement, determined through calculation of percentage-agreements in scoring, was measured among three assessor groups (self-, peer, and expert assessors) in formative assessment and between two assessor groups (self- and expert assessors) in summative assessment two weeks apart, allowing for reflective practice and completion of an SGSP between assessments. Validity for improving self-assessment was determined by measuring significance in positive shifts of agreement between self- and expert assessors. The results showed that interrater agreement between the self- and expert assessors increased significantly: from a 28% agreement in formative assessment to a 60% agreement in summative assessment. Significance in percentage shifts between assessments was demonstrated with a McNemar score of 0.26 (p<0.001). These results suggest that the described MSA tool and reflective process in an SGSP may be valid methods for improving skill in student self-evaluation of competence in caries excavation on extracted teeth.
Keywords: assessment; dental caries; dental education; operative dentistry; preclinical education; self-assessment.