New machine-marked tests for selection into core medical training: evidence from two validation studies

Clin Med (Lond). 2009 Oct;9(5):417-20. doi: 10.7861/clinmedicine.9-5-417.

Abstract

This study examined whether two machine-marked tests (MMTs; a clinical problem-solving test and situational judgement test), previously validated for selection into U.K. general practice (GP) training, could provide a valid methodology for shortlisting into core medical training (CMT). A longitudinal design was used to examine the MMTs' psychometric properties in CMT samples, and correlations between MMT scores and CMT interview outcomes. Independent samples from two years were used: in 2008, a retrospective analysis was conducted (n=1711), while in 2009, CMT applicants completed the MMTs for evaluation purposes (n=2265). Both MMTs showed good reliability in CMT samples, similar to GP samples. Both MMTs were good predictors of CMT interview performance (r = 0.56, p < 0.001 in 2008; r = 0.61, p < 0.001 in 2009) and offered incremental validity over the current shortlisting process. The GP MMTs offer an appropriate measurement methodology for selection into CMT, representing a significant innovation for selection methodology.

Publication types

  • Validation Study

MeSH terms

  • Adult
  • Clinical Competence*
  • College Admission Test*
  • Family Practice / education*
  • Female
  • Humans
  • Judgment
  • Longitudinal Studies
  • Male
  • Predictive Value of Tests
  • Problem Solving
  • Psychometrics
  • Reproducibility of Results
  • School Admission Criteria*
  • United Kingdom