Aim: To investigate possible bias due to gender combination of students, role players and examiners in a high-stakes assessment.
Setting: Valid oral interactive contextualized examinations (VOICEs) is a long-station OSCE-style exam in general practice (GP). At the time of writing it consisted of 65% of the student's final GP mark.
Method: In the VOICE, students undertake six tasks--four vivas and two role-plays. "Patient" roles are taken by professional role players who work regularly on the undergraduate curriculum. During the role-play, the student's clinical competence is assessed by an observing GP examiner. The communication skills marks are awarded by the role player and the examiner together, by negotiation. Data have been recorded detailing the role player's initial marks, the examiner's initial marks and their final (awarded) agreed marks for 1024 consultations.
Sample: 512 final year medical students, 28 role players and 48 examiners. There were no inclusion or exclusion criteria. All those present on exam day became part of the data.
Results: There was a significant relationship between gender and performance for some, but not all, stations. Correlations for multiple comparisons removed the significance. Female students perform better across the board than male students. While not always significant, this did affect grading. There was no significant association between the genders of role players and examiners with the question choices. There has been a significant worsening of male results since 1999. Differences exist in the way that pairs of mixed or single genders score students.