Background: Evaluating the impact of simulation-based education (SBE) has prioritised demonstrating a causal link to improved patient outcomes. Recent calls herald a move away from looking for causation to understanding 'what else happened'. Inspired by Shorrock's varieties of human work from patient safety literature, this study draws on the concept of work-as-done versus work-as-imagined. Applying this to SBE recognises that some training impacts will be unexpected, and the realities of training will never be quite as imagined. This study takes a critical realist stance to explore the experience and consequences, intended and unintended, of the internal medicine training (IMT) simulation programme in Scotland, to better understand 'training-as-done'.
Methods: Critical realism accepts that there is a reality to uncover but acknowledges that our knowledge of reality is inevitably our construction and cannot be truly objective. The IMT simulation programme involves three courses over a 3-year period: a 3-day boot camp, a skills day and a 2-day registrar-ready course. Following ethical approval, interviews were conducted with trainees who had completed all courses, as well as faculty and stakeholders both immersed in and distant from course delivery. Interviews were audio-recorded, transcribed verbatim and analysed using critical realist analysis, influenced by Shorrock's proxies for work-as-done.
Results: Between July and December 2023, 24 interviews were conducted with ten trainees, eight faculty members and six stakeholders. Data described proxies for training-as-done within three broad categories: design, experience and impact. Proxies for training design included training-as-prescribed, training-as-desired and training-as-prioritised which compete to produce training-as-standardised. Experience included training-as-anticipated with pre-simulation anxiety and training-as-unintended with the valued opportunity for social comparison as well as a sense of identity and social cohesion. The impact reached beyond the individual trainee with faculty development and inspiration for other training ventures.
Conclusion: Our findings highlight unintended consequences of SBE such as social comparison and feeling 'valued as a trainee, valued as a person'. It sheds light on the fear of simulation, reinforcing the importance of psychological safety. A critical realist approach illuminated the 'bigger picture', revealing insights and underlying mechanisms that allow this study to present a new framework for conceptualising training evaluation.
Keywords: Critical realism; Immersive simulation; Internal medicine; Simulation-based mastery learning; Training evaluation.
© 2024. The Author(s).