Defensibility, credibility and feasibility of three standard setting procedures for OSCE: Developing evidence-informed assessments
Publication Details
Tor, E., MacNish, J., Wrtight, A., & Steketee, C. (2011). Defensibility, credibility and feasibility of three standard setting procedures for OSCE: Developing evidence-informed assessments. Association for Medical Education in Europe 2011.
Abstract
Background: Standard setting is integral in clinical skill assessments in outcome and competency based MBBS curriculum. To determine the most credible and feasible standard setting method for clinical examinations (OSCEs), the outcomes from three different methods were compared.
Summary of work: Modified Angoff (MAM), Borderline Group (BGM), and Borderline Regression (BRM) standard setting methods were applied to nine OSCE stations for 103 first year and 106 second year student. The same set of five examiners standard set each OSCE station using each of the three methods.
Summary of results: Pass marks and standard errors for MAM were higher than the other two methods in most stations. Pass / fail decisions agreement between MAM and BGM/BRM was statistically significant, but marginally convincing (Kappa = 0.488, p
Conclusions: Empirical evidence seems to indicate that BGM/BRM is more credible and defensible than MAM for standard setting pass mark in OSCE.
Take-home messages: Evidence-informed optimization of defensibility, credibility and feasibility of standard setting procedures for OSCE should be an on-going goal.
Keywords
Abstract Only
Comments
Further information about the AMEE conference may be accessed here