Improving reliability of the objective structured clinical examination (OSCE) in first year medical students

Abstract

Objective structured clinical examination (OSCE) can be used for formative and summative assessment in medical education. The reliability score (internal consistency) for 60 minutes of OSCE has been shown to be 0.47 (Petrusa, 2002). The aim of the study was to investigate whether we could improve the reliability of the 60 minutes of OSCE by utilizing various standardisation strategies in implementation of the examination. The School of Medicine, Sydney, University of Notre Dame examined a cohort of 112 graduate-entry medical students (Year 1, 2008) using OSCE. There were eight stations with total examination duration of 64 minutes. In the preparation for the OSCE, we employed the following strategies to improve the reliability: setting up a multidisciplinary panel to standard set all the questions, utilizing a detailed criterion based marking guide for each station, conducting training sessions to ensure consistency in both the examiners and actors and having external standard setters to standardise the marking. The reliability score was calculated using the SPSS statistical package (SPSS Inc., Chicago, Illinois, USA). The overall reliability of the 64 minutes OSCE was 0.69 (Cronbach’s alpha). This was higher than the historical data and comparable to a 120 minutes OSCE (Petrusa, 2002). By employing strategies to standardize the OSCE, we were able to improve the reliability of the OSCE examination. This would be particularly useful in situations where there are constraints on the ability to run longer exam periods. Increasing the number of stations and duration of future OSCE could further improve the reliability.

Keywords

objective structured clinical examination, reliability

Link to Publisher Version (URL)

10.1111/j.1365-2923.2010.03782.x

This document is currently not available here.

Find in your library

Share

COinS