Improvement of psychometric properties of the objective structured clinical examination when assessing problem solving skills of surgical clerkship

OBJECTIVE: To improve the psychometric properties of the objective structured clinical examination (OSCE) and elevate staff motivation.

METHODS: The OSCE was conducted in May 2006 at the Department of Surgery, King Saud University, Riyadh, Saudi Arabia as a pilot study for 95 students split over 2 consecutive days lasting 120 minutes each day. There were 15 actual stations on each day consisting of 10 stations that addressed problem solving skills, and 5 addressed clinical skills testing.

RESULTS: The stability of the OSCE measured by Cronbach's alpha on day one was 0.87, and 0.88 on day 2. The internal consistency of the OSCE measured by Carmine's theta on day one was 0.76, and 0.79 on day 2. Overall, OSCE's reliability for each day was high (r=0.8), without a significant difference between the 2 days. Credibility and comprehensiveness of the the OSCE were considered good by both staff and students. Accuracy of the OSCE measured by Pearson's correlation with multiple choice question scores was 0.65. Feasibility of the OSCE has also improved remarkably compared with previous OSCEs.

CONCLUSION: The OSCE proved to be highly reliable, and a valid format when more problem solving skills testing has been emphasized for the final year surgical clerkship. Feasibility has also improved remarkably after using this strategy, marked by more staff participation and enthusiasm.