Browse by author
Lookup NU author(s): Professor Brian Lunn
Full text for this publication is not currently held within this repository. Alternative links are provided below where available.
A retrospective analysis was made of the predicted performance of borderline candidates by standard setters, compared to how a nominal borderline group actually performed in SBA papers (including Common Content items). Candidates performing 2% around the examination cut-mark were identified for each assessment. Their performance on each question was compared to the prediction of standard setters for these questions. Looking at whole exam prediction standard setters tended to underestimate the ability of candidates by 1-4%. When the facility of questions were analysed it was noted that the higher the facility the more standard setters underestimated candidate performance (range 19-29% difference) and when the facility was lower the more candidate performance was overestimated (range 24-43% difference). The closer to the cut-mark the closer standard setters were to predicting performance. These data suggest that standard setters assessments tend to cluster around a nominal point where the cut-mark has been historically, rather than predicting borderline candidate performance. There is no gold standard for standard setting but the Angoff model is often cited as the best of the options available. This and similar methodologies depend on the ability of standard setters to gauge candidate performance and are resource and time intensive. If evidence suggests that the model does not hold to the theory then perhaps it is time to consider whether this methodology is the best choice? This will be discussed in the context of move to a National Medical Licensing Assessment and consideration of standard setting such an exam.
Author(s): Lunn B
Publication type: Conference Proceedings (inc. Abstract)
Publication status: Published
Conference Name: European Board of Medical Assessors Annual Academic Conference: Crossing Boundaries: Assessment in Medical Education
Year of Conference: 2016
Pages: 36-36
Acceptance date: 01/04/2016
Publisher: University of Exeter Medical School
URL: http://medicine.exeter.ac.uk/media/universityofexeter/medicalschool/cpdevents/assessmentinmedicaleducation/EBMA_conference_programme_2016.pdf
