Hi Matt,
This is helpful and in the example you shared from your own curriculum, that is exactly the situation. Students who have failed a block course exam in M1 / M2 years for example are required to retake the exam. And as everyone probably knows, writing questions is difficult and time-intensive. We have a home-grown system that houses testing, evaluation, etc. and our database developer has created something similar where students can review exams and quizzes for a short period of time.
It's helpful to know your guide that no more than 25% can be from previous exams and quizzes. I will pass this on to the faculty members who asked me about it. I'm curious if other schools have a different threshold.
Happy Weekend everyone!
Max
------------------------------
Max Anderson
Dir of Instructional Design and Learning Innovation
University of Illinois College of Medicine
------------------------------
Original Message:
Sent: 01-07-2022 13:29
From: Matthew Cunningham
Subject: Number of new exam items in a retake exam vs original exam
Hi Max,
It's hard to know how to answer without knowing more context around what constitutes a "retake". I can tell you that in our pre-clinical curriculum, when students fail a block they often remediate by taking an alternate cumulative exam. We have a rule-of-thumb that no more than 25% of that remediation exam can be questions that they saw on the exams they took during the block. The remainder must be questions that were not used on block exams for that year (often questions in item pools that have been used in previous years and cycled out). One of the reasons we limit re-use of questions in this way is because we use Examsoft for testing, and students are allowed to do a secure review (i.e., see correct answers and rationales) immediately following the exam. I'm not sure how much of that secure review students actually remember when it comes time for the remediation exam (which is often many months later), but we've felt it's important to minimize the impact of anything they do remember.
Hope this helps,
Matt
------------------------------
Matthew Cunningham
Director of Educational Evaluation
University of Washington School of Medicine
Original Message:
Sent: 01-06-2022 17:02
From: Max Anderson
Subject: Number of new exam items in a retake exam vs original exam
Hi everyone and Happy New Year!
A question was posed to me this week & since I don't have all the answers (much to my own chagrin), I am asking the experts (you).
In exam construction for retakes specifically, we are looking for how many items should be new out of the total number that were in the original exam. Do you all have a ballpark or an 'official' number / ratio you go by?
How do you select the content for the new items?
Thank you for your help.
Max
------------------------------
Max Anderson
Dir of Instructional Design and Learning Innovation
University of Illinois College of Medicine
------------------------------