Make Multiple Choice Quizzes Less Terrible
It is obvious why schools, workplaces, MOOCs and many places that test knowledge use multiple choice questions.
It’s not because they’re great for students. It is best to ask them to explain and demonstrate their experience, with as few prompts as possible.
With multiple options, a student can get lucky…or use some simple tricks to get around it.
If you choose ‘All of the above’ when it is an option and the answer with more words when it is not, I wonder how many exams you can pass without reading the questions…
But they have a clear advantage:
Ease of marking.
There is no ambiguity or subtlety. Either they did it right or they didn’t.
It makes it easy for humans to rate…and trivial for machines to do so.
So, ideal or not, sometimes you need to settle for them.
Here’s how to make them less terrible:
There are no false answers
Many tests include obviously wrong answers.
The type that no one would choose unless they are randomly guessing.
If the test is about the history of law, this is the question that asks: “What is the Magna Carta?” and option (B) says a soup recipe.
Or one of those ‘workplace behavior’ courses, where you’re asked if punching someone in the stomach is an example of respectable professional behaviour.
Get rid of these and replace them with something that tests your understanding.
I like to have the wrong answers evenly distributed. I don’t know if few or most students will get a question right… but my goal is for all wrong answers to have the same stats.
For example, if 70% choose the correct answer from (A), I want 10% to choose (B), 10% to choose (C), and 10% to choose (D).
If computers score the exam, this should be trivial to track. If hardly anyone chooses a given wrong answer, then that answer is useless. Cut it out and replace it.
Not ‘all of the above’
If there are five options and the first two are correct, the student can stop thinking. They might take a look at the others for a quick sanity check, but they already know the answer.
I like to make students consider each answer on its own merits, you know, make them think.
This makes ‘none of the above’ a better option, but I still don’t like it. Although that is more for personal reasons. I’m not entirely sure why, maybe because it seems lazy to me.
Specify multiple answers (but not how many)
If a question has more than one correct answer, it’s best to let students know.
Make it clear, even for those who are stressed, rushed, and distracted.
Put it in bold in the question, something like “choose ALL answers that apply”.
But don’t say “choose the two correct answers”.
Why not?
As before, I like to have students consider each answer on its own merits. If they know that only two of the five answers are correct, they can ignore other options without even thinking about them.
no ambiguity
This can make or break the test.
Ambiguous questions or answers are horrible. Someone who doesn’t understand the material may be puzzled… but your expert students should definitely know which answer(s) is correct.
If you find yourself thinking, ‘It depends on…’ or ‘Technically, all/none of these are possible…’ then you are no longer testing the experience.
There is no easy answer for this.
If one person writes the questions and answers, have other people on your team review them. Find experts outside of your team to test your quiz.
What is clear to the author is not always clear to everyone.