Saturday, 9 January 2016

Multiple choice (UKPSF K3 K4 V3)

It has been a very long time since I was at School, but I can dimly remember doing some exams which were assessed with multiple-choice questions.  I didn't really like them. I think, I found it confusing to try and understand the wordings of the questions. I don't remember that much of the exams at Imperial College, when I was an undergraduate there, but I don't think that there were many multiple choice questions.

So I have always been a bit biased against multiple choice questions, versus free answers. In Physics and Mathematics assessments, we very rarely ask the student to write essays. In the course I inherited, there were multiple choice questions, but these were mostly for numerical answers to calculations.

On issue within multiple choice questions is what happens when students guess between the typically four possible choices. I have been to a talk, by a person from the medical school, where they gave students -1/4 if they got an answer wrong.  The famous SAT test in the USA, also uses a system where 1/4 is subtracted for an incorrect answer. I have not had time to study it, but there are statistical tests, to try and discover, where a student is just guessing.

More interestingly, there are banks of multiple choice questions called concept inventories. The teaching method for maths and physics is to show some theory, then work some examples. The students then attempt to solve additional problems.  What has been found is the students focus too much on  the problem solving and don't attempt to master  the underlying physics or theory. So banks of multiple choice questions have been designed to test a student's knowledge without any calculation. The physics education community have also carefully studied student's misconceptions of physics and thus the questions are designed to contain common incorrect answers.

So perhaps I should revise my dislike of multiple-choice questions.

Of course, if you think that knowledge has nothing to do with learning, then you will not want to use multiple choice methods, and just rely on the students writing their opinions in an essay.