Problem Design for Multiple Choice Questions
I gave my students a problem from the 2002 AMC 10-A:
Tina randomly selects two distinct numbers from the set {1, 2, 3, 4, 5}, and Sergio randomly selects a number from the set {1, 2, …, 10}. The probability that Sergio’s number is larger than the sum of the two numbers chosen by Tina is: (A) 2/5, (B) 9/20, (C) 1/2, (D) 11/20, (E) 24/25.
Here is a solution that some of my students suggested:
On average Tina gets 6. The probability that Sergio gets more than 6 is 2/5.
This is a flawed solution with the right answer. Time and again I meet a problem at a competition where incorrect reasoning produces the right answer and is much faster, putting students who understand the problem at a disadvantage. This is a design flaw. The designers of multiple-choice problems should anticipate mistaken solutions such as the one above. A good designer would create a problem such that a mistaken solution leads to a wrong answer — one which has been included in the list of choices. Thus, a wrong solution would be punished rather than rewarded.
Readers: here are three challenges. First, to ponder what is the right solution. Second, to change parameters slightly so that the solution above doesn’t work. And lastly, the most interesting challenge is to explain why the solution above yielded the correct result.
Share:
bons:
well the right way to do it is to multiply the probability that Tina’s two numbers sum to x, where x is between 3 and 9, by the probability that Sergio beats Tina, and add those probabilities together. I suppose the solution above yields the correct result because 6 is the median number as well.
23 December 2010, 5:44 pmJonathan:
Solution to problem:
1/2 wins or ties 3/10
1/3 wins or ties 4/10
1/4 wins or ties 5/10
2/3 wins or ties 5/10
1/5 wins or ties 6/10
2/4 wins or ties 6/10
2/5 wins or ties 7/10
3/4 wins or ties 7/10
3/5 wins or ties 8/10
4/5 wins or ties 9/10
As each pair occurs with probability 1/10, we get (3+4+5+5+6+6+7+7+8+9)/10/10 = 60/100
Same problem, slightly altered, average still 6, but answer is not 6/10
If we change the first set to {0,1,3,4,7} the average holds, but the result falls.
0/1 wins or ties 1/10
0/3 wins or ties 3/10
0/4 wins or ties 4/10
1/3 wins or ties 4/10
1/4 wins or ties 5/10
0/7 wins or ties 7/10
3/4 wins or ties 7/10
1/7 wins or ties 8/10
3/7 wins or ties 10/10
4/7 wins or ties 10/10
As each pair occurs with probability 1/10, we get (1+3+4+4+5+7+7+8+10+10)/10/10 = 59/100
Discussion:
23 December 2010, 9:07 pmIn the first example, the probability that a sum is greater than or equal to a random number from 1 to 10 is directly proportional to the sum (all sums are between 3 and 10). In the second example, one sum is greater than 10, breaking the symmetry that previously kept the probability proportional to the sum.
Jonathan:
Hmm, first pairs of numbers should be the sum from the first set, so not
24 December 2010, 9:04 am1/2 wins or ties 3/10,
but rather
1+2 wins or ties 3/10 of the time.
JBL:
Nice counter-example, Jonathan. Of course, we can write down an argument that sounds very much like the student’s argument but is actually valid, and in fact this argument is in some respects more elegant than a brute-force solution — we build off Jonathan’s discussion and don’t have to do any arithmetic.
27 December 2010, 10:22 amJBL:
Here are two follow-up questions:
1) Let S be a finite multiset of integers (i.e., I can have repetitions) with mean 6. Tina randomly selects a number from S, and Sergio randomly selects a number between 1 and 10. Let p be the probability that Sergio picks a larger number than Tina. As S varies, what is the range of possible values of p?
2) Let S be a finite set of integers with mean 3. Tina randomly selects two distinct numbers from S, and Sergio randomly selects a number between 1 and 10. Let p be the probability that the number Sergio picks is larger than the sum of the numbers Tina picks. As S varies, what is the range of possible values of p?
27 December 2010, 12:32 pmTanya Khovanova:
Here is a comment from a_shen in my LJ discussion on the same subject:
27 December 2010, 10:11 pmOnce I heard another example: how many faces will have the union of a pyramid based on a square whose faces are equilateral triangles and a regular tetrahedron that is glued to one of these faces: it was said that the right answer to this problem had a negative correlation with the result of the entire test…