Tanya Khovanova's Math Blog

Testing in the US is dominated by multiple-choice questions. Together with the time limit, this encourages students to stop thinking and go for guessing. I recently wrote an essay AMC, AIME, USAMO Contradiction, in which I complained about the lack of proofs in the first two rounds of math competitions.

Is there a way to improve the situation? I grew up in the USSR, where each round of the math competition had the same format: you were given several hours to write proofs for three or four difficult problems. There are two concerns with organizing a competition in this way. First, the Russian system is much more expensive, whereas the US’s multiple choice tests can be inexpensively checked by a computer. Second, the Russian system is prone to unfairness. You need many math teachers to check all these papers on the highest level. Some of these teachers might not be fully qualified, and it is difficult to ensure uniform checking. This system can’t easily be adopted in the US. I am surprised I haven’t heard of lawsuits challenging USAMO results, but if we were to start having proofs at the AMC level with several hundred thousand participants, we would get into lots of trouble.

An interesting compromise was introduced at the Streamline Olympiad. The problems were multiple choice, but students were also requested to write proofs. Students got two points for a correct multiple choice answer, and if the choice was correct the proof was checked. Students could get up to three points for a correct proof. This idea solves two issues. The writing of proofs is rewarded at an early stage and the work of the judges is not as overwhelming as it would have been, had they needed to check every proof. However, there is one problem that I discussed in previous posts that this method doesn’t solve: with multiple choice, minor mistakes cost you the whole problem, even though you might have been very close to a solution. If we want to reward thinking more than accuracy, the proof system allows us to give credit for partial solutions.

I can suggest another approach. If the Russians require proofs for all problems and the Americans don’t require proofs for any problem, why not compromise and require a proof for one problem out of the set.

But I actually have a bigger idea in mind. I think that current development in artificial intelligence may soon help us to check the proofs with the aid of a computer. Artificial intelligence is still far from ready to validate that a mathematical text a human has produced constitutes a proof. But in this particular case, we have two things working for us. First, we can use humans and computers together. Second, we do not need to check the validity of any random proof; we need to check the validity of a specific proof of a simple problem that we know in advance, thus allowing us to prepare the computers.

Let us assume that we already can convert student handwriting into computer-legible text or that students write directly in LaTeX.

Here is the plan. Suppose for every problem, we create a database of some sample right, wrong and partial solutions with corresponding scores. The computer checks the students’ solutions against the given sample. Hopefully, the computer can recognize small typos and deviations that shouldn’t change the point value. If the computer encounters a solution that is significantly different from the ones in the sample, it sends the solution to human judges. Humans decide how to score the solution and the solution and its score is added to the sample database.

For this system to work, computers should be smart enough not to send too many solutions to humans. So how many is too many? My estimate is based on the idea that we wouldn’t want the budget of AMC to go too much higher than the USAMO budget. Since USAMO has 500 participants, judges check just a few hundred solutions to any particular problem. With several hundred thousand participants in AMC, the computer would have to be able to cluster all the solutions into not more than a few hundred groups. The judges only have to check one solution in each group.

As a bonus, we can create a system where for a given solution that is not in the database, the computer finds the closest solution and highlights the difference, thus simplifying the human’s job.

In order to improve math education, we need to add proofs when teaching math. My idea might also work for SATs and for other tests.

Now that there is more money available for education research, would anyone like to explore this?

A stick has two ends. If you cut off one end, how many ends will the stick have left?

This pre-kindergarten math problem was given to me by Maxim Kazarian who lives in Moscow, Russia. That got me thinking about math education in the US. Actually, just about anything can get me thinking about education in our country. One of our math education patterns is to provide simplified templates and to train kids to plug numbers into them without thinking.

Math education should be about thinking. We need to give kids a lot of math problems that do not fit into standard templates, in order to encourage creative thinking. Here is another puzzle from Maxim:

A square has four corners. If we cut one corner off, how many corners will the remaining figure have?

I invite my readers to invent additional problems that sound as if a subtraction by one is needed, when, in fact, it is not. Here is my contribution:

Anna had two sons. One son grew up and moved away. How many sons does Anna have now?

To get to the national swimming championship, you need to win the state running championships.

What? Is that a joke? Perhaps you’re having the same reaction. Because this is exactly what is happening with math competitions. The official USA math competition has three rounds: AMC, AIME and USAMO.

AMC is a multiple-choice competition with 25 problems in 75 minutes. To be good at it, you need speed, accuracy and the ability to guess well.

AIME is 3 hours long and has 15 problems. The problems are a different level of difficulty and guessing will not help you. Though AIME is also multiple-choice, unlike AMC where you choose out of 5, in AIME you choose out of 1,000. But you still need speed and accuracy. A small arithmetic mistake will cost you the whole problem.

USAMO is a competition that runs for 9 hours and has 6 problems. The problems are much harder and you have to write proofs. Proofs? What proofs? Where are the proofs coming from? It is like you got to the national swimming championship because you are a great runner, but you do not know how to swim.

As the result of this system of selection, the USA team at the International Math Olympiad has diverse skills: these kids are good at all three types of the math competitions. It is like taking an Olympic triathlon team to the Olympic swimming event.

However, the US loses by selecting in this way. There are many kids who are great mathematicians: they may be good at difficult problems but not that good at speed-racing problems. An arithmetic mistake costs you only one point at IMO, but a whole problem at AIME. There are kids who are deep mathematicians prone to small arithmetic mistakes. They could get a gold medal at IMO, but they can’t pass AMC or AIME.

Not only that. As many AMC problems are boring and do not require ideas, AMC might discourage kids from all math competitions at an early stage.

I will write later with my ideas about how to change AMC. Right now I would like to offer a solution to a smaller problem. I am sure that the US math team organizers know many cases in which a non-senior kid does great at USAMO and is potentially a team member for the next year’s US IMO team, but, oops, next year he can’t pass AMC.

I suggest the following: USAMO participants are allowed to go to next year’s AIME no matter what their AMC scores are. USAMO winners are allowed to go to the next year’s USAMO no matter what their AIME results are. This way kids who have proved that they are great swimmers do not need to compete in running again.

I bought the book “Math Doesn’t Suck: How to Survive Middle School Math Without Losing Your Mind or Breaking a Nail“ by Danica McKellar because I couldn’t resist the title. Sometimes this book reads like a fashion magazine for girls: celebrities, shopping, diet, love, shoes, boyfriends. At the same time it covers elementary math: fractions, percents and word problems.

You can apply math to anything in life. Certainly you can apply it to fashion and shoes. I liked the parallel between shoes and fractions that Danica used. She compared improper fractions to tennis shoes and mixed numbers to high heels. It is much easier to work with improper fractions, but mixed numbers are far more presentable.

Danica is trying to break the stereotype that girls are not good at math by feeding all the other stereotypes about girls. If you are a typical American girl who hates math and missed some math basics, this book is for you. If you want to discover whether the stars are on your side when you are learning math, the book even includes a math horoscope.

I ran an experiment. I copied multiple choices from the 2007 AMC 8 into a file and asked my son Sergei to try to guess the answers, looking only at the choices. I allowed him to keep several choices. The score I assigned depended on the number of leftover choices. If the leftover choices didn’t contain the right answer, the score for the problem was zero. Otherwise, it was scaled according to the number of choices he left. For example, if he had four choices left and the right answer was among them he got 1/4 of a point. Here are the choices:

1. 9, 10, 11, 12, 13.
2. 2/5, 1/2, 5/4, 5/3, 5/2.
3. 2, 5, 7, 10, 12.
4. 12, 15, 18, 30, 36.
5. 24, 25, 26, 27, 28.
6. 7, 17, 34, 41, 80.
7. 25, 26, 29, 33, 36.
8. 3, 4.5, 6, 9, 18.
9. 1, 2, 3, 4, cannot be determined.
10. 13, 20, 24, 28, 30.
11. Choose picture: I, II, III, IV, cannot be determined.
12. 1:1, 6:5, 3:2, 2:1, 3:1.
13. 503, 1006, 1504, 1507, 1510.
14. 5, 8, 13, 14, 18.
15. a+c < b, ab < c, a+b < c, ac < b, b/c = a.
16. Choose picture: 1, 2, 3, 4, 5.
17. 25, 35, 40, 45, 50.
18. 2, 5, 6, 8, 10.
19. 2, 64, 79, 96, 131.
20. 48, 50, 53, 54, 60.
21. 2/7, 3/8, 1/2, 4/7, 5/8.
22. 2, 4.5, 5, 6.2, 7.
23. 4, 6, 8, 10, 12.
24. 1/4, 1/3, 1/2, 2/3, 3/4.
25. 17/36, 35/72, 1/2, 37/72, 19/36.

It is clear that if you keep all choices, your score will be 5, which is the expected score for AMC if you are randomly guessing the answers. Sergei’s total score was 7.77, which is noticeably better than the expected 5.

There were two questions where Sergei felt that he knew the answer exactly. First was question number two with choices: 2/5, 1/2, 5/4, 5/3, 5/2. All but one of the choices has a 5 in it, so 1/2 must be wrong. Numbers 2/5 and 5/2 are inverses of each other, so if organizers expect you to make a mistake by inverting the right answer, then one of these choices must be the right answer. But 5/4 and 5/3 are better suited as a miscalculation of 5/2, than of 2/5. His choice was 5/2, and it was correct. The second question for which he was sure of the answer was question 19, with his answer 79. I still do not have a clue why.

Sergei’s result wasn’t too much better than just guessing. That means that AMC 8 organizers do a reasonably good job of hiding the real answer. You can try it at home and see if you can improve on Sergei’s result. I will publish the right answers as a comment to this essay in a week or so.