What should we do about this difference? How do we evaluate the question, so that we can made pedagogical and policy decisions?
One way, the least thoughtful way, perhaps the most common way, is to go by our gut. That really just means going by what we already believe on the question, without examination or consideration. The problem with this approach is that no one ever learns anything, and there is no way to convince anyone -- not even ourselves -- of anything new.
I am a big of another way: the thought experiment. I like other approaches, too. But the thought experiment strikes me as the essence of the more thoughtful approach. Wikipedia tells me that its origins are different than I had thought, and that my understanding of it might not be archetypal. But I'll stick with my approach.
Think about what you are proposing. What would that imply down the road? What does it rely upon or assume? If it were true, what else would have to be true? How would you test if it were true (that's more archetypal) and has that test already been done? Does any of this contradict what we already know to be true?
I suppose that this a mathematical approach to proving something -- not surprising, considering that math was my first discipline. Work through the proof, and if you find a contradiction, the supposition must be false.
This has usually been my approach to merit-pay for teachers. For this to work as a reform, teachers would have to motivated by financial reward. The problem is that the given the widespread belief that teaching is a poorly compensated job -- what that is true or not -- it is hard to believe that the kinds of people who go into teaching are likely to be motivated by financial rewards. In fact, those that are would already have chosen another profession, for precisely that reason. Some supporters then counter that this would attract those very people to teaching. So, I ask whether the level of financial rewards that people discuss for merit pay would be sufficient to attract such people. I mean, are they likely to attracted because of the size of merit-based bonus, or the size of total compensation? Obviously the latter, right? So, how large would these bonuses need to be such that, when added to the base salary, they attract such people to the profession?
I like this approach, but it is not the final answer. A though experiment can be quite useful, but it does not always lead to a final answer, and even when it does others may not be convinced.
Luckily, there often are real studies, or even real experiments. When it comes to motivation, the work has already been done. For example, my attention was recently brought to a paper by a bunch of researcher who have nothing to do with educational policy or pedagogical issue. Actually, the paper is published by the Federal Reserve Bank of Boston, and is by a bunch of business school professors.
Ariely, Gneezy, Lowenstein & Mazer (2005) looked the impact of external financial motivation on performance. They considered tasks with requiring varying degrees of concentration, creativity and problem-solving. They found that for high rewards actually tended to degrees performance. That's right. Decrease.
First, let me point out that that is a problem. Teaching requires a lot of creativity and problem-solving. If high contingent financial rewards hinder performance with such tasks, we would not want to bring to to our schools.
Second, this experimental finding really poses a problem for offering merit-pay at a high enough level to attract the sorts of people motivated by financial rewards. It is like being between a rock and hard place. Attracting them takes a lot of money, but a lot of money decreases performance.
Third, I want to add a bit more discussion. Why is there such fervent belief in the power of of financial rewards for performance? I can think of two ones, one which the research literature addresses, and one which it does not -- or at least I have found it.
Clearly, in some situations extrinsic rewards can make a difference. When it comes to effort and narrow concentration, we often see an impact. Simple tasks, repetitive task, physical tasks. All of these can be very responsive to extrinsic motivation. But creative and problem solving tasks? The kinds of things where we need to wider our focus, in order to find solutions -- or at least next step? The research has made clear the extrinsic motivation does not really help there, and can even hurt.
So, why do we believe in it so? I think that there's another issue. I think that it might work in some cases. I think extrinsic motivation (i.e. rewards, prizes or even avoiding punishments) can help to identify the absolutely best performers -- at least best at performing when in such environments. But the fact that they might even be able to raise the performance of those who do best in such environments does not actually make them a good idea. If they make some small fraction of performers a bit better, but make most performers worse, that is a problem. Obviously, there are some configurations of that dynamic where the tradeoffs are worth it, not that is not always the case.
So, the more thoughtful approach means we consider not just the benefits, but the costs as well. In the case of merit pay for teachers, is it likely to help most students or help those student most in need? Or will it help a random minority of students at the expense of the rest?