Check Training Hours Member Portal Contact Us Search

Page 20

Thomas Ahn - on the effectiveness of incentive pay for teachers

In conversation with ...

In Conversation With…features an interview between a leader or figure involved in public education and a staff member of the Kentucky School Advocate.

This month’s conversation is with Dr. Thomas Ahn, an assistant professor of economics at the University of Kentucky who co-authored a study on North Carolina’s pay-for-performance teacher-bonus program. Ahn, and study co-author Jacob L. Vigdor, a professor of public policy and economics at Duke University, found that North Carolina was able to avoid the pitfalls normally associated with incentive-based pay programs. Ahn, who was traveling overseas, answered questions about the study’s findings via e-mail.

Q. Briefly outline the teacher pay-for-performance system used in North Carolina.

A. Two aspects of the pay-for-performance system in North Carolina are worth noting: the emphasis on growth of test scores and payouts aimed at the school level.

The accountability system in North Carolina allows the state to track an individual student’s performance from year to year. Because of the longitudinal nature of the data, the state does not have to rely on just looking at the students’ performance in one given year. Instead, it can look to see how much students have improved since last year, which is a more accurate measure of the work that teachers have put in during the current year.

The bonus, which can be as large as $1,500 per year for each teacher, is disbursed at the school level. That is, the academic improvement of the school as a whole is what determines whether a teacher will receive a bonus.

These two characteristics interact in interesting ways to reveal some surprising insights about whether pay-for-performance works and how to design a good incentive system to maximize teacher effort and student performance.

Q. Summarize the findings of your study. They appear to have gone against conventional wisdom.

A. We find that teachers respond to the bonus by exerting more effort, leading to higher academic performance by students. In fact, the bonus program seems to be very cost effective, giving good bang-for-buck, when compared to other possible reforms, such as class-size reduction. Surprisingly, school-level incentives elicit higher effort from teachers and higher increases in test scores, (at least in the case of North Carolina) compared to individual (class-level) incentives.

The conventional wisdom is that school-level incentives should not work all that well. We find a powerful reason why they should. There is a subtle motivational problem when we have individual incentives.

Imagine a school with two teachers: one teacher finds it difficult to coax test score improvement from her class and the other teacher finds it easy to get her students to perform well. If we set a test score improvement goal at the class level with a cash bonus as the reward, the likely outcome is that neither teacher will try hard.

Why? For the first teacher, the bonus is out of reach, since she will not be able to elicit the required test score increase. She may as well take it easy, since no money is coming her way. The second teacher, on the other hand, is virtually assured of the bonus, due to her ability. Since she will get the money no matter what, her optimal response in this situation is also to take it easy.

Now, what happens when the improvement is evaluated at the school level? That is, what if we average the increase of the two teachers? Now, suddenly, the first teacher has a realistic shot at the bonus, because her numbers will be buoyed by the other teacher. The second teacher, on the other hand, is now forced to try harder, because the bonus is no longer a sure thing.

Q. How does school size factor into this?

A. School size is a concern for school-level incentives, because of the “free-rider” problem. In a large school, one teacher has limited ability to affect schoolwide performance. Whether the teacher is an outstanding educator or an indifferent one, her students’ achievements will be averaged with the performance of students from other classes. The larger the school, the smaller the portion of the schoolwide performance her students will account for. This can lead a teacher to think that she cannot make much of a difference on whether the school is rewarded or not, leading to lower effort and poor academic performance of her students.

Q. How does this system factor into teacher evaluation?

A. We can envision choosing between individual-level and school-level incentives as a tug-of-war. No system is better than the other in all cases across all dimensions. Individual incentives can eliminate free-rider problems but has the possibility of introducing the motivational problem described above. School incentives suffer from free-rider effects but do a better job of making sure that more teachers have a realistic chance at qualifying for the bonus (thus leading to higher effort). In the case of North Carolina, it turns out that school-level incentives result in higher achievement compared to individual-level incentives, but this need not be the case for Kentucky.

Q. How critical to this system is the fact that North Carolina tracks individual students longitudinally rather than group performance in a given year?

A. Tracking students from one year to the next is critical to the North Carolina system. The longitudinal nature of the data is what allows the state to measure academic growth. Without it, an accountability system which is limited to measuring proficiency emerges.

There are potentially large problems with measuring proficiency instead of growth. A school serving low-performing students that improves its test scores dramatically (but fails to reach the proficiency goal) may never be rewarded, while a school serving high-performing students that basically treads water may receive the bonus (despite little growth in academic achievement) every year under such a system.

Q. How will the recent cases of teacher cheating on standardized tests in Atlanta and elsewhere affect the way your study is viewed and the prospects for more widespread teacher rewards based on school-level performance?

A. It is human nature to be tempted into taking short cuts. Teachers are people too, and there will always be some bad apples, especially if administrators are complicit in the deception. I do not see the cheating scandals as damning indictments against the moral fiber of our teachers or school-level performance pay, but these incidents do show us that we have to exercise care in how we set up the system. If we do not pay attention to the details, at best, pay-for-performance will be an inefficient use of tax dollars, and at worst, it will encourage these types of deviant behaviors, from teaching to the test to outright fraud, as in the Atlanta case. I believe there are at least three things to consider going forward:

First, make sure the test measures what the students learn, and make sure the teachers teach what the students are supposed to learn. Addressing this issue requires leadership from the administration as well as grassroots involvement from teachers and parents. We need more efforts devoted to developing a solid curriculum, designing better standardized tests, improving communication between administration and teachers, and lining up what the public expects from our teachers to the realities of what is actually within the realm of possibility.

Second, measure growth in academic achievement, as the North Carolina system does. A teacher has no control over which students arrive in her classroom on the first day of school, but she should be held responsible (and rewarded) for how she impacts the students once they are there. Measuring growth will also make it easier to detect cheating, as anomalous growth in individual scores is easier to analyze.

Third, be aware of the motivation problem in setting the bar for bonus qualification. Making it too easy or too difficult to qualify for the bonus will only result in discouraged or indifferent teachers and wasted resources. The standard must be set such that it is challenging, but not impossible to meet.

A well-designed incentive system where teachers are fairly rewarded will go a long way to deter cheating scandals like this in the future