Check out this piece today by Robert Reischauer and Michael McPherson. Reischauer, former director of the Congressional Budget Office, and McPherson, president of the Spencer Foundation, argued that one of the major problems for education reform is the way we collect data.
It’s important to measure outcomes, they write–
If you don’t track your performance, you can’t tell whether you’re improving, and you have no reliable way to know if your improvement strategies are having the desired effects. Resistance to measurement can often reflect reluctance to face up to the need for sometimes difficult, but vitally important, institutional change.
–but just because you’re measuring something doesn’t mean that thing is valid or useful.
A big problem arises in attempts to measure college performance. The college graduation rate is low, so it makes sense to evaluate (and presumably reward and punish) institutions of higher learning on their graduation rates. But such performance measures could be very tricky:
Some have suggested that federal and state policy should provide financial incentives for colleges that improve their graduation rates. But if we reward colleges for improving their graduation rates, college administrators may respond by simply reducing admissions of students who face significant academic challenges. If all colleges were to follow such a policy, they all might wind up with higher graduation rates, but the total number of students graduating would be smaller and many young people would be denied an opportunity for economic and social mobility.
Another risk in rewarding colleges for graduating more students is that such a policy may induce them to lower their standards. Most colleges have strong internal checks and balances to guard against that response, but nonetheless there is a real risk of erosion over time.
There are ways around these problems, for sure–Reischauer and McPherson recommend adjusting the graduation rate to account student background and tracking average earnings graduates and admissions rates to graduate school–but it’s important to design measurement systems to keep such things in mind. Measurement is only useful if it leads to real performance improvement. Anything else is pretty much a waste of effort.
