There’s a good reason for the American fixation with rankingsif done correctly, they can help tell us what’s working and what’s not. Of course universities ought to be judged. The key is judging the right things.
All of the existing college rankings have the same aimto help overwhelmed parents and students sift through the thousands of colleges and universities in this country by giving them some yardstick for judging the best schools. Whether the guides actually do measure academic excellenceas opposed to, say, academic reputation (not always the same thing)is debatable at best (see Broken Ranks by Amy Graham and Nicholas Thompson, September 2001). The publishers of these guides argue that they are providing a valuable consumer service. Parents who will shell out tens of thousands of dollars to put their teenagers through college need to know they are spending their money wisely.
How much more important, then, is it for taxpayers to know that their moneyin the form of billions of dollars of research grants and student aidis being put to good use? These are institutions, after all, that produce most of the country’s cutting-edge scientific research and are therefore indirectly responsible for much of our national wealth and prosperity. They are the path to the American dream, the surest route for hard-working poor kids to achieve a better life in a changing economy. And they shape, in profound and subtle ways, students’ ideas about American society and their place in it. It seemed obvious to us that these heavily subsidized institutions ought to be graded on how well they perform in these roles, so we set out to create the first annual Washington Monthly College Rankings. While other guides ask what colleges can do for students, we ask what colleges are doing for the country.
The first question we asked was, what does America need from its universities? From this starting point, we came up with three central criteria: Universities should be engines of social mobility, they should produce the academic minds and scientific research that advance knowledge and drive economic growth; and they should inculcate and encourage an ethic of service. We designed our evaluation system accordingly. (See “A Note on Methodology”.)
Given our very different way of measuring success, we suspected that the marquee schools routinely found at the top of U.S. News‘s list might not finish at the very top of oursbut even we were surprised by what the data revealed. Only three schools in the 2006 U.S. News top 10 are among our highest-ranked: MIT, Stanford, and the University of Pennsylvania. In addition, while the private colleges of the Ivy League dominate most rankings of the nation’s best colleges, they didn’t dominate oursonly Cornell and the University of Pennsylvania made our top 10, and Princeton (tied with Harvard for the top slot on U.S. News‘s current list) was all the way down at #44, a few slots behind South Carolina State University.
Our list was also more heavily populated with first-rate state schools (the University of California system scored particularly well) than that of U.S. News, which has no public universities within its top ten. UCLA finished second in our overall ranking, UC-Berkeley third, Penn State University sixth, Texas A&M seventh, UC-San Diego eighth and the University of Michigan tenth. Each of our highest-rated schools are, by any reasonable national measure, academically serious schools. But they are not the super-elitethe Harvards and Yalesthat normally dominate lists of the nation’s best universities.
The schools that topped our list didn’t necessarily do so for the reasons you might expect. MIT earned its number one ranking not because of its ground-breaking research (although that didn’t hurt), but on the basis of its commitment to national servicethe school ranked #7 in that category, far better than most of its elite peers. Similarly, UCLA, which finished second in our overall rankings, excelled in research and came in first in our social mobility rating because of its astoundingly successful graduation rate for lower-income students. (Schools in the University of California’s system were consistently high performers in this area: UCLA took top honors, with UC-Berkeley, UC-San Diego, UC-Davis, and UC-Riverside not far behind.) At the same time, Princeton finished behind schools such as the University of Arizona and Iowa Stateschools with which it probably does not often consider itself to be in competitionnot just because of its comparatively low research numbers, which are perhaps to be expected given that the university doesn’t have a medical school and considers its mission to be teaching, not research. What really did in Princeton were mediocre scores on national service and social mobility, categories in which it should have excelled.
Princeton’s comparatively low ranking is evidence of something else indicated by our numbers. Schools that are similar in size, prestige, and endowment end up in very different places on The Washington Monthly College Rankings, largely because of decisions they have made about how to prioritize their resources or focus their energies. When it comes to social mobility, for instance, Harvard has about the lowest percentage of Pell Grant recipients in its student body of any school in the country. By comparison, Columbia, whose institutional ambitions and prestige are similar to Harvard’s, has twice as many lower-income students as its counterpart on the Charles River; Cornell has nearly three times the number. Public universities provide some equally interesting data: Both Indiana University and the University of Virginia are the most elite public institutions in states with populations of roughly similar wealth, yet the percentage of IU students who are Pell Grant recipients is nearly twice that of UVa.
On research, as well, the results are interesting. The big state schools finished somewhat higher than we had expected, and the super-elite schools (the Cal Techs and Harvards) fell somewhat lower. Even so, we were caught off-guard by some of the top finishers, including University of California’s San Diego campus. UCSD is not normally considered among the elite UC campusesUCLA and UC-Berkeley have that distinctionmuch less top-tier national schools. But it has quietly rounded up a formidable team of scholars. Nine Nobelists are on faculty at UCSD (Dartmouth, by comparison, has none), and the National Research Council recently ranked its Oceanography, Neurosciences, Physiology, and Bioengineering departments either first or second in the country. This concentration of talent translates into direct benefits for the surrounding community: Forty percent of the companies in San Diego’s biotech corridor are spin-offs of research based at UCSD. These accomplishments landed UCSD in the sixth slot for research grants, and eighth on our overall rankings.
Perhaps the most striking data, however, is found in national service. Our measures here were simple: whether a school devotes a significant part of its federal work study funding to placing students in community service jobs (as the original work study law intended); the percentage of students enrolled in ROTC; and the percentage of graduates currently enrolled in the Peace Corps. All schools, large and small, are capable of excelling in these areas. In fact, we found that while some very small and nationally unknown schools have made an aggressive commitment to national service, most of the highest ranking U.S. News schools have not. The University of Portland, for example, finishes third in national service while Harvard lingers down at #75. Harvard obviously has far more resources than the University of Portland, and there’s no question that it could match Portland’s remarkable performance on service, if it chose to make a similar commitment to emphasizing that value among its students. But, at least by the criteria we set, it has not.
These service results haven’t changed much since the first time we rated colleges on their commitment to national service (see “The Other College Rankings” by Joshua Green, Jan./Feb. 2002). But there’s one nice surprise: MIT leaped from near the bottom of the pack three years ago to near the top today.
A word on our criteria. This is the first Washington Monthly College Rankings. In future years, we would prefer to expand our criteria and develop an even more comprehensive measure of the qualities by which colleges and universities enrich our country. There’s only one problem: Many of these data aren’t available. We would love, for example, to add a category measuring academic excellence. It’s nearly impossible, however, to directly gauge the quality of education a student receives at a given school. Most ranking systems rely on measures of inputssuch as the average SAT scores of the incoming class or the size of faculty salaries. But as Amy Graham and Nicholas Thompson noted in these pages four years ago, [t]hat’s like measuring the quality of a restaurant by calculating how much it paid for silverware and food: not completely useless, but pretty far from ideal.
There is one existing set of data that would do a great deal to answer that question: the National Survey of Student Engagement (NSSE). NSSE compiles such information as the average number of hours students at a particular school spend doing homework or meeting with professors outside of classmeasures which, studies show, are highly correlated with academic achievement. Unfortunately, the vast majority of colleges and universities refuse to grant NSSE permission to release their schools’ scores to the public, and legislation to force them to do so, sponsored by Sen. Ted Kennedy (D-Mass.), has been stifled in Congress. Rather than use data that we believe doesn’t accurately measure the quality of an undergraduate education, we chose not to include that category in this year’s ranking. We hope to be able to add in those measures in the future.
And while we’re putting together our wish list, we have a few additional requests. We would prefer that the federal government require every school to report the percentage of Pell Grant recipients who actually graduate, but it doesn’t. We would love it if schools kept a systematic count of which professions their graduates enteredsuch as teachingbut they don’t. And we would be thrilled if the federal government tabulated how many of its employees came from which schools.
Still, we have tried to abide by the best principles of social science and used the best data available to generate the closest possible measures of the qualities we value. It pleased us to use metrics for success that were almost all within the means of even the most modest of our nation’s universities. For that is more or less the point of this exercise. Succeeding on the Washington Monthly ranking (and succeeding at serving the country) is within the reach of most schools. Granted, most colleges are unlikely to catch up to Johns Hopkins on research overnight. But when it comes to service, Portland finishes well because it has made an institutional commitment to values that work. And in terms of social mobility, schools such as Alabama A&M and South Carolina Statehardly considered academic powerhousesscore very high because their graduation rates are well above what their Pell Grant numbers would have predicted.
The U.S. News rankings, and others like them, have had an impact. A growing body of reporting and scholarship shows that the criteria these guides use have sent administrators scurrying to increase the amount of money given by their alumni or the SAT scores of their incoming freshman in order to improve their score. Such measures have arguably very little impact on how well a school serves its student body, but as schools compete for students, every little thingincluding rising or dropping two spots on a listcounts.
Imagine, then, what would happen if thousands of schools were suddenly motivated to try to boost their scores on The Washington Monthly College Rankings. They’d start enrolling greater numbers of low-income students and putting great effort into ensuring that these students graduate. They’d encourage more of their students to join the Peace Corps or the military. They’d intensify their focus on producing more Ph.D. graduates in science and engineering. And as a result, we all would benefit from a wealthier, freer, more vibrant, and democratic country.
