FLAT MAXIMA…. SOLVED!…. Yesterday I asked whether NFL scouts were really able to choose the best players out of a college draft populated exclusively by the finest athletes in the country. Or were the differences between the top players so small that they were essentially picking randomly?
The chart on the right shows the answer. It’s from a paper by Cade Massey and Richard Thaler, who mapped the performance of all NFL players drafted from 1991-2002. They chose five measures of performance, and by all of those measures it turned out that players chosen in the first round of the draft did better than those chosen in the second, second round picks outperformed third round picks, etc. The measures aren’t perfect, but taken together they seem to make it clear that even when confronted by a small group of the most elite performers on the planet, scouts can indeed predict future performance reasonably well.
What this appears to show is that, at least in a case where scouts have truckloads of information on each prospect, the principle of the flat maximum doesn’t hold up. Yes, everyone chosen in the first few rounds of the NFL draft is a phenomenal athlete, but there are still differences large enough to predict future performance with at least moderate accuracy.
On the other hand, the operative word is still “moderate.” If you’re interested in more (and who wouldn’t be?), scroll down to Figure 7 in the paper, which compares players by position and draft order. (That is, it compares them not merely by draft round, but by the actual order they’re picked.) Players picked earlier do get more starts than players picked later, but their Pro Bowl performance is nearly identical. Within the first draft round, the first player chosen at a given position is only barely more likely to make the Pro Bowl than a player chosen four spots later. Within all rounds, the difference is a little greater, but still only about 55%. The difference in pro performance between the third linebacker chosen and the seventh linebacker chosen is very, very small.
Of course, in most situations decisionmakers don’t have nearly as much information about their prospects as NFL scouts. If this is as good as they can do, how likely is it that a college admissions committee can choose between a group of students who are all like this? Probably not very.
