FOXES AND HEDGEHOGS….In a New Yorker piece put online a few days ago, Louis Menand reviews Philip Tetlock’s new book, Expert Political Judgment: How Good Is It? How Can We Know? His conclusion:
People who make prediction their business ? people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables ? are no better than the rest of us….Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert?s predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote.
Still, some people score higher when it comes to predicting the political future and some score lower. What accounts for the difference? According to Tetlock:
Low scorers look like hedgehogs: thinkers who ?know one big thing,? aggressively extend the explanatory reach of that one big thing into new domains, display bristly impatience with those who ?do not get it,? and express considerable confidence that they are already pretty proficient forecasters, at least in the long term. High scorers look like foxes: thinkers who know many small things (tricks of their trade), are skeptical of grand schemes, see explanation and prediction not as deductive exercises but rather as exercises in flexible ?ad hocery? that require stitching together diverse sources of information, and are rather diffident about their own forecasting prowess.
Menand points out that Tetlock’s hedgehogs are wrong more often than his foxes, but that’s not the end of the story. “The upside of being a hedgehog, though, is that when you?re right you can be really and spectacularly right.” Which explains why Time magazine named Power Line their blog of the year for 2004.
So there’s your lesson for the day. Avoid ideologues on both left and right. Stay away from people who have unshakable faith in their convictions. The more confident someone sounds, the more likely they are to be wrong. Steer clear of cranks with big theories. Pay more attention to statistical and actuarial formulas than to expert opinion. And ignore the folks at Power Line. They aren’t due to be right again for a long time.
