YES, JOHN LOTT AGAIN….If you are an econometrician ? a person who evaluates the real world using complex statistical models ? there are two basic ways you can go about your job:
-
You can do your best to figure out which statistical model does the best job of mirroring the real world, and then plug in your data and see what pops out. We will call this methodology tolerably honest.
-
You can plug in your data first, and then tweak your model until it provides the results you want. We will call this methodology dishonest bullshit.
The alert reader has probably guessed that I am talking here about the latest sad chapter in the John Lott saga, and indeed I am. The indefatigable Tim Lambert is on the case, and assuming I have been able to put the timeline together correctly, here’s what’s happened:
-
Lott and two coauthors produced a statistical model (“Model 1”) that showed significant crime decreases when states passed concealed carry gun laws.
-
Back in April, two critics discovered that there were errors in the data Lott used. When the correct data was plugged into Lott’s model, his results went away.
-
After a long silence, Lott admitted the data errors and posted a table with new results. Oddly, though, his new results were similar to his old ones and continued to show significant drops in crime. So who’s right, Lott or his critics?
-
Answer: his critics. It turns out that since he really had no choice but to use the corrected data, and the corrected data erased his results, he decided to invent a different model (“Model 2”) for use in this new table ? but without disclosing the fact that he had switched to a new model specifically constructed to keep his results intact. Note: In less refined circles this would be called “lying.”
-
When Tim discovered that Lott had surreptitiously changed his model, he emailed Lott. No response.
-
It turns out Lott was busy covering his tracks. How? By quietly removing the corrected table from his website and replacing it with a new corrected table. This one uses Model 2 but has the old, incorrect data.
-
Here’s where you have to pay attention. Why would Lott do this?
Answer: this new table claims to be “corrected: April 18, 2003,” and it turns out that Lott is trying to pretend that this was the original table he had posted all those months ago. That way, he could claim that he had never changed his model at all. Model 2 is the one he’s been using all along!
-
Unfortunately, when Lott changed the revision date on the document to make it look like it had been created on 4/18/03, he changed it to 1/18/04 instead. What’s more, Lott apparently doesn’t know that you can check the create date of PDF documents anyway, and this one was created on 9/2/03. That is, it was created in September, not April.
Basically, Lott wants to pretend that Model 2 is the one he’s always used. That way, when he corrects the data errors, his results still hold up. Unfortunately for Lott, his attempts to rewrite history were as clumsy as they were dishonest. His original table did use Model 1, his results do go away when the corrected data is plugged in, and he did respond to this by furtively devising a new model that would continue to give him the results he wanted.
If you’re not sure you understand what’s going on here, reread the timeline. Reread it five or six times. Eventually it will all become clear.
And a note to Glenn Reynolds, who has said he is “not sufficiently knowledgeable to opine on the statistical questions”: my timeline deliberately avoids discussing the validity of the competing econometric models, which I’m not competent to judge either. Rather, it simply shows how Lott works, something that anyone is competent to judge. He’s a liar and a cheat, and merely being “quite reluctant” to rely on him is far too weak a response.
The evidence is clear. John Lott should be fired from the American Enterprise Institute forthwith and banned from polite society.
