Social scientists often write papers where at some point they pivot from a well-founded but narrow claim to a broad conclusion that is unsupported by theory or data. That pivot is called “story time.”
An economist offers a theoretical model explaining how it works. Ole Rogeberg writes:
Here`s a blogpost regarding a new paper (embellished with video and an essay) where a colleague and I try to come up with an explanation for why the discipline of economics ends up generating weird claims such as those you`ve blogged on previously regarding rational addiction.
From Ole’s blog:
The puzzle that we try to explain is this frequent disconnect between high-quality, sophisticated work in some dimensions, and almost incompetently argued claims about the real world on the other. . . .
Our explanation can be put in terms of the research process as an “evolutionary” process: Hunches and ideas are turned into models and arguments and papers, and these are “attacked” by colleagues who read drafts, attend seminars, perform anonymous peer-reviews or respond to published articles. Those claims that survive this process are seen as “solid” and “backed by research.” If the “challenges” facing some types of claims are systematically weaker than those facing other types of claims, the consequence would be exactly what we see: Some types of “accepted” claims would be of high standard (e.g., formal, theoretical models and certain types of statistical fitting) while other types of “accepted claims” would be of systematically lower quality (e.g., claims about how the real world actually works or what policies people would actually be better off under).
In our paper, we pursue this line of thought by identifying four types of claims that are commonly made – but that require very different types of evidence (just as the Pythagorean theorem and a claim about the permeability of shale rock would be supported in very different ways). We then apply this to the literature on rational addiction and argue that this literature has extended theory and that, to some extent, it is “as if” the market data was generated by these models. However, we also argue that there is (as good as) no evidence that these models capture the actual mechanism underlying an addiction or that they are credible, valid tools for predicting consumer welfare under addictions. All the same – these claims have been made too – and we argue that such claims are allowed to piggy-back on the former claims provided these have been validly supported. We then discuss a survey mailed to all published rational addiction researchers which provides indicative support – or at least is consistent with – the claim that the “culture” of economics knows the relevant criteria for evaluating claims of pure theory and statistical fit better than it knows the relevant criteria for evaluating claims of causal or welfare “insight”. . . .
If this explanation holds up after further challenges and research and refinement, it would also provide a way of changing things – simply by demanding that researchers state claims more explicitly and with greater precision, and that we start discussing different claims separately and using the evidence relevant to each specific one. Unsupported claims about the real world should not be something you`re allowed to tag on at the end of a work as a treat for competently having done something quite unrelated.
Or, as Kaiser Fung puts it, “story time.” (For a recent example, see the background behind the claim that “a raise won’t make you work harder.”)
This (Ole’s idea) is just great: moving from criticism to a model and pointing the way forward to possible improvement.
[Cross-posted at The Monkey Cage]
