McKinsey insider: study has no predictive value

We’ve been keeping a close eye on the controversy surrounding McKinsey & Company and the firm’s report on the Affordable Care Act. To briefly recap, McKinsey published a highly dubious study showing nearly a third of American businesses will stop offering health coverage to their employees as a result of the new reform law.

House Speaker John Boehner (R-Ohio) is touting the results of the survey, and Rep. Cliff Stearns (R-Fla.) argued this week that the company’s credibility — McKinsey routinely has government contracts — means the report should get the benefit of the doubt.

At this point, congressional Republicans are the only people in Washington who believe this, and I’m including much of McKinsey’s staff.

Last week, several McKinsey insiders quietly acknowledged that the firm’s usual methodology was ignored when preparing this report, and sent out word they were troubled by its publication. Today, Jonathan Cohn moves the ball forward a bit more.

The report is at odds with predictions from more respected authorities, including the Congressional Budget Office, the Rand Corporation, and the Urban Institute. But McKinsey has declined to divulge more details about the report, despite requests from the White House and congressional Democrats, while company spokesmen have declined to answer questions from reporters, including yours truly. Why the silence? One possibility is that the conclusion can’t withstand scrutiny. […]

McKinsey employees are telling TNR similar things, sometimes in even blunter terms. “Trust me,” says one of the firm’s insiders. “The survey is not a good tool for prediction.”

Another senior firm employee defends the report’s “quality and rigor,” describing it as an “independent, professionally conducted, extensive survey.” But this employee also says that, notwithstanding the hype, the survey was not designed to produce the sort of reliable forecasts that CBO and other authorities make. “We are not making a point prediction or forecast about employer behavior after the implementation of health reform.”

Except, of course, that McKinsey’s report is being used exactly that way, and presented its findings to suggest it’s a warning about policy changes to come.

Keep in mind, McKinsey & Company still refuses to allow an independent analysis of the survey’s methodology, and we still don’t know what the questions were, who wrote them, or who paid for all of this.

The firm has cited proprietary concerns as the basis for the secrecy, but as Greg Sargent reminded us yesterday, McKinsey claims it did not conduct the survey for an outside client, which means there should be no barriers to subjecting the research to scrutiny.

And yet, McKinsey continues to refuse all requests for information. For a firm that’s never had a problem with the peer-review process in the past, this is just bizarre, and the story will continue to dog them until they allow an objective evaluation of the report.