Were the cost estimates for Waxman-Markey overstated by 200-300%?

Jesse and I both come from the Sustainable Development PhD Program at Columbia, which has once again turned out a remarkable crop of job market candidates (see outcomes from 2012 and 2011). We both agreed that their job market papers were so innovative, diverse, rigorous and important that we wanted to feature them at FE.  Their results are striking and deserve dissemination (we would probably post them anyway even if the authors weren't on the market), but they also clearly illustrate what the what the Columbia program is all about. (Apply to it here, hire one of these candidates here.) This is the first post.

Good policy requires good cost-benefit analysis. But when we are developing innovative policies, like those used to curb greenhouse gas emissions, it's notoriously difficult to estimate both costs and benefits since no analogous policies have ever been implemented before.  The uncertainty associated with costs and benefits tends to make many forms of environmental policy difficult to implement in part because the imagined costs (when policy-makers are considering a policy) tend to exceed actual costs (what we observe after policies are actually implemented). Kyle Meng develops an innovative approach, linking Intrade predictions about the success of Waxman-Markey with stock-market returns and abrupt political events, to measure the cost of the bill to firms as predicted by the market. This is very different from standard technocratic approaches used by the government to assess the cost of future policies, which rely on parameterized models of technology and econometric models of behavior ("structural models").

By relying on the market, Meng infers what players in affected industries actually expect to happen in their own industry. The result is a bit surprising: Meng estimates that standard costs-estimates for WM (produced before it failed to pass) are 200-300% larger than what players in the industry actually expected it to cost them.  But this still didn't stop industry players from fighting the bill -- one of the ways that Meng validates his approach is to use lobby records to show that firms which expect to suffer more from the bill (as recovered using his approach) spend more money to fight it.

It's tough to tell whether Meng's approach or the structural models are more accurate predictors of firm-level costs since WM was never brought into law, so the outcomes will remain forever unobserved. But he does show that for several similar laws (eg. the Montreal Protocol), the structural predictions tended to overestimate the actual costs of implementation (which were observed after the law was implemented and outcomes observed) by roughly a factor of two. This doesn't prove that Meng's approach is more accurate, but it shows that his estimate for the bias of the structural approach (with regard to WM) is consistant with the historical biases of these models.

The paper:

The Cost of Potential Cap-and-Trade Policy: An Event Study using Prediction Markets and Lobbying Records
Kyle Meng

Abstract: Efforts to understand the cost of climate policy have been constrained by the limited number of policies available for evaluation. This paper develops an empirical method for forecasting the expected cost to firms of a proposed climate policy that was never realized. I combine prediction market prices, which reflect market beliefs over regulatory prospects, with stock returns in order to estimate the expected cost to firms of the Waxman-Markey cap-and-trade bill, had it been implemented. I find that Waxman-Markey would have reduced the market value of a listed firm by an average of 2.0%, resulting in a total cost of $165 billion for all listed firms. The strongest effects are found in sectors with greater carbon and energy intensity, import penetration, and exposure to U.S. product markets, and in sectors granted free allowances. Because the values of unlisted firms are not observed, I use firm-level lobbying expenditures within a partial identification framework to obtain bounds for the costs borne by unlisted firms. This procedure recovers a total cost to all firms between $110 and $260 billion. I conclude by comparing estimates from this method with Waxman-Markey forecasts by prevailing computable general equilibrium models of climate policy.

In figures...

Abrupt political events that affect the expected success of WM are quantified by looking at expectations in Intrade markets:

click to enlarge

When WM appears more likely, the stock prices of CO2 intensive firms falls on average:

click to enlarge

Firms that are more CO2 intensive are affected more strongly:

click to enlarge

Firms whose stock prices are more responsive to WM lobby harder against it:

click to enlarge

How these cost estimates compare with structural cost estimates, and similar statistics for historical regulations that actually passed into law.

click to enlarge

Take home summary: Cap and trade in the USA probably would have been cheaper to implement than we thought, according to the firms it was going to regulate.