Regression coefficient stability over time (in Stata)

If you're estimating a regression model with time series or panel data, you often would like to know if the coefficient you're interested in is changing over time or if its stable for subsamples of the time series or panel.

Here's my simple but handy script that let's you see if your coefficient is stable or changing with a single command.  You specify your multiple regression model (for OLS, but you can change it easily to run a different estimator), which coefficient to examine.  It does a sequence of regressions for a moving window of specified length and stores the changing coefficient, SE and CI (t-test).

For example, the figure at right shows the coefficient from the model

GDP_it = B * cereal_yields_it + e_it

for a panel of all countries estimated for a moving window that covers 3 years at a time.

The description of the code is just commented out in the script and pasted below the fold.

[7/14/2011: A small but important error in the code was fixed. Thanks to Kyle for catching it.]

AGU Climate Q&A Service for Journalists

Last year the American Geophysical Union (of the superlatively massive AGU Fall Meeting held every December in San Francisco) piloted something they called The Climate Q&A Service for Journalists during their fall meeting. The idea was to set up a website where journalists could submit questions they had about climate science and climate scientists could sign up for slots where they'd monitor questions coming in and answer them.

Apparently it was sufficiently successful that they're doing it again this year, here, except that it's been expanded to run from Oct 24th - Jan 21st. If you're a climate scientist looking to do a good deed and help people understand how the climate works, hop on over and sign up for a time slot. Or if you're a reporter looking to get some questions about climate science asked, go on over and submit a question.

Photoessay on the African Middle Class

I stumbled across a great photo essay on the African middle class over at classesmoyennes-afrique.org. It's in French but there's an English version here.

Nothing terribly deep on the analytical takedown here aside from the fact that it's nice to be reminded that African countries aren't entirely populated by desperately poor farmers and kleptocrat dictators.

Plus the pictures are great.

Prison privatization, political economy, and interest group creation

NPR has a great ongoing series exploring how privatization can have extraordinary externalities, namely the role of private prisons in the drafting of Arizona's immigration law:

Thirty of the 36 co-sponsors received donations over the next six months, from prison lobbyists or prison companies — Corrections Corporation of America, Management and Training Corporation and The Geo Group.

By April, the bill was on Gov. Jan Brewer's desk.

Brewer has her own connections to private prison companies. State lobbying records show two of her top advisers — her spokesman Paul Senseman and her campaign manager Chuck Coughlin — are former lobbyists for private prison companies. Brewer signed the bill — with the name of the legislation Pearce, the Corrections Corporation of America and the others in the Hyatt conference room came up with — in four days.

Brewer and her spokesman did not respond to requests for comment.

This is an excellent example of why economics is both an extraordinary tool for analysis but also one that is easily abused. The primary argument for privatization almost always comes down to one of efficiency: the public sector is slow, it's bloated, taxpayers pay $1000 for toilet seat installation, etc. That is not a concern to be belittled, and removing all constraints and checks from public sector workers is clearly a horrible idea.

But the flip side is that there are deep political economy concerns any time you privatize the provision of a public good. The incentives society as a whole faces (i.e., let's *not* incarcerate everyone all the time) are the exact opposite of what a for-profit prison faces (from the article: "They talk [about] how positive this was going to be for the community," Nichols said, "the amount of money that we would realize from each prisoner on a daily rate."). Since politicians respond to incentives, too, and for-profit companies have money to spend on campaign donations, political ads, etc., an already difficult problem is made much more complex (and, I'd argue, welfare decreasing) by following a simple welfare-enhancing efficiency argument.

Now, the flip side of the flip side (since I doubt most people reading a "sustainable development" blog are hungering for privatization) is that the same argument applies for all organized interest groups, not just private ones. This is very similar to the classic argument against unions, for example: New York's widely disliked (even by a lot of teachers) United Federation of Teachers is able to exert a huge amount of political pressure to support policies that are almost certainly harmful to educational outcomes, e.g., making firing bad teachers extraordinarily hard. The problem is the same: once an interest group comes into existence it will do its best to influence policy in its favor.

So what do we do? Legitimately, I think the two solutions are the obvious, difficult ones: transparency in campaign finance and mobilization of counter-interest groups. The first is hard for all of the obvious reasons (including most organized interest groups being against it) and the second is hard because often the counter-interest faces not just asymmetric funding (there's no private interest in keeping people *not* in prisons) but also a fundamental public goods problem: damages tend to be dispersed and costs of abatement concentrated, so anyone joining the counter-interest is either going to be doing it altruistically (e.g., a non-profit), because they were one of the unlucky few who got hit with particularly concentrated damages (e.g., the family of someone unjustly imprisoned), or because they reap some metabenefit (e.g., the NPR reporters covering this story) .

Which is to say, surprise, it's a fairly intractable and complex problem. If there's a lesson to extract I think that it is, as it so often seems to be, to always think about how incentives align, especially before you make large, difficult-to-reverse decisions. In particular, I think it's important to remember that creating a monied interest group is one of the most difficult-to-reverse decisions there is.

Statistical inference isn't easy, either

I was just playing around with the citation management software / website Mendeley (recommended by Amir, and worth checking out for the auto-formatting of citations alone) when I trolled over to their "most read articles in all disciplines" section and saw that the 3rd most read article was a PLoS Medicine piece titled "Why most published research findings are false: author's reply to Goodman and Greenland," by Ioannidis. Ignoring the fact that it's the response and not the original paper (huh?) and led on by the rather provocative title, I poked around and discovered that Ioannidis' work just got written up in The Atlantic and was covered in pretty nice detail by Marginal Revolution back when it came out. So blogging about it does feel a bit like trying to review a restaurant that's already been covered by Frank Bruni and Food and Wine, but I'm going to go ahead and do so anyway since the point is so worthwhile.

The crux of the paper rests on a pretty simple idea: if you're running a huge number of one-off statistical tests (i.e., not testing the same hypothesis over and over) a fraction of your results proportional to the power of your test will be false positives (i.e., type I error). This is pretty straightforward a concept for anyone doing applied work: if you're checking to make sure you've got balance across treated and controlled populations in a randomized trial, for example, having an occasional statistically significant difference between the two populations isn't a huge deal as long as the percentage of variables that turn up that way is proportional to the significance level you're setting. Yes, you should follow through as a good little applied researcher and make sure something's not hiding there, but some portion of your results will always end up that way due to random variation.

The nice step that Ioannidis takes is to look at the entire field of medical research and apply the same logic, effectively viewing the suite of randomized trials as a game where we keep picking new potential tests for the same problems over and over again, some subset of which are guaranteed to be incorrectly not-rejected. To quote Alex Tabarrok's pithy wording of it in the Marginal Revolution post:

Want to avoid colon cancer? Let's see if an apple a day keeps the doctor away. No? What about a serving of bananas? Let's try vitamin C and don't forget red wine.

Moreover, since the number of things that actually, say, help avoid colon cancer is likely small, and the number of tests being run to find things which do is large, Ioannidis concludes that a large portion ("most") results are in fast false positives and thus meaningless. It's a pretty simple premise which leads to a pretty deep statement about how we think about learning about the world.

So the solutions to this are, of course, pretty intuitive: don't trust small sample size studies; insist on retesting hypotheses; be skeptical of results in any field where a large number of researchers are pursuing solutions to the same problem. In short, demand robustness checks on everything, and make sure that what's being shown is not just an artifact of your specific data set. Good lessons that all applied researchers should have tattooed across their proverbial chests already, but nonetheless a nice thing to be reminded of.

The structure of human knowledge

Following up on Jesse's post:

After dinner today I told Brenda that I wanted a network map of all papers ever written so we could see where the biggest gaps in human knowledge were. In moments she had us browsing the site well-formed.eigenfactor.org looking at a coarser approximation of my dream (see picture).

I highly recommend any academic or casual intellectual browse the highly interactive site, it is simply too interesting, beautiful and [maybe] important to ignore.

Perhaps the two most striking observations one can make from simple visual inspection are that (1) biologists write a lot of papers and (2) social sciences/mathematics/computer science are extremely insular (observe the big "hole" in the network picture).

I'll let the data speak for itself (please please look at the site); but the only thing I'll say is that if anyone wanted to create a new field, bridging the social and physical sciences looks like a conspicuously good place to start.

Interdisciplinarity isn't easy

I'm currently wrapping up edits on a paper on interdisciplinarity and research success and came across a pretty cool paper for my lit review with a couple of choice quotes:

"[T]he young scientist, who grows up in the midst of a competition between university departments and amidst competition within his department, who inherits the individualistic research tradition and graduates without having had an opportunity to develop skills in cooperative thinking and collaborative study, is poorly prepared to participate in the activities of a committee or a research team.

"Over and above this pressure from the outside, there are important scientific grounds why interdisciplinary (and interdepartmental) research should become a greater concern of the universities. The assertion that institutes of an interdisciplinary character will be associated more often with industrial enterprises than with universities may be correct in the statistical sense, but it should not imply that cooperative research is an industrial prerogative.

"For the research worker who has grown up in the traditional departmentalized university and who is anxious to take part in interdisciplinary work, the first step is to get a bird's-eye view of the neighboring fields and to obtain familiarity with the problems which are currently the foci of interest. However, text-book acquaintance is not enough; some contact with actual work methods is essential. "

I think these are all reasonably fair points. The problem is that this article is from the December 8th, 1944 issue of Science. Reading through it and noting how little change there's been in the language around "interdisciplinary research" is fairly shocking, and makes me appreciate not only how difficult working outside of one discipline is, but also the extent to which the road towards doing quality work combining the social and natural sciences (which is probably the best way to describe the specific flavor of interdisciplinarity that Sol and I are in) has been a long and arduous one.

Not that there hasn't been any progress, mind you. The flip side of interdisciplinary work is field creation. Climatology, neuropsychology, behavioral economics, and an untold number of additional academic disciplines were all, at one point, "inter-discipline." It's just nice to be reminded of the fact that establishing those fields isn't easy.

Cleanup cost from Haiti earthquake

I was struck by this NYT article stating that just the cost of debris removal in Haiti is estimated at $1B

By late summer, however, the need to tackle the earthquake damage directly became so glaring that some initial steps were taken. The government tendered its very first cleanup contract to Mr. Perkins’s Haiti Recovery Group. Worth $7.5 million to $13.5 million — nobody would be more precise — the contract represented a minuscule piece of a debris removal operation expected to cost $1.2 billion.

This is an incredible sum, when you consider that income for the entire country is about $7B



The cost of cleanup is just what is paid for the removal of damaged property and excludes the value of the lost assets and lost revenue due to the destruction of assets.

Sustainable Development PhD Research Symposium Oct 28th

Here is the Earth Institute announcement for an upcoming event at Columbia University on October 28th.

Sustainable Development Ph.D. Research Symposium

Date: Thursday, October 28th

Time: 4.00-6.30 PM

Location: Jed D. Satow Conference Room; 5th Floor, Lerner Hall; Columbia University

---

---

The first annual Sustainable Development Ph.D. Research Symposium has been scheduled for Thursday, October 28th, 4.00-6:30 PM, in the Jed D. Satow Conference Room (5th Floor, Lerner Hall).

The purpose of the symposium is to showcase the pioneering research of the Ph.D. Program in Sustainable Development’s 5th and 6th year doctoral candidates to the wider Columbia University community and invited guests from the private sector, governments, and NGOs. It will be attended by: the Director of the Earth Institute, Prof. Jeffrey Sachs; the Dean of the School of International and Public Affairs, Prof. John Coatsworth; the program’s Academic Directors, Prof. John Mutter and Prof. Wolfram Schlenker; and many of the program’s core faculty.

The symposium will consist of a series of short presentations, followed by short question and answer sessions and a general discussion.  The topics of the presentations will cover many of the most pressing global sustainability issues, including the global economic losses to tropical cyclones, the future of India’s dwindling groundwater resources, drought and floods and poverty traps in rural Mexico, the effects of climate change on Indian agriculture and the connections between Malaria ecology and demography.

---

---

SCHEDULED SPEAKERS:

(1) Chandra Kiran Krishnamurthy: A Quantile Regression Approach to Estimating Climate Change Impacts on Crop Yields. [Link to Chandra's profile].

(2) Gordon McCord: Improving Empirical Estimation of Demographic Drivers: Fertility, Child Mortality & Malaria Ecology. [Link to Gordon's profile].

(3) Anisa Khadem Nwachuku: The Materialism Paradigm: Neither Sustainable, nor Development. [Link to Anisa's profile].

(4) Marta Vicarelli: Exogenous Income Shocks and Consumption Smoothing, Strategies Among Rural Households in Mexico. [Link to Marta's profile].

(5) Jesse Anttila-Hughes: The Long Term Fertility Impacts of Natural Disasters. [Link to Jesse's profile].

(6) Ram Fishman: How Low Will It Go?  The Future of Groundwater Tables and Irrigation in India. [Link to Ram's profile].

(7) Solomon Hsiang: Global Economic Losses to Tropical Cyclones. [Link to Solomon's profile].

(8) Aly Sanoh: Municipal Taxes, Income, and Rainfall Uncertainty. [Link to Aly's profile].

Political institutions evolve only incrementally

Image copyright Nature 2010

I really enjoyed this interesting and creative article in Nature this week:

Rise and fall of political complexity in island South-East Asia and the Pacific

Thomas E. Currie, Simon J. Greenhill, Russell D. Gray, Toshikazu Hasegawa & Ruth Mac

Abstract: There is disagreement about whether human political evolution has proceeded through a sequence of incremental increases in complexity, or whether larger, non-sequential increases have occurred. The extent to which societies have decreased in complexity is also unclear. These debates have continued largely in the absence of rigorous, quantitative tests. We evaluated six competing models of political evolution in Austronesian-speaking societies using phylogenetic methods. Here we show that in the best-fitting model political complexity rises and falls in a sequence of small steps. This is closely followed by another model in which increases are sequential but decreases can be either sequential or in bigger drops. The results indicate that large, non-sequential jumps in political complexity have not occurred during the evolutionary history of these societies. This suggests that, despite the numerous contingent pathways of human history, there are regularities in cultural evolution that can be detected using computational phylogenetic methods.

Probably the most fun aspect of the article is that they take some clever statistical techniques to evaluate a complex archeological/anthropological data set that is never analyzed with quantitative methods. Their main result is that it seems very unlikely that political institutions develop complexity with leaps and bounds. Rather, institutions appear to evolve very slowly with only incremental changes in complexity.  However, they find that the reverse is not true: sometimes it looks as though complex political institutions may collapse rapidly to much simpler institutions.

They basically use maximum-likelihood techniques to estimate the probability that political systems make specific transitions in a simple markov-chain model of political institutions.  Because the probability of certain transitions between nodes in the chain seem very unlikely, they are able to rule out certain models of political development (leaps and bounds) that otherwise seemed plausible.  Very interesting.

African maritime infrastructure

A recent article by Michael Lyon Baker in Foreign Affairs make an unconventional but interesting point: African maritime ports are in bad shape and this produces a bottleneck for trade and economic growth.

Africa has the least efficient ports in the world. Dwell times -- the amount of time a ship must stay in port -- for the loading and unloading of cargo exceed global averages by several days and are nearly quadruple those of Asian ports, thus driving up shipping costs through delays. No African port can be found on the list of the top 70 most productive in the world. As a result, shipping companies send smaller, older, and cheaper ships to Africa in an effort to reduce their losses.

A number of factors are to blame: poor harbor maintenance, bureaucratic red tape, inadequate maritime law enforcement, and lax security....

Baker points out that poor infrastructure in ports means that shipping companies reallocate their fleet in such a way that further slows trade:

Moreover, many African ports cannot handle ships of median size due to infrastructure limitations. Meanwhile, the global shipping industry has been modernizing its fleets, scrapping obsolete vessels for newer mega-carriers. This means that shipping companies will continue deploying their remaining smaller and slower ships for transport to and from Africa.... In this environment, companies producing goods in Africa cannot reliably or efficiently get their wares to market. This plays a large role in explaining why Africa garners only 2.7 percent of global trade despite its cheap labor force, cheap commodities, and proximity to major markets.

However, if governments really want to get shipping companies to use their bigger and better ships in Africa, it is probably the case that infrastructure and market opportunities would need to not only improve, but surpass opportunities elsewhere (such as China).  This may not be realistic.

Probably too much of the article focuses on piracy, but his mention of labor market opportunities seems to agree with many of the things we know about predation:

A crucial means of countering piracy, oil theft, narcotics trafficking, and terrorism along the African coast is creating better job opportunities. Development approaches usually focus on land-based projects such as agriculture. To be sure, African states need to attract investors for manufacturing companies, but they must also entice shipping companies to get those goods to market. The poor state of Africa’s maritime sector is the most important factor stifling the continent’s growth.

I think it is extremely interesting to think about African maritime infrastructure, because Baker is right that most of the policy focus in Africa is land-based. Although I'm not sure we have enough evidence that dilapidated ports are the "most important factor" for African development. It is notoriously difficult to pin down the effects of infrastructure on economic development since infrastructure is frequently developed in response to increasing demands, itself a result of development.

Two more data websites

Permanently added to Meta-Resources:

Michael Kremer's data set website (economics)

The EPA Data Finder for US environmental and pollution data.

Misuse of "control variables" in multi-variable regression

Yesterday I was talking to my friend Anna who had a great question about multi-variate regression:

"When you have a regression of Y on X and Z, what happens to the variations in X and Z that influence Y but are perfectly correlated?"

This is a serious question that doesn't seem to be taken seriously enough by many researchers.  I think in econometrics 101, they teach us the Frisch-Waugh Theorem because they want us to think about this, but few of us do.

Anna was concerned about this because many people run a regression of Y on X, observe a correlation, then include a "control variable" Z and observe that the correlation between Y and X vanishes.  They then conclude that "X does not affect Y."  This conclusion need not be true, Anna was right.

If you use multi-variable regression in research, I would be sure you understand why Anna was right. If you don't, I would sit and think about Frisch-Waugh until you do.

If you don't believe me and you know how to use Stata, run this code. It might help.



/*WHY YOU SHOULD THINK VERY HARD ABOUT THE FRISCH-WAUGH THEORM IF YOU USE MULTIPLE REGRESSION FOR CAUSAL INFERENCE*/
/*SOLOMON HSIANG, 2010*/


clear
set obs 1000


/*Here, X is the true exogenous variable*/
gen X = runiform()


/*Let Z and Y be influenced by X*/
gen Z = X
gen Y = 2*X


/*There are three independant sources of observational error*/
gen e_x = 0.1*runiform()
gen e_z = 0.1*runiform()
gen e_y = 0.1*runiform()


/*Then the following observations are observed*/
gen x = X + e_x
gen z = Z + e_z
gen y = Y + e_y


/*To be completely clear about what is observable, let's throw
away the fundamental variables and only keep the observed variables*/
drop X Y Z e_x e_y e_z


/*Suppose you thought that a unit change in x would increase y, so you estimate:*/
reg y x


/*This would give you a fairly good estimate of the coefficient 2, which is correct.*/
/**/
/*Now suppose you are anxious because someone tells you that you haven't controlled 
for every variable in the world. In your panic, you concede and include the variable 
z in your regression:*/
reg y x z


/*This is bizzare.  We know that Z was not involved at all in the creation of Y. But
including z in our regression suggests it is not only highly significantly correlated,
but it also dramatically changes the coefficient on x to half of its true value.*/

Madagascar

Jesse and I have been working hard and haven't posted anything in a while. I don't have much time to offer comments, but this is a really interesting piece in last month's NGM.  It follows groups of loggers and others in Madagascar and I think it does a good job capturing some of the challenges of development and the political economy of resource extraction.

For anyone who's interested in commitment problems of political economy, this is a good quote:

In September 2009, after months during which up to 460,000 dollars' worth of rosewood was being illegally harvested every day, the cash-strapped new government reversed a 2000 ban on the export of rosewood and released a decree legalizing the sale of stockpiled logs. Pressured by an alarmed international community, the government reinstated the ban in April. 

And I also liked this breakdown of who gets to keep what from the forest:

For weeks they camp out in small groups beside the trees they've singled out for cutting, subsisting on rice and coffee, until the boss shows up. He inspects the rosewood, gives the order. They chop away with axes. Within hours a tree that first took root perhaps 500 years ago has fallen to the ground. The cutters hack away at its white exterior until all that remains is its telltale violet heart. The rosewood is cut into logs about seven feet long. Another team of two men tie ropes around each log and proceed to drag it out of the forest to the river's edge, a feat that will take them two days and earn them $10 to $20 a log, depending on the distance. While staggering through the forest myself, from time to time I come upon the jarring apparition of two stoic figures tugging a 400-pound log up some impossible gradient or down a waterfall or across quicksand-like bogs—a hard labor of biblical scale, except that these men are doing this for money. As is the man the pair would meet up with at the river, waiting to tie the log to a handcrafted radeau, or raft, to help it float down the rapids ($25 a log). As is the pirogueman awaiting the radeau where the rapids subside ($12 a log). As is the park ranger whom the timber bosses have bribed to stay away ($200 for two weeks). As are police at checkpoints along the road to Antalaha ($20 an officer). The damage to the forest is far more than the loss of the precious hardwoods: For each dense rosewood log, four or five lighter trees are cut down to create the raft that will transport it down the river.

At a bend in the river, the pirogues pull up to shore. A man with a mustache squats in a tent, smoking a hand-rolled cigarette. His name is Dieudonne. He works with the middleman, the boss on the ground, entrusted by the timber baron to select the trees for cutting and oversee the logs from the riverbank to the transport trucks. There have been 18 trucks this morning. Thirty or so rosewood logs lie scattered around Dieudonne's tent. His cut is $12 a log.

A new mechanism to consider when measuring climate impacts on economies

[A shorter (and more heavily copy-edited) version of this post was published in EARTH Magazine, read it here.]

My paper Temperatures and tropical cyclones strongly associated with economic production in the Caribbean and Central America was recently published in the Proceedings of the National Academy of Sciences. Because the paper is a little technical, here is a presentation of the results that everyone should be able to understand.

Following countries over time, years with higher than

normal temperatures during the hottest season 

(Sep-Oct-Nov) exhibit large reductions in output across  

several non-agricultural industries.

Central finding:
Economic output across a range of industries previous thought of as "not vulnerable to climate change" respond strongly to changes in temperature.  The data suggest that the response is driven by the direct human response to high temperatures: people generally are less productive and tire faster when it's hot.  This impact, which appears to be quite large, has not been factored into any previous estimates for the global cost of climate change.

Background
Governments and organizations around the world are trying to figure out how much money we should spend to avoid climate change.  The answer isn't obvious.  On the one hand, climate change seems ominous and we'd like to spend lots of money to avoid it. But on the other hand, if we spend money on avoiding climate change, we can't spend it on other important things. For example, imagine that the United Nations has a million dollars it can spend. Should it spend it on building solar panels or building schools?  Both are clearly important. But if we want to get the most "bang for our buck," we need to figure out what the benefits of these two types of investments are.

A whole research industry has sprung up around the cost-benefit analysis of preventing climate change.  How much money should be spent to prevent climate change by investing in more expensive low-carbon technologies? Who should pay for it and when should they pay for it?  A tremendous amount of intellectual machinery has been applied to this problem by many extremely smart people.  The basic approach is to build models of the world economy-climate system and try to see what happens to the climate and the economy under different global policies.  These models are used by governments around the world to determine what they think the best climate policies are and how much they should spend on the problem.

However, there is something of a dark secret to this approach: we don't really know what will happen to us if the climate changes.  We have a fairly good grasp of how much it might cost to implement different energy policies. And we've learned a lot about how different energy policies will translate into global climate changes.  But when it comes to figuring out how those climate changes translate into costs to society (both financial and non-monetary), we end up having to do a lot of guesswork.

It's unfair to say we know nothing about the costs of climate change, but what we understand well is limited to certain types of impacts.  For example, we have been doing extensive research on the possible agricultural impacts for years. We've also done studies for a lot of the health impacts.  But most research stops there.  For example, we only are beginning to learn about the effect of climate on people's recreation and perceived happiness.  We're also only beginning to learn about the effect of climate on violence and crime.  We know a lot (but not nearly everything) about the effect of climate on ecosystems, but we don't really understand how ecosystems affect us, so we still can't estimate this impact on society. The list goes on.

Now we know a lot about climate impacts on health and agriculture because people have studied those impacts a lot.  Why did we study those kinds of impacts so much? I'm not sure. Maybe because the importance of climate on health and agriculture is obvious (eg. my plants on the windowsill died after just two days of this summer's heat wave).

The fact that we only really understand agricultural and health impacts of climate change is very important in the cost benefit analyses I mentioned earlier.  When governments are trying to figure out the best policies, they add up the known costs of preventing climate change and they add up the known benefits of preventing climate change.   If the costs outweigh the benefits, then that suggests we shouldn't spend much money to stop climate change.  But there is a natural asymmetry in this comparison between costs and benefits: we know all (or most of) the costs but only know the health and agricultural benefits.  So when we add up the costs of energy policies, the numbers tend to look very big.  But when we add up the known benefits of those policies, we add up the health benefit and the agricultural benefits, but we have to stop there because we don't know what else will be affected by climate change.  Maybe it shouldn't be surprising that many cost-benefit analyses find that climate change is not worth spending a lot of money on.

But what we know about climate impacts in non-health and non-agricultural sectors is slowly improving.  In a 2009 working paper, Dell, Jones and Olken did something very simple and got very surprising results.  They compared the economic output of countries over time with year-to-year changes in the weather of those countries.  They found that in poor countries, small increases in the annual average temperature of a country lead to large drops in economic output of that country.  The approach sounds simple, right? It is.  But the results are startling because they found such a large effect of temperature. They estimate that a 1C increase in average temperatures decreases a poor country's gross domestic product (GDP) by 1.1% in the same year. To get a sense of how big this effect is, recall that the economy of the Unites States shrank by 2.4% in 2009 and people are upset about the state of the economy.

Because the effect found by Dell et al. is so large, many people have been skeptical that it represents something real (note from my own unpublished work: I can corroborate their results using different data sets from the ones they used).  To check these results further, in 2010, Jones and Olken tried to looking for a similar effect in the exports of these countries and found that they also responded strongly to temperature changes.  Do people believe the general result yet? I'm not sure.  But part the skepticism seems to persist because its hard to know why poor countries should be so strongly affected by temperature.  One reason for this is that it's very hard to know what mechanisms are at work when one is only looking at macro-economic data.  Further, thinking of ways in which temperature affects economics this strongly and systematically across countries seems to be hitting the limits of many peoples' imaginations. This is where my study comes in.

Quick Hit: Naxalites and India's Regional Resource Curse

Foreign Policy has a great article on the effect of mining (principally coal, but also iron and other metals) on two of India's poorest states, especially in fueling the ongoing Maoist / "Naxalite" rebellion. There's a couple of things that are pretty interesting in the article

1. This is a pretty strictly regional "resource curse" dynamic. Normally one is taught to think of the resource curse operating on national levels (Dutch Disease, kleptocrat dictators, etc.) but it's interesting to see that that need not be the case. That made me appreciate how much the political economy and power dynamics in Appalachia are really just a developed-country, regional resource curse.
2. Who knew India had that much coal? A classic "folk empirical result," if you will, of geographically-inclined development economists is that the countries which prospered during the industrial revolution had huge coal reserves. Not that China doesn't, too, but still: India? I clearly need to read more
3. Calling the Maoist rebels "Naxalites" is apparently not accurate, which is interesting to hear since I've heard that term used by multiple people in the past few years.
4. "If you want to be somebody in Jharkand, just kill an aid worker" manages to pack so much of the troubles of development work into so few words that it should count as haiku.

The article also has some great photography. Worth clicking through.

Scalability and path dependency in urban planning

Providing a cogent reminder of the fact that scale matters in things like economics and urban planning just like it does in the natural world, a traffic jam between Jining and Beijing is now entering its 9th straight day of existence. Hat tip on that goes to Tyler Cowen at Marginal Revolution.

I lived in Beijing for the better part of a year after college and the traffic was awful even then. I remember one of the first things I heard about transport in the city was that the percent of urban area devoted to roads (or "road area ratio" in planning parlance) was vastly less than in other comparably-sized cities. Poking around a bit on Google Scholar yields a figure around 12% vs. cities like London being in the range of 35% and American commuter cities being in the range of 45% (Ge and Ping, 2008).

Abstracting a bit more, news like this makes me more sanguine about predictions that India and China are seeking to completely emulate American lifestyles, thereby placing an unbearable load on the Earth's resource base. If very deeply-rooted structural differences make owning personal vehicles this much of a pain this early in a country's development trajectory then I think the long-term prospects have to be fundamentally different. Recall that the Americas (ok, and Australia) were built more or less from scratch over the past two centuries while a lot of new transport and housing technologies were being developed and demographic changes were being realized. Expecting countries with longer histories and different cultural biases to have their pre-existing large cities turn into L.A. just because per capita incomes have gone up seems a bit simplistic.

Though I'm sure traffic in L.A. sometimes seems to last for days on end.

*Image copyright AP.

timetric.com

A new website launched by a group in the UK makes accessing, analyzing and sharing time series data simple:

http://timetric.com/

Similar to gapminder.org, the target audience is the public.  The site is well put together, letting me make this graph and embed it in about a minute.





According to their About page:

timetric.com's full of indexes and indicators about nearly every part of your life and your business, from finance to weather to politics to sport. We're here to help you find the data you need, to use it and make sense of it, and to compare it with your own data. (The fancy name for this sort of thing is time series analysis).

We want to make your insights easy for you to share, so you can embed the indexes on timetric.com into your blogs and websites, or share them on Twitter, with just a couple of clicks.

And so you can compare your own data with the indexes we've collected, it's really easy to import your own data; you can upload Excel (or CSV) spreadsheets or enter it in by hand.

For the programmers out there, we've got an API, and we support all the good stuff like OAuth, OpenID and OpenSearch, so it's easy to integrate Timetric with other services.

We're here to help you understand the world better, and to help you make better decisions through statistics, and we're just getting started.

I'm a big supporter of all these efforts to get data out into the public domain and accessible to everyone.  Its extremely difficult to get a sense of "the world" just looking at newspapers and other media, so making this kind of data accessible is another step in the right direction.

I've added the site to our Resources page.

Geoengineering?

I was just on the website of EARTH Magazine (a magazine published by the American Geological Institute and whose target audience is the scientifically-inclined) and in the upper right corner was a simple online poll for visitors to click on.  The question and results are below.  I'm not even sure how I feel about geoengineering, but I find this simple graph fascinating.  

Two and three-dimensional non-parametric regressions

A pet peeve of mine are people who run dozens of linear regressions but never check whether linearity  is a good assumption.  Often, people will run linear models and then look for non-linearities later.  But there's no point in estimating lots of linear models first if they might be meaningless.  For example, imagine a U-shaped curve of Y as a function of X.  In this case, estimating and declaring that a linear model estimates no relation between Y and X is not meaningful.  As a general rule, it's probably a better strategy to look for non-linearities early and often.

With the goal of searching-for-specifications in mind, its useful to have a method of non-parametric regression that's fast since the researcher will have to try many things. Locally weighted polynomial regressions are fairly slow, so the faster and simpler Nadaraya-Watson estimator (Nadaraya, 1964 and Watson, 1964) comes in handy.

My code for the NW estimator in Matlab is here.
(If your data is in Stata and you want to move it to Matlab, see my code for that here).

It contains 4 .m files. The file test.m will generate some random data and estimate the conditional mean with the NW estimator.  The code uses bootstrap-resampling to estimate confidence intervals. All models have flexible but fixed bandwidth.

With one dependent variable, NWbootstrap.m (with a normal kernel) and NWbootstrap_epkv.m (with an Epinechnikov kernel) are appropriate.

With two dependent variables,  NWbootstrap_epkv_3D.m is useful.  This should be used, for example, if you are looking for "interaction terms" in a multiple regression model (the slope of a function changes as a function of another variable).

UPDATE: A problem with dropping missing variables has been fixed.  The code drop_missing_Y is now in the NW toolbox to solve the problem.

All files have helpfiles with syntax. Below is a demonstration of the code.