FAQs for "Reconciling disagreement over climate–conflict results in Africa"

[This is a gues blog post by my coauthor Kyle Meng.]

Sol and I just published an article in PNAS in which we reexamine a controversy in the climate-conflict literature. The debate is centered over two previous PNAS articles: the first by Burke et al. (PNAS, 2009) which claims that higher temperature increases conflict risks in sub-Saharan Africa and a second PNAS article by Buhaug (PNAS, 2010) refuting the earlier study.

How did we get here?

First, a bit of background. Whether climate change causes societies to be more violent is a critical question for our understanding of climate impacts. If climate change indeed increases violence, the economic and social costs of climate change may be far greater than what was previously considered, and thus further prompt the need to reduce greenhouse gas emissions. To answer this question, researchers in recent years have turned to data from the past asking whether violence has responded historically to changes in the local climate. Despite the increasing volume of research (summarized by Sol, Marshall Burke, and Ted Miguel in their meta-analysis published in Science and the accompanying review article in Climatic Change) this question remained somewhat controversial in the public eye. Much of this controversy was generated by this pair of PNAS papers.

What did we do?

Our new paper takes a fresh look at these two prior studies by statistically examining whether the evidence provided by Buhaug (2010) overturns the results in Burke et al. (2009). Throughout, we examine the two central claims made by Buhaug:
1) that Burke et al.'s results "do not hold up to closer inspection" and
2) climate change does not cause conflict in sub-Saharan Africa.  
Because these are quantitative papers, Buhaug’s two claims can be answered using statistical methods. What we found was that Buhaug did not run the appropriate statistical procedures needed for the claims made. When we applied the correct statistical tests, we find that:
a) the evidence in Buhaug is not statistically different from that of Burke et al. and
b) Buhaug’s results cannot support the claim that climate does not cause conflict. 
A useful analogy

The statistical reasoning in our paper is a bit technical so an analogy may be helpful here. Burke et al's main result is equivalent to saying "smoking increases lung cancer risks roughly 10%". Buhaug claims above are equivalent to stating that his analysis demonstrates that “smoking does not increase lung cancer risks” and furthermore that “smoking does not affect lung cancer risks at all”.

What we find, after applying the appropriate statistical method, is that the only equivalent claim that can be supported by Buhaug’s analysis is "smoking may increase lung cancer risks by roughly 100% or may decrease them by roughly 100% or may have no effect whatsoever". Notice this is a far different statement than what Buhaug claims he has demonstrated in 1) and 2) above. Basically, the results presented in Buhaug are so uncertain that they do not reject zero effect, but they also do not reject the original work by Burke et al.

Isn’t Buhaug just showing Burke et al.’s result is “not robust”?

In statistical analyses, we often seek to understand if a result is “robust” by demonstrating that reasonable alterations to the model do not produce dramatically different results. If successful, this type of analysis sometimes convinces us that we have not failed to account for important omitted variables (or other factors) that would alter our estimates substantively.

Importantly, however, the reverse logic is not true and “non-robustness” is not a conclusive (or logical) result. Obtaining different estimates from the application of model alterations alone does not necessarily imply that the original result is wrong since it might be the new estimate that is biased.   Observing unstable results suggests that there are errors in the specification of some (or all) of the models.  It merely means the analyst isn’t working with the right statistical model.

There must exist only one  “true” relationship between climate and conflict, it may be a coefficient of zero or a larger coefficient consistent with Burke et al., but it cannot be all these coefficients at the same time. If models with very different underlying assumptions provide dramatically different estimates, this suggests that all of the models (except perhaps one) is misspecified and should be thrown out.

A central error in Buhaug is his interpretation of his findings.  He removes critical parts of Burke et al.’s model (e.g. those that account for important differences in geography, history and culture) or re-specifies them in other ways and then advocates that the various inconsistent coefficients produced should all be taken seriously. In reality, the varying estimates produced by Buhaug are either due to added model biases or to sampling uncertainty caused by the techniques that he is using. It is incorrect to interpret this variation as evidence that Burke et al.’s estimate is “non-robust”.

So are you saying Burke et al. was right?

No. And this is a very important point. In our article, we carefully state:
“It is important to note that our findings neither confirm nor reject the results of Burke et al.. Our results simply reconcile the apparent contradiction between Burke et al. and Buhaug by demonstrating that Buhaug does not provide evidence that contradicts the results reported in Burke et al. Notably, however, other recent analyses obtain results that largely agree with Burke et al., so we think it is likely that analyses following our approach will reconcile any apparent disagreement between these other studies and Buhaug.”
That is, taking Burke et al’s result as given, we find that the evidence provided in Buhaug does not refute Burke et al. (the central claim of Buhaug). Whether Burke et al. was right about climate causing conflict in sub-Saharan Africa is a different question. We’ve tried to answer that question in other settings (e.g. our joint work published in Nature), but that’s not the contribution of this analysis.

Parting note

Lastly, we urge those interested to read our article carefully. Simply skimming the paper by hunting for statistically significant results would be missing the paper’s point. Our broader hope besides helping to reconcile this prior controversy is that the statistical reasoning underlying our work becomes more common in data-driven analyses.


Fourth Interdisciplinary Ph.D. Workshop in Sustainable Development

The students of the Columbia Sustainable Development Ph.D. program have put out the call for papers for the Fourth Interdisciplinary Ph.D. Workshop in Sustainable Development. It's a great opportunity for Ph.D. students to meet colleagues from a broad array of disciplines, and a bunch of our younger colleagues will be there. Please pass it along.

Fourth Interdisciplinary Ph.D. Workshop in Sustainable Development
April 25th-26th, 2014: Columbia University in the City of New York, USA

The graduate students in the Sustainable Development PhD program at Columbia University are convening the Fourth Interdisciplinary Ph.D. Workshop in Sustainable Development (IPWSD); scheduled for April 25th-26th, 2014, at Columbia University in New York City.

The IPWSD is a conference open to graduate students working on or interested in issues related to sustainable development.  It is intended to provide a forum to present and discuss research in an informal setting, as well as to meet and interact with similar graduate student researchers from other institutions.  In particular, we hope to facilitate a network among students pursuing in-depth research across a range of disciplines in the social and natural sciences, to generate a larger interdisciplinary discussion concerning sustainable development.  If your research pertains to the field of sustainable development and the linkages between natural and social systems, we encourage you to apply regardless of disciplinary background.

For details, please see the call for papers, or visit our conference website where a detailed list of topics, conference themes and other information is available.

Please share this information widely with graduate students and other interested parties. We look forward to seeing you in New York City in April!

With kind regards,

The Fourth IPWSD Planning Committee
Sustainable Development Doctoral Society, 
Columbia University
Contact: cu.sdds.ipwsd@gmail.com


Reconciling disagreement over climate–conflict results in Africa

Kyle and I have a paper out in the Early Edition of PNAS this week:

Reconciling disagreement over climate–conflict results in Africa
Solomon M. Hsiang and Kyle C. Meng
Abstract: A recent study by Burke et al. [Burke M, Miguel E, Satyanath S, Dykema J, Lobell D (2009) Proc Natl Acad Sci USA 106(49):20670– 20674] reports statistical evidence that the likelihood of civil wars in African countries was elevated in hotter years. A following study by Buhaug [Buhaug H (2010) Proc Natl Acad Sci USA 107 (38):16477–16482] reports that a reexamination of the evidence overturns Burke et al.’s findings when alternative statistical models and alternative measures of conflict are used. We show that the conclusion by Buhaug is based on absent or incorrect statistical tests, both in model selection and in the comparison of results with Burke et al. When we implement the correct tests, we find there is no evidence presented in Buhaug that rejects the original results of Burke et al. 
Related reconciliation of different results in Kenya.

A brief refresher and discussion of the controversy that we are examining is here.


Climate-conflict research, before the IRB...

Finding old papers on temperature manipulation is turning into a hobby of mine. I actually had to go to the library to dig up this gem. From Rohles, Frederick H. "Environmental psychology: A bucket of worms." Psychology Today 1.2 (1967): 55-63.


Reconciling temperature-conflict results in Kenya

Marshall, Ted and I have a new short working paper out. When we correct the coding of a single variable in a previous study (that uses a new data set), we obtain highly localized temperature-conflict associations in Kenya that are largely in line with the rest of the literature. I think this is a useful example for why we should be careful with how we specify interaction terms.

Reconciling temperature-conflict results in Kenya
Solomon M. Hsiang, Marshall Burke, and Edward Miguel
Abstract: Theisen (JPR, 2012) recently constructed a novel high-resolution data set of intergroup and political conflict in Kenya (1989-2004) and examined whether the risk of conflict onset and incidence responds to annual pixel-level variations in temperature and precipitation.  Thiesen concluded that only extreme precipitation is associated with conflict incidence and that temperature is unrelated to conflict, seemingly at odds with recent studies that found a positive association at the pixel scale (O'laughlin et al., PNAS 2012), at the country scale (Burke et al., PNAS 2009), and at the continental scale (Hsiang et al., Nature 2011) in Africa.  Here we show these findings can be reconciled when we correct the erroneous coding of temperature-squared in Thiesen. In contrast to the original conclusions presented in Theisen, both conflict onset and conflict incidence are significantly and positively associated with local temperature in this new and independently assembled data set.