What is the debate over climate and conflict about?

Last week, Andrew Solow published a Nature comment titled "A call for peace on climate and conflict." In the article, Solow raises many important points that I whole-heartedly agree with, such as trying to avoid data-mining, looking deep into statistical models when they disagree, engaging with qualitative researchers, and presenting and publishing across research communities. My coauthors and I agree so strongly with these latter points that we regularly present and engage with researchers outside of our field -- e.g. Marshall Burke recently presented at the International Studies Association (a political science meeting) and I recently presented at the Association of American Geographers, at an interdisciplinary water resources conference at UCSD, and I will be presenting to a community of medical doctors at Harvard today.

However, I worry that Solow's comment may confuse readers as to why there is controversy in the field. Solow begins his comment:

Among the most worrying of the mooted impacts of climate change is an increase in civil conflict as people compete for diminishing resources, such as arable land and water [1]. Recent statistical studies [2–4] reporting a connection between climate and civil violence have attracted attention from the press and policy-makers, including US President Barack Obama. Doubts about such a connection have not been as widely aired [5–7], but a fierce battle has broken out within the research community.

The battle lines are not always clear, but on one side are the ‘quants’, who use quantitative methods to identify correlations between conflict and climate in global or regional data sets. On the other side are the ‘quals’, who study individual conflicts in depth. They argue that the factors that underlie civil conflict are more complex than the quants allow and that the reported correlations are statistical artefacts. 

Where the papers he is referencing to are

1. Homer-Dixon (Princeton Press, 1999).
2. Miguel, Satyanath, Sergenti, J.Polit. Econ. (2004).
3. Burke, Miguel, Satyanath, Dykema,  Lobell, D. B. Proc. Natl Acad. Sci. USA (2009).
4. Hsiang, Meng, Cane, Nature  (2011).
5. Buhaug, Proc. Natl Acad. Sci. USA (2010).
6. Theisen, Holtermann, Buhaug, Internatl Secur. (2011).
7. Buhaug, Hegre, Strand, (Peace Research Institute of Oslo, 2010). 

Thus, the dispute that motivates the comment (referenced in the first paragraph) is the disagreement between Miguel-Burke-Hsiang et al vs Buhaug-Theisen-Buhaug et al while the transition in the second paragraph then shifts the discussion to a dispute between ‘quants’ and ‘quals’ (which is the topic of most of the text).  Because these two discussions are so intermingled, a careless reader might incorrectly conclude that the Miguel-Burke-Hsiang vs. Buhaug-Theisen debate is the qual vs. quant debate. This is not the case. Miguel-Burke-Hsiang et al and Buhaug-Theisen et al are all quantitative research groups. The debate between the two groups is about how quantitative research should be executed and interpreted. It is not a debate over whether quantitative or qualitative methods are better.

Because the Miguel-Burke-Hsiang vs. Buhaug-Theisen debate is raised in the comment, but not outlined, I summarize the papers that Solow cites here:

2004: Miguel et al. demonstrate that annual fluctuations in rainfall are negatively correlated with annual fluctuations in GDP growth and positively correlated with civil conflict in African countries. Miguel et al argue that rainfall changes influence conflict through this economic channel.

2009: Burke et al. (which includes Miguel and Satyanath, both authors on the 2004 paper) revisit this problem but include growing season temperature in their statistical model, motivated in part by other findings that temperature is a strong predictor of agricultural performance (even once rainfall is controlled for). They find that temperature appears to have an even stronger effect on conflict than rainfall. They conduct a number of robustness checks and project how conflict might change under global warming.

2010: Buhaug (PNAS) argues that Burke et al. arrive at incorrect conclusions because they should not include country fixed effects or country-specific trends in their statistical model. Buhaug instead advocates for a model that assumes all countries are identical (with respect to conflict) except for GDP and an index of political exclusion. Using this model, Buhaug argues that temperature has zero effect on conflict. Buhaug concludes his article with the statement:

"The challenges imposed by future global warming are too daunting to let the debate on social effects and required countermeasures be sidetracked by atypical, nonrobust scientific findings and actors with vested interests."

This is when the debate begins to get attention (eg. here)

2010: Buhaug et al. (PRIO) examine several additional dimensions of the result in Burke et al., such as its out of sample prediction and how results look when other measures of civil conflict are used. The authors conclude:

"In conclusion, the sensitivity assessments documented here reveal little support for the alleged positive association between warming and higher frequency of major civil wars in Africa… More research is needed to get a better understanding of the full range of possible social dimensions of climate change."

2011: Thiesen et al. revisit civil conflict in Africa by trying to pinpoint the locations where the first battle deaths in major wars occurred. Theisen et al examine whether the 0.5 degree pixels where these first deaths occurred were experiencing drought at the time of these deaths.  The authors follow Buhaug and do not use fixed effects, instead they use a model that assumes all pixels are identical except for six control variables (e.g. democracy, infant mortality). The authors do not find a statistically significant association between drought and the location of first battle death, so they conclude that climate does not affect civil conflict in Africa.

2011: Hsiang et al. examine whether the global climate (not local temperature) has any effect on global rates of civil conflict. Hsiang et al. identify the tropical and sub-tropical regions of the world that are most strongly affected by the El Nino-Southern Oscillation (ENSO) and then examine the likelihood that countries in this region start new civil conflicts, conditional on the state of ENSO. They find that in cooler/wetter La Nino years the rate of conflicts is half of what it is in hotter/drier El Nino years -- but only in the tropical and sub-tropical regions that are affected by this global cimate oscillation. The authors show that the additional conflicts observed in El Nino years only occur after El Nino begins and are focused in the poorest countries.

Some of my thoughts on the above debate (in no particular order):

1. Clearly, this discussion is all based on statistical evidence -- it is not a debate as to whether quals or quants are better suited to answer this question.
2. No statistical evidence undermining the findings of Hsiang et al has been released or published in the last two years (to my knowledge). Many authors have casually stated in reviews that "there are issues with the paper" or that Buhaug (2010) or Theisen et al (2011) disprove our findings (eg. here). But valid "issues" have not been pointed out to me, publicly or privately, and I do not see how these other papers can possibly be interpreted as disproving our results. Since I'm fairly certain that these authors have been trying to find problems with our paper, but have not released them anytime in the last two years, I am gaining confidence that our findings are extremely robust. Furthermore, one of Chris Blattman's graduate students recently replicated our paper successfully for an econometrics assignment.
3. Buhaug and Theisen et al. generally overstate their findings. The estimates they obtain are extremely noisy, so they have very large confidence intervals, preventing them from rejecting a "zero effect"or very large effects. This is far from proving there is zero effect. For example, saying that X is somewhere between -100 and 100 is not evidence that X is exactly equal to 0. 
4. Buhaug and Theisen et al.'s approach of dropping fixed effects, and assuming Africa is homogenous except for a handful of controls, is easily rejected by the data. A simple F-test for the joint significance of the fixed effects in Burke's model easily rejects their hypothesis that these effects are the same throughout Africa. 
5. I think the paper by Thiesen et al is very difficult to interpret, since they are assigning all the potential causes of a conflict to conditions within the 50 x 50 km pixel where the first battle death occurred. Regardless of what results they report or whether the statistical techniques are sound, I'm not sure how I would interpret any of their results since I tend to think that many factors located beyond that pixel would affect the likelihood of civil war in a country.
6. There is a general argument underlying all the Buhaug-Theisen articles that "because regression coefficients change a lot across our models, the result of Burke must be non-robust." But this is faulty statistical logic. If the regression coefficients are changing between models, this means that all the models (or all but one) are mis-specified because they have different omitted variables, which is causing a different amount of bias in each model (and thus the different regression coeffs). This does not imply that the "true effect"of climate is equal to zero. There can only be one true effect. A good model might identify this effect and be robust to small variations in the model, but the true relationship between any X and Y cannot be generally "non-robust" and presenting non-robust estimates certainly does not prove that the true effect is zero.
7. Plotting the results in Burke et al. is pretty compelling evidence. There is some noise (which is what drives the Buhaug claims) but just plotting the data early on might have prevented all this controversy (perhaps I am dreaming). 
8. I think Miguel and Satyanath should be praised for revisiting their 2004 findings, including an additional and important control variable and then altering their conclusions based on their new findings.

Note: I am not opposed to qualitative research. However, I do think that qualitative researchers must carefully consider the limited extent of their observations when drawing inferences.  Large scale political conflict is a rare event, so it is unlikely that a randomly sampled case study will observe conflict in association with climatic events, even if there is a strong relationship. More discussion of this point is here.

My coauthor Marshall Burke has some additional thoughts on Solow's Comment and the general debate on G-FEED.