I was a little surprised by this, since publishing in Nature and Science is sometimes viewed as a "second tier" prospect in the social sciences:
An average social science article published in Nature (2000-2010) is more heavily cited than an average article in any social science journal, with a "citation impact" of 51 (see the Reuter's article here).
The second best place to get your social science article? Science (citation impact = 35).
Nature and Science don't publish many social science articles, 65 and 80 respectively over the last decade, but the articles they publish seem to do well. For reference, note the citation impact for top economics-only journals (Quarterly Journal of Economics is top in 2008 with citation impact = 5) and the top political science-only journals (Political Analysis is top in 2007 with citation impact = 2.5).
A standard critique of Reuter's citation impact measure is that it counts an article's citations over a fairly short window of time just following that article's publication (2 years). In the social sciences, articles may remain as unpublished working papers for several years, preventing many of their citations from being counted in Reuter's analysis. Is this long lag in publication timing driving Reuters' finding? Probably not. Reuter's also publishes a measure of impact that spans a longer time window following an article's publication: the 28 years from 1981-2008. Giving an article in the Quarterly Journal of Economics almost thirty years to accumulate citations still leaves its average citation count (49) just behind a the two-year citation count for an article in Nature (51). Similarly, the long-run citation count for The American Political Science Review is 31, just behind the two-year citation count for Science (35).
[If you're interested in citations and the structure of human knowledge more generally, see this earlier post.]
In some sense, this is a problem. Some social scientists publish in natural science journals because they have a specific message appropriate for those outlets, but sometimes that is not the case. Sometimes social science articles are published in natural science journals that could not be published in economics journals (data limitations, analysis not sufficiently rigorous...) Whatever the reason, the results may be far from conclusive but in a natural science journal be more likely to be treated as a scientific discovery and the broader coverage could be harmful.
ReplyDeleteJarrod,
ReplyDeleteI'm not certain that I agree this is a problem, per se. I'm familiar with the concerns you mention, they are points that many people bring up. But I've never actually seen clear evidence that social science papers published in these interdisciplinary outlets are of a systematically lower quality. Of course, there is a distribution of article quality that is published in Science and Nature, as there would be with any strictly social science journal, so it's easy to find particular instances of a Nature paper that is lower quality than a QJE paper. But that alone does not characterize a systematic difference in the two distributions.
I also think there is some misconception about how many social science articles get through the review process in these journals. The stats above suggest that Nature only published 6.5 social science articles a year (avg. over the last decade). Given that Nature is a weekly publication, this suggests that Nature only publishes a social science article once every eight weeks. To me, this doesn't suggest they are simply letting all sorts of low quality papers through the review process.
Finally, I think many social science researchers who do not read these journal regularly have an inaccurate sense of which social scientists are publishing in these journals. Just to provide a handful of examples from economics (since I know it), recent highly-cited articles in these journals were published by established giants in the field, such as Alan Kreuger (also here), Steven Levitt and John List, Jeffrey Sachs, Larry Goulder and Rob Stavins and Geoffrey Heal. Particularly in my world of environmental economics, younger researchers like Chris Costello and Wolfram Schlenker have had publications in these journals with enormous impact. Also, the Proceedings of the National Academy of Sciences (PNAS), which is very similar to Nature and Science, has had recent publications by famous economists such as William Nordhaus (also here) and Roy Radner. An additional fun fact along these lines: one of the most famous economics publications ever written, the paper in which John Nash described the solution concept now known as a "Nash Equilibrium" which is now used everywhere in economics, was published in PNAS (see it here).