Incentives work for online labor, too

A nice experimental paper on online labour market incentives just got presented at the ACM Conference on Computer Supported Cooperative Work (CSCW). It's by John Horton, Daniel Chen and Aaron Shaw and covers different incentives' impact on success rates for a set of simple Mechanical Turk questions. Shaw summarizes it nicely at his blog :

The results surprised us. They suggest that workers perform most accurately when the task design credibly links payoffs to a worker’s ability to think about the answers that their peers are likely to provide.

That priming technique is referred to catchily as "Bayesian Truth Serum" and its setup (from the paper online) behooves a little extra attention:

Bayesian Truth Serum or BTS (financial) “For the following five questions, we will also ask you to predict the responses of other workers who complete this task. There is no incentive to misreport what you truly believe to be your answers as well as others’ answers. You will have a higher probability of winning a lottery (bonus payment) if you submit answers that are more surprisingly common than collectively predicted.”

BTS was designed by Drazen Prelec at MIT, and you can get the original paper (I believe) here.

In short, the setup exemplifies what I think is particularly great about data analysis these days: Shaw, Horton, and Chen manage to put together a very tight, deeply informative and even slightly controversial (psych and sociological priming are not so effective) paper that really required nothing more than an internet connection, some computational power for the stats, and a clever eye. Go check it out.

via boingboing.

Google Correlate

Google Correlate goes live at Google Labs today. It provides correlative search by time or US state and returns the top most-correlated search terms and / or data series. For particularly Fight Entropy-ish content, check out terms that correlate with:

• state level annual rainfall
• Kyoto Protocol
• hurricane
• record high temperature (note the top hit)
• El Nino Southern Oscillation (Sol will like this one)

Also: their explanatory webcomic manages the rare feat of being simultaneously cute and informative.

TRMM Satellite picks up Joplin tornado

Our colleague and occasional guest poster Kyle Meng points out that NASA's TRMM satellite (previously here) passed over the supercell that generated this past weekend's megatornado in Joplin, MO, generating a map of the resultant precipitation. You can find out about it here.

WeatherSpark

WeatherSpark combines two of our favorite topics (data visualization and climate/weather) in one very compelling package. A few quick and interesting things to do with it in under 5 minutes:

1. Check out predicted cyclicality of temperatures over the 24 hour cycle
2. Note the difference between observed weather (black line in the past) versus predicted
3. Check out weather at different physical locations (close to / far from coast; east of / west of mountains; north / south; etc.)
4. Play around with the other, less intuitive variables (pressure, humidity...)
5. Do 1. 2. and 3. together to detect rural vs. urban heat island effects (particularly cool)

Enjoy!

Data-oriented search engine

Jon Goldhill sent me a link to his company's new data-oriented search engine, which I think has interesting potential: zanran.com

According to their website:


What is Zanran?
Zanran helps you to find ‘semi-structured’ data on the web. This is the numerical data that people have presented as graphs and tables and charts. For example, the data could be a graph in a PDF report, or a table in an Excel spreadsheet, or a barchart shown as an image in an HTML page. This huge amount of information can be difficult to find using conventional search engines, which are focused primarily on finding text rather than graphs, tables and bar charts.


Put more simply: Zanran is Google for data.


How it works… technology overview
Zanran doesn't work by spotting wording in the text and looking for images – it's the other way round. The system examines millions of images and decides for each one whether it's a graph, chart or table – whether it has numerical content.
The core technology is patented computer vision algorithms that decide whether an image is numerical – and they're accurate (about 98%). But the huge majority of images on the internet are not graphs etc. So even though the accuracy is high, you will still get some non-numerical images.
In comparison, looking for tables is relatively simple. Once we've found a table we then have to decide whether it's essentially numerical - and we have algorithms for that.
Our programmes then take suitable text near that image and build the search engine around that text. At present, we extract tables and images from HTML, PDF and Excel files and will be processing PowerPoint and Word documents in the near future.



I've added a link on our resources page. Check it out.

Columbia University's Sustainable Development PhD Graduates of 2011!

On Saturday, six of us from Columbia University's PhD program in Sustainable Development graduated amid great fanfare. In reverse alphabetical order (since Ram got to walk across the stage first):

Marta Vicarelli wrote the dissertation "Essays on Climate Risks and Vulnerability-Reduction Strategies" and will be an Assistant Professor of Economics at University of Massachusetts Amherst following a Postdoc at Yale University.

Anisa Khadem Nwachuku wrote the dissertation "Critiquing Economic Frameworks in Sustainable Development: Health Equity, Resource Management and Materialism" and is working for McKinsey & Company.

Gordon McCord wrote the dissertation "Essays on Malaria, Environment and Society" and will be an Assistant Professor at the School of International Relations & Pacific Studies at University of California San Diego.

Chandra Kiran Krishnamurthy wrote the dissertation "Essays on Climatic Extremes, Agriculture and Natural Resources" and will be a Postdoc at Umeå University of Sweden.

Solomon Hsiang wrote the dissertation "Essays on the Social Impacts of Climate" and will be a Postdoc at Princeton University.

Mukul Ram Fishman wrote the dissertation "Theoretical and Applied Dimensions of Natural Resource Management" and will be a Postdoc at Harvard University.

Jump if you're a doctor!

Are Temperature and Incentives Compliments or Substitutes? Evidence from 1947

Here's another gem from the lab of Norman Mackworth (which in my head is looking increasingly like the Dharma Initiative): "High Incentives Versus Hot and Humid Atmospheres in a Physical Effort Task" N. H. Mackworth (British Journal of Psychology, 1947).  The setup is very similar to my last post, except this time the workers are doing arm curls until complete exhaustion:

But Mackworth varied whether the worker was encouraged to push themselves to do more (verbally and visually).  In all cases, the workers with encouragement did more arm curls.  But at high temperatures, this margin dropped substantially (see fig). In this case, it seems like moderate temperatures and incentives are compliments.  This sounds like unfortunate news for hot developing countries...

[For a review of what's happened since 1947 in this field of ergonomics, see the book chapter here.]

Which health supplements actually work?

Since we're pretty big on data visualization here on Fight Entropy I thought I'd point out that the guys over at Information is Beautiful have updated their great Snake Oil infographic on nutritional supplements. It does an excellent job of summarizing the current state of research on a fairly sprawling and contentious topic. Go check it out.

Now if they'd only do one on climate phenomena...

Are Temperature and Human Capital Compliments or Substitutes? Evidence from 1946

My own work has focused on whether economic productivity can be influence by temperature through its impact on worker productivity. I recently dug up "Effects of Heat on Wireless Telegraphy Operators Hearing and Recording Morse Messages" by N. H. Mackworth (British Journal of Industrial Medicine, 1946) and have decided its one of my favorite papers. The British military asked Norman Mackworth to put Morse code operators in rooms of varying temperature and humidity and recorded the number of errors they made. These guys were put to work in 3 hour stretches, decoding random sequences of letters and numbers for 5-7 weeks (see fig).  

The main result is stark.  The operators who were considered "exceptionally skilled" (based on their error rates at low temperatures) were hardly affected by the heat. But the less skilled operators had error rates that went through the roof when temperatures rose (see below). Apparently, moderate temperatures and human capital are substitutes (at least in this situation). This is good news for many of the hot, developing countries around the world...

How important is brain temperature? Ask a large predator fish

This is the kind of thing I can't believe they didn't teach me in elementary school.  I knew the platypus was exceptional among mammals for its beak and eggs, but I never knew that some cold-blooded animals (ectotherms) were actually partially warm-blooded (regional endothermy is the technical name). 

I was doing a literature review of the effects of temperature on brain function and cognition, when I ran across this amazing literature. In a famous 1982 Science article, Francis Carey of WHOI demonstrated that swordfish had evolved a specialized organ that simply sat next to their brain and kept it warm by burning calories quickly.  This was important, he argued, because these fish rapidly dove to depths where the water temperature was 20 degrees (C) colder than their normal environment. Because brain function is important to these hunters, it seemed reasonable that it was worth it for them to evolve specialized organs to keep their brains warm.

Sound too fantastic? Since then, cranial endothermy has also been documented in some varieties of sharks and tuna.  Because these species are so far from one another (in an evolutionary sense) and many of their more closely related relatives didn't have brain heaters, its thought that these different groups evolved similar organs independently from one another. This would suggest that maintaining a relatively more stable brain temperature might have large benefits.  Now, does this apply to humans?...

Temper and Temperature in Major League Baseball

Another recent (slightly amusing but) good one from Psychological Science. (Related here and here).

Temper, Temperature, and Temptation
Heat-Related Retaliation in Baseball
Richard P. Larrick, Thomas A. Timmerman, Andrew M. Carton and Jason Abrevaya

Abstract: In this study, we analyzed data from 57,293 Major League Baseball games to test whether high temperatures interact with provocation to increase the likelihood that batters will be hit by a pitch. Controlling for a number of other variables, we conducted analyses showing that the probability of a pitcher hitting a batter increases sharply at high temperatures when more of the pitcher’s teammates have been hit by the opposing team earlier in the game. We suggest that high temperatures increase retaliation by increasing hostile attributions when teammates are hit by a pitch and by lowering inhibitions against retaliation.

Seriously?

This came out in March in the journal Psychological Science.  My amazement is with the people in the study (not the people doing it).

Local Warming Daily Temperature Change Influences Belief in Global Warming

Ye Li, Eric J. Johnson
Center for Decision Sciences, Columbia University

Lisa Zaval
Psychology Department, Columbia University

Abstract: Although people are quite aware of global warming, their beliefs about it may be malleable; specifically, their beliefs may be constructed in response to questions about global warming. Beliefs may reflect irrelevant but salient information, such as the current day’s temperature. This replacement of a more complex, less easily accessed judgment with a simple, more accessible one is known as attribute substitution. In three studies, we asked residents of the United States and Australia to report their opinions about global warming and whether the temperature on the day of the study was warmer or cooler than usual. Respondents who thought that day was warmer than usual believed more in and had greater concern about global warming than did respondents who thought that day was colder than usual. They also donated more money to a global-warming charity if they thought that day seemed warmer than usual. We used instrumental variable regression to rule out some alternative explanations.

Al Gore's "Our Choice" app

By chance, I ran across across Al Gore's new high profile ipad/iphone app. Its a new kind of ebook (which presents the material from his paper book with the same title) where interactive graphics, maps, movies and data visualizations are all embedded in the text.  After browsing it for just a few minutes, it becomes clear that it deserves the praise it's been getting for its innovation in design (regardless of whether or not you agree with its message).  For complex multidimensional problems, like global climate change, getting an average person up to speed on the issues is costly.  Design like this can help reduce this cost and get more people involved in the discussion in a meaningful way.  For someone who wants a rapid and concise intro to the issues [and is willing to tolerate a few subjective/political interpretations of the issues], this is a fun way to go.  You can download it here for $5. Trailer videos below.