Map resources

I spoke with some librarians today who pointed me to two excellent online resources for historical maps, the University of Texas Map Library and the David Rumsey Map Collection.  There is so much untapped information on these sites its overwhelming.  Below were two favorites that I found on Rumsey's site.

The Distribution of Wealth, 1870

Ranking of States by Income, Debt, Literacy, etc, 1880

Bibliometrics and book shrink

In the middle of doing a bunch of bibliometrics work I came across a site called BookShrink that parses a text to choose the sentences in it that are most representative of the whole. The alogorithm basically assigns a score to each sentence based on the total frequency count of all words in the book and then normalizes by sentence length. So no, it doesn't look like it has anything positional and no, it doesn't condense the book into Cliff Notes-style summary (and it doesn't read pdf's, so don't think you're getting out of doing your lit review using just some bash scripts) BUT it is pretty cool. Loading in the entirety of The Wealth of Nations yields:

1 (1.000000): The taxes upon expense, therefore, which fall chiefly upon that of the superior ranks of people, upon the smaller portion of the annual produce, are likely to be much less productive than either those which fall indifferently upon the expense of all ranks, or even those which fall chiefly upon that of the inferior ranks, than either those which fall indifferently upon the whole annual produce, or those which fall chiefly upon the larger portion of it.

2 (0.961065): I shall endeavour to give the best account I can, first, of those taxes which, it is intended should fall upon rent; secondly, of those which, it is intended should fall upon profit; thirdly, of those which, it is intended should fall upon wages; and fourthly, of those which, it is intended should fall indifferently upon all those three different sources of private revenue.

3 (0.765899): From the end of the first to the beginning of the second Carthaginian war, the armies of Carthage were continually in the field, and employed under three great generals, who succeeded one another in the command; Amilcar, his son-in-law Asdrubal, and his son Annibal: first in chastising their own rebellious slaves, afterwards in subduing the revolted nations of Africa; and lastly, in conquering the great kingdom of Spain.

4 (0.708201): You must pay, too, for the tax upon the salt, upon the soap, and upon the candles which those workmen consume while employed in your service; and for the tax upon the leather, which the saltmaker, the soap-maker, and the candle-maker consume, while employed in their service.

...

No, nothing about pins, but still pretty interesting.

NOAA Environmental Visualization Laboratory

Just completing my data visualization hat trick for the week.  A colleague of mine, Amir Jina, recently pointed me to what is now one of my favorite websites: NOAA's Environmental Visualization Laboratory.

The group takes huge quantities of real-time data and transforms it into beautiful images designed to teach students and the public about the global environment. Perhaps most impressive is the constant stream of new images.  Some elegant highlights below.

Nitrogen dioxide concentrations showing China's superlative industrial pollution.

The absence of atmospheric water vapor over the equatorial Pacific illustrates the current La Nina.

3D plot of snow-depth in the US following some of the recent storms (related to the La Nina).

Composite image of some 2010 hurricanes.

North Atlantic sea surface temperatures were extraordinarily warm this past October.

Cropscape Data Visualization

Micahel Roberts recently sent me a link to this relatively new data browser for land-use in the US, produced by the National Agricultural Statistical Service. Its not available for all years, but the resolution is impressive, allowing you to zoom in so you can see the individual fields of farmers.

The announcement of the dataset was here:

“CropScape delivers data visualization tools directly into the hands of the agricultural community without the need for specialized expertise, GIS software or high-end computers,” said Mark Harris, NASS Research and Development Division director. “This information can be used for addressing issues related to agricultural sustainability, land cover monitoring, biodiversity and extreme events such as flooding, drought and hail storm assessment.”

NASS produced the 2010 CDL using satellite image observations at 30-meter (0.22 acres per pixel) resolution and collected from the Resourcesat-1 Advanced Wide Field Sensor (AWiFS) and Landsat Thematic Mapper. The collection of images was categorized using on-the-ground farm information including field location, crop type, elevation, tree canopy and urban infrastructure. All prior CDL products dating back to 1997 are also hosted by CropScape.

CropScape was developed in cooperation with the Center for Spatial Information Science and Systems at George Mason University, Fairfax, Va. The research and development ofCropScape and the NASS partnership with George Mason University reflect NASS’s continued commitment to improve U.S. agricultural production, sustainability and food security. 

CropScape is operated by NASS’s Research and Development Division and hosted and maintained by the Center for Spatial Information Science and Systems at George Mason University. For more information about CropScape, and the Cropland Data Layer visit http://nassgeodata.gmu.edu/CropScape.

Refugee flow visualization

A colleague and I are doing some work on refugees and ran across this excellent vizualization of UNHCR data here. This is how they describe it:

Since 1950, the United Nations High Commissioner for Refugees is mandated with the coordination of aid and assistance for refugees worldwide. According to its self-description, its primary purpose is to safeguard the rights and well-being of refugees. Over 6,000 people in more than 110 countries work for the UNHCR. Founded in 1951 as a means to assist the more than one million people who were still uprooted after World War II, the agency's mandate covered about 10 million refugees in 2009. This visualization attempts to give a comprehensive overview of the phenomenon of flight and expulsion, an ongoing issue of global scale and extreme complexity. 

Based on the annual UNHCR Refugee Report, the application allows views from different perspectives on the extensive dataset, highlighting different aspects. The idea for this visualization originated from a class project on the topic of mapping global tendencies at Potsdam University of Applied Sciences in 2008. The current application's interface was completely rebuilt in late 2009.

What it means to be in a nondemocratic developing country

Like I assume most of the people who read this blog, I've been watching events unfold in Egypt with a mix of (mostly) hope and (some) worry. Ignoring for the moment possible outcome trajectories, the specifics of what's been going on, the response from other countries, and everything else, I think it's interesting to focus for a second on why and how Egypt got here:

"... a carnival atmosphere descended on the square, where vendors offered food at discount prices and protesters posed for pictures in front of tanks scrawled with slogans like, '30 years of humiliation and poverty.'"

That's from an online NY Times article from yesterday that's been getting continuously updated (and now no longer shows the original quote). The set of events that have to go down to bring about the kind of social unrest we're seeing in Egypt aren't trivial. People routinely toss around the idea of "revolution" when discussing topics like a change in US political party or new regulations (e.g., of gun control, carbon taxation, health care...) but the costs of revolution, in particular in terms of the uncertainty of its outcome, are huge. So I think it bears thinking about for a second:

30 YEARS of humiliation and poverty. Notoriously brutal secret police. Elite capture. Enough frustration to push your way across a bridge guarded by hundreds of cops willing to turn a water cannon on you.

We often talk about development, its constraints, and the various political, economic, and historical forces which result in countries being "developed" or not. Sometimes those conversations can get very abstract. It's worth remembering that being in a typically underdeveloped, undemocratic country means dealing on a day to day basis with issues so debilitating and frustrating that you're motivated to spray paint your grievances on the side of a tank.

*Note: the graffiti on the side of the tank says "Down with Mubarak," and comes from Ramy Raoof's flickr stream, here.

Weather-driven pirate risk maps

A variety of sources point out that Naval Research Labs' James Hansen* has come up with a piracy-risk model that takes as its two major sources of input current and forecasted weather and in-field reports on pirate activity:

"Usually, I'm doing theoretical stuff down in the weeds," said Hansen, a Seattle-area native and applied mathematician at the Naval Research Laboratory in Monterey, Calif. "This is the only project where I can actually show pictures of the impact," he said, projecting images of Somali boatmen armed with missile-launchers and automatic weapons.

The quote is from an article in the Seattle Times about a presentation that Hansen gave at the American Meteorological Society meeting. The model is driven by the fact that most if not all modern piracy depends on small, fast boats that can evade long-distance detection and can outpace larger slower cargo ships. In heavy weather the boats can't operate. Combine weather forecasts with observed pirate activity and voilà.

I can't locate any sort of working paper at present, so if any of y'all locate it please do send it to me. And as I wrote to the sdev-internal email list: someone please please please write a paper using this ASAP. Just be careful of your exclusion restrictions.

* As Sol points out: "How many climate scientists are named James Hansen? I now know of 3."