Letting crows do the work for you

Let’s say you wanted to gather all the loose change within a 5-mile radius, but you lack the time to do it yourself (which seems likely).  Why not enlist crows to do the scavenging for you?

That’s exactly what Josh Klein has done.  For his master’s thesis at NYU Klein invented a device that, through operant conditioning, trains crows to gather coins and drop them into a slot.  The device then disburses peanuts as a reward, like a vending machine.  It’s a fantastically efficient arrangement, since he only needs to keep the machine stocked and the crows take care of the rest.

C. Brachyrhynchos, the American crow

C. brachyrhynchos, the American crow

This is formally described as “synanthropy”, adapting animals to work within human environments.  But I think it’s an awesome demonstration of an even larger strategy: getting an organism to do a useful job while pursuing an unrelated goal.  The crow doesn’t share our interest in money, it’s just trying to get some peanuts.

Luis Von Ahn applies the same strategy to solve difficult computational problems, using human beings as crows.   A researcher at Carnegie Mellon, he’s developed a series of “Games with a Purpose” that surreptitiously gather useful metadata while engaging players in 2-person online computer games.  As with Klein’s crows, the players are pursuing their own goal — in this case to have a bit of fun.  But through the games’ design, they create a byproduct that has real value.  All of the games attach human meaning to data that machines can’t read.

His games are:

Games with a Purpose

Games with a Purpose

  • The ESP game, where two players are shown the same image and asked to suggest tags describing it, then awarded points when they match up.  When more than one person has independently chosen the same tag, that implies it’s a reliable descriptor of the image.  Search engines can then use those tags to return more relevant image results.
  • Tag a Tune plays a musical track for both players.  The players need to figure out whether they’re listening to the same track or different tracks by sending each other descriptions of what they hear.  The value of the game is that in the process they’ll provide useful tags like “piano”, “airy”, and “female vocals” to indicate qualities that would otherwise be indecipherable to a computer.
  • Verbosity is a password-like game that gives one player a word to describe to the other player, who has to guess it.  The game provides a number of canned relationships that can be used to describe the word, like “It is a type of ___”, or “it is the opposite of ___”, or “it looks like ___”.  For example, if the word were “cough” then the clues might be “it is a type of physical reaction” and “it is used to clear your airway”.  This game assigns meaningful ontology relationships (known as “triples”) to words, which can be used to deepen computer understanding of human language.
  • Squigl takes the images and tags created in the ESP game and asks players to trace the portion of the image in which the tagged object appears.  So for a picture of a woman walking her dog that’s tagged “leash”, both players would be tracing a similar area of the picture.  Points are awarded depending upon how well the areas overlap.  There are several possible utilities for the data gathered, from deciding what proportion of an image is relevant to a tag to informing image-reading applications.
  • Matchin simply shows both players two pictures and asks them to pick the one they like best.  Each consecutive time they pick the same one, they’re awarded an increasing pool of points.  This game provides data along the lines of Flickr’s “interestingness” function, adding human subjectivity to digital images.

Von Ahn compares the design of these games to the design of an algorithm.  Each one ensures that its data output is correct, and has a certain level of efficiency associated with it.  Like Klein’s device, they also take advantage of operant conditioning by awarding players points on the site’s leaderboard.  It’s a simple currency and less tangible than peanuts, but effective because it’s promoted as a measure of prestige.  Profiles even allow players to use the site as a matchmaker, pairing people who see the world in similar ways.

There are plenty of other examples of people acting as crows on the Web:

  • Search logs of the most commonly submitted queries provide Web designers with a prioritized list of the things users expect to find on the site, expressed in their own words.
  • Flickr users feed image-processing applications when they tag their images for personal use.
  • Foreign language translations of documents are used to inform probabilistic translation applications.

In all of these cases, the people doing the work have an objective that’s unrelated to the way in which someone else makes use of their work.

User-distributed work is emerging as a central component of Web 2.0, but some jobs are too laborious, too tedious, or too unfulfilling for people to pursue them on their own merits.  Learning how to use crows will be an important part of governing the role of the human being in future applications.


No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: