Webcast: Extending Game Design to Business Applications

I love games and think they have a lot to teach us about user interface design.  This July, I’ll be webcasting a revised and updated version of my talk on extending the gaming experience to conventional UI’s through Rosenfeld Media and Smart Experience as a part of their Future Practice series.  I debuted it last year at the IA Summit and EuroIA, and got a lot of great feedback from people who found it fun, interesting, and useful.

Details are available on the webcast’s description page.  You can watch it individually, or project it in a conference room for as many people as you like on the one registration.  If you decide you’re interested, be sure to use the code FERRARAWBNR to get a 15% discount.

If you’re interested in hearing something different, insightful, and kind of neat I think you’ll find this one fits the bill!

Letting crows do the work for you

Let’s say you wanted to gather all the loose change within a 5-mile radius, but you lack the time to do it yourself (which seems likely).  Why not enlist crows to do the scavenging for you?

That’s exactly what Josh Klein has done.  For his master’s thesis at NYU Klein invented a device that, through operant conditioning, trains crows to gather coins and drop them into a slot.  The device then disburses peanuts as a reward, like a vending machine.  It’s a fantastically efficient arrangement, since he only needs to keep the machine stocked and the crows take care of the rest.

C. brachyrhynchos, the American crow

This is formally described as “synanthropy”, adapting animals to work within human environments.  But I think it’s an awesome demonstration of an even larger strategy: getting an organism to do a useful job while pursuing an unrelated goal.  The crow doesn’t share our interest in money, it’s just trying to get some peanuts.

Luis Von Ahn applies the same strategy to solve difficult computational problems, using human beings as crows.   A researcher at Carnegie Mellon, he’s developed a series of “Games with a Purpose” that surreptitiously gather useful metadata while engaging players in 2-person online computer games.  As with Klein’s crows, the players are pursuing their own goal — in this case to have a bit of fun.  But through the games’ design, they create a byproduct that has real value.  All of the games attach human meaning to data that machines can’t read.

His games are:

Games with a Purpose

• The ESP game, where two players are shown the same image and asked to suggest tags describing it, then awarded points when they match up.  When more than one person has independently chosen the same tag, that implies it’s a reliable descriptor of the image.  Search engines can then use those tags to return more relevant image results.

• Tag a Tune plays a musical track for both players.  The players need to figure out whether they’re listening to the same track or different tracks by sending each other descriptions of what they hear.  The value of the game is that in the process they’ll provide useful tags like “piano”, “airy”, and “female vocals” to indicate qualities that would otherwise be indecipherable to a computer.

• Verbosity is a password-like game that gives one player a word to describe to the other player, who has to guess it.  The game provides a number of canned relationships that can be used to describe the word, like “It is a type of ___”, or “it is the opposite of ___”, or “it looks like ___”.  For example, if the word were “cough” then the clues might be “it is a type of physical reaction” and “it is used to clear your airway”.  This game assigns meaningful ontology relationships (known as “triples”) to words, which can be used to deepen computer understanding of human language.

• Squigl takes the images and tags created in the ESP game and asks players to trace the portion of the image in which the tagged object appears.  So for a picture of a woman walking her dog that’s tagged “leash”, both players would be tracing a similar area of the picture.  Points are awarded depending upon how well the areas overlap.  There are several possible utilities for the data gathered, from deciding what proportion of an image is relevant to a tag to informing image-reading applications.

• Matchin simply shows both players two pictures and asks them to pick the one they like best.  Each consecutive time they pick the same one, they’re awarded an increasing pool of points.  This game provides data along the lines of Flickr’s “interestingness” function, adding human subjectivity to digital images.

Von Ahn compares the design of these games to the design of an algorithm.  Each one ensures that its data output is correct, and has a certain level of efficiency associated with it.  Like Klein’s device, they also take advantage of operant conditioning by awarding players points on the site’s leaderboard.  It’s a simple currency and less tangible than peanuts, but effective because it’s promoted as a measure of prestige.  Profiles even allow players to use the site as a matchmaker, pairing people who see the world in similar ways.

There are plenty of other examples of people acting as crows on the Web:

• Search logs of the most commonly submitted queries provide Web designers with a prioritized list of the things users expect to find on the site, expressed in their own words.
• Flickr users feed image-processing applications when they tag their images for personal use.
• Foreign language translations of documents are used to inform probabilistic translation applications.

In all of these cases, the people doing the work have an objective that’s unrelated to the way in which someone else makes use of their work.

User-distributed work is emerging as a central component of Web 2.0, but some jobs are too laborious, too tedious, or too unfulfilling for people to pursue them on their own merits.  Learning how to use crows will be an important part of governing the role of the human being in future applications.

In Defense of Documentation

At recent conferences and in discussion groups, I’ve found it’s become increasingly popular to deride design documentation as time-wasting, inscrutable, and pedantic.  There’s a growing drive to jump directly into development, and manage design on the fly.  Few people are providing cogent arguments to the contrary.  As an IA who prides himself on the quality his documents, I feel the need to address the role of the spec in the age of Agile.

Bad documentation is real, and must die

There’s no doubt that there are some absolutely terrible documentation practices out there.  If you’re writing several dozen pages of design strategy in plain prose, you shouldn’t be surprised when no one bothers to read it.  A core measure of a development effort’s success is its efficiency, and documents that slow down the process should be hastily ditched.  It makes no sense to compound the time lost in writing poor documentation with even more time spent reading it.

Good documentation is also real

Good documents are completed quickly, express ideas efficiently, expedite development, build trust among team members, and allow people to achieve things that would otherwise be impossible.  People enjoy reading and discussing them, and teams of designers, developers, writers, and clients work as a whole to review and iterate them.

An analogy

Two years ago my wife and I decided to upgrade to an HDTV.  The problem was that we were going from a 24” tube set to a 46” flat panel, and the new set wouldn’t fit our existing furniture.  Even worse, the money we were spending on the TV precluded us from buying new stuff.  So I decided that I’d build a TV stand myself.

"Wireframe" of our TV stand

Now you need to understand that I’m not the sort of person you might describe as “handy”.  I’d never built so much as a birdhouse, and my high school didn’t even have a wood shop.  I had absolutely no idea how to construct a physical object.  Furthermore, my wife strictly forbade the use of any saws or sharp objects for fear I might cut off something important.

So I decided to create a physical object in the same way I would create any virtual one: I drew up a wireframe in Visio.  Before picking up a hammer or even buying the wood, I first mapped out the whole vision on paper.  I shared early top-level designs with my wife, we talked them through, and I modified the plans until we agreed on the approach, and then I filled in all the low-level details.

The completed TV stand

I’m pretty pleased with the result (see the picture at right).  It’s not exactly Herman Miller, and it weighs about 50,000 pounds, but it meets our needs precisely and cost me just under $100 to make.  And this example illustrates a few key advantages to a good documentation process over going straight to development:

• Faster completion time. The documentation itself only took two days to develop, and since everything was planned out beforehand there was no wasted development time.  I got the project done in six weeks, much faster than I could have without the documentation.

• Control of design. There’s a terrific symmetry between the design on paper and the finished product.  This was very important, because a lot of factors had to be balanced in the design: ensuring structural integrity, fitting all of the components in without making the stand too tall, and creating aesthetic beauty.  Jumping straight in, I wouldn’t have been able to manage that balance.

• Planning around restrictions. It’s hard to build something out of wood when you can’t use a saw.  Lowe’s could cut the wood for me, but they could only do straight cuts at 90° angles to the board.  This was a built-in hardware limitation – not a deal-killer, but it required some creative planning.  Accounting for restrictions in the documentation keeps them from becoming show-stoppers later on.

• Lower risk. Of course I could have jumped straight into the build, but that would have been much more risky.  I would have wasted a lot of lumber.  It could have come out looking terrible.  I could have discovered halfway through that there was a much better way of doing it.  Designing it on paper first was harmless, and maximized my chances of success later on.

Hmm, now what could that have to do with UI design?

By now this connection should be abundantly obvious.  In fact, from the most general point of view of process there is no useful distinction between building a TV stand and building a website.  Both require design and development efforts.  Both involve costs and risks.  Both produce an end product that needs to satisfy many needs equitably.

Quality documentation allows people to achieve things that are otherwise impossible.  The human brain can only do so much on its own – we need to extend its capabilities by processing things on paper, with our hands, and through conversation.  The task of designing software is often so complex that you can’t even grasp the full problem until you’ve written enough of it down.

But perhaps most critically, a good document provides for a foundation of trust to be built between team members.  Developers shouldn’t have to do design – it’s a distraction from their responsibilities, and exposes them to accountability for decisions that should have been handled by someone who’s paid to make them.  If the developers can trust that all foreseeable design decisions have been anticipated and built into the documentation, they’ll stick to it zealously.

What about Agile?

“Agile” is the name given to a process of rapid iterative prototyping and efficient communication.  I think it’s a terrific thing, because it allows concepts to be tried out and massaged into good shape in short order.  I employed a bit of Agile while building my TV stand – using scrap wood to figure out to make dowel joints and apply a stain, and grabbing whatever was within reach to get a job done.

But Agile has spawned a certain animosity toward even very high quality, efficiently produced, time-saving documentation.  I don’t think that’s Agile’s intention.  I think it rightfully seeks to eliminate time wasted producing documents that no one will bother reading, but perhaps its mantra has been repeated so often that it’s grown beyond its original meaning.  Played out to its logical extreme, ditching documentation would lead to disaster on all but the simplest projects.

So what should we do?

We need to reestablish the role of documentation in the product development lifecycle.  To do that, we first need to come to grips with the fact that what people have received in the past has been found wanting.  So we need to make a few key changes:

• Crank it out. We need to be able to turn documents around faster than developers can build to them, or else we’ll become a bottleneck.  Our documents should be done in the simplest form possible, with no excess ornamentation or verbosity.  Ugly is efficient.

• Communicate efficiently. We do after all work in usability, so our top priority should be to keep the needs of the document’s consumers in mind.  Integrate images with text effectively, speak clearly, and edit out anything that doesn’t solve a problem.

• Know your medium. We don’t need to be coders, but if we understand how HTML, CSS, scripting, and databases work we’ll be able to take advantage of the strengths of these technologies.  Marshall McLuhan was right – the medium is the message.

• Make people’s jobs easier. Bad documentation adds to other people’s workloads.  Good documentation lightens them.  Everyone will be happy to have something that reduces the total number of decisions they need to make.  Our documents should take control of everything that will affect the usability of the system.

• Produce gold. People need to believe you’ll design something that will make everyone involved look good, and bring them to a better outcome than they could hope for without us there.  Get everyone’s input into the document as it’s developed, and iterate it repeatedly to build confidence in the finished product.

Interview with Eric Matthews, Creative Director of Sony Computer Entertainment Europe

"Kung Foo" allows players to punch and kick at ninjas overlaid onscreen. (Image from Gamespot)

In 2002, Sony Computer Entertainment’s (SCE) London Studio took on the unique challenge of developing video games for a very different type of controller.  The EyeToy USB camera for the PlayStation 2 projects a player’s image onscreen, where he or she can interact with overlaid objects by moving around and physically touching them. 

The result was a compilation of novel games called “EyeToy: Play”, the first release for the controller.  In “Wishy Washy” the players wipe soap suds off of dirty windows.  “Kung Foo” has players chop and kick at hordes of miniature attacking ninjas.  “Mirror Time” confounds players’ visual-motor control by having them grab at objects while their image is repeatedly flipped backward, upside down, and back again.

In a market where such models for player experience had no precedent, the EyeToy became an international hit selling millions of units.  Last month I spoke with Eric Matthews, the director of creative development for Sony Computer Entertainment Europe.  He works with the team at SCE London Studio, which has led innovation in the video game industry with titles including “SingStar”, “The Getaway”, and the “EyeToy: Play” series, now in its fifth iteration.  We discussed user interface design, the development of the EyeToy, and the role of usability in game design.

How did the idea for the EyeToy get its start?

"Mirror Time" inverts the player's image repeatedly to confuse their sense of left, right, up, and down. (Image from Gamespot)

The idea of using a webcam as a video game interface originated from SCE’s research & development facilities in the US.  Dr. Richard Marks had been experimenting with things like motion control, color tracking, and augmented reality.  He had developed some prototypes using cameras, but there wasn’t yet any game experience associated with it.  Phil Harrison, then the head of Sony Computer Entertainment Europe development, showed Rick’s work to the various game development studios to see if anyone could think of directions it could be taken.

Ron Festejo, who’s one of the people on our team, really wanted to look into it.  From there, Rick assisted on the technical side, but it was London that made it into a game.

How did designing for the EyeToy differ from design for more conventional games?

We indentified four things to focus on.

First, you are the star of the game.  You’re seeing yourself up onscreen, so there’s a bit of vanity there.

Second, it had to be as much fun to watch as it was to play.  We wanted it to be a great social experience, and something where parents and kids could both engage in play together.

Third, the games couldn’t be better played on a controller.  It wouldn’t be worth doing if you could only do the very same things as you could in any other game, so it needed to be something unique to the controller.
 
Fourth, it had to be easy for people to intuit, so you don’t need to have the rules explained to you.   You couldn’t have a control scheme as complicated as you do for other games, where you’ve got sixteen buttons on a controller.  And of course that has the advantage of making it more accessible to people who normally aren’t as comfortable with games.

How did you design gameplay for the EyeToy?

We started by doing a bit of competitive research, looking at the landscape and what things may have been done along the same lines. 

Then we asked what is it good at, and what are its weaknesses?  For one, it had a magical “wow” factor of seeing yourself onscreen interacting with the game world, which got people particularly excited.  But while it could pick up big broad movements, but it was not very good at fine granular motion.  Another disadvantage was that it was sensitive to noise in the background.  We felt it worked well as a short-form medium, with fast play and a lot of variation in the games. 

Then we ran some brainstorming sessions to see where we could take the ideas.   We wanted to see how many different forms of interaction we could create, so we went really broad.  From there we would say okay, we’ve brainstormed an idea for a boxing game, why is that a valuable thing?  Why would you want to play that? 

After that we went into a very experimental prototyping phase so we could get a playable experience onscreen quickly to see if the ideas were fun and engaging.  We developed around thirty prototypes, including some wild stuff that never worked.  We ended up with twelve in the final game.

Can you give an example of how one of the games was designed?

Well “Wishy Washy” evolved from an idea where you had to climb to the top of a building.  I sat there with Pete Marshall, the lead programmer, and we started thinking “Okay, how would you see that?”  And we said well maybe the camera’s on the inside of the building looking out through the windows at you.  Then that turned into, well perhaps you’re washing the windows as you go up.  Then it was, you need to clean the window on each floor before you could go to the next one.  Through that process of just talking it through over a half hour late at night, an evolution of thinking turned it into a window washing game.

How did you interact with marketing?

We got marketing involved to collaborate on what games had the greatest potential and harvesting the best ideas, and getting out to look for an audience.  They found that it was something parents could play with their kids, that could bring down barriers and get people moving.  After the game was finished they put demonstrations of it in clubs and shopping malls for the launch, because it’s one of those things you don’t really get until you’ve played.

One of the defining moments came at The PlayStation Experience, a London event where we first showed the prototypes to the public.  We showed three games and the press went nuts.  Everyone from small children to hardcore Tony Hawk skaters with multiple piercings were lining up to play it.  The gaming press wrote that it was something very unlike what they’d seen before, and there was a lot of excitement.  At that point we started to think okay, we may have something here.

Up to then, no one had thought it was going to sell millions.  Well, let me amend that – we thought it would either sell less than 100,000 or more than a million but not in between.  It turns out it sold somewhere between 3 and 4 million units.

Did you do any testing of the game with users?

This is going to sound terrible, but EyeToy: Play was the first game where we did formal user testing, and that was only once the game was finished.  We had done ad hoc testing using the people in the QA department, children of coworkers, things like that.  But this was the first time we recruited real users and set up at a facility behind the one-way glass and so forth. 

And it was an absolute nightmare.  Once people were into the game they had a great time playing it, but they couldn’t get there quickly enough because the flow of the menus was too long winded.  People also couldn’t figure out the right distance to stand, and then someone would walk across the room in front of the camera and inadvertently trigger something onscreen.  It came to a head when the players kept accidentally cancelling the setup process for their profiles.  Around then Ron Festejo got up and said “Stop the blasted thing, I can’t bare it anymore!”

So we all found the first test very tough, but it was a very valuable learning experience.  From that point on, we’ve tested all of the games we develop.  I’m a big proponent of the value of testing with users, and we’re building our own facility in London.

How did you develop the process for testing games?

There was no company in Europe that focused on testing video games.  They were all working in consumer software or Web-based usability.  And some of that’s relevant, but some of it isn’t.  They had a heavy task orientation, where you’re given situation A, now show me how you’d do B, and that’s not as relevant to games.  They just hadn’t had the experience of applying their methodology to a game.  How do you test for fun?

So we worked with an agency that had a lot of experience in Web called AmberLight, and collaborated to create a process that we use to test all of our games.

So what is that process?

First, you decide what you want to test.  That includes obvious things – the fundamental control system, how it feels and how it compares to competitors.  Then there are things like signposting, navigation, how long it takes to complete objectives – primary, secondary, tertiary.  We do a peer group review, and get the experts to look at it before going into testing for what we think is going to work well, and what we think might not work so well.

Then we test what’s called the first publishable, which is typically one complete level of near-publishable quality with all of the controls and visuals in place – a vertical slice of the game.  This comes about halfway through the development lifecycle.  We test it with 10 people playing simultaneously, but separately from one another.

Following the test we have them complete a questionnaire.  We ask primarily about usability concerns, but also look at appeal to some extent.  We finish the testing by holding a roundtable with those people, though we’ll sometimes do individual interviews as well.

In the end, we document all of the issues we’ve found, make a top ten list from it and say okay, these are the things we want to fix for the next time round.  We’ll do testing 2-3 times in the lifecycle with 10 people each time, bringing in different demographics.

Alongside that, Mark Cerny had laid the foundations for testing at Sony in the US over the last 10 years.  They’re doing a lot of data capture as well, recording how long it takes to complete each level and looking for things like death clustering, which is when everyone’s dying at the same places in the game.  There’s now a strong culture of user testing at studios including Foster City and Santa Monica, and that’s been a big part of enabling them to increase the quality and relevance of the titles they produce.

And what’s the future of the EyeToy?

The PS3 now has its own USB camera — the PlayStation Eye, which was released with a game called Eye of Judgment that recognizes playing cards placed on a surface. 

We’ve also continued releasing EyeToy games for the PS2.   Two of them make use of an evolution of the EyeToy color tracking technology that identifies both motion and color, which gives a much deeper level of control.  One is EyeToy: Play PomPom Party, which is a cheerleading game that comes with a set of pom poms, pink and green.  Along with that is EyeToy:Play Hero, which comes with a green sword that you use to fight your way through the game.  On screen the sword can light up, take on flames, and so forth. 

And we’ve just announced EyePet, which is generating some excitement.  You might want to take a look at the video.

Games as production

In my presentation at the 2008 IA Summit, I discussed how many human activities can be understood as games, and benefit from adopting their characteristics. When we think of games as being specifically unproductive, we’re missing the opportunity to engage users at a level beyond what can be achieved in more conventional interfaces.

In fact games can serve as catalysts of production. Take fold.it, which is a puzzle game that challenges players to find the best ways to fold proteins. This is in fact among the most difficult problems in modern biology, as a protein can take on very different characteristics depending upon its shape. For example, mad cow disease is caused by proteins that already exist in the body, but which have been folded into irregular shapes that make them agents of the disease.

A screenshot from fold.it

People who play fold.it are actually contributing to science, because the game uses the real physical properties of the proteins as its rules. Players are awarded points for things like reducing the size of the protein efficiently, or turning certain types of molecules so they all face inward. The New York Times notes that it’s plausible that by playing this game, you could actually win a Nobel prize (even if you know nothing of biochemistry).

The real pioneer in the productive use of games, though, is Luis Von Ahn of Carnegie Mellon University. I’ll discuss his work in depth in an upcoming posting.

Final Fantasy User Interface

Over the past few weeks I’ve been playing a game for the PS2 called Final Fantasy XII (sounds dirty, but it’s not). I haven’t been a big fan of the series, but this one is really very good. I’m finding it especially interesting from the perspective of user interface design.

FFXII is a role-playing game, in the vein of Dungeons & Dragons. Generally speaking, these can be fairly complex endeavors. You control multiple characters, each of whom has its own attributes that evolve over the course of the game, and you need to attend to their complement of weapons, armor, and accessories, as well as micromanaging their individual actions in battle (I know, fun times). Creating a UI that allows players to sort through all of this complexity is a real challenge, but the design in this game is just fantastic. I expand on a few examples below.

Screenshot 1: Equip

Equip screen

This first screen shows the interface for equipping a character with body armor (click the screenshot to enlarge it). The list of available armor is on the right side, and colored dots show what each of the 6 characters is currently wearing. So character 4 is currently wearing bronze armor, and we’re considering upgrading him to diamond armor (the current position of the cursor). The list of attributes on the left shows what the exact effect of that upgrade will be, with the blue numbers indicating the new values. The character’s defense will go up fivefold, while his strength will increase by a more modest amount. Players can compare different armors just by moving the cursor up and down and noting the attribute changes each one affords.

Another little nuance is that certain characters are only allowed to wear certain types of armor. This is indicated by a circle under that character’s face. If the character can’t wear the armor, we see a dot instead of a circle. So we can tell at a glance that Basch is the only person who can wear diamond armor, while anyone is welcome to leather clothes. I like too that faces are used as column headings here: it’s both space-efficient and takes advantage of the fact that people are innately good at recognizing faces.

Screenshot 2: Gambits

Gambits

Final Fantasy XII includes a novel gameplay element called the gambit system, which is a sort of very simple programming language. Rather than laboriously commanding each character action by action in battle, you can set up a list of things they should do automatically whenever a certain condition is true. This is the gambit screen for a character named Penelo, who is currently assigned five active commands. The fifth one says that if any foe is nearby, she should attack it. But it’s superceded by all of the commands listed above it, which include healing poisoned allies or curing them if their health drops below 70%. It’s kind of funny that the condition statements are purchased or won over the course of the game, with the most helpful ones being more difficult to attain.

The system is flexible and very easy to use. You just pair a condition statement with an action, turn the gambit on and you’re ready to roll. Each gambit can be picked up and repositioned in the list to ensure the actions are executed in the proper order. The interplay of different characters in different battles against different enemies makes the gambits rather involved. It’s a great use of programming logic, allowing the player to assemble a sequence of if-then statements and then test out their effectiveness in different scenarios.

Screen 3: Battle

Battle head-up display

When in battle, an overlay provides a remarkable amount of information without obscuring your view of the action, as shown in screen 3. You’re able to keep track of each character’s current and maximum health, current and maximum magic power, available actions, next action, how soon an action will be performed, which character is leading, whether their gambits are active, whether a character is being targeted, the enemy’s health, and your character’s position in a map of the world. That’s really quite a lot, yet it doesn’t feel at all obtrusive. The economy of space strikes me as very Tufte.

Scratching the surface

That’s really just the beginning; I could go on at some length describing the menu structure, automated mapping, selection controls, cursor behavior, etc. There are so many good things happening in this game, many of them very small touches that are easy to overlook because they feel completely natural. Far from treating the game as frivolous, it’s clear that the designers put real care into this UI.

Amazon’s product rating system

A recent article on Boxes & Arrows discusses rating and review systems on sites like NetFlix, eBay, and Amazon. I think these types of systems are a fantastic basis for a socially constructed experience. They take advantage of the Web as a mass medium, they can include both quantitative and qualitative components, and they can be really useful to people.

The article tweaks Amazon a bit for its “most helpful review vs. most critical review” idea. Authors Alex Kirtland and Aaron Schiff argue that people writing those reviews can be evaluating a product on completely different criteria, so a reader can’t depend on them to be comparable.

I think they’re missing the real beauty in Amazon’s design. There are three components to it:

• A star rating. This creates a quantitative value that, in aggregate, gives users a basis for quickly comparing buyers’ overall happiness with a product.
• A review. This answers the question of why a particular rater gave a product a positive or negative review.
• A thumbs up/down rating. Assigned by people who read the reviews, and rate it as either helpful or unhelpful. This is a rating system on top of the rating system, and simply counts the total number of positive against negative votes.

The real kicker in Amazon’s design is that they use the thumbs up/down rating to juxtapose the most highly rated positive rating with the most highly rated negative rating. In effect, they create a top-level debate giving the best reasons for and against buying a product. What a clever bunch of designers they have working there.

I also don’t have the same issue with comparability that Kirtland and Schiff are citing. While positive and negative reviews of the same product will often discuss completely different criteria, it’s also implicit that they’re both rating the product as a whole. The reviews are helpful because they disambiguate the reasoning underlying the rating, giving the readers the opportunity to decide which criteria are most important to them.

Kirtland and Schiff’s article is a good piece, but I think it does a bit of a disservice to a very impressive social component.

Complexity in “Spore”

Today Andrew Hinton shared a great video of Will Wright demonstrating “Spore” for the Long Now Foundation. It’s an amazingly ambitious game, and showcases the potential for games to change the way you look at the world.

As software, I think it’s interesting how they’ve distributed the burden of complexity. On the one hand, the developers have created relatively simple algorithms that produce very complex, multivariate outputs. The game allows players to create their own creatures that will live in the Spore universe, body part by body part.  For example:

A product of Spore's creature creator

The interesting thing is that the little critters are imbued with this sort of mystical mathematical soul. Wherever you position legs on a creature’s body, the computer figures out how that would affect its gait, balance, and stride. When you map a stripes onto its skin, it crunches a few formulas to figure out how they should flow from body part to body part. Simple algorithms are processed by the player’s computer to produce very complex results.

On the other hand, the rich diversity of life in the game is a product of the collective efforts of the many people who will be playing it. Rather than limiting themselves to a handful of creatures they prepackage with the game, they leave it open to the masses of players to build a complex game experience for each other.

So in a way, Maxis is working in two fundamental raw materials: the computational power of the players’ computers, and the limitless capacity of human imagination. The human investment is amplified by the algorithms, and the result is something much more complex than any person or software developer could ever create on their own.

Generative design

My favorite presentation from this year’s IA Summit was Leah Buley’s “How to Be a UX Team of One“. She discussed the generative design methods employed at Adaptive Path. Typically design will start with one idea, which is then iterated it until it evolves into some ideal form. But generative methods focus on maximizing the volume and variety of ideas up front, then pulling together the best concepts into a single solution. Brilliant.

Catriona Cornett recently ran an exercise with Vanguard’s information architects to demonstrate a generative method. She timeboxed it to 10 minutes, and instructed us to each come up with 10 ideas for a problem in that time – bing bang boom. It was a really good experience. One thing I loved was that at the end of the 10 minutes, we had a ton of things to talk about. By contrast, I’ve been in meetings that went on for hours with nothing to talk about.

I did something along similar lines in my work on Vanguard’s search engine and intranet. In the process of evaluating the existing site and conducting a comparative analysis, a lot of ideas would just hit me. Without chewing on them too much, I wrote a short description of each on one of those really tiny 1″ post-its and stuck it to a whiteboard in my cubicle. At the end of each day I’d look over the ideas, grouping together similar ones and seeing if they spurred more ideas still when juxtaposed. After two weeks, the board was full and I felt extremely ready to start jumping into some snazzy designs. But AP’s methods are much more structured and group-oriented than this.

I strongly suspect generative methods are a flat-out better way to do design. If you haven’t seen Leah’s presentation, it’s definitely worth checking out the slidecast.

Testing search

My sense is that information architects generally have less influence over the core experience of search than they do over other aspects of a website’s interface, and I think that’s a bit of a shame since it’s such a critical resource for so many users. Sometimes a website’s search engine will be a user’s primary experience of the site’s architecture, but here we are on the sidelines watching as the implementation teams wield control over the search experience.

In our work on Vanguard’s search engine, we developed a collection of methods for testing the quality of the results it was bringing back. These became critical tools underlying optimization efforts and functional design strategy, and ingrained the IA’s deeply in the quality of search. I think that these testing methods can bring a lot of value to the user experience community, so I’ve proposed a pair of articles for Boxes and Arrows explaining them.

Both tests are premised on the idea that the search engine should return the best matches available for a user’s expressed interest at the top of the results page. That sounds really simple, but there are a lot of complicating factors that weigh into it:

• How closely does any of the available content actually match the user’s real interest?
• Has the user done a good enough job describing what he or she wants?
• How does the search engine parse the user’s input?
• What standards does the search engine use to determine what constitutes a best match?

As a result of these kinks in the system, search is inevitably much more hit-or-miss than we’d like it to be. So from the user experience perspective, the first questions we should ask are how often does the engine return the very best matches at the top of the results list, and how good are the matches it does return?

Well it turns out we can measure those things, and it’s not especially hard to do. You can put a number to it, then set objectives for how much you’ll improve over time. (The critical resource here is search logs — which is why I’m psyched that Lou Rosenfeld has a book coming out on just that.) In the process, the strategy shakes out too. You’ll find exactly which searches are underperforming, which among those are the most important to people, and the best ways to fix the system. That puts IA’s right back in the center of the search experience.

Those evaluation methods are what I’ll cover in the articles. I’d greatly appreciate any feedback submitted on the B&A site, and incorporate any input into the text when writing them.