Interview with Eric Matthews, Creative Director of Sony Computer Entertainment Europe

"Kung Foo" allows players to punch and kick at ninjas overlaid onscreen. (Image from Gamespot)

In 2002, Sony Computer Entertainment’s (SCE) London Studio took on the unique challenge of developing video games for a very different type of controller.  The EyeToy USB camera for the PlayStation 2 projects a player’s image onscreen, where he or she can interact with overlaid objects by moving around and physically touching them. 

The result was a compilation of novel games called “EyeToy: Play”, the first release for the controller.  In “Wishy Washy” the players wipe soap suds off of dirty windows.  “Kung Foo” has players chop and kick at hordes of miniature attacking ninjas.  “Mirror Time” confounds players’ visual-motor control by having them grab at objects while their image is repeatedly flipped backward, upside down, and back again.

In a market where such models for player experience had no precedent, the EyeToy became an international hit selling millions of units.  Last month I spoke with Eric Matthews, the director of creative development for Sony Computer Entertainment Europe.  He works with the team at SCE London Studio, which has led innovation in the video game industry with titles including “SingStar”, “The Getaway”, and the “EyeToy: Play” series, now in its fifth iteration.  We discussed user interface design, the development of the EyeToy, and the role of usability in game design.

How did the idea for the EyeToy get its start?

"Mirror Time" inverts the player's image repeatedly to confuse their sense of left, right, up, and down. (Image from Gamespot)

The idea of using a webcam as a video game interface originated from SCE’s research & development facilities in the US.  Dr. Richard Marks had been experimenting with things like motion control, color tracking, and augmented reality.  He had developed some prototypes using cameras, but there wasn’t yet any game experience associated with it.  Phil Harrison, then the head of Sony Computer Entertainment Europe development, showed Rick’s work to the various game development studios to see if anyone could think of directions it could be taken.

Ron Festejo, who’s one of the people on our team, really wanted to look into it.  From there, Rick assisted on the technical side, but it was London that made it into a game.

How did designing for the EyeToy differ from design for more conventional games?

We indentified four things to focus on.

First, you are the star of the game.  You’re seeing yourself up onscreen, so there’s a bit of vanity there.

Second, it had to be as much fun to watch as it was to play.  We wanted it to be a great social experience, and something where parents and kids could both engage in play together.

Third, the games couldn’t be better played on a controller.  It wouldn’t be worth doing if you could only do the very same things as you could in any other game, so it needed to be something unique to the controller.
 
Fourth, it had to be easy for people to intuit, so you don’t need to have the rules explained to you.   You couldn’t have a control scheme as complicated as you do for other games, where you’ve got sixteen buttons on a controller.  And of course that has the advantage of making it more accessible to people who normally aren’t as comfortable with games.

How did you design gameplay for the EyeToy?

We started by doing a bit of competitive research, looking at the landscape and what things may have been done along the same lines. 

Then we asked what is it good at, and what are its weaknesses?  For one, it had a magical “wow” factor of seeing yourself onscreen interacting with the game world, which got people particularly excited.  But while it could pick up big broad movements, but it was not very good at fine granular motion.  Another disadvantage was that it was sensitive to noise in the background.  We felt it worked well as a short-form medium, with fast play and a lot of variation in the games. 

Then we ran some brainstorming sessions to see where we could take the ideas.   We wanted to see how many different forms of interaction we could create, so we went really broad.  From there we would say okay, we’ve brainstormed an idea for a boxing game, why is that a valuable thing?  Why would you want to play that? 

After that we went into a very experimental prototyping phase so we could get a playable experience onscreen quickly to see if the ideas were fun and engaging.  We developed around thirty prototypes, including some wild stuff that never worked.  We ended up with twelve in the final game.

Can you give an example of how one of the games was designed?

Well “Wishy Washy” evolved from an idea where you had to climb to the top of a building.  I sat there with Pete Marshall, the lead programmer, and we started thinking “Okay, how would you see that?”  And we said well maybe the camera’s on the inside of the building looking out through the windows at you.  Then that turned into, well perhaps you’re washing the windows as you go up.  Then it was, you need to clean the window on each floor before you could go to the next one.  Through that process of just talking it through over a half hour late at night, an evolution of thinking turned it into a window washing game.

How did you interact with marketing?

We got marketing involved to collaborate on what games had the greatest potential and harvesting the best ideas, and getting out to look for an audience.  They found that it was something parents could play with their kids, that could bring down barriers and get people moving.  After the game was finished they put demonstrations of it in clubs and shopping malls for the launch, because it’s one of those things you don’t really get until you’ve played.

One of the defining moments came at The PlayStation Experience, a London event where we first showed the prototypes to the public.  We showed three games and the press went nuts.  Everyone from small children to hardcore Tony Hawk skaters with multiple piercings were lining up to play it.  The gaming press wrote that it was something very unlike what they’d seen before, and there was a lot of excitement.  At that point we started to think okay, we may have something here.

Up to then, no one had thought it was going to sell millions.  Well, let me amend that – we thought it would either sell less than 100,000 or more than a million but not in between.  It turns out it sold somewhere between 3 and 4 million units.

Did you do any testing of the game with users?

This is going to sound terrible, but EyeToy: Play was the first game where we did formal user testing, and that was only once the game was finished.  We had done ad hoc testing using the people in the QA department, children of coworkers, things like that.  But this was the first time we recruited real users and set up at a facility behind the one-way glass and so forth. 

And it was an absolute nightmare.  Once people were into the game they had a great time playing it, but they couldn’t get there quickly enough because the flow of the menus was too long winded.  People also couldn’t figure out the right distance to stand, and then someone would walk across the room in front of the camera and inadvertently trigger something onscreen.  It came to a head when the players kept accidentally cancelling the setup process for their profiles.  Around then Ron Festejo got up and said “Stop the blasted thing, I can’t bare it anymore!”

So we all found the first test very tough, but it was a very valuable learning experience.  From that point on, we’ve tested all of the games we develop.  I’m a big proponent of the value of testing with users, and we’re building our own facility in London.

How did you develop the process for testing games?

There was no company in Europe that focused on testing video games.  They were all working in consumer software or Web-based usability.  And some of that’s relevant, but some of it isn’t.  They had a heavy task orientation, where you’re given situation A, now show me how you’d do B, and that’s not as relevant to games.  They just hadn’t had the experience of applying their methodology to a game.  How do you test for fun?

So we worked with an agency that had a lot of experience in Web called AmberLight, and collaborated to create a process that we use to test all of our games.

So what is that process?

First, you decide what you want to test.  That includes obvious things – the fundamental control system, how it feels and how it compares to competitors.  Then there are things like signposting, navigation, how long it takes to complete objectives – primary, secondary, tertiary.  We do a peer group review, and get the experts to look at it before going into testing for what we think is going to work well, and what we think might not work so well.

Then we test what’s called the first publishable, which is typically one complete level of near-publishable quality with all of the controls and visuals in place – a vertical slice of the game.  This comes about halfway through the development lifecycle.  We test it with 10 people playing simultaneously, but separately from one another.

Following the test we have them complete a questionnaire.  We ask primarily about usability concerns, but also look at appeal to some extent.  We finish the testing by holding a roundtable with those people, though we’ll sometimes do individual interviews as well.

In the end, we document all of the issues we’ve found, make a top ten list from it and say okay, these are the things we want to fix for the next time round.  We’ll do testing 2-3 times in the lifecycle with 10 people each time, bringing in different demographics.

Alongside that, Mark Cerny had laid the foundations for testing at Sony in the US over the last 10 years.  They’re doing a lot of data capture as well, recording how long it takes to complete each level and looking for things like death clustering, which is when everyone’s dying at the same places in the game.  There’s now a strong culture of user testing at studios including Foster City and Santa Monica, and that’s been a big part of enabling them to increase the quality and relevance of the titles they produce.

And what’s the future of the EyeToy?

The PS3 now has its own USB camera — the PlayStation Eye, which was released with a game called Eye of Judgment that recognizes playing cards placed on a surface. 

We’ve also continued releasing EyeToy games for the PS2.   Two of them make use of an evolution of the EyeToy color tracking technology that identifies both motion and color, which gives a much deeper level of control.  One is EyeToy: Play PomPom Party, which is a cheerleading game that comes with a set of pom poms, pink and green.  Along with that is EyeToy:Play Hero, which comes with a green sword that you use to fight your way through the game.  On screen the sword can light up, take on flames, and so forth. 

And we’ve just announced EyePet, which is generating some excitement.  You might want to take a look at the video.

Games as production

In my presentation at the 2008 IA Summit, I discussed how many human activities can be understood as games, and benefit from adopting their characteristics. When we think of games as being specifically unproductive, we’re missing the opportunity to engage users at a level beyond what can be achieved in more conventional interfaces.

In fact games can serve as catalysts of production. Take fold.it, which is a puzzle game that challenges players to find the best ways to fold proteins. This is in fact among the most difficult problems in modern biology, as a protein can take on very different characteristics depending upon its shape. For example, mad cow disease is caused by proteins that already exist in the body, but which have been folded into irregular shapes that make them agents of the disease.

A screenshot from fold.it

People who play fold.it are actually contributing to science, because the game uses the real physical properties of the proteins as its rules. Players are awarded points for things like reducing the size of the protein efficiently, or turning certain types of molecules so they all face inward. The New York Times notes that it’s plausible that by playing this game, you could actually win a Nobel prize (even if you know nothing of biochemistry).

The real pioneer in the productive use of games, though, is Luis Von Ahn of Carnegie Mellon University. I’ll discuss his work in depth in an upcoming posting.

Final Fantasy User Interface

Over the past few weeks I’ve been playing a game for the PS2 called Final Fantasy XII (sounds dirty, but it’s not). I haven’t been a big fan of the series, but this one is really very good. I’m finding it especially interesting from the perspective of user interface design.

FFXII is a role-playing game, in the vein of Dungeons & Dragons. Generally speaking, these can be fairly complex endeavors. You control multiple characters, each of whom has its own attributes that evolve over the course of the game, and you need to attend to their complement of weapons, armor, and accessories, as well as micromanaging their individual actions in battle (I know, fun times). Creating a UI that allows players to sort through all of this complexity is a real challenge, but the design in this game is just fantastic. I expand on a few examples below.

Screenshot 1: Equip

Equip screen

This first screen shows the interface for equipping a character with body armor (click the screenshot to enlarge it). The list of available armor is on the right side, and colored dots show what each of the 6 characters is currently wearing. So character 4 is currently wearing bronze armor, and we’re considering upgrading him to diamond armor (the current position of the cursor). The list of attributes on the left shows what the exact effect of that upgrade will be, with the blue numbers indicating the new values. The character’s defense will go up fivefold, while his strength will increase by a more modest amount. Players can compare different armors just by moving the cursor up and down and noting the attribute changes each one affords.

Another little nuance is that certain characters are only allowed to wear certain types of armor. This is indicated by a circle under that character’s face. If the character can’t wear the armor, we see a dot instead of a circle. So we can tell at a glance that Basch is the only person who can wear diamond armor, while anyone is welcome to leather clothes. I like too that faces are used as column headings here: it’s both space-efficient and takes advantage of the fact that people are innately good at recognizing faces.

Screenshot 2: Gambits

Gambits

Final Fantasy XII includes a novel gameplay element called the gambit system, which is a sort of very simple programming language. Rather than laboriously commanding each character action by action in battle, you can set up a list of things they should do automatically whenever a certain condition is true. This is the gambit screen for a character named Penelo, who is currently assigned five active commands. The fifth one says that if any foe is nearby, she should attack it. But it’s superceded by all of the commands listed above it, which include healing poisoned allies or curing them if their health drops below 70%. It’s kind of funny that the condition statements are purchased or won over the course of the game, with the most helpful ones being more difficult to attain.

The system is flexible and very easy to use. You just pair a condition statement with an action, turn the gambit on and you’re ready to roll. Each gambit can be picked up and repositioned in the list to ensure the actions are executed in the proper order. The interplay of different characters in different battles against different enemies makes the gambits rather involved. It’s a great use of programming logic, allowing the player to assemble a sequence of if-then statements and then test out their effectiveness in different scenarios.

Screen 3: Battle

Battle head-up display

When in battle, an overlay provides a remarkable amount of information without obscuring your view of the action, as shown in screen 3. You’re able to keep track of each character’s current and maximum health, current and maximum magic power, available actions, next action, how soon an action will be performed, which character is leading, whether their gambits are active, whether a character is being targeted, the enemy’s health, and your character’s position in a map of the world. That’s really quite a lot, yet it doesn’t feel at all obtrusive. The economy of space strikes me as very Tufte.

Scratching the surface

That’s really just the beginning; I could go on at some length describing the menu structure, automated mapping, selection controls, cursor behavior, etc. There are so many good things happening in this game, many of them very small touches that are easy to overlook because they feel completely natural. Far from treating the game as frivolous, it’s clear that the designers put real care into this UI.

Amazon’s product rating system

A recent article on Boxes & Arrows discusses rating and review systems on sites like NetFlix, eBay, and Amazon. I think these types of systems are a fantastic basis for a socially constructed experience. They take advantage of the Web as a mass medium, they can include both quantitative and qualitative components, and they can be really useful to people.

The article tweaks Amazon a bit for its “most helpful review vs. most critical review” idea. Authors Alex Kirtland and Aaron Schiff argue that people writing those reviews can be evaluating a product on completely different criteria, so a reader can’t depend on them to be comparable.

I think they’re missing the real beauty in Amazon’s design. There are three components to it:

• A star rating. This creates a quantitative value that, in aggregate, gives users a basis for quickly comparing buyers’ overall happiness with a product.
• A review. This answers the question of why a particular rater gave a product a positive or negative review.
• A thumbs up/down rating. Assigned by people who read the reviews, and rate it as either helpful or unhelpful. This is a rating system on top of the rating system, and simply counts the total number of positive against negative votes.

The real kicker in Amazon’s design is that they use the thumbs up/down rating to juxtapose the most highly rated positive rating with the most highly rated negative rating. In effect, they create a top-level debate giving the best reasons for and against buying a product. What a clever bunch of designers they have working there.

I also don’t have the same issue with comparability that Kirtland and Schiff are citing. While positive and negative reviews of the same product will often discuss completely different criteria, it’s also implicit that they’re both rating the product as a whole. The reviews are helpful because they disambiguate the reasoning underlying the rating, giving the readers the opportunity to decide which criteria are most important to them.

Kirtland and Schiff’s article is a good piece, but I think it does a bit of a disservice to a very impressive social component.

Complexity in “Spore”

Today Andrew Hinton shared a great video of Will Wright demonstrating “Spore” for the Long Now Foundation. It’s an amazingly ambitious game, and showcases the potential for games to change the way you look at the world.

As software, I think it’s interesting how they’ve distributed the burden of complexity. On the one hand, the developers have created relatively simple algorithms that produce very complex, multivariate outputs. The game allows players to create their own creatures that will live in the Spore universe, body part by body part.  For example:

A product of Spore's creature creator

The interesting thing is that the little critters are imbued with this sort of mystical mathematical soul. Wherever you position legs on a creature’s body, the computer figures out how that would affect its gait, balance, and stride. When you map a stripes onto its skin, it crunches a few formulas to figure out how they should flow from body part to body part. Simple algorithms are processed by the player’s computer to produce very complex results.

On the other hand, the rich diversity of life in the game is a product of the collective efforts of the many people who will be playing it. Rather than limiting themselves to a handful of creatures they prepackage with the game, they leave it open to the masses of players to build a complex game experience for each other.

So in a way, Maxis is working in two fundamental raw materials: the computational power of the players’ computers, and the limitless capacity of human imagination. The human investment is amplified by the algorithms, and the result is something much more complex than any person or software developer could ever create on their own.

Generative design

My favorite presentation from this year’s IA Summit was Leah Buley’s “How to Be a UX Team of One“. She discussed the generative design methods employed at Adaptive Path. Typically design will start with one idea, which is then iterated it until it evolves into some ideal form. But generative methods focus on maximizing the volume and variety of ideas up front, then pulling together the best concepts into a single solution. Brilliant.

Catriona Cornett recently ran an exercise with Vanguard’s information architects to demonstrate a generative method. She timeboxed it to 10 minutes, and instructed us to each come up with 10 ideas for a problem in that time – bing bang boom. It was a really good experience. One thing I loved was that at the end of the 10 minutes, we had a ton of things to talk about. By contrast, I’ve been in meetings that went on for hours with nothing to talk about.

I did something along similar lines in my work on Vanguard’s search engine and intranet. In the process of evaluating the existing site and conducting a comparative analysis, a lot of ideas would just hit me. Without chewing on them too much, I wrote a short description of each on one of those really tiny 1″ post-its and stuck it to a whiteboard in my cubicle. At the end of each day I’d look over the ideas, grouping together similar ones and seeing if they spurred more ideas still when juxtaposed. After two weeks, the board was full and I felt extremely ready to start jumping into some snazzy designs. But AP’s methods are much more structured and group-oriented than this.

I strongly suspect generative methods are a flat-out better way to do design. If you haven’t seen Leah’s presentation, it’s definitely worth checking out the slidecast.

Testing search

My sense is that information architects generally have less influence over the core experience of search than they do over other aspects of a website’s interface, and I think that’s a bit of a shame since it’s such a critical resource for so many users. Sometimes a website’s search engine will be a user’s primary experience of the site’s architecture, but here we are on the sidelines watching as the implementation teams wield control over the search experience.

In our work on Vanguard’s search engine, we developed a collection of methods for testing the quality of the results it was bringing back. These became critical tools underlying optimization efforts and functional design strategy, and ingrained the IA’s deeply in the quality of search. I think that these testing methods can bring a lot of value to the user experience community, so I’ve proposed a pair of articles for Boxes and Arrows explaining them.

Both tests are premised on the idea that the search engine should return the best matches available for a user’s expressed interest at the top of the results page. That sounds really simple, but there are a lot of complicating factors that weigh into it:

• How closely does any of the available content actually match the user’s real interest?
• Has the user done a good enough job describing what he or she wants?
• How does the search engine parse the user’s input?
• What standards does the search engine use to determine what constitutes a best match?

As a result of these kinks in the system, search is inevitably much more hit-or-miss than we’d like it to be. So from the user experience perspective, the first questions we should ask are how often does the engine return the very best matches at the top of the results list, and how good are the matches it does return?

Well it turns out we can measure those things, and it’s not especially hard to do. You can put a number to it, then set objectives for how much you’ll improve over time. (The critical resource here is search logs — which is why I’m psyched that Lou Rosenfeld has a book coming out on just that.) In the process, the strategy shakes out too. You’ll find exactly which searches are underperforming, which among those are the most important to people, and the best ways to fix the system. That puts IA’s right back in the center of the search experience.

Those evaluation methods are what I’ll cover in the articles. I’d greatly appreciate any feedback submitted on the B&A site, and incorporate any input into the text when writing them.

The amazing WiiMote even makes julienne fries

Johnny Lee is a researcher at Carnegie Mellon who has been working on ways that you can use the Wii’s controller to create interfaces for PC’s.  If you haven’t seen his demonstration videos before, they’re definitely worth viewing.  Trust me, you’ll be blown away.

You can connect the WiiMote to any PC with a bluetooth receiver; the software you create from that point is bounded only by your imagination.  The results here are pretty remarkable.  This is a perfect, and particularly literal example of a game innovation extended to conventional UI’s.

Video game presentation uploaded!

I’ve updloaded a PDF version of my presentation from the 2008 IA Summit,  Extending the Gaming Experience to Conventional UI’s (6mb).  It’s shown in notes view, since much of the content was conveyed through speech with the slides serving as visual aids.  Download it here:

• video-game-experience_printable

Also, once I receive the audio file I’ll set up a slidecast that will allow the presentation to run automatically with the actual recording from the live session (fun!).  That’s really going to be the ideal way to view it, but it may take a little while.  I will post it here once it’s ready.

Game UI design discussion group

I’ve set up a discussion group for game user interfaces on Yahoo.  For those of you who’ve expressed interest in continuing the conversation on extending the game interface to conventional UI’s, this forum will give us a way to share in it collectively.

I’ll also be starting a game UI pattern library on Flickr.  More to come on that soon.