Amazon’s product rating system

A recent article on Boxes & Arrows discusses rating and review systems on sites like NetFlix, eBay, and Amazon. I think these types of systems are a fantastic basis for a socially constructed experience. They take advantage of the Web as a mass medium, they can include both quantitative and qualitative components, and they can be really useful to people.

The article tweaks Amazon a bit for its “most helpful review vs. most critical review” idea. Authors Alex Kirtland and Aaron Schiff argue that people writing those reviews can be evaluating a product on completely different criteria, so a reader can’t depend on them to be comparable.

I think they’re missing the real beauty in Amazon’s design. There are three components to it:

• A star rating. This creates a quantitative value that, in aggregate, gives users a basis for quickly comparing buyers’ overall happiness with a product.
• A review. This answers the question of why a particular rater gave a product a positive or negative review.
• A thumbs up/down rating. Assigned by people who read the reviews, and rate it as either helpful or unhelpful. This is a rating system on top of the rating system, and simply counts the total number of positive against negative votes.

The real kicker in Amazon’s design is that they use the thumbs up/down rating to juxtapose the most highly rated positive rating with the most highly rated negative rating. In effect, they create a top-level debate giving the best reasons for and against buying a product. What a clever bunch of designers they have working there.

I also don’t have the same issue with comparability that Kirtland and Schiff are citing. While positive and negative reviews of the same product will often discuss completely different criteria, it’s also implicit that they’re both rating the product as a whole. The reviews are helpful because they disambiguate the reasoning underlying the rating, giving the readers the opportunity to decide which criteria are most important to them.

Kirtland and Schiff’s article is a good piece, but I think it does a bit of a disservice to a very impressive social component.