The fickle critic

benkaras — Wed, 03 Jan 2007 23:00:00 GMT

Personally, I have trouble using the "I liked it" scale ala Netflix. A movie that garners 5 stars one week only finds 4 another week. In a way, this arises from ambiguity present even within the "I liked it" scale. Did I like it because I was in the mood for a happy film? Or it an enjoyable serious film? Would I like to see it again?

What this scale needs are actions that go along with each rating:

- Loved it! I watch it every weekend instead of sleeping. I press my friends to come over and see it too. I have an imaginary copy of this DVD on my bookshelf. e.g. Seven Samurai or Rosencrantz and Guildenstern are Dead
- Liked it a lot. I suggest the movie to friends. I watch it occasionally when I'm in the mood, or not. e.g. Tsotsi
- Worth renting. I watch this one if my friends invite me over. I find myself forced to rent it just once to see what it was all about, and often find it superficially satisfying. e.g. Braveheart
- Blasé. I didn't finish the movie. These movies evoke nothing from me and I found it boring. I would rather go watch a car wash. I guide my friends away from these. e.g. Finding Neverland
- Yech. I have a physical hatred of this movie. This is usually a personal dislike of an otherwise popular movie. But I will not see this again even if friends are renting it. e.g. Garden State
- Why? I don't want to see this one. Just not my style. e.g. Click

You will probably point out that Garden State, Click, and Finding Neverland were all blockbuster movies. I'm not saying they were bad. I just want to make a statement about how I relate to them. Again, ratings have multiple interpretations, which is a perennial frustration of mine.

How do you interpret your ratings?

The split personality critic

benkaras — Mon, 01 Jan 2007 23:00:00 GMT

The 5 star rating control is misused in most applications today. How can I say that? What's more natural than selecting 5 stars for a movie or song? Actually, the intuitive nature of the rating star control is the cause of this subtle usability issue.

Ratings have multiple interpretations. Does 5 stars mean I loved a movie? Does it mean I would recommend it to my friends? Does it mean Netflix should send more movies like that one? Does it mean the cinematography/main actor/director alone was worth watching? Does it mean I would watch the movie if I'm sad/happy/with friends? 5 stars cannot imply all these things. And therein lies the trouble.

Intuitively, a 5-star control uses the "Liked it" scale. I expect most people treat it this way: 5 stars means I loved a movie. 1 star means I hated it. This, at least, works.

Meanwhile, software interprets these numbers. Netflix tries to send me movies that resemble other 5-star movies. iTunes tries to play 5-star songs more often than 3-star songs. The act of imposing this correlation introduces a tension into my decision. If I rate this movie as 5-stars, I now know that I'll get a bunch more like it, even if I didn't really want them.

An example helps: Braveheart is not a 5 star movie in my book. Yet I want to see more movies of that caliber. So should I give it the 3-stars on the "I liked it" scale, or the 5-stars on the "Send me more" scale? I liked Borat, but I usually hate that genre of movie. So I could give that movie 4 stars or 2 stars depending on the interpretation.

What is the proper way to incorporate a ratings system into an application if the end-user interpretation may differ from how the application makes use of the rating? I don't have a good answer other than to suggest that applications, including Netflix, should think very hard about how to incorporate the ratings system into their interface. There is a balance to strike between simplicity in design and accomplishment and with complexity and nuanced meaning.

The Great Flying Tortoise : Tagging

The fickle critic

The split personality critic