Thursday, July 24, 2008

Angry voters on imdb

Nick thinks that imdb is flawed because so many people give 1's to movies. He views this as evidence that the votes are subjective, and therefore that the system is ruined. In his mind, people are gaming the system out of their personal beliefs, instead of voting based purely on the quality of the movie, and it "completely defeats the point."

I agree (partially) with his premise, but disagree with his conclusion. First, you need to look at why voters would game the system. Is it because they are involved in the movie business and are trying to make a profit based imdb scores? This is highly unlikely, given the large mass of votes and the relative dearth of technologically literate actors.

The next possibility is that they vote movies down because they disliked the movie, or disagreed with what it stood for. So they will give a movie that really isn't all that bad a much lower score than it "objectively" deserves. In essence, they are punishing the movie too much for some small flaws. To me, this scenario is much more likely.

Now, if you accept the second scenario, then we begin to witness the beauty of imdb. Even though these individual voters may be subjective, they are subjective in varying ways, which means that their opinions are independent of one another.

Let's say that you watch Pulp Fiction and you hate the scene with the gimp, because you think it's distasteful. So you give the movie a 1, even though somewhere in your head you recognize that the movie probably deserves a higher score.

If you were the only one rating the movie, this would spell trouble. But since you are just one voter of many, this vote only lowers the arithmetic mean score by some fractional amount.

Moreover, this is not entirely a bad thing. If lots of people hate a movie (and award it a 1), that's not a very good sign for the movie! It should not be in the top 250. This is why The Titanic has dropped, and good riddance I say.

As long as individuals are not attempting to game the system out of self-interest, their low scores represent a valid, independent opinion on the movie. Therefore, votes of 1 (on a scale of 1-10) don't represent a system failure. Instead, they allow users the ability to punish a movie if it is highly annoying or unpleasant.

Bottom Line: The fact that some voters give movies a 1 does not screw up the whole system. One vote has a small effect, and moreover, subjective individuals can sum to an objective population.