Any serious athlete or fan has had multiple experiences dealing with referees (or umpires, or whatever) that they thought had a vendetta against their team. In fact, you probably have dealt with a couple of refs whose faces you've have memorized in case you see them in a dark alley 10 years from now. We've all been there.
Having been on both sides of the equation, as a soccer referee, I can attest that it's not as easy as it looks. You have so many different things to keep track of--was that offsides, does that slide tackle warrant a yellow card, is that cute older sister in the stands talking to her boyfriend or just some dude, and all the while you just want the game to be over with so you can get paid.
Unfortunately, nobody takes any of that into account. I had parents--of 9 year olds!--yell at me from across the field well after the game, when they were walking back to their cars. One minute its, "here, Johnny, have a couple of orange slices, make sure you stay hydrated slugger," and the next its, "hey ref, you suck, take out your whistle next time, you blew the game!" It's crazy. But what you learn to realize as a ref is that everybody loves to vent some anger by getting out of control now and then. Reason #45357 why an iPod is the best money you've ever spent.
Anyway, I've come up with a few biases to try to explain why people get so angry at the refs at sporting events (all definitions from Wikipedia, the premiere source for information, whether broad academia is willing to admit it or not):
"Illusion of control — the tendency for human beings to believe they can control or at least influence outcomes that they clearly cannot."
This is a big one in sports, where people at home and in the stands believe that they have an impact on what happens in the games based on their cheering, whether or not they watched a given play, or whatever. While it's fun to feel like part of the team, it probably means that you take any referee errors a little bit too personally.
"Impact bias — the tendency for people to overestimate the length or the intensity of the impact of future feeling states."
When a referee makes a questionable decision in the game, people have the right to feel a little bit upset. Bill Simmons has a famous Levels of Losing column in which he pretty much talks about how awful it is to lose, and he's right. But generally nobody seems willing to admit that there will be another play, and another game. If they were, they might not care about a couple of bad calls, even if the ref does clearly need LASIK eye surgery.
"Actor-observer bias -- the tendency to attribute their own behavior to their circumstances, but tend to attribute other people's behaviors to their dispositions."
When a ref makes a call, we automatically assume that it must be their fault--they weren't paying attention or they are racist, even when it's just as likely that the ref had a tough angle or was forced to make the decision on the move in a split second.
Any other reasons why we chastise referees so much?