How To Find Which Basketball Stats Matter

Dave Johns today summarized whether basketball stats can actually tell how much a player is helping his team. It seems that there are two key problems:

1) The one stat we know must be good in the long run, adjusted plus-minus, is too noisy to make good inferences based on short run data.

2) The stats that we do have lots of short-run game data on, like points, rebounds, and FG%, we don't know how to interpret in terms of how much they actually help the team. For example, Kevin Love is piling up boards, but does that actually help the Timberwolves win ballgames?

One approach to solve these problems would be to use a large set of training data for both adjusted +/- and summary stats, spanning many years. For each player and each game (or even each quarter), you try to use the statistics from (2), like points and rebounds, as features to try to predict the player's adjusted +/- in that time period. Some of the statistics will be able to predict the +/- really well, whereas others won't. So going forward, we'll be able to say which of the stats are good short-run proxies for long-run +/- and which are not. That's it. It will be beautiful.

For a few minutes I thought about trying to do this myself, but I couldn't find easy enough access to raw +/- data. That is to say, screen scraping nba.com does not sound like much fun. If anyone knows of a nice and clean data set, holla acha boy.

Frankly I was surprised Johns didn't mention this approach in his article (thus this post), but I assume that it's what teams using bball sabermetrics are doing. The approach is similar to the netflix prize or to many articles in machine learning, like Burstein et al '09, who predict the function of proteins based on training features. I'm pasting their figure 1 below for a schematic of the process, although note theirs is binary whereas our classification system would be continuous. Think of the "features" as either simple stats like assists, or more complicated ones like under-/over-shooting, and think of "classification algorithms" as either naive things like "how many points did the player score?", or more complicated things like John Hollinger's PER.

doi:10.1371/journal.ppat.1000508

Kobe In Crunch Time: Efficiency vs Predictability

Henry Abbott breaks it down, showing that Kobe isn't that legit in late game scenarios, despite popular opinion, and his own. But here's what Abbott is missing. Kobe probably would have the highest probability of making the last shot of any player in the NBA, if defenses paid equal attention to everyone. Kobe gets his rep for what he could do in this situation, and that matters for something.

The problem is that defenses adjust for this ability. "Doubling-teaming" isn't always a binary yes or no--it's also about being more apt to help off your man in case of a drive, hedge on off-ball screens, and keep Kobe in your plane of vision. Of course, all of this hinders how well the defense can guard Kobe's teammates. So what Kobe should do is look for them. But he doesn't, because he gets too caught up in thinking about how efficient he can be, and forgets that this makes him predictable.

This is a pretty clear demonstration of the trade off. Crunch time offense is a game, and in any game you want a mixed strategy. That means that with some probability you'll do one thing, and with some probability you'll do something else. If you're the Lakers and you commit to Kobe always shooting, or even assign it a higher probability than you should, the other team will be able to take advantage.

Bill Simmons On Uniqueness

He gripes in his most recent chat:

[I]t's funny to take heat from soccer fans that I'm a bandwagon Tottenham fan. I mean... of course I am. I am something like 17 months into this thing. But what I don't get about sports like the UFC/soccer/NHL (and even baseball with the saber community towards people who just like baseball and don't want to dive into the stats) is why the diehards are so protective/condescending towards casual fans. What's the goal there? To just drive away everyone who might like the sport and want to become more of a fan? I think there's a difference between local bandwagon fans (like the Pink Hat Red Sox fans) and "I am starting to like your sport, I genuinely want to follow it and learn about it" fans and it would just seem like the diehards should embrace the latter group. Or am I crazy?...

I do think that diehard fans tend to exclude newcomers - the same phenomenon works with music, you always want your favorite band to be the little band that not everyone knows about (and never have them get to the U2 level). I think Kings of Leon are a good recent example of this and even the band members hated that they became "mainstream" because it brought in fans that they didn't necessarily want. The best breakdown of this was in Steve Martin's book about his standup career when he talks about becoming hugely famous and how he started dreading doing his shows because he felt like people weren't there for the right reasons. It's an interesting topic I think.

Once more people join a given group, affiliation loses lots of its signaling benefits due to diffusion. So it makes sense that fans who have invested in a team / band would discourage newbies from joining.

But why do the musicians / comedians themselves not want more followers? That's a bit trickier. Simmons mentions Steve Martin being wary of his new fans, but the same is even more true of Dave Chappelle. When he skipped out on the third season of his show, he turned down millions.

Any theory to explain this phenomenon also has to account for the fact that neither athletes nor academics tend to express these sentiments. Jordan, Manning, Hawkings, Volkow--they do not worry about "going mainstream." Indeed they tend to welcome it.

Perhaps the musicians / comedians are signaling loyalty to their core constituents, and the real emotional and financial costs they pay in doing so just makes their signaling more credible. So it seems that the less badly you want to go mainstream, the more your pursuit is about signaling as opposed to results.

Lessons From FIFA's Ineptitude

Today, two calls in the World Cup (a missed goal here, a missed offside here) were clear mistakes and require us to assign blame. There are a number of ways to respond. The comforting but ultimately misguided tact is to blame the individual refs. It is statistically unlikely that the majority of refs would be so much better than the refs today.

Or, one might blame the goals on FIFA's failure to allow instant replay. Replay definitely would have helped. But that tact also strikes me as not general enough. Instead, I see the problem as yet another example of the status quo bias against new technology. To me, inertia and rationalization are the real culprits.

The only reason that FIFA's decision making seems so dumb is because the games are public enough to allow for constant scrutiny. But imagine if millions of fans were watching as yet another patient's info was copied down by hand instead of being uploaded to a time-saving server. They too would be vitriolic. Or, imagine if announcers called the action as somebody wasted minutes every day by visiting websites individually instead of setting up an RSS reader. They too would be unimpressed.

So, sure, condemn FIFA for its failure to adapt to useful new technology. But be careful not to frame your blame as for a singular case of stupidity. Indeed, we can all fall victim to the status quo bias against adopting new technologies.

Short / Long Term Effects Of Better Competition

Jonah Lehrer's article in the WSJ presents some counter-intuitive data: golfers tend to do worse when they go up against a great player like Tiger. Apparently, "whenever Mr. Woods entered a tournament, every other golfer took, on average, 0.8 more strokes." This and other one-shot experimental data goes against the common observation that competition increases performance. So how do we resolve the paradox?

Here's one explanation. In the short run (like one golf tournament), better competition makes us overly anxious, sometimes leading to paralysis by analysis. But in the long run this extra focus might make us better, because it forces us to take our games to the next level, assuming that we make it over the "dip." We come to expect the anxiousness in any given tournament and practice harder to overcome it. Any thoughts?

Rating Interceptions Probabilistically

Just as it's hard to emphasize the process instead of the results in medicine, it's hard to correctly weight the role of luck in determining the outcome of a given play in sports. But, especially in sports with small sample sizes like football, it is crucial to isolate just the effect that the player himself had on the play.

One way to do this is by having watchers rate the expected value that a player contributes to a given play. For example, instead of just seeing a binary 0 or 1 interceptions on a given play, a continuous rating system should give more information.

So, give 0.5 expected interceptions for a throw that would have been picked off 50% of the time by an average defense / rest of offense (as wideouts play a role, too), 0.05 for a ball that probably wouldn't have been picked off unless the wide receiver hadn't tipped the ball off his hands, or 0.9 for a ball that was basically thrown right to the defender. Of course these ratings will be rough and subjective. But by and large they will be better than the status quo raw stats.

For example, this interception would be rated as ~ 0.9 expected interceptions, this one would be rated as ~ 0.02 interceptions, and this non-interception would be rated as ~ 0.6 expected value interceptions. You then divide the number of expected value interceptions by the number of passing attempts, possibly regress for the style of offense, and you'll have a much more reliable measure of the propensity of the QB to throw an interception than the raw interception stats.

This is what most scouts already intuitively do. But this rating method quantifies it so that the knowledge is more transparent and accessible. I'm willing to bet that this expected value measure, if adopted, would predict the QB's actual interceptions in the following season better than his actual interception total from previous seasons. Is anyone willing to bet against me on that?

This generalizes to other sports statistics, like the expected outs from an outfielder's throw to a runner speeding to home plate, the expected goals of a shot in soccer, or even the expected points from a jump shot in basketball (i.e., air balls are like 0, and points that hit off the rim more than once before going in are less than a swish). It generalizes beyond sports too, to emphasizing the process over the results as an important part of any performance analysis.

Mark McGwire and the Self Serving Bias

He's admitted to steroid use but denies that they helped his performance:

McGwire insisted the performance-enhancing drugs he used did not actually enhance his performance. The dosages were too low and his physical ability too divine, turns out, for the drugs to have an impact on his body, particularly as it related to his hitting. “I was given the gift,” he told Costas, “to hit home runs.” He said he would have hit every single one of them had he never injected a drop of anything. “Absolutely,” he said. “I truly believe so.”

Some sportscasters are calling him out for lying in order to boost his chances of getting into the HOF, or something. I do not believe that he is consciously lying, in part because I have read about how powerful the self-serving bias can be. For example, children given methylphenidate attribute their success on impulsivity tests to effort and ability much more so than to medication, even when a double blind design ensures that in fact the medication leads to significantly fewer errors than placebo.

This is clearly adaptive--attributing success to internal factors builds your confidence and helps you perform better the next time. In the case of McGwire, it allowed him to use steroids off and on while minimizing anxiety that he'd perform worse without them.

As with Kobe, I wonder: Is McGwire's profligate self-serving bias merely an aberration? Or is it in fact one of the main reasons that he was able to have so much success in the first place?

Seeing Selection Biases, Not Treatment Effects

I'm not sure if this is a good thing or not but this is how my views typically diverge from the views of others: Where others see are more likely to see treatment effects, I am more likely to see selection biases.

Let's consider sports. A basketball team has just won four games in a row, and they are winning the first half of a fifth. The talking heads during the halftime show are talking about how they are using their "momentum." Basically they are arguing that the collective confidence advantage (i.e., the treatment effect) from winning the first four games is what is allowing them to win the fifth.

I admit that this might be a very slight factor. But much more important to me is the selection bias that comes from having won those first four games: They must have been doing something right! They probably had good players, a good strategy, or something else going for them. Maybe, in the case of the NBA, they have been paying off the refs.

There are of course different forms of treatment effects and selection biases. Hypothetically a Bayesian truth-seeker shouldn't "usually" stand on one side or the other, but I tend to weigh the selection bias much more than most folks, and I wonder why that is. Where do you stand?

Variation in Sports Highlights

Now that the NBA season has started and I have easy access to cable, I've been watching some Sportscenter. I love the highlights, but there's so little suspense, because basically you know that every shot they show is going to go in. They should throw in some missed shots, like 10% of the time, just so that the viewers are still a little bit on their toes.

A Case Study in Secret Telling: Tim Donaghy

The former NBA referee has written some remarks about the biases of NBA refs, that, if true, would be quite damning. Apparently some refs bet on who would be the first to call a technical foul in a given game! More troublesome are the systematic biases, like the favorable treatment of Kobe or pushing a playoff series to seven games on purpose. But are his allegations likely to be true? Let's evaluate the evidence:

In Donaghy's favor: The details he lists seem too exhaustive and specific to be completely fabricated; Heat-Mavs, 2006 finals; former FBI agent Philip Scala's endorsement; the fact that the NBA has a huge public relations incentive to make his allegations appear false; if any of the specific allegations he made could be proven false it would be a huge blow to him, yet none of them have been.

Against Donaghy: Random House/Triump books canceled publication of his book after an apparently independent review process; the NBA denies his allegations; he stands to make a profit from book sales and potentially a higher profit if the claims are more outrageous; if "everyone was doing it" then he looks less bad to the public in comparison; the NBA claims that the FBI found no other "criminal conduct" among other referees in its review, although some of the stuff he alleges may not be criminal per se as much as it would be unfair.

When Jose Canseco made his allegations about steroids in baseball they were at least verifiable in the long run, as players could recieve blood tests for steroids. Donaghy's biggest problem seems to be that he has no similar way to prove his points--unless he can convince other referees who also have an in to corroborate them.

Tim Donaghy could either represent a case study in libel or a case study in how a large group (i.e., the NBA admin) could keep a secret. I think it is more likely the latter with a probability of about 80%. So I'd bet 4-1 that yes, most of what he says is true, if you could find some way of verifying it. If he is indeed telling the truth, the fact that Random House canceled publication of his book shows how defecting from the group and revealing a secret via a tell-all book is much harder in practice than in theory. Telling a secret really is hard. This makes other conspiracy theories seem more likely, too.

Contact Sports and the Brain

Malcolm Gladwell's new article about head injuries in NFL and college football players is gripping as always. Long term brain trauma has long been known to be prevalent in former boxers, too, but since the sport remains popular people still engage in it. Hits are worse when individuals aren't prepared. There's a debate in the academic lit as to how big of an issue this is for soccer players. Concussions among pro's occur at a rate of ~ 0.5 per 1000 player hours, usually due to elbow to head contact or head to head contact in "heading duels." I can tell you that when I used to play soccer I dreaded heading the ball back off of long goalie punts. Apparently the damage is worse when you fail to time it right and tense up your neck correctly. Here's one breakdown for how players were concussed during basketball:

How do you get a concussion running without the ball or during warm-up!?

Binge drinking should probably be considered a contact sport too, because the odds ratio for brain trauma among individuals with a BAC of 200+ mg/dl (around 10 drinks for 190 pound person) rises to 9.23! This is mainly due to a higher risk for assault (likely just fights in general), biking accidents, and simply falling. Be careful out there.

Kobe and the Self-Serving Bias

In the post-game interview tonight following the third game of the NBA finals, Kobe Bryant expressed his frustration over messing up in one of the crucial last plays. Loosely, he said, "I'm used to coming through in these situations... my team trusts me to come through in these situations." So there is little doubt that he thinks that he's good in late game situations.

However, the data does not confirm his intuition. 82games.com recently posted an analysis of game winning shots (HT: Simmons) over the past six years and Kobe's 25% field goal percentage is tied for 71st out of the 76 players analyzed. That means that, relatively speaking, he's pretty bad in the late game. The simplest explanation for the discrepancy between his perception and reality is the faithful self-serving bias.

Nevertheless, he is clearly a pretty good player overall, rated via player efficiency ratings as the sixth best in the league this year. Now, is his overall greatness the reason that he is unable to see the facts as they are? Or, conversely, is his self-serving bias a prominent cause of his overall greatness? Or, is it neither, and he just so happens to be an arrogant asshole? These are the questions I stay up late at night pondering.

A Measure of Referee Directed Irrationality

I've long argued that one exemplary case study for irrationality can be found in individual's responses to referees. My personal experience as a youth soccer referee led me to believe that if your measure of success is the reactions of players, parents, and coaches, there is simply no way to succeed. On one occasion I recall a parent yelling at me after the game about some stupid call on the way back to the parking lot, following an upperhouse (i.e., not the highest level) under-10 soccer game. Of course, I may have simply been a bad referee, but that strikes me as unlikely because I witnessed it happening to my fellow soccer referees as well.

You can measure an individual's level of irrationality towards referees in real life based on how they respond to referees in sports video games. You will find that, especially in basketball, people often complain about the calls being made. Of course, there is no a priori reason for the game's algorithms to favor one player over another, so we can prove that this is completely ridiculous. Whereas in real life we must face the confound of the complainers actually having a point. If you find yourself or your friends complaining about the referees in sports video games, that should be strong evidence to you of their irrationality in this regard, and you should feel free to generalize this both to real life responses to referees and indeed other random walk based outcomes as well.

The Opportunity Cost of Hard Work

Razib has some fascinating thoughts expanding upon David Brook's recent tautology of a column:

You read a book like Until It Hurts: America's Obsession with Youth Sports and How It Harms Our Kids, and you understand the opportunity cost of focusing on a task where those with innate talent can easily beat those who have spent years honing their skills at the upper levels. Not that greatness in sports doesn't require hard work, obviously it does, but among the best the small hardwired differences mean that those who aren't as fast, strong or quick by their nature will lose.

More generally, the solution to all of life's problems cannot be simply more hard work. As Seth Godin points out in The Dip, sometimes you should quit, to focus on other stuff. I've found it to be more fun to focus on the niches in which I have a comparative advantage.

Best Column I've Read All Year: Moneyball Meets Mr. Battier

Michael Lewis's new column in the NYT about why Shane Battier is underrated combines three of my favorite things: basketball, statistics, and Michael Lewis. I don't want to focus on the outcome, I just want to note that you have a very high probability of enjoying it. Unfortunately there are too many nuggets to quote just one. I hate it when bloggers say this but you really must read the whole thing.

How long would the soccer-ball-in-afro trick work?

As I detailed in My Contribution to Humanity, the soccer world is due for a shake down. One day, a tall player will grow a huge afro, cut a round hole in the middle of it, juggle the ball into the hole, and run across the field unopposed. This will happen eventually, all we can hope is that a US player does it first.

How long will it take before the quasi-nefarious tactic is banned? Before recently there was little historical precedent to use for predictions.

But earlier this year hockey player Sean Avery distracted the goalkeeper face-on, a hitherto unused tactic that technically speaking broke no rules.

The hockey referees did not call any infraction, and it is hard to see what exactly they could have called, although onlookers agreed that it was highly unsportsmanlike. However, the very next day the NHL created a rule colloquially known as the Sean Avery Rule banning face-on distraction of the goalie.

So the historical precedent is that the trick would work for one game and then the world soccer community would ban it. Now, soccer is more of an international game, and there is a lot of bureaucracy involved in any international affair. Just look at the UN. So it is possible that the tactic could be useful into the second or third games, but beyond that I don't see much hope.

Bottom line: Aspiring soccer players should continue to work hard on their overall game, but they should also devote some effort to growing really long hair that can be sculpted with enough gel into an afro. The Sean Avery incident indicates that this tactic would work for between one and three games, meaning that there is an almost infinite first mover advantage.

The curse of height

You may have heard of the curse of oil, also called the resource curse more generally. It is the idea that for many countries access to oil is more of a hindrance than a blessing, due to the corruption, conflict, and volatility that often follow from its discovery. A cursory glance at many of the major regions where there is civil unrest (Sudan, Iraq, etc.) lends credence to the hypothesis.

But what I'm interested in is more of a local level. Can individuals fall victim to a resource curse?

I think that in most cases it is probable. Take for instance extremely tall young men and women, who are pressured beyond belief into playing competitive basketball. Once they take the court, they are often ridiculed for being clumsy or not being able to dunk the ball in traffic, which of course every 5'10" person would be able to if they extrapolated their height while miraculously maintaining the same athleticism and coordination.

Excelling at sports is potentially commendable in its own right, and exuberantly tall individuals will have at least a leg up. But many teams will take on otherwise unqualified tall players as "projects" or simply to intimidate the opposing team in warm-ups, and they often wither away on the bench. This certainly does not do wonders for self-esteem, especially when fans have high expectations.

Bottom Line: Much like the curse of oil, being four standard deviations above the average range of height can end up as more of a curse than a blessing. We should adjust our expectations of these individuals accordingly.

Behavioral sportonomics in NBA contracts

Sports teams often explain that they had to overbid on a player because they were worried about a fan backlash against management if that player was let go. While discussing the NBA, Bill Simmons explained why he thinks this analysis is flawed,

When do you think sports franchises will break out of the "We need to him put butts in seats!" mindset and realize that winners are the only things that put butts in seats?

Simmons explains overbidding as a tactical error, but perhaps a simpler explanation could be found in human psychology. Loss aversion posits that individuals will be more averse to losing an item than they would be excited about gaining that same item. Why can't we apply it to the contract decisions of sports teams?

One way to test this hypothesis would be to see whether new GMs who did not sign the player themselves are more likely to let a player go than an long-tenured GM who actually signed or drafted the player himself. If an old GM is more likely to go to all ends to resign a player than a new GM, that would be evidence for loss aversion. Also, teams that rely on pure metrics might be less likely to fall victim to this problem than teams that rely on their "gut instinct," but that would be harder to quantify.

What do you think of the new science of "behavioral sportonomics?"

Focal foul calls in biased refereeing

It is naive to assume that referees only want to make the correct calls. What they really want is for all players and spectators to think that they are unbiased. The typical optimization strategy is to appear unbiased, not to actually be unbiased. This explains why in so many games two teams will have the same number of fouls called although it is highly likely that one team is objectively fouling more.

Once you accept this model, one surprising prediction arises: it is actually beneficial in the long run if your team has a bad, focal call made against them. One horrible call probably won't mean much in terms of the overall score, but the referee will have to "make up" for that call for some time. The more focal the call, the more the referee will "make up" for in order to appear unbiased.

For example, last week I was playing a competitive 3 on 3 game of basketball and one of my opponents made an awful call (it was self-refereed but the same principle applies). He claimed that I had traveled when in fact I had clearly not moved my pivot foot. My teammates and some spectators went wild with indignation, but I was enthused about the call. When my opponent offered to reverse it, I insisted that the call stand, ostensibly out of honor but really because I knew it would help my team's cause.

Since that one focal call had been made against me, I was able to get away with a litany of small calls or noncalls in my favor. The very next play I drove to the basket and probably took one extra step as I scooped in the layup. No call. Buckets.

Why aren't there more knuckleballers?

Economic theory posits that in the long-run no company can earn a profit in a competitive environment, because somebody else will come in and copy them. But Tim Wakefield has been a fairly successful pitcher for the Red Sox as a knuckeballer for the past 6 years, eating innings and more importantly earning a major league salary at the age of 41. He has beaten or tied the league average in adjusted ERA for the last eight years. But, there aren't many knuckleballers: besides Wakefield, only R. A. Dickey (who plays sparingly for Texas) appears to be currently pitching with this style. If there are any parents out their who want their kids to play in the pros, I have the following advice: teach them to kick footballs, help them to grow an afro, or tell them to throw a knuckleball.