Saturday, December 19, 2009

Seeing Selection Biases, Not Treatment Effects

I'm not sure if this is a good thing or not but this is how my views typically diverge from the views of others: Where others see are more likely to see treatment effects, I am more likely to see selection biases.

Let's consider sports. A basketball team has just won four games in a row, and they are winning the first half of a fifth. The talking heads during the halftime show are talking about how they are using their "momentum." Basically they are arguing that the collective confidence advantage (i.e., the treatment effect) from winning the first four games is what is allowing them to win the fifth.

I admit that this might be a very slight factor. But much more important to me is the selection bias that comes from having won those first four games: They must have been doing something right! They probably had good players, a good strategy, or something else going for them. Maybe, in the case of the NBA, they have been paying off the refs.

There are of course different forms of treatment effects and selection biases. Hypothetically a Bayesian truth-seeker shouldn't "usually" stand on one side or the other, but I tend to weigh the selection bias much more than most folks, and I wonder why that is. Where do you stand?