Wednesday, September 17, 2008

Nassim Taleb on uncertainty in statistics

In his recent essay in Edge, the never dull Taleb claims that you shouldn't trust statistical methods that were created before the invention of computers:
[These methods] are based on "variance"/"standard deviation" and terms invented years ago when we had no computers. One way I can prove that anything linked to standard deviation is a facade of knowledge: There is a measure called Kurtosis that indicates departure from "Normality". It is very, very unstable and marred with huge sampling error: 70-90% of the Kurtosis in Oil, SP500, Silver, UK interest rates, Nikkei, US deposit rates, sugar, and the dollar/yen currency rate come from 1 day in the past 40 years. This means that no sample will ever deliver the true variance. It also tells us anyone using "variance" or "standard deviation" (or worse making models that make us take decisions based on it) in the fourth quadrant is incompetent.
It is true that computers have fundamentally changed the ballgame, and this analysis is good fodder for those who think introductory statistics is a little bit archaic. Where I disagree with his analysis is not whether we are currently able to model this "fourth quadrant," but whether we will one day be able to. I believe in progress, one non-bailout at a time.