True altruism in door holding

Society widely considers it polite to hold doors open for people as they walk through the door. We consider it such an inherently kind act that not offering thanks to the door holder is considered highly rude.

But just because society considers this a kind act does not make it so--society has been wrong before and will be wrong again. And, some critics have wisely pointed out that if you hold the door open for somebody who is a long distance away, you make them feel guilty and awkward. So the altruist now has to correctly estimate both the distance and speed of the walker, and factor that into his decision to hold the door open.

But the true altruist can do even better. If you anticipate that somebody behind you will be at that awkward distance where you don't want to open the door, you should slow down so that they will reach the door just after you enter. Pretend to check your cell phone, retie your shoes, itch your scalp--just make sure that it is timely and surreptitious.

At some point you will still have to decide if slowing down to hold the door is worth your time as an altruist. Remember that altruism comes in many flavors, be it sending aid to impoverished nations, developing a cure for a disease, or sitting at your computer writing a blog post about altruism that will be read around the world.

Filtering out fanatical politicos

Many people have strong convictions about politics, and I see nothing wrong with that. Strong convictions are more rational than they are generally given credit for. But they become counter-productive when the politico is no longer willing to listen to outside viewpoints and update their opinions to new information.

This effect can be difficult to gauge in real time, but I have a different question to ask that thrusts at the same point. Ask, "would you prefer that the country receives a negative short run shock in some key indicator, so that your candidate has a better chance of winning?"

Obama supporters might prefer for the economy to continue to fall (even slightly) to boost his chances of winning. McCain supporters might hope for a (small) terrorist attack on a US embassy abroad to boost his chances of winning.

If somebody hopes for a negative short run turn of events for the country just so that their candidate will win, they have gone over the edge. They may argue that over the long run it will be for the benefit of society, but they are fooling themselves. There's no way they can accurately predict future events with that kind of precision; if we can avoid an undeniably negative event now we should do so without hesitation.

Of course, politics isn't about policy.

Empathy has no backstop

Last year my friend Ben blogged about which traits increase in value "nonstop," meaning that more of the trait is always better. He correctly pointed out that few traits fit this criteria. For example, it is good to have more flexibility up to a point, but too much and you'll never get anything done. To him, persistence was one the few traits without this backstop.

I think that empathy is another one. Sure, you can be too kind (telling someone "you're welcome" all the time gets annoying), but you can't be too empathic. Indeed, boosting your empathy is undoubtedly a positive personality change, since it will put you more at peace with your own place in the world.

Of course this is largely semantics but it's still a stimulating thought exercise. Can you think of any other such traits?

(Thanks to Joey D for sparking a conversation about this idea.)

Random chance fuels innovation

From Wikipedia's page on mustard gas:

In 1943, during the Second World War, a US stockpile exploded aboard a supply ship that was bombed in an air raid in the harbor of Bari, Italy exposing and killing thousands of civilians and 628 Allied troops. The deaths and incident were classified Top Secret for 55 years and labeled as a mystery illness. It was noted by the U.S. Army's medical workers that the white cell counts of exposed soldiers were reduced, and mustard gas was investigated as a therapy for Hodgkin's lymphoma, a form of cancer. Study of the use of similar chemicals as agents for the treatment of cancers led to the discovery of mustine, and the birth of anticancer chemotherapy.

A random event allowed for a natural experiment which led to a new treatment for disease. It is random that I stumbled across it, and it is random that you have now read it. It's random--but that doesn't mean it doesn't matter.

The thin tail of imdb's top movies

The average score for movies on imdb is 6.7, and all of the movies on imdb's top 250 hold a score of 7.9 or above (adjusted), which means that they are truly the cream of the crop. But it is intriguing that even among these top movies certain ones are able to stand out as the best. Here is a frequency diagram that shows the distribution:

The general trend throughout is that the higher the score, the lower the number of movies are able to achieve it. Note that I chopped off all of the 7.9s because the scores ended there so it was impossible to know how many 7.9s there are on imdb as a whole. It is truly amazing to me that some movies are rated so much more highly than all others.

Since there is no rating system like imdb for most forms of art, this distribution raises some questions. Does this distribution apply to all art forms or just movies? If it just applies to movies, what makes them so special--is there less cognitive dissonance?

If it applies to other art forms as well, what does that say about our tastes? Perhaps there is something inside of us that is able to know when art is of high quality, even when we are unable to put our thoughts into words. And that is where a pure numerical rating system shines the brightest.

On boredom

Marcus Aurelius says we shouldn't be bored, because we can always retreat into our own mind. But we are not all philosopher-kings; there are certain things that we must listen to even though we don't want to. Consider two case studies:

1) A student who must pay attention in class even though he knows the majority of the material. The trouble with spacing out, he figures, is that the teacher may at any time say something he doesn't know and he wouldn't want to miss it.

2) When politeness dictates that you must listen to a full story even though we can already see where it is going, or experience tells us that the punch line will be a let-down.

I think we can overcome much of our boredom in both of these circumstances. In the second case, remember that politeness is overrated--if you are bored with a story then your fellow interlocutor may in fact be bored in telling it. Don't be afraid to interrupt if you think of something more interesting to discuss. Cutting off the convo early can be mean, but moving it in a different direction can be a net positive.

In the first case, I think we overrate both our own memory and the potentially utility of whatever the upcoming fact may be. If you space out you probably will not miss a "eureka" moment of clarity, as these are uncommon in the real world. Many subjects are simply a collection of small facts that add up to a coherent whole. If you don't care much about the coherent whole, there's no need to trouble yourself with the small facts. Space out more, feel less guilty about it, and you'll end up more productive.

(Full disclosure: I wrote this post word-for-word while sitting through a physics lecture in Sanders late this afternoon.)

Nassim Taleb on uncertainty in statistics

In his recent essay in Edge, the never dull Taleb claims that you shouldn't trust statistical methods that were created before the invention of computers:

[These methods] are based on "variance"/"standard deviation" and terms invented years ago when we had no computers. One way I can prove that anything linked to standard deviation is a facade of knowledge: There is a measure called Kurtosis that indicates departure from "Normality". It is very, very unstable and marred with huge sampling error: 70-90% of the Kurtosis in Oil, SP500, Silver, UK interest rates, Nikkei, US deposit rates, sugar, and the dollar/yen currency rate come from 1 day in the past 40 years. This means that no sample will ever deliver the true variance. It also tells us anyone using "variance" or "standard deviation" (or worse making models that make us take decisions based on it) in the fourth quadrant is incompetent.

It is true that computers have fundamentally changed the ballgame, and this analysis is good fodder for those who think introductory statistics is a little bit archaic. Where I disagree with his analysis is not whether we are currently able to model this "fourth quadrant," but whether we will one day be able to. I believe in progress, one non-bailout at a time.

The myth of the rational movie rater

Bryan Caplan's The Myth of the Rational Voter is a short, brilliant book. He described four ways that the average voter is biased as compared to the "enlightened public," a group who has more factual information about how the government works.

Because I like his ideas so much, I've copied them and applied them to movie rating. Here are the four most prominent biases that affect your typical movie rater:

1) Availability bias: Nobody on earth has seen every movie ever made--there simply are not enough hours in the day, given our current lifespan. Individually, each of our relative rankings must be flawed because we inherently have imperfect information.

2) Snobbery bias: I am always surprised at the number of people that will automatically disdain a movie because it's too low class. People generally only watch the best movies--you'd be surprised at the percentage of movies in the top 250 that are in the average DVD collection. So when a movie is in some way not "great," it is disdained. The effect of this is that many raters lump movies as either "incredible" or "terrible," without the necessary continuum.

3) Anchoring-based bias: If a movie wins an Oscar, the average rater will assume that it must be at least pretty good. If a movie is already rated highly on imdb, subsequent raters will give it higher rankings as well. These are both sad but unavoidable consequences of the rating incompleteness theorem.

4) Anti-foreign language bias: Dubbed films are awful and subtitles annoy many people, so this bias is a natural consequence of the unfortunate fact that the world is not yet filled with anglophones. Memo to the rest of the world: hurry up with those English classes!

Availability bias and snobbery bias are mitigated in large part on imdb because of the diverse voting population. Snobbery bias hurts a lot of professional movie critics, in my unprofessional opinion. Snobbery bias also explains why you see so many 10s and 1s on imdb. Anti-foreign language bias is a cross-cultural phenomenon, and it probably hurts imdb overall because so many speakers are from the United States. But there are some foreign language films on the Top 250, and the site offers over 500 languages, so there is hope yet.

I was tempted to include an "anti-black and white bias," but I decided that this is actually overplayed. Look at the success of Sin City on imdb, especially with the young raters that are supposed to be so resistant. Those under 18 gave the movie a 8.7, although I will admit that I have not accounted for the Jessica Alba factor.

What biases am I missing?

Selfish Self-Experimentation

There are two ways to go about self-experimenting. One is a quest of self-improvement, testing various methods and tools to gauge how much they will help you. The other kind is on the surface more altruistic: using your own body as an incubator to see how others will react to similar treatments.

An example of the first kind is measuring your weight gains with and without Creatine to see whether or not it is worth the long-term risks. An example of the second kind is volunteering to swallow various chemicals to measure what they will do to the average human. Hopefully the effects in rats are congruent to the effects in humans! It's no surprise that there are lots more of the selfish kind than the altruistic kind.

If self-experimentation is ever to achieve a broad scale, it will be because someone convinces a large mass of people that it is in their own best interest. There is a large corpus of data in the workout logs of bodybuilding.com, but it is inserted so haphazardly. Even a small amount of control in that data and it's value would increase exponentially. But how do you convince people to record their workouts methodologically?

Explaining individual variance within species

A Primate's Tale is an autobiographical tale of Robert Sapolsky's trips to Kenya to study hormonal patterns in baboons. Robert is bursting with both scientific curiosity and liberal idealism, which makes it all the more amusing when he makes fun of himself.

One especially thought-provoking section of the book is when he considers what evolutionary psychology can and cannot explain:

Sociobiology is often faulted for the Machiavellian explanations it gives for some of the most disturbing of social behaviors. And for the suggestion that some of those horrendous behaviors are highly rewarding to their skillful practitioners. Less noticed is that it also generates just as valid (or invalid) explanations for some of the most selfless, altruistic, caring of behaviors and shows the circumstances under which those are highly rewarding behavioral strategies to follow. Yet, nothing about that science at this stage can begin to explain the individual differences--why did Isaac take the strategy that we recognize as being such a "nice" one, while Nebuchanezzar behaved in a manner that was vicious and rotten? At this stage, as a trained scientist, all I can conclude is that Nebuchanezzar was a shit on some fundamental level.

Indeed, some people may also be best described as "shits." But there has to be some biological explanation for this individual variance. Did they try to share at a young age and not recieve a reward for their kindness? Are they less able to project the consequences of their actions into the future because of an inability to properly store long-term memory?

Some people say that the nature versus nurture debate is over because the truth is clearly a combination of both. I'll buy that, but this is still a fascinating field--what factors determine personality?

The wisdom of Ezra Klein: don't read Ezra Klein

Tyler Cowen and Ezra Klein had an IM conversation in the New York magazine a couple of days ago about the current state of politics. I initially read the piece because of Cowen, but it was Klein who turned out to be the star. He noted that,

I think the primary failure in political reporting is the focus on the presidency. It's totally misleading. But it's easier to construct a narrative around one person than 535. But if you're a domestic-policy guy, like I am, the real problem is Congress, not the president. It's not like no one has tried health reform before....

Three weeks ago, I left the country for seven days. When I came back, nothing had changed in the election. Nothing! But before I'd left, it felt like all sorts of things were going on. There was plenty to write about... My main conclusion has been that it's a huge analytical mistake to pay much attention to politics. But it's also my job, so what're you going to do?

These two points really stick out to me--that 1) most pundits focus way too much on the president, and that 2) paying much attention to day-to-day politics is a "huge analytic mistake." I agree, but it's so hard to adhere to either when everybody around you is discussing the relative merits of McCain and Obama!

I also love the format here of instant messaging. Videos are largely a waste of time (for me), but I find the structure of a dialectic to be fascintating. Pitting your ideas against another persons' in real time is challenging, but it yields great results. The format would be even better if they displayed how long each person took to send each message, but would that put too much pressure on the participants?

The cognitive dissonance of long books

Peruse many of the lists of the top books of all-time and you will find lots of very long books. War and Peace, Ulysses, and Don Quixote are on seemingly every list, and each of them tops 700 pages. This is incredible, given that undertaking such an epic novel is such a time sink. Because of this conundrum, I have long wondered whether there is a bias favoring long books over shorter ones.

So why do people rate long books so highly? I think it's a case of cognitive dissonance. Readers know full well how long the book will be before they decide to read it. They then spend so much time and energy reading the book that they come to believe it must have been good.

There are other possibilities, such as a selection bias, where people only rate books that they have finished. People may be more likely to stop reading a longer book that they don't like, as opposed to a shorter book that they may simply power through. This strikes me as flawed for two reasons. One, many readers are cursed with the terrible affliction of finishing all books they start, and two, people are liable to form an opinion after reading nary a sentence.

The best evidence for a cognitive dissonance effect in long books may come from movies, where longer flicks are not as disproportionately represented on the "best of" lists. Before watching a movie, you are much less likely to know exactly how much longer the movie will be, and you may become bored goes on and on. But since you didn't know beforehand how long the movie will be, it's somehow not your fault if the movie is too long for your taste.

What conclusions can we draw? My advice is to take opportunity cost into consideration when recommending a book. For every doubling of book length, the value extracted should also double. Unless it's Godel, Escher, Bach: An Eternal Golden Braid, a long book which I recently gave a 5/5 on Shelfari. But that one must have been good... I spent over a month reading it!

Anti-nihilistic effects of "classical" education

One of the reasons put forward for teaching the classics in school is that it teaches children the right moral code. By reading Socrates and other more contemporary writers, schools expect students to logically decide that all humans are meant to live the good life. Many people deride religious education in public schools, but stand by idly while secular moral systems are imposed.

It seems strange to me that these moral codes are taught so unabashedly in school. If our moral system is in large part a function of cultural and societal mores, then why do we impose a homogeneous one on nearly every student? A more practical, skills-based education would not only boost our nation's productivity, but it also would be the right thing in promoting the freedom of our citizens.

After all, if somebody wants to adhere to nihilism, who can tell him that he is wrong?

The digg mentality and Veritocracy

Veritocracy is a new site (still in beta version) that aspires to become the next great content aggregator. Users will submit pages, other users will vote them up or down, and the top articles will appear at the top of the main site. I've never enjoyed their kind of article much--the "ape escapes from zoo and visits the local Starbucks" stories don't do it for me--but they have an interesting way of dealing with "gaming." From TechCrunch:

If you submit crap, miscategorize your articles, or even vote for other people’s crap, readers won’t be voting for the same things you are (and may even vote down things you vote up) and thus the the system will uncorrelate you from everyone (or won’t correlate you to them in the first place). This will make sure your content and votes have less chance of effecting what other users see in the future.

This is a step in the right direction because it would discriminate against bad users who simply don't care. But it would not discriminate against gaming by smart users, who could generally vote for all of the highest rated content and then selectively vote for articles that they would like to see at the top. Indeed, this system would probably actually help smart gamers because it increases their pull.

Weeding out the dumb gamers is doable, but fighting against smart users on this type of site is a highly intricate. And for me, this conundrum reduces the utility of the aggregator.

Demystifing terrorism

Wise words from Jim Harper at the Cato blog:

Reading terrorists their rights, and treating them with scrupulous fairness, would help start to make them boring, and it would keep the focus on their wrongdoing. This would enervate terrorism and deprive terrorist groups of recruits and support. On these grounds alone, we should be all for reading terrorists their rights.

This is spot-on analysis. An overreaction is exactly what a terrorist group aims for when they attack. 9/11 wasn't a hugely successful military action until we made it so by invading Afghanistan and Iraq. If we treat terrorists as ordinary international criminals, then their appeal would drop. The Sayyid Qutb's of the world might start looking for other solutions to their problems.

As we move into post-Bush politics, the question should become less and less, "whose fault is it that we made the mistake of declaring war?" Instead, we should focus more and more on how we will avoid this kind of overreaction the next time that there is a terrorist attack on our soil. This will not be easy, as it will necessitate overriding our innate desire for revenge with cold rationality. Can we do it? Given that it is not even currently a talking point, I have my doubts.

Dreamfullness

Daydreams are useful to help us solve problems creatively, while our dreams at night are more negative than our everyday experience. I don't see any reason to assume that our night dreams are any more negative than our day dreams. So both dreams could fall under the same category as pragmatic problem solving tools. They don't sugarcoat the problem like us humans normally do, making them effective.

I see myself writing up a huge report on these ideas, replete with randomized groups and graphical representations, I see myself anxiously awaiting the day I receive the editor's verdict, and I see the paper returning, with two words in red ink on the back. Keep dreaming.

Defining the ratedness of a movie

Questions of whether or not a movie is "overrated" or "underrated" is difficult, because it first presupposes so much about how others have rated it. Tyler Cowen just linked to one list of underrated science fiction movies, most of which I cannot comment on because I haven't seen. However, I feel that I must ask, where are the metrics?

There seem to be two types of "underrated": movies that have not been watched enough, and movies that have been watched but not fully appreciated enough. Both of these methods could be measured using imdb. If a movie has a high score but few views, it has not been watched enough. And if you gave a movie a high score (or some other group did, say the Top 1000 voters), but others did not, then the movie would be underappreciated.

Determining if a movie has not been watched enough is more complicated (you'd have to define an average), while determining if a movie is underappreciated is more subjective (who says that your opinion is the end all and be all?). But both are doable empirically; look out for it in a Tuesday Statisticz coming to a blog near you.