Bill Simmons On Uniqueness

He gripes in his most recent chat:

[I]t's funny to take heat from soccer fans that I'm a bandwagon Tottenham fan. I mean... of course I am. I am something like 17 months into this thing. But what I don't get about sports like the UFC/soccer/NHL (and even baseball with the saber community towards people who just like baseball and don't want to dive into the stats) is why the diehards are so protective/condescending towards casual fans. What's the goal there? To just drive away everyone who might like the sport and want to become more of a fan? I think there's a difference between local bandwagon fans (like the Pink Hat Red Sox fans) and "I am starting to like your sport, I genuinely want to follow it and learn about it" fans and it would just seem like the diehards should embrace the latter group. Or am I crazy?...

I do think that diehard fans tend to exclude newcomers - the same phenomenon works with music, you always want your favorite band to be the little band that not everyone knows about (and never have them get to the U2 level). I think Kings of Leon are a good recent example of this and even the band members hated that they became "mainstream" because it brought in fans that they didn't necessarily want. The best breakdown of this was in Steve Martin's book about his standup career when he talks about becoming hugely famous and how he started dreading doing his shows because he felt like people weren't there for the right reasons. It's an interesting topic I think.

Once more people join a given group, affiliation loses lots of its signaling benefits due to diffusion. So it makes sense that fans who have invested in a team / band would discourage newbies from joining.

But why do the musicians / comedians themselves not want more followers? That's a bit trickier. Simmons mentions Steve Martin being wary of his new fans, but the same is even more true of Dave Chappelle. When he skipped out on the third season of his show, he turned down millions.

Any theory to explain this phenomenon also has to account for the fact that neither athletes nor academics tend to express these sentiments. Jordan, Manning, Hawkings, Volkow--they do not worry about "going mainstream." Indeed they tend to welcome it.

Perhaps the musicians / comedians are signaling loyalty to their core constituents, and the real emotional and financial costs they pay in doing so just makes their signaling more credible. So it seems that the less badly you want to go mainstream, the more your pursuit is about signaling as opposed to results.

Our Regrets Change Over Time

Ben Casnocha recently posted about the regrets of the dying, and Robin Hanson replied that in fact people on their deathbed do not spontaneously offer such regrets. Robin is technically correct, but he is taking the claim far too literally.

Indeed, there is broader data to suggest that the regrets of older people are quite different from the regrets of younger people. In particular, as time since a decision grows, people tend to shift their regrets towards not making the hedonistic decision.

An '06 study (link, pdf) shows how the intensity of regrets towards work or enjoyment changes as time passes. For events last week, people express slightly higher levels of regret towards enjoying instead of working (2.2 vs 2.0 out of 6). But for events five years ago, people feel more regret for working instead of enjoying (3.4 vs 1.4 out of 6). This change is even more pronounced for feelings of guilt vs missing out, and there's lots of replicating data (for example, see Ran Kivetz's other papers here).

This is part of what makes using the regret heuristic so complicated. One must not only project future regrets for a decision, weighted by the probability of each outcome, but also consider how those regrets might change in direction and strength over time, and integrate over all probabilistic future time points. If this computation were easy, there wouldn't be so much demand for strategies to get an approximate answer.

P-Value Polemics

As I am always up for a good scholarly debate, I was quite pleased, after reading this '05 article calling for a replacement to the p-value called p-rep (cited 200+ times), to see a somewhat vitriolic '09 rebuttal (pdf). First, the abstract of the '05 paper by P. Killeen:

"The statistic Prep estimates the probability of replicating an effect. It captures traditional publication criteria for signal-to-noise ratio, while avoiding parametric inference and the resulting Bayesian dilemma. In concert with effect size and replication intervals, Prep provides all of the information now used in evaluating research, while avoiding many of the pitfalls of traditional statistical inference."

A rather bold claim! And, shortly after its publication, the journal Psychological Science (6th highest psyc impact factor) recommended that authors report p-rep instead of the traditional p-value. Which makes the rebuttal article by Iverson et al that much more tantalizing. They write:

"This probability of replication prep seems new, exciting, and extremely useful. Despite appearances however prep is misnamed, commonly miscalculated even by its progenitors, misapplied outside a common but otherwise very narrow scope, and its seductively large values can be seriously misleading. In short, Psychological Science has bet on the wrong horse, and nothing but mischief will follow from its continued promotion of prep as a scientifically informative predictive probability of replicability."

Now that is what I call a take down! These same authors calm down quite a bit in their '10 article and even make the level-headed suggestion that p-rep is a step in the right direction, but that is uncool so I won't quote from it.

####

Reading about p-values makes me want to start a blog about them (how does such a blog not already exist?!). A good subtitle could be "where one in every twenty posts will be significant by chance alone."

Trade Off #15: Acquiring Info vs Altering Subject

Paradoxically, when you ask someone a question, their answer will likely be different because you've asked them. On reflection this phenomenon is remarkably general; in acquiring or assessing info about a property of a system, we must be willing to accept the consequences of altering that system. A smattering of examples follow: 

• Increasing the energy voltage in transmission electron microscopy can lead to higher image resolution (meaning more info), but it also does more damage to the tissue. (see here; there are similar trade offs in lots of med imaging techs, like PET scans, see here)
• When a model of a complex psychological phenomenon becomes widespread, reality often begins conforming to the model. This is often called performativity, and it is perhaps why Keynes called economics a "moral" science. (see here) 
• The Heisenberg uncertainty principle says that the more precisely the position of a particle is measured, the less precisely its momentum can be, and vice versa. The explanation for this is controversial, but it's likely due to observer effects--the measuring apparatus delivers a force to the particle which alters it. (see here)

Most examples that fit into this category are true to at least some degree, but it's often tricky to determine precisely how much variance they explain. For example, even when you run a placebo-controlled drug trial, you must be wary of side effects that can tip the participants off to their treatment status (see here). Regardless, this is a widespread trade off, and it now assumes its rightful place in the canon.

(photo is of a peptide fiber, taken with EM, credit to Christoph Meier)

We're All Individuals Now

Christian Jarrett explains how we tend to value our uniqueness, 

Whether it's a gift for small talk or a knack for arithmetic, many of us have something we feel we're particularly good at... this strength then becomes important for our self-esteem... children tend to choose friends who excel on different dimensions than themselves, presumably to protect their self-esteem from threat... [W]hen making hiring decisions, people tend to favour [sic] potential candidates who don't compete with their own particular strengths... Participants tricked into thinking they'd excelled at the maths [sic] tended to choose the potential team member who was weak at maths but stronger verbally, and vice versa for those participants fed false feedback indicating they'd excelled verbally. 

I think this drive can explain two anomalies:

1) We often agree to disagree. Aumann's classic disagreement theorem (pdf) says that rational truth-seekers cannot and will not agree to disagree; given the same priors and the same data, they must each reach the same conclusions. Cowen and Hanson (pdf) discuss how this disagreement result is quite robust. What's the deal? Disagreeing allows us to show off our independence and individual intelligence, which are among the most credible ways to establish uniqueness.

2) Our opinions oscillate in cycles away from what we perceive as the current consensus. A recent PLoS Comp Bio paper shows that in order to explain this "hype cycle" you need to model the preference for individuals to feel unique:

[W]e identify a missing ingredient that helps to fill this gap: the striving for uniqueness. Besides being influenced by their social environment, individuals also show a desire to hold a unique opinion. Thus, when too many other members of the population hold a similar opinion, individuals tend to adopt an opinion that distinguishes them from others....

[T]here is a third, pluralistic clustering phase, in which individualization prevents overall consensus, but at the same time, social influence can still prevent extreme individualism. The interplay between integrating and disintegrating forces leads to a plurality of opinions, while metastable subgroups occur, within which individuals find a local consensus. Individuals may identify with such subgroups and develop long-lasting social relationships with similar others.

There is a trade-off between uniqueness and accuracy in beliefs, so we typically seek uniqueness on opinions for which the cost of being wrong is low: art, politics, sports, etc. That's why there is less of drive for uniqueness regarding med. Few would claim that covering an open wound is a bad idea.

Before you say, "but I'm not an individual!", you need to check out this classic Monty Python scene.

Trust The Ratings Of Others

An '09 paper (link here, pdf here, HT to TC) claims that people make more accurate emotional predictions about a future event when they are simply told how someone else reacted to that event, as opposed to when they are given info about the event. This is somewhat counter-intuitive, so let's look at their evidence.

One of their tests was speed dating. Each guy submitted a photo and some demographic info about himself. Then, each girl predicted how much she would enjoy the date based on either this photo / info or the enjoyment rating of a girl who earlier had gone on a date with the same guy. Next, the guy and girl had their five minute date, (ignore the heteronormativity, my fellow Vassar alums), and finally the girl rated how much she enjoyed herself on a sliding scale of 1 - 100. 

The authors define prediction error as the difference between the girl's predicted and actual enjoyment ratings. Participants made more accurate predictions when they used the first girl's enjoyment rating to predict their own (the avg error was 11.4 +/- 8.7) than when they predicted their enjoyment on the basis of the info (an avg error of 22.4 +/- 10.8).

In classic psyc study fashion, they also asked their participants to say which condition they thought would lead to more accurate predictions. 75% said the info would be more useful than the rating of a girl who had already been on a date with that guy. Oops. Now, indulge me in a few reactions:

1) Why do people underestimate the value of someone else's rating? Probably because people think of themselves and their opinions as more unique than they actually are. This sets up my public choice theory for why popular critics like Anthony Lane tend to be negative and contrarian. Although on the surface this annoys readers, people on a deeper level prefer to read opinions about art that they disagree with, because it allows them to think of their own opinions as more unique.

2) There is only one specified relationship between the study participants: they are all undergrads at the same school. So although the authors toss the word "social network" in towards the end of the paper, their results do not speak to the predictive power of a friend's opinion as opposed to a stranger's opinion. This remains an open question--in predicting their own enjoyment, will people find the opinion of someone in their network more valuable than the average opinion of strangers? Even if you say yes, you must take into account the trade-off of sample size, which is larger when you listen to the masses. The high valuation of sites like facebook relies in large part on the assumption that we'll prefer recommendations from those in our network, but I'm not so sure.

3) A subsequent study looked at how people combine their own mental simulations and third-person reports of other's experiences in making judgments. Corroborating the results of this study, they found that people assign far too much weight to their own simulation of how an event will play out as opposed to the feelings of other people who have actually experienced the event. I myself find this all very relevant to imdb's movie ratings. Remind me again why you trust yourself to judge a movie instead of deferring to the aggregated ratings of others?