Behavioral fMRI Has No Clothes?

Jonah Lehrer in Wired on Dec 21 tells which brain regions activate in fMRI when physics and non-physics majors watch a falling ball:

[U]ndergraduates... watch a couple of short videos of two different-size balls falling... Galileo’s metal balls all landed at the exact same time — a refutation of Aristotle, who claimed that heavier objects fell faster. While the students were watching the footage, Dunbar asked them to select the more accurate representation of gravity... [U]ndergraduates without a physics background disagreed with Galileo.... [I]n an fMRI machine... showing non-physics majors the correct video triggered a particular pattern of brain activation... to the anterior cingulate cortex.... With physics majors... their education enabled them to see the error, and for them it was the inaccurate video that triggered the ACC...

When physics students saw the Aristotelian video with the aberrant balls, their DLPFCs kicked into gear and they quickly deleted the image from their consciousness.... However, when it comes to noticing anomalies, an efficient prefrontal cortex can actually be a serious liability. The DLPFC is constantly censoring the world, erasing facts from our experience. If the ACC is the “Oh shit!” circuit, the DLPFC is the Delete key. When the ACC and DLPFC “turn on together, people aren’t just noticing that something doesn’t look right,” Dunbar says. “They’re also inhibiting that information.” The lesson is that not all data is created equal in our mind’s eye: When it comes to interpreting our experiments, we see what we want to see and disregard the rest.

In my interview with Colin Marshall on Oct 23 I explained why exactly this doesn't make sense to me:

I have read about some research on human subjects making decisions and undergoing fMRI, but it doesn't make sense to me. Why does it matter that the PFC is lighting up but not the ACC in some given task? Surely it matters to some computational neuroscientist trying to falsify theories, but I can't grok the lay fascination with brain regions.

The experiment is fairly interesting, although it does support what we'd expect a priori, and some numerical group averages would be nice. But I still don't see what the fMRI data adds to it. Yes it's possible that the ACC activating at the same time as the DLPFC can correlate with specific behavioral outcomes, but how does this change our understanding of how people interpret data? It is well known that certain brain regions will activate consistently in certain tasks, and there is little reason that detecting errors in this ball watching paradigm should differ greatly from detecting errors in other paradigms.

Lehrer's article is high quality: so far it has been retweeted 860 times, and it was even discussed on OB, currently the best blog on the internet. But to me the fMRI stuff just seems like black box fluff. Why does it matter to Wired readers whether the brain activation occurs in Brodmann areas 46 and 24 as opposed to any others?

Addendum: See Bob's useful ideas in the comment section, causing me to see how some lay people could find this stuff interesting. I'm leaving the post as is, but with less confidence.

Eleven Fringe Ideas of 2009

There are a bunch of ideas kicking around in my cell phone's memos section that haven't yet made it onto AMTB. In the interest of entering 2010 with a "clean slate", I'm going to aggregate them here: 1) What does it mean to be good "on paper"? In job or school apps it means that you have an impressive resume but lack traits that are difficult to quantify, like interpersonal skills. In sports it means that a team has lots of big names but they either lack chemistry or are mostly past their primes. In "life," it means that you've got it all but for some reason still aren't satisfied. Being good "on paper" is especially troubling because it means that you'll consistently lose the expectation game. 2) Do self-serving biases make the world go round? One couple's experience in marriage therapy, as described in the NYT here, suggests that there may be some validity to my aphorism. 3) Similar to how you are more likely to see 1's than other digits in nature, you are more likely to see people starting than finishing books on airplanes. When I start books on airplanes I worry that my fellow passengers will assume that I am some nublar, which encourages me to plow through and get to a respectable page number quickly. 4) Would raising the speed limit and saying that you'll actually ticket people accomplish the laudable goal of allowing people to drive with the traffic and not break the limit? Or would it merely shift the equilibrium up? 5) Let's assume that people are bound to have status competitions over something. This is a reasonable assumption based on, say, the entire written record of humans in groups. If that's true, then competition over who can give the most to charity would be a good way of channeling this tendency and should be encouraged. I can see two plausible ways of doing so: a) Banning anonymous donations so that there is no expectation of or mechanism for counter-signaling, and b) Contractually delineating each family's donations each year in the local paper, possibly as a percentage of overall income. 6) Are micro delusions necessary for the self-serving bias as a whole? Meaning, is it possible to have clear eyes on a micro level but happily self-serving on a macro level? The evidence tentatively points to no: Clinically depressed people, who have the lowest group averages of self-serving biases, are less likely to fall victim to trivial delusions. Eek! 7) For 2008 one of my NYR's was to read 50 books. Full transperency: I only read 45. It's OK though, I still love myself. :) 8) Advice givers should be more cognisant of the shelf-life of their advice. That means communicating how long what they say will be relevant for, and the possibility that the world may have already changed since their first-hand experiences. 9) Eliezer Yudkowsky's line: "True dissent doesn't feel like going to school wearing all black, it feels like going to school wearing a clown suit." This would get its own quote of the day post, if I were uncool enough to do that kind of stuff. 10) The less likely a phenomenon is to occur, the less likely it is that multiple factors contributed to it. This statement is true if the relevant factors are completely independent, but as the factors become more and more correlated the statement becomes less and less true. Those two sentences explain the various reactions to the 2008 meltdown very well, in my opinion. 11) Are young people really more able to change their mind? Via a blog comment on EconLog, Robin Hanson told Bryan Caplan that he was too old to change his mind about philosophy. Specifically he said, "People are a lot more impressionable about such things when young, which you aren't anymore." I'd love to see some data. A Psychological Science article mentions that, "adult learners often struggle to acquire any level of proficiency in a second language... constraints on L2 acquisition... have been hypothesized to result from a biologically sensitive period for language learning." But, the authors don't offer any data-oriented citations and anyway language learning is a specific skill, not necessarily relevant to the plasticity of opinions in general. If this is indeed a myth, it's a pervasive and dangerous one. And if it's not a myth, it seems to be an important trend to openly acknowledge.

The Real Top Ten Movies of the Aughts

Using imdb's Bayesian scores:

10) The Departed, 2006. Factors: Star-studded cast with DiCaprio, Damon, Nicholson, and the elder Sheen combining to star in 7 other top 250 movies, plus Scorsese has directed 4; stole its plot from Infernal Affairs, another top 250 movie; some hilarious scenes, especially ones including Walberg and Baldwin; legitimate plot twists. Ratings-wise does well across the board and picks up the female vote. Have to assume that the love story angle helped it there.

9) Amélie, 2001. Factors: Very cute and romantic, with the rare female protagonist; French so more explicit than you might expect; flat out dominates the female vote, especially 18-29, but doesn't get betrayed by the male vote like Beauty and the Beast does.

8) Wall-E, 2008. Factors: Fits well into current technophobic apocalyptic dogma; sweet beginning with no dialogue; "fun for the whole family"; refreshingly short. Surprisingly, rated rather poorly by those aged 45+, especially females, who give it a measly 6.1.

7) LotR: Two Towers, 2002. Factors: Best fight scene of the decade in the last scene, with Legolas pulling off the coolest moves. Not surprising that males under 18 give this one the highest rating.

6) Memento, 2000. Factors: Sick topsy-turvy plot; eminently rewatchable; Chris Nolan directed (he kills it on imdb); confusing for non-movie watchers making it an "insiders club"; street cred for having a solid psychological basis (see patient HM, who also learned to do mirror writing); and great ending.

5) Avatar, 2009. Factors: Innovative 3D presentation; fits well within current environmental dogma; James Cameron directed (he quasi kills it on imdb as the director of 2 top 250 movies). This is a controversial pick as the rating is inflated because it just came out and the score will probably fall quite a bit. Wasn't sure what to do so it remains here for now.

4) LotR: Fellowship of the Ring, 2001. Factors: Best visuals of the trilogy and a rather scary scene when the hobbits first hide from the dark horses. Does quite well with females considering it's so action oriented--females 18-29 give it the highest ratings of any cohort.

3) City of God, 2002. Factors: Lots of violence and a long winding story with a sympathetic narrator. This movie went viral--countless people listed it as their favorite movie for a while in the mid-2000s. Of course the conformity theory makes no exceptions and now the public has generally cooled on it. But, the respect remains.

2) LotR: Return of the King, 2003. Factors: Visually impressive, plus a huge nerd following. Could have easily dissapointed (see Star Wars VI) but did not. Very polarizing, though, with 5.2% ratings of 1.

1) The Dark Knight, 2008. Factors: Heath Ledger's posthumous performance guarantees goosebumps; Chris Nolan directed (see #6); Batman Begins was top 100 so it has a strong pedigree, Christian Bale starring (he's in two other top 250 movies); and round two of a trilogy so execs OK'd the ambiguous ending. The perfect storm. Did not drop as much in score post-DVD as I thought it would, with demonstrated staying power into the next decade.

Rubbernecking and Curiosity

Let me be perfectly clear: Drivers attempting to gawk at the aftermath of a traffic incident can be dangerous. Wikipedia quotes a study finding that rubbernecking led to 16% of all "distraction related" auto accidents. Truly, that's too bad.

But imagine a world in which humans did not have the innate tendency to look to the side of the road. Imagine a world in which we weren't so curious about something for no instrumental reason.

I don't think we as a society would have advanced far. So many of our inventions have been due to chance, when unknown explorers sought uncharted territory. If you took away our insatiable curiosity, what else would we lose along with it?

So, as I rubberneck the rubberneckers one more time, I think to myself: Maybe this is a mistake. But all in all, the human tendency to want to know what is going on in the world is a good thing. And for that, I pay my respects to the blind god of natural selection.

(Thanks to Chris Whitney for a convo about this)

Prefer Distillation

Tyler Cowen argues that you should always read the original source of material that you are studying, and offers nine reasons for doing so. The phrase conspicuously absent from his post is "opportunity cost." In fact, I'll quote Tyler himself from Discover Your Own Economist to explain this beautifully,

We must ignore the carping of the sophisticates. Well-educated critics may claim that pictures cannot be ranked, value is multidimensional or subjective, or that such talk represent a totalizing, colonizing, possessive, postcapitalist, hegemonic Western imperalist approach. All of those missives are beside the point. When it comes to the arts, dealing with the scarcity of our attention is more important than anything, including respecting the artists.

Exactly: Ignore sophisticated bloggers who want to tell you what to do with your scare attention! Read the Wikipedia pages or Spark Notes of famous thinkers whenever possible. If a book is truly ranked high, then you should maybe consider reading it. Otherwise, hail the modern distilling process, and only settle for the highly vaporizable stuff that reaches chamber #8:

Seeing Selection Biases, Not Treatment Effects

I'm not sure if this is a good thing or not but this is how my views typically diverge from the views of others: Where others see are more likely to see treatment effects, I am more likely to see selection biases.

Let's consider sports. A basketball team has just won four games in a row, and they are winning the first half of a fifth. The talking heads during the halftime show are talking about how they are using their "momentum." Basically they are arguing that the collective confidence advantage (i.e., the treatment effect) from winning the first four games is what is allowing them to win the fifth.

I admit that this might be a very slight factor. But much more important to me is the selection bias that comes from having won those first four games: They must have been doing something right! They probably had good players, a good strategy, or something else going for them. Maybe, in the case of the NBA, they have been paying off the refs.

There are of course different forms of treatment effects and selection biases. Hypothetically a Bayesian truth-seeker shouldn't "usually" stand on one side or the other, but I tend to weigh the selection bias much more than most folks, and I wonder why that is. Where do you stand?

An Open Letter to the President of the United States

Greetings Mr. Obama,

Thank you for giving me the opportunity to sound off about open access to scientific research, a subject that I consider near and dear to my heart. I know that you are planning on boosting public R&D funding up to 3% of US GDP, which is an effort that I laud. The following are some thoughts that I think could help scientific progress without spending much additional money:

1) All data should be made completely public, in simple text format, so that it can be easily used with the best open access statistical software, R. This is not being discussed but it would be a non-trivial improvement. It will make meta analyses about a million times more effective and trustworthy (p<0.0001), and it will improve the pace of science via crowdsourcing. Suddenly everyone with internet access and half a brain will be able to contribute to science. Many scientists themselves are unlikely to support this--it would decrease barriers to access and thus hurt the mystique and possibly prestige of scientists. Be not weary of the ruthless hand of creative destruction, lest ye fail to grab the low hanging fruit.

2) Somehow, posting science info to Wikipedia should be incentived to yield higher social status, especially within academic circles. There are huge positive externalities. Scholarpedia has some OK article but it smacks of unnecessary elitism. Perhaps NIH funding and promotion decisions could be made in part by looking at the Wikipedia update history of the applicant, in addition to more formal publications. Uni's would likely follow suit.

3) All publically-funded research, and moreover research from any source that will be considered reputable, must be registered *before* it begins. In this way null results or fangled results due to high intrasubject variance are to be taken just as seriously as positive results, and published somewhere on the web with a link from the central depository of PubMed. Viola, no more publication bias.

4) It should be made more explicit what bloggers are allowed to reproduce on their websites. This is an issue I worry about on Brains Lab and it's a dumb one--I shouldn't have to waste any mental effort on it. PLOS One's model is money. Everyone should be able to take figures and reproduce them if the funding for the research came from a public source. Private sources should of course not be forced to do this, but they should be shunned and possibly blacklisted for their insolence.

5) In the ideal world, non-national security-dependent research would be made public right away so that the world could begin reacting to it without the distilling process. In the world we live in, it should be made public at the latest six months after its publication, following the Journal of Neuroscience method. Voluntary encouragement to self-archive articles doesn't work... only about 15% of scientists comply. If it is mandated 95% comply. Fact is, creative destruction is a beautiful thing. If a business model is no longer viable or necessary, then that's all there is to it.

Yours Affectionately,

Andrew Thomas McKenzie

Most Interesting Sections of Year in Ideas

I've read this for three years now and it's perennially my favorite "best of" list, surpassing any list of books, video games, or movies. Here is the 2009 edition. These were the best ones:

• cognitive illiberalism,
  
• (the) cul de sac ban (no safer and causes more traffic congestion),
• drunken ultimatums,
  
• forensic polling analysis (by this point it should be common knowledge that if you want to create false numbers, use a random number generator),
  
• guilty robots,
  
• killer earth (agree that we do not want to go "back to nature"!),
  
• lithium in the water supply,
  
• man made greenery (let's celebrate the diversity of approaches to geoengineering!),
  
• (the) myth of the deficient older employee,
• printable batteries (small but efficient batteries--which this is a step towards--will be the biggest invention of the next 20 years),
  
• random promotions,
  
• resomation,
  
• (the) sound cannon, and
  
• weapons of mosquito destruction

The best one overall is "massively collaborative mathematics." Basically, some mathematician posted a tough math theorem on his blog and, 1000+ comments later, it was proven. Crowdsourcing needs to become the name of the game in every scientific endeavor. There's too many smart, educated people out there wasting their time with nothing better to do. More on this to come.

What's Your Probability of 2.5° C Warming?

Phil at Statistical Modeling asks everyone to get a little bit more quantitative about their estimates of global warming. He suggests:

Maybe we should start characterizing people by a single number... What probability do you assign to the following statement: increasing the atmospheric carbon dioxide concentration above 800 ppm will change the global average surface temperature by more than 2.5 degrees C (4.5 F)? This would imply a climate sensitivity somewhat below the extreme low end of what the Intergovernmental Panel on Climate Change says is credible... I've chosen 800ppm, which is a bit less than triple the pre-industrial level. It's likely that we're going to zoom right past 800ppm, too.

His number is 90%. Unfortunately the question is slightly incomplete because he doesn't include a deadline. Here are a few possibilities for the deadline: 1 year after we pass 800 ppm, 10 years after we pass 800 ppm, or 30 years after we pass 800 ppm. This would be a "last chance" deadline. Meaning, if the global average temperature increased by more than 2.5° C before then, that would count too. The global average temperature in 2008 was 14.3° C according to the UK's Met; a Google search of this took way too long to find the answer to such a simple question due to the hackery and charts of "temperature anamolies."

Hmm... so let's take the 10 years past 800 ppm deadline for checking to see if global temps have risen. Here's my probability, then: 20%. This is poorly calibrated, but I don't like the idea of either going higher or lower, so I'm sticking with it. Note also that I still favor higher prices for fossil fuels via a revenue-neutral carbon tax. I don't support subsidies, cap-and-trade, or other measures that epitomize the tryanny of the minority over the majority. In an ideal world, we would vote on values, but bet on these kinds of beliefs.

One Explanation of the Flynn Effect

From the NYT's bits blog:

[T]he average American consumes 34 gigabytes of content and 100,000 words of information in a single day... This doesn’t mean we read 100,000 words a day — it means that 100,000 words cross our eyes and ears in a single 24-hour period. That information comes through various channels, including the television, radio, the Web, text messages and video games... [O]n average, most Americans consume 11.8 hours of information a day... Overall, from 1980 to 2008, the number of bytes we consume has increased 6 percent each year, the researchers said, adding up to a 350 percent increase over 28 years.

Given that most of us are only awake for 16-18 hours a day, it is hard to imagine adding more than 4-6 hours more of bit consumption per day. There have been recent suggestions that memories can be strengthened during sleep, but the effect sizes in that procedure are modest--an improvement in object placement accuracy from a deviation of 1.23 +/- 0.10 cm without the sound cues during sleep to 1.07 +/- 0.08 cm with them.

So, eventually the only way to increase our overall bit consumption will be to improve in its efficiency. And increasing efficiency of bit consumption probably means finding some method more efficient than reading. Of course, that's assuming that increasing overall bit consumption is a goal for many of us. And a preference utilitarian who looked at this massive increase in consumption would have to assume that it is. Actions speak louder than words.

(HT to Larry for the story)

Unorthodox RSS Feeds I've Been Reading

The RSS invasion is upon us. More and more people are realizing every day that it doesn't make sense to visit any other websites when you can read it all from the comfort of your feed reader. One pervasive but misguided assumption remains that your only option for RSS subscriptions comes in the form of blogs. Oh no, my friend. You are mistaken.

In the interest of reversing this myth, here are my four favorite RSS feeds from non-blog sources:

• Ben Casnocha's del.icio.us feed. Many of his astute blog posts come from here first, so you will feel smart and informed when you already know what he's talking about.
  
• Cleaveland Clinic Journal of Medicine PubMed feed. Most of these are highly specific and the feed doesn't update much, but every now and then there is a legitimate must-read, like this one about public data reporting at hospitals. Very quant, plus all of the articles are free.
  
• Colin Marshall's twitter feed. As I said in my interview with Colin, it's possible that he's a better tweeter than blogger, and that's saying something. I especially like when he defines a previously ambiguous word. For example, "Procrastination: the temporary displacement of tasks at which it is possible to fail with tasks at which it is not possible to fail."
• Robin Hanson's LessWrong comment feed. This one is tricky because he is replying to someone else's writing and context is sometimes hard to pick up. But for OB fans it's a must, and there are some money quotes, like, "Conversation is a highly evolved system and random changes are usually for the worse."

Let me know if you know of any good unorthodox RSS feeds.

How to Read Books

Non-classical Top 1000 Amazon Reviewer Irfan Alvi reviews Rong Fan's How We Read here and offers bullet points of the best advice he found from the book. In order to further condense the knowledge, here are what I consider the key points from his review of the book I haven't yet read:

3. Since time is limited, choose books very carefully. Books of intermediate size, written by a single expert author, are often best. Use resources like Amazon to help find suitable books (since you're reading this review, you already know that).

6. Mastering the general principles of a subject is more important than remembering details, so read accordingly. For this purpose, read introductory chapters and paragraphs very carefully.

9. Take regular breaks during your reading sessions, even if you'd rather push on. Allow longer reading sessions for more complex topics.

12. Don't bother with speed-reading. Read at the proper pace to learn well, with slower usually being better.

21. Recognize that reading must be balanced with hands-on experience. Reading informs experience so that more is gained from experience, but reading is never a substitute for experience. For that matter, balance reading and experience with other aspects of your life, including rest and recreation. In other words, don't become a bookworm.

I just happened upon Irfan's reviews of books, and they're awesome--he clearly knows how to read books quite well himself. Here he reviews Secrets of a Buccanner-Scholar (and offers a defence of academia), here he reviews How Doctors Think (huge uncertainty means medicine is not a science), and here he reviews Creativity in Science, which is the best book I read all year.

Here he reviews Quantum Enigma and trashes it (correctly, from my opinion on QM and consciousness), but still gives it three stars! This indicates that you need to be fairly positive about a book even when you don't like it to get to the top 1000 reviewers. And it is more evidence for why Amazon needs to go to a ten star system--the five stars just aren't enough to fully indicate one's opinion.