"What's more, there is a well documented asymmetry between the impact of bad first impressions versus good ones. Consider the following: We are quicker to both form and recall bad impressions, and are also more likely to do so. We also tend to be more confident about bad impressions, take less time to arrive at them, and require less information to be convinced of them -- that is, relative to good impressions. Finally, once a bad impression is formed, we seal it away from revision or interference."
That's from a post on The Splintered Mind, a blog on experimental philosophy. There is more about ways to avoid these impressions and counteract the bias.
I would add that this is especially important when we begin to discount people's ideas due to our dislike of their personality. It is even more reason to be a close reader and critical thinker. History is chalk full of scientific advancements that weren't accepted for decades because the person espousing them was considered uncharismatic.