What you want to believe has a big influence on what you believe, and what benefits you personally (or at least, what you think will benefit you personally) has a great influence in what you want to believe. I'm not talking about dishonesty or self-delusion, but a genuine phenomenon, that a conflict of interest can mislead even the most honest and careful thinker. You see it in politics and religion and economics all the time, it happens in science too, although not as much. It's because science is easier.

Think of an example we're all familiar with (but that avoids some of the controversies that frequent these pages!): debugging a piece of software. We have software that fails, and we have clues as to the cause of the failure by noting the type and frequency of errors we get, and if they are correlated to some condition that aggravates them. We use this to suggest a working hypothesis, and try to duplicate the error by controlling the conditions and compartmentalizing the code, until we have finally isolated what we feel is the error and eliminated all other possibilities.

Then we fix it, but we immediately conduct a test incorporating our fix. Those of us who have actually gone through this process know that more often than not, we get it wrong, and the error persists. Indeed, our fix may even aggravate the problem. So the troubleshooting process continues, but this time we at least know what wasn't the cause of the failure. Sooner or later we locate the problem, make a final change, and correct the flaw. But we must always test, because until we can test repeatedly under different conditions without a failure, we can never be assured we found the problem. Of course, if the problem is an intermttent one, as often occurs in real life, or if data is spotty and unclear, this exercise may be lengthy and difficult and take a long time. It involves intuition, non-linear thinking, often at the subconscious level where we are not aware of it. It's not all cold, clear logic. A lot of creative pruning goes on to zero in on the most promising strategies for investigation. And sometimes, that works against us by deluding us.

From a professional and ego standpoint, we like to think we've solved a problem, it makes us feel good and useful, and this desire to succeed often deludes us into giving data which supports our preconceptions more weight than that which contradicts them. I don't think this a human flaw; in fact, I think we're programmed that way by evolution. Nature has selected for problem solvers that tend to follow the most-likely-to-succeed chain of reasoning than those thinkers that give all potentialities equal weight. We trade off an occasional wild-goose chase for abandonment of a lot of almost-certain blind alleys. But this also extends to the conflict-of-interest. That's why we have to test, again and again to convince ourselves we've got it right. And we haveto avoid subconsciously rigging the tests to give us the answers we want instead of those we need. This is very hard.

One of the ways you know you've finally grown up is when you catch yourself asking yourself
"am I really picking this course of action because it is the best, or most likely to succeed, or most honest", or whatever criterion you have selected as important. Or do you fear you may have picked that strategy because it was easier, quicker, or more profitable. When you start asking yourself that question, and when you start answering it correctly more often than not, then you are a man, my son.

Soon after that, you usually die of old age. If you're lucky.