CENTER FOR RESEARCH IN
COGNITION & NEUROSCIENCES/ConnexConnex

Certain cognitive biases are extraordinarily powerful and widespread. It should thus come as no surprise that scientists and experts, like everyone else, are subject to their influence. In this talk I advance the hypothesis that two such cognitive biases are likely to be major contributors to the replicability problem in the sciences (i.e. the fact that across a range of disciplines, roughly a third of published scientific findings fail to replicate) and to systematic error in theory formation. The first alleged culprit is the "confirmation bias," which is a near universal human tendency to seek out information which one believes or desires to be true. The second alleged culprit is a (related) bias to overestimate the likelihood of obtaining a significant result in an experimental setting. For each type of bias I first demonstrate experimentally that working scientists suffer from such biases. Using computational modelling and meta-analytic techniques, I then explore the large scale impact that these may have at the levels of data production, publication practice, and belief formation. I finally present a new framework for thinking about interventions which attempt to limit the impact of these cognitive biases in scientific practice.