We like to think of ourselves as rational decision makers, using patterns of evidence to discern meaning and to understand and shape our environment. The case made in this video is that that is at best a half truth. The reality is that our powers of explanation are much weaker than we tend to recognise or care to admit and that in looking for patterns we are too ready to overlook random variation.
That’s not just an abstract or theoretical concern: the crisis of replication in science is a real and alarming symptom of the problem; the challenge to the very concept of statistical significance is closely related.
This video is a thirty minute summary by Michael Blastland of the ideas in his recent book, followed by a discussion with Matthew Taylor which is also well worth watching. That’s a rather bland description of a talk which was anything but – these are challenging ideas, powerfully presented, which anybody who creates or uses evidence for public policy needs to understand.