This appendix provides an introduction to the basic principles of statistical reasoning involved in the evaluation of research evidence (statistical significance, effect sizes, correlation coefficients, and so on). Such concepts are indispensable for the critical appraisal of research evidence.

Statistical significance and the genesis of scientific facts

We will begin this tutorial with an imaginary experiment, focusing on the genesis and authentication of ‘factual knowledge’ produced by the experiment. Although fictitious, the experiment is based on a real scientific study (Hikida et al, 2007).

We invite the reader to consider the following make-believe. You are a researcher interested in the biological basis of schizophrenia, and you believe from previous research that a certain gene is ‘implicated’, the Disrupted-In-Schizophrenia-1 gene (DISC1), which is thought to have a role in working memory. Your experiment involves a comparison of three genetically engineered mice (tg) with a mutant version of the gene, compared with three controls, with normal versions of the gene (wild-type: wt). Let us imagine that MRI scans at six weeks showed small reductions in hippocampal volume, but not other brain areas.

Behavioural analyses showed a mixed pattern: spatial memory in the Y maze paradigm showed no differences between tg and wt; time to find a hidden food pellet was found to be significantly longer for tg and they also remained immobile for longer (250 seconds versus 150) in the forced swim test (the mice are placed in a cylinder filled with water, from which they cannot escape). To the lay reader, this latter test may seem odd, but immobility in the forced swim test is seen by animal researchers (not uncontroversially) as a behavioural correlate of negative mood, representing a kind of hopelessness in the animal. The data for the imaginary swim test are shown in table A1.

Looking at the data, it would appear that a ‘fact’ has been established, the greater immobility of the tg mice in the swim test, from which we may seek to draw some wider conclusions. But let us pause, as alternative explanations for the pattern of results must surely be considered and ruled out before definitive conclusions may be drawn.