Correctly identifying the appropriate model to use (see page 57) is the principal hurdle in any analysis, but running the chosen model in your favourite statistics package also presents a number of potential pitfalls. If you encounter problems when using a statistics package, do refer to its help routines and tutorials in order to understand the input requirements and output formats, and to help you interpret error messages. If that fails then look to see if you have encountered one of these common problems.

Problems with sampling design

If I just want to identify any differences amongst a suite of samples, can I do t tests on all sample pairs? No, the null hypothesis of no difference requires a single test yielding a single P-value. Multiple P-values are problematic in any unplanned probing of the data with more than one test of the same null hypothesis, because the repeated testing inflates the Type I error rate (illustrated by an example on page 252). If an ANOVA reveals a general difference between samples, explore where the significance lies using post hoc tests designed to account for the larger family-wise error (page 245).