Skip to main content Accessibility help
×
Home
  • Print publication year: 2006
  • Online publication date: January 2010

2 - Exploratory data analysis

Summary

Background

It is important to understand and, if necessary, process the data before beginning any cluster or classification analysis exercises. The aim of these preliminary analyses is to check the integrity of the data and to learn something about their distributional qualities. The methods used to achieve these aims come under the general heading of exploratory data analysis or EDA. One of the greatest pioneers of EDA methods was J. W. Tukey. He defined EDA as an attitude and flexibility (Tukey, 1986). It is well worth reading Brillinger's (2002) short biography of Tukey to get a better understanding of why he held these views so strongly. If one wants to find out more about EDA methods a good, comprehensive, starting point is the online NIST Engineering Statistics handbook (NIST/SEMATECH). Do not be put off by the title, this is an excellent resource for biologists. According to the NIST handbook EDA is an approach or philosophy that employs a variety of techniques (mostly graphical) to:

maximise insight into a data set;

uncover underlying structure;

extract important variables;

detect outliers and anomalies;

test underlying assumptions;

develop parsimonious models; and

determine optimal factor settings.

The first step in any EDA is to generate simple numerical or, even better, graphical summaries of the data. In addition to univariate analyses (histograms, box plots, etc.) it is generally worthwhile considering multivariate analyses. At the simplest these will be bivariate scatter plots, possibly with different symbols indicating group membership (e.g. sex).