Introduction to pattern classification

The statistical methods discussed in the book so far have had the common feature of trying to best characterize the dataset at hand. For example, when we apply the general linear model to a dataset, we use methods that determine the model parameters that best describe that dataset (where “best” means “with the lowest mean squared difference between the observed and fitted data points”). The field known variously as machine learning, statistical learning, or pattern recognition takes a different approach to modeling data. Instead of finding the model parameters that best characterize the observed data, machine learning methods attempt to find the model parameters that allow the most accurate prediction for new observations. The fact that these are not always the same is one of the most fundamental intuitions that underlies this approach.

The field of machine learning is enormous and continually growing, and we can only skim the surface of these methods in this chapter. At points we will assume that the reader has some familiarity with the basic concepts of machine learning. For readers who want to learn more, there are several good textbooks on machine learning methods, including Alpaydin (2004), Bishop (2006), Duda et al. (2001), and Hastie et al. (2001).