Overview

One feature of HCI research is that a broad range of methods have been developed to support the design and evaluation of interactive systems. In this chapter, we focus on evaluation methods – not because they are more important than design methods, but because we have more experience of them. In particular we consider the excitements and challenges of research on developing and testing new methods for evaluating interactive systems.

Evaluation methods can be broadly classed on three dimensions:

• with or without the active involvement of users

• with or without a running system

• with or without a realistic context of use.

Most methods fit into one position in this three-dimensional space. For example, controlled experiments typically involve the active participation of users with a running system in a laboratory setting (a non-realistic context of use), while contextual inquiry (Beyer and Holtzblatt, 1998) involves users with a running system in a realistic context. Conversely, interviews involve users but typically not a running system (though they may sometimes take place within the intended context of use). Most of the techniques that involve the active participation of users have drawn on and adapted methods initially developed within the social sciences, and methodological development has been relatively minor and definitely incremental.

In this chapter we focus on the other set of methods – those which do not require the active involvement of users, being applied, rather, by HCI experts.