Introduction

The idea that we should try to learn by our experiences is both natural and intuitive, but one we frequently seem to ignore in the software development and test context.

One particular approach to learning by experience in a project environment is to collect and use metrics observed during the course of the project, such as how much effort was expended in completing the project, how many defects were found during testing, or how many defects were found by the client following delivery of the tested system. This information subsequently can be used to determine whether the testing process is improving over time, for example, by comparing the current effectiveness in finding defects against previously observed results.

Metrics can also be used to investigate the benefits of adopting a specific approach to testing – such as the introduction of automated testing – by comparing the efficiency of the testing process before introduction with that observed following rollout of the tool. The Crown Management Systems case study (Chapter 17) provides just such an example of the role and use of metrics in a testing project environment.

If no formal, quantitative measurements are made, it is only possible to make qualitative statements about the effectiveness of the testing process. This may assure senior management in the short term, but in the long term will not help to improve the testing process.