Statistical hypothesis testing is a process to accept or reject a hypothesis based on observations, where multiple hypotheses are proposed to characterize observations. Taking observations as realizations of a certain random variable, each hypothesis can be described by a different probability distribution of the random variable. Under a certain criterion, a hypothesis can be accepted for given observations. Signal detection is an application of statistical hypothesis testing.

In this chapter, we present an overview of signal detection and introduce key techniques for performance analysis. We mainly focus on fundamentals of signal detection in this chapter, while various signal detection problems and detection algorithms will be discussed in later chapters (e.g. Chapters 7, 8, and 9).

Elements of hypothesis testing

There are three key elements in statistical hypothesis testing, which are (i) observation(s); (ii) set of hypotheses; and (iii) prior information. With these key elements, the decision process or hypothesis testing can be illustrated as in Fig. 2.1.

In statistical hypothesis testing, observations and prior information are all important and should be taken into account. However, in some cases, no prior information is available or prior information could be useless. In this case, statistical hypothesis testing relies only on observations.

Suppose that there are M(≥ 2) hypotheses. Then, we have an M-ary hypothesis testing in which we will choose one of the M hypotheses that are proposed to characterize observations and prior information under a certain performance criterion. There are various hypothesis tests or decision rules depending on criteria.