Consider spatiotemporal pattern representation in the framework of template matching, the oldest and most common method for detecting an object in an image. According to this method the image is searched for items that match a template. A template consists of one or more local arrays of values representing the object, e.g. intensity, color, or texture. A similarity value between these templates and certain domains of the image is calculated, and a domain is associated with the template once their similarity exceeds a given threshold.

Despite the simple and straightforward character of this method, its implementation requires us to consider two fundamental problems. The first relates to what features should be compared between the image S0(x, y) and the template Si(x, y), i ∈ I. The second problem is how this comparison should be done.

The normative answer to the question of what features should be compared invokes solving the issue of optimal image representation, ensuring the most effective utilization of available resources and, at the same time, minimal vulnerability to uncertainties. Solutions in principle to this problem are well known from the literature and can be characterized as spatial sampling. For example, when the resource is the frequency bandwidth of a single measurement mechanism, the optimality of spatially sampled representations is proven in Gabor's seminal work (Gabor 1946). In classification problems, the advantage of spatially sampled image representations is demonstrated in Ullman et al. (2002).