Introduction

Estimating the three-dimensional (3D) structure of the environment is challenging because the third dimension – depth – is not directly available in the retinal images. This “inverse optics problem” has for years been a core area in vision science (Berkeley, 1709; Helmholtz, 1867). The traditional approach to studying depth perception defines “cues” – identifiable sources of depth information – that could in principle provide useful information. This approach can be summarized with a depth-cue taxonomy, a categorization of potential cues and the sort of depth information they provide (Palmer, 1999). The categorization is usually based on a geometric analysis of the relationship between scene properties and the retinal images they produce.

The relationship between the values of depth cues and the 3D structure of the viewed scene is always uncertain, and the uncertainty has two general causes. The first is noise in the measurements by the visual system. For example, the estimation of depth from disparity is uncertain because of internal errors in measuring retinal disparity (Cormack et al., 1997) and eye position (Backus et al., 1999). The second cause is the uncertain relationship between the properties of the external environment and retinal images. For example, the estimation of depth from aerial perspective is uncertain because various external properties – for example, the current properties of the atmosphere and the illumination and reflectance properties of the object – affect the contrast, saturation, and hue of the retinal image (Fry et al., 1949).