COSCH Case Study that has employed this technology: Germolles

Definition

Augmented Reality (AR) is a group of imaging techniques designed to blend virtual and real elements in a coherent scene. Usually, an Augmented Reality application allows a user to navigate in a real environment with an augmented visualization of synthetic elements. This technology has numerous applications, from digital entertainment and video games to specialized professional training and simulation. At the time of writing, AR applications are mainly implemented on smartphones or tablets, which offer an ideal combination of hardware (sensors, cameras, graphics processor, and powerful processing units) for this task.

Description

From a practical point of view, augmenting reality means synchronizing and fusing heterogeneous representations of both actual and virtual scenes. Depending on the targeted application, this fusion needs to be precise and/or complete. For instance, specific applications may require the virtual elements to be positioned very precisely. Other applications may need to be visually convincing, with realistic shading, occlusion, and shadows.

Augmented Reality covers many different software and hardware technologies and is therefore difficult. Its challenges may be presented through three important principles: co-localization, co-occlusion, and co-lighting.

Co-localization means aligning the real scene with the virtual scene: the camera of the virtual scene is placed at the same coordinate location as the physical camera (fig. 13.1). This process is generally achieved through a combination of computer vision techniques with specialized sensors: accelerometers, GPS, gyroscopes, depth cameras, etc. It may also be facilitated by placing custom markers throughout the environment. These markers look like two-dimensional barcodes and present high contrast to enable easy detection. By comparing their projection onto the camera image plane with a reference picture, it is possible through basic linear algebra to retrieve the position of the user. Marker-based co-localization is usually more robust than marker-less and sensor-based techniques, but it is also more intrusive.

Co-occlusion means taking into account the geometry of the real world when rendering virtual elements. Specifically, virtual and real elements should occlude each other, depending on their respective shapes and locations in the environment. Since the shapes of the virtual elements are known, the significant challenge is to acquire the geometry of the real elements. This is generally done with dedicated scanners or depth cameras, offline (in advance, ahead of usage) or in real time if the device has the necessary hardware.