We present our novel generic approach for interfacing web components on mobile devices in order to rapidly develop Augmented Reality (AR) applications using HTML5, JavaScript, X3D and a vision engine. A general concept is presented exposing a generalized abstraction of components that are to be integrated in order to allow the creation of AR capable interfaces on widely available mobile devices. Requirements are given, yielding a set of abstractions, components, and helpful interfaces that allow rapid prototyping, research at application level, as well as commercial applications. A selection of various applications (also commercial) using the developed framework is given, proving the generality of the architecture of our MobileAR Browser. Using this concept a large number of developers can be reached. The system is designed to work with different standards and allows for domain separation of tracking algorithms, render content, interaction and GUI design. This can potentially help groups of developers and researchers with different competences creating their application in parallel, while the declarative content remains exchangeable.

With "House of Olbrich" we present an iPhone Augmented Reality (AR) app that visualizes the compelling history of Darmstadt's unique Jugendstil (Art Nouveau) quarter with video-see through Augmented Reality. We propose methods for enabling high performance computer vision algorithms to deploy sophisticated AR visuals on current generation Smartphones by outsourcing resource intensive tasks to the cloud. This allows us to apply methods on 3D feature recognition for even complex tracking situations outdoors, where lightning conditions change and tracked objects are often occluded. By taking a snapshot of the building, the user learns about the architect, design and history of the building. Historical media, like old photographs and blueprints are superimposed on the building's front, depicting the moved history of the famous House of Olbrich, which was destroyed during World War II and has been only rudimentary restored. Augmented Reality technology allows tourists to jump back in time visually by using their Smartphones: Mixing Realities emphasizes the user's experience and leads his attention to the impressive historical architecture of the Art Nouveau. In addition, we ease interaction means by superimposing snapshots. Tourists may view and read information also in a relaxed position without the need to front-up their mobiles all the time.

With the current generation of smartphones augmented reality (AR) finally gets in the hands of end users. This is a giant leap for cultural heritage presentation. But due to software and hardware limitations of consumer devices the AR experience is still lacking the quality we have seen in research projects over the last years. In this paper we are proposing a scalable method for high quality AR presentations for cultural heritage on a wide range of consumer devices: Snapshot Augmented Reality. Instead of a live video stream superimposed with jittering annotations we are freezing the scene and enabling Augmented Reality Photography. The result is an interactive scene superimposed on a still image taken by a visitor. In order to outsource processing power and deliver content for a wide range of smartphones most of the sophisticated software works in the cloud. We are describing a reliable and scalable server infrastructure for tracking objects and environments and delivering context aware content to the visitors’ devices.

Multi-touch interaction on tabletop displays is a very active field of todays HCI research. However, most publications still focus on tracking techniques or develop a gesture configuration for a specific application setup. Very few explore generic high level interfaces for multi-touch applications. In this paper we present a comprehensive hardware and software setup, which includes an X3D based layer to simplify the application development process. We present a robust FTIR based optical tracking system, examine in how far current sensor and navigation abstractions in the X3D standard are useful and finally present extensions to the standard, which enable designers and other non-programmers to develop multi-touch applications very efficiently.

Many of the recent real-time markerless camera tracking systems assume the existence of a complete 3D model of the target scene. Also the system developed in the MATRIS project assumes that a scene model is available. This can be a freeform surface model generated automatically from an image sequence using structure from motion techniques or a textured CAD model built manually using a commercial software. The offline model provides 3D anchors to the tracking. These are stable natural landmarks, which are not updated and thus prevent an accumulating error (drift) in the camera registration by giving an absolute reference. However, sometimes it is not feasible to model the entire target scene in advance, e.g. parts, which are not static, or one would like to employ existing CAD models, which are not complete. In order to allow camera movements beyond the parts of the environment modelled in advance it is desired to derive additional 3D information online. Therefore, a markerless camera tracking system for calibrated perspective cameras has been developed, which employs 3D information about the target scene and complements this knowledge online by reconstruction of 3D points. The proposed algorithm is robust and reduces drift, the most dominant problem of simultaneous localisation and mapping (SLAM), in real-time by a combination of the following crucial points: (1) stable tracking of longterm features on the 2D level; (2) use of robust methods like the well-known Random Sampling Consensus (RANSAC) for all 3D estimation processes; (3) consequent propagation of errors and uncertainties; (4) careful feature selection and map management; (5) incorporation of epipolar constraints into the pose estimation. Validation results on the operation of the system on synthetic and real data are presented.

Recent studies have estimated that television and related equipment account for 1.8% of global greenhouse gas (GHG) emissions and Information and Communication Technology is responsible for 2% of global GHG emissions. Both these sectors are forecast to grow as the developing world increases its uptake of technology.This study estimates the carbon footprint of two different ways of watching television: using broadcast digital terrestrial television (DTT) and video-ondemand (VOD) over the Internet. It compares the two distribution methods and the corresponding consumer equipment. It uses the principles of life cycle assessment (LCA) to derive the carbon footprints using a bottom-up analysis of the system applied to the BBC’s television services. This was the only environmental impact considered and was mainly from electricity use. Equipment manufacturing was not included.

Since the last ten years product development in automotive industry is changing radically. Most physical mock-ups have vanished and are now replaced by digital ones. But they are still needed for final evaluations or issues, which cannot be adequately simulated. During their production, deviations from the CAD model may be made. Since digital and real mock-up must match for the further product development, the transfer of differences between physical and digital mock-up to the CAD format is a crucial issue. In this paper an Augmented Reality (AR) based tool-chain is presented, which allows matching the CAD data with real mock-ups and documents the differences between them. Essential functions like measurement and online construction are provided, allowing the end-users to create information in AR space and feeding them back into the CAD model.

In this paper we introduce a novel architecture for rapid development and assessment of advanced 3D visual tracking systems. Indeed, we notice that it does not exist up to now a universal tracking approach that fulfills the requirements of all possible application scenarios at the same time. On contrary, very specific and performing solutions can be developed for given situations and uses. Therefore, software for visual tracking must be designed as a highly flexible system that can be quickly re-configured in order to enable the development of optimised solutions in terms of accuracy, robustness, frame rate and delay, to this purpose we designed an architecture that offers many functionalities, which can be combined together, and thus build a new processing chain. The overall system offers numerous advantages, such as interactive programming, real-time access to the data and parameter at runtime.

In order to insert a virtual object into a TV image, the graphics system needs to know precisely how the camera is moving, so that the virtual object can be rendered in the correct place in every frame. Nowadays this can be achieved relatively easily in postproduction, or in a studio equipped with a special tracking system. However, for live shooting on location, or in a studio that is not specially equipped, installing such a system can be difficult or uneconomic. To overcome these limitations, the MATRIS project is developing a real-time system for measuring the movement of a camera. The system uses image analysis to track naturally occurring features in the scene, and data from an inertial sensor. No additional sensors, special markers, or camera mounts are required. This paper gives an overview of the system and presents some results.

Accurate acquisition of camera position and orientation is crucial for realistic augmentations of camera images. Computer vision based tracking algorithms, using the camera itself as sensor, are known to be very accurate but also time-consuming. The integration of inertial sensor data provides a camera pose update at 100 Hz and therefore stability and robustness against rapid motion and occlusion. Using inertial measurements we obtain a precise real time augmentation with reduced camera sample rate, which makes it usable for mobile AR and See-Through applications.This paper presents a flexible run-time system, that benefits from sensor fusion using Kalman filtering for pose estimation. The camera as main sensor is aided by an inertial measurement unit (IMU). The system presented here provides an autonomous initialisation as well as a predictive tracking procedure and switches between both after successfull (re)-initialisation and tracking failure respectively. The computer vision part performs 3D model-based tracking of natural features using different approaches for yielding both, high accuracy and robustness. Results on real and synthetic sequences show how inertial measurements improve the tracking.

In this paper, we present an overview of several visual tracking methods for industrial augmented reality applications. We show that no universal algorithm can deal with the large number of possible scenes, and that the different methods have to be seen as complementary approaches that all have their strengths and weaknesses. The main difficulty, then, consists in combining existing building blocks in the right manner so that the overall system enables stable tracking. This paper addresses each phase of the tracking, i.e. Initialization, Tracking, ReInitialization, and proposes a first choice of appropriate algorithms. Finally, a global system is designed, tested and evaluated with help of video sequences of different real environments.

In this paper we present the Augmented Reality Ocular, an extension of the traditional coin-operated binocular with augmented reality capabilities. The Augmented Reality Ocular enables a real implementation of the augmented reality paradigm and makes the technology available to a large public. After a review of the state of the art and a presentation of current related projects, we describe our approach of implementation. Finally different concept and purpose of applications are described in order to demonstrate the wide field of application of this new augmented reality device.

Computer assisted operation planning systems are gaining increasing recognition in the field of surgery. These systems offer new possibilities for preparing an intervention, with the goal of reducing the amount of expensive operating-room time required for the intervention. The safest and most effective surgical approach should always be selected, but it is often difficult to transfer the output of the planning system to the intra-operative situation so that the planning results can be considered during the actual intervention. At the Fraunhofer Institute for Computer Graphics (IGD) in Darmstadt and the Centre for Advanced Media Technology (CAMTech) in Singapore methods are being developed to bridge the gap between the external planning session and the intra-operative case: Augmented Reality (AR) techniques are used to overlay preoperative scanned image data, as well as results of the planning session, on the operation field.

Rapid development of medical field, expanding knowledge base and new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. To overcome the Current drawbacks of traditional training systems (on-the-job training, plastic models etc.) for laparoscopy/hysteroscopy an intelligent adaptable training environment has been realized using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

In this paper we introduce an architecture for rapid development and assessment of advanced 3D visual tracking algorithms. We claim, that no universal tracking approach exists, that fulfills the requirements of all possible application scenarios at the same time. On the contrary, very specific and working solutions can be developed for given situations and uses. Therefore, software for visual tracking must be designed as a highly flexible system that can be quickly re-configured in order to enable the development of optimized solutions in terms of robustness, accuracy, frame rate and delay. To this purpose we designed an architecture that offers many functionalities, which can be combined together to build a new processing chain. The overall system offers numerous advantages, such as interactive programming, run-time access to data and parameters and easy interfacing with other libraries or applications.