The term user-contributed content (UCC) indicates content which is created by, collected from, distributed to, and used by users. Several types of content are currently being utilized as UCC, such as textual information from online encyclopedias and pictures from online image sites. Our goal is to demonstrate the use of live panoramas as UCC for social augmented reality (AR). To achieve this goal, we will enhance the creation, assembly, dissemination, and use of panoramas by individuals. We first address the assembly and dissemination of data, through the creation of a network architecture to collect and distribute geographic information with an emphasis on panoramas. This architecture forms the structural support for the remainder of the work in this thesis. In order for a type of data to be utilized as a form of UCC it must be reasonably created by users. We therefore provide tools which allow for the robust creation of UCC. Our goal is not only to allow for the creation of UCC, but also to ensure that the creation process is robust enough to perform comfortably for a large range of users. To this end we introduce several techniques to enhance the robustness of panorama collection. We also examine the definition of robustness as perceived by users, and examine performance in the context of a range of users. Once content has been created, collected, and distributed future users must be able to utilize the stored content. One central issue to the use of such data is localization of the camera. We therefore provide solutions for the localization of the camera using stored panorama data. As UCC is necessarily user-centric, we also investigate the performance of users with respect to various distributions of localization data. Finally, we demonstrate that with a complete cycle of UCC creation, assembly, distribution, and use we are able to create a unique experience for viewers of UCC through the use of a proof-of-concept client interface to the proposed framework.

Natural feature tracking systems for augmented reality are highly accurate, but can suffer from lost tracking. When registration is lost, the system must be able to re-localize and recover tracking. Likewise, when a camera is new to a scene, it must be able to perform the related task of localization. Localization and re-localization can only be performed at certain points or when viewing particular objects or parts of the scene with a sufficient number and quality of recognizable features to allow for tracking recovery. We explore how the density of such recovery locations/poses influences the time it takes users to resume tracking. We focus our evaluation on two generalized techniques for localization: keyframe-based and model-based. For the keyframe-based approach we assume a constant collection rate for keyframes. We find that at practical collection rates, the task of localization to a previously acquired keyframe that is shown to the user does not become more time-consuming as the interval between keyframes increases. For a localization approach using model data, we consider a grid of points around the model at which localization is guaranteed to succeed. We find that the user interface is crucial to successful localization. Localization can occur quickly if users do not need to orient themselves to marked localization points. When users are forced to mentally register themselves with a map of the scene, localization quickly becomes impractical as the distance to the next localization point increases. We contend that our results will help future designers of localization techniques to better plan for the effects of their proposed solutions.

We describe an online repository we have developed for evaluating image-based orientation tracking methods. We have collected many videos which contain rotation-only camera movement under a wide variety of conditions, such as changing illumination, position, and rotation speed and direction. The dataset is useful for testing the robustness of orientation tracking systems, as well as other systems which use panoramas as a data source. In this paper we discuss the design of the repository and give examples of various uses of the imagery and other data it contains.

In this paper, we present an analysis of four orientation tracking systems used for construction of environment maps. This paper focuses on the methodology used to collect data for a detailed analysis of these tracking methods. This analysis consists of three parts. Two methods of qualitative analysis are used, one based on the panoramas generated by each system, and another based on an expert evaluation of a live demo. A ground truth evaluation of the systems is also performed. Finally, we present an analysis of the performance of these methods, and a comparison of the results of each test.

We present a multimedia solution for easily adding virtual annotations to class lectures through the use of augmented videoconferencing and tracked physical props. These props may be any object normally used in a lecture, such as toy cars or tops (physics), ball and stick molecules (chemistry), or frogs to be dissected (biology). In the classroom, the actions of the instructor are captured by one or more cameras. We then use a normal desktop computer to add virtual data to the camera image. Our software solution tracks the physical objects and allows for overlays of relevant information, optionally deriving information from the movement of the objects. For example, a toy car may be tracked in order to determine its velocity, which may then be displayed as a 3D arrow (vector) directly on top of the video showing the moving car. The resulting video may be sent either to a projector or monitor (to be viewed in class) or over the Internet (to be viewed by remote students). Additionally, our solution allows students to interact with the virtual data through the augmented video, even when distributed over the Internet.

The acquisition of surround-view panoramas using a single hand-held or head-worn camera relies on robust real-time camera orientation tracking and relocalization. This paper presents robust methodology and evaluation for camera orientation relocalization, using virtual keyframes for online environment map construction. In the case of tracking loss, incoming camera frames are matched against known-orientation keyframes to re-estimate camera orientation. Instead of solely using real keyframes from incoming video, the proposed approach employs virtual keyframes which are distributed strategically within completed portions of an environment map. To improve tracking speed, we introduce a new variant of our system which carries out relocalization only when tracking fails and uses inexpensive image-patch descriptors. We compare different system variants using three evaluation methods to show that the proposed system is useful in a practical sense. To improve relocalization robustness against lighting changes in indoor and outdoor environments, we propose a new approach based on illumination normalization and saturated area removal. We examine the performance of our solution over several indoor and outdoor video sequences, evaluating relocalization rates based on ground truth from a pan-tilt unit.

We present an analysis of four orientation tracking systems used for construction of environment maps. We discuss the analysis necessary to determine the robustness of tracking systems in general. Due to the difficulty inherent in collecting user evaluation data, we then propose a metric which can be used to obtain a relative estimate of these values. The proposed metric will still require a set of input videos with an associated distance to ground truth, but not an additional user evaluation.

Currently, we are witnessing a potent confluence of trends in technology and society. First, hardware manufacturers are trending towards powerful, connected, yet ultra-portable mobile devices. These devices have advanced sensors for video, GPS, and orientation, and support a fast and available network connection. At the same time, there is a growing familiarity with and desire for social information sharing. People of all ages are becoming accustomed to broadcasting a multitude of personal data such as what they are doing and where, photos, and videos, as well as interactive input such as comments on other people’s information streams, links, and recommendations. Because of their ubiquity and capabilities, advanced mobile devices such as smart phones have great potential for improving and extending interactions on social networks. Already, users can upload text, photos, videos from wherever they are with their phone. However, this type of interaction only captures a small part of what could be achieved. In fact, researchers in the augmented reality (AR) and virtual reality (VR) fields have long studied the best ways to share experiences and communicate using high immersion devices. We believe that environment maps (or panoramas) provide a great starting point for combining social interaction with mobile mixed reality. Panoramas already can be considered one of the first public successes of VR [1]. They are useful for remote collaboration and exploration, because they provide a high level of immersion (surround-view imagery), but can be stored compactly in any common image format.

The acquisition of surround-view panoramas using a single hand-held or head-worn camera relies on robust real-time camera orientation tracking. In absence of robust tracking recovery methods, the complete acquisition process has to be re-started when tracking fails. This paper presents methodology for camera orientation relocalization, using virtual keyframes for online environment map construction. Instead of relying on real keyframes from incoming video, the proposed approach enables camera orientation relocalization by employing virtual keyframes which are distributed strategically within an environment map. We discuss our insights about a suitable number and distribution of virtual keyframes, as suggested by our experiments on virtual keyframe generation and orientation relocalization. After a shading correction step, we relocalize camera orientation in real-time by comparing the current camera frame to virtual keyframes. While expanding the captured environment map, we continue to simultaneously generate virtual keyframes within the completed portion of the map, as descriptors to estimate camera orientation. We implemented our camera orientation relocalizer with the help of a GPU fragment shader for real-time application, and evaluated the speed and accuracy of the proposed approach.

We seek to support creativity in science, engineering, and design applications by building infrastructure that offers new capabilities for creative collaborative exploration of complex data in a variety of non-traditional computing environments. We describe particular novel environments and devices, including the Allosphere and the interactive Fogscreen, the software components to support collaborative interaction in mixed-media environments, and several key application scenarios that will leverage these capabilities. Our main focus is on supporting integrated visualization, sonification, and interaction capabilities in and across novel computing environments.

This paper presents methodology for integrating a small, single-point laser range finder into a wearable augmented reality system. We first present a way of creating object-aligned annotations with very little user effort. Second, we describe techniques to segment and pop-up foreground objects. Finally, we introduce a method using the laser range finder to incrementally build 3D panoramas from a fixed observerpsilas location. To build a 3D panorama semi-automatically, we track the systempsilas orientation and use the sparse range data acquired as the user looks around in conjunction with real-time image processing to construct geometry around the userpsilas position. Using full 3D panoramic geometry, it is possible for new virtual objects to be placed in the scene with proper lighting and occlusion by real world objects, which increases the expressivity of the AR experience.

We propose a novel approach to mixed-reality teleconferencing that focuses on 3D augmentations of multi-camera 2D video streams. We exchange information about the geometric layout of all participating meeting sites, as well as the camera parameters for surveying these spaces. We can then correctly overlay 3D graphics on top of the video feeds, representing meeting content as well as highlight annotations and interaction tools for scene manipulation and bookkeeping of meeting contributions and decisions. Our primary contribution is a standard for the delivery and interaction with such data, which will allow for immersive meeting participation on a diverse set of devices, ranging from special-purpose 3D immersive environments to ultra-mobile platforms, such as cell phones and portable video players.

We present a technique that allows a user to look beyond occluding objects in arbitrary 3D graphics scenes. In order to control this form of virtual x-ray vision, the user interactively cuts holes into the occluding geometry. The user can rapidly define a cutout shape or choose a standard shape and sweep it over the occluding wall segments to reveal what lies behind them. Holes are rendered in the correct 3D perspective as if they were actually cut into the obstructing geometry, including border regions that give the cutout shape physical depth, simulating penetration of a physical wall that possesses some generic thickness.

Nonviral gene delivery has great potential for replacement of recombinant protein therapy. In many cases, gene therapies would be a considerable improvement over existing therapies because of putative advantages in dosing schedule, patient compliance, toxicity, immunogenicity, and cost. Development of a nonviral gene delivery vehicle capable of efficient, cell-specific delivery will be a valuable addition to the clinical armamentarium.KeywordsNuclear Magnetic Resonance SpectroscopyCationic PolymerCharge RatioLithium HydroxidePhoenix PharmaceuticalThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.