Combined efforts in the fields of neuroscience, computer science, and biology allowed to design biologically realistic models of the brain based on spiking neural networks. For a proper validation of these models, an embodiment in a dynamic and rich sensory environment, where the model is exposed to a realistic sensory-motor task, is needed. Due to the complexity of these brain models that, at the current stage, cannot deal with real-time constraints, it is not possible to embed them into a real-world task. Rather, the embodiment has to be simulated as well. While adequate tools exist to simulate either complex neural networks or robots and their environments, there is so far no tool that allows to easily establish a communication between brain and body models. The Neurorobotics Platform is a new web-based environment that aims to fill this gap by offering scientists and technology developers a software infrastructure allowing them to connect brain models to detailed simulations of robot bodies and environments and to use the resulting neurorobotic systems for in silico experimentation. In order to simplify the workflow and reduce the level of the required programming skills, the platform provides editors for the specification of experimental sequences and conditions, environments, robots, and brain–body connectors. In addition to that, a variety of existing robots and environments are provided. This work presents the architecture of the first release of the Neurorobotics Platform developed in subproject 10 “Neurorobotics” of the Human Brain Project (HBP).1 At the current state, the Neurorobotics Platform allows researchers to design and run basic experiments in neurorobotics using simulated robots and simulated environments linked to simplified versions of brain models. We illustrate the capabilities of the platform with three example experiments: a Braitenberg task implemented on a mobile robot, a sensory-motor learning task based on a robotic controller, and a visual tracking embedding a retina model on the iCub humanoid robot. These use-cases allow to assess the applicability of the Neurorobotics Platform for robotic tasks as well as in neuroscientific experiments.

Developing neuro-inspired computing paradigms that mimic nervous system function is an emerging field of research that fosters our model understanding of the biological system and targets technical applications in artificial systems. The computational power of simulated brain circuits makes them a very promising tool for the development for brain-controlled robots. Early phases of robotic controllers development make extensive use of simulators as they are easy, fast and cheap tools. In order to develop robotics controllers that encompass brain models, a tool that include both neural simulation and physics simulation is missing. Such a tool would require the capability of orchestrating and synchronizing both simulations as well as managing the exchange of data between them. The Neurorobotics Platform (NRP) aims at filling this gap through an integrated software toolkit enabling an experimenter to design and execute a virtual experiment with a simulated robot using customized brain models. As a use case for the NRP, the iCub robot has been integrated into the platform and connected to a spiking neural network. In particular, experiments of visual tracking have been conducted in order to demonstrate the potentiality of such a platform.

This work intends to describe and demonstrate a haptic prototype specifically developed to work with the software application Augmented Chemical Reactions. The main goal of the application and the additional sensorial device is to improve the user immersion in chemistry learning by offering 3D molecules visualization, on a monitor, by using augmented reality technology. The application allows students to manipulate the molecules using patterns for the interface instead of the common mouse and keyboard interface.

This paper presents a new and innovative gesture-based text-input concept designed for high-performance blind-typing on mobile devices with a touch-sensitive surface on their back-side. This concept is based on the Gestyboard concept which has been developed by the Technische Universität München for stationary use on larger multi-touch devices like tabletop surfaces. Our new mobile concept enables the user to type text on a tablet device while holding it in both hands, such as the thumbs are in the front of the tablet and the other eight fingers are in the back. The user can hence type text using these fingers on the back of the device. Although, the gesture-based finger movements are quite unfamiliar and the participants need to mentally rotate the QWERTY layout by -90 and 90 degrees respectively, our multi-session evaluation shows that despite the fact that their fingers are occluded by the tablet, our concept enables the users to blind-type and that they improve their performance in each session. Consequently, the user can use all ten fingers simultaneously to type text on a mobile touchscreen device while holding it comfortably in both hands. This implies that our concept has a high potential to yield to an high-performance text-input concept for mobile devices in the near future.

This paper presents the second version of the Gestyboard, which is an innovative approach of text entry on multi-touch devices like tabletops or tablets. To overcome the lack of tactile feedback, we use unique gesture-to-key mappings for each finger according to the ten-finger touch-typing method. As a key feature, the Gestyboard only accepts keystrokes when they are performed with the finger corresponding to the ten-finger touch-typing method. This way, missing a keystroke is not possible, and therefore blind typing is naturally supported by the concept. The first version of the Gestyboard was optimized according to the qualitative and quantitative results of our first formal evaluation. This paper presents two new evaluations which give new insights on the comparative performance and conceptual improvements of the Gestyboard. In the second evaluation, our participants reached a speed of 108 cpm (characters per minute [21.6wpm]) and an error rate of 4% which is close to the performance of standard users on classic touchscreen keyboards. The third evaluation additionally revealed that our participants increased their typing speed with the Gestyboard by 44% and decreased their error rate by 48% in just 3 trial sessions. This steep learning curve is mostly due to the familiarity to the QWERTY layout.

The spatial understanding of chemical molecules is crucial for learning chemistry at school. With a good 3D understanding of molecules, chemical processes become obvious compared to a 2D representation in textbooks or just the molecular formula. With the increasing spread of computers, smartphones and tablets, the field of computer aided learning becomes more and more important. Common molecular viewers such as Jmol (Jmol, 2012) present chemical simulations as 3D renderings on a regular computer screen in combination with desktop-based user interfaces using a mouse and a keyboard to manipulate 3D molecules. Such interfaces may be cumbersome to use since users have to associate 2D mouse motion and key presses with 3D object motions. In this paper we investigate the hypothesis that the understanding of spatial structures of molecules is enhanced by Augmented-Reality-based 3D user interfaces with which students can directly manipulate the virtual 3D molecules by freely moving and rotating a 3D object in air with their hands. Our results show that a direct manipulation 3D user interface improves the 3D understanding in comparison to the traditional desktop-based user interface with mouse and keyboard.

This demonstration shows an Augmented Reality tool to support teaching chemistry. The understanding of spacial relations in and between molecules is an essential part that has to be understood by students to learn chemistry As nowadays the techniques to show and simulate molecular behaviors get faster and better, 3D applications to show the molecules become more and more popular also in schools. Augmented Chemical Reactions is an application that shows the 3D spatial structure of molecules as well as the dynamics of the atoms in and between molecules. This application does not use the commonly used 3D user interface of mice and keyboards to move and rotate the virtual objects but it makes use of an intuitive 3D user interface. This 3D User interface is an direct manipulation user interface using the augmented reality technique. With this user interface it enables the users to better understand the spacial structure of the shown geometries.

Accurate, accessible, and realtime information on the number, location, and medical condition of patients are critical for the successful management of mass casualty incidents (MCIs), where the number of patients exceeds the capacity of the emergency management service (EMS). We present a concept of a collaborative infrastructure which generates a common operation picture in realtime. A complex, stressful and uncommon situation like an MCI creates strong psychological influences and burdens on the rescue workers. Based on our psychological findings we derived eleven special requirements for efficient and intuitive user interfaces in unstable, time-critical emergency situations. Taking the requirements into consideration we developed a concept to overcome the MCI through the combination of multiple devices. The devices are carefully chosen according to the task of the EMS personnel in the field as well as in the incident command post. Three different interfaces - PDAs for the rescue units in the field, tablet PCs for the incident commanders and a multitouch table in the incident command post - help the entire rescue team to gain efficient situational awareness.

The calibration of optical see-through head-mounted displays is an important fundament for correct object alignment in augmented reality. Any calibration process for OSTHMDs requires users to align 2D points in screen space with 3D points in the real world and to confirm each alignment. In this poster, we present the results of our empiric evaluation where we compared four confirmation methods: Keyboard, Hand-held, Voice, and Waiting. The Waiting method, designed to reduce head motion during confirmation, showed a significantly higher accuracy than all other methods. Averaging over a time frame for sampling user input before the time of confirmation improved the accuracy of all methods in addition. We conducted a further expert study proving that the results achieved with a video see-through head-mounted display showed valid for optical see-through head-mounted display calibration, too.

Systems using two-handed spatial manipulation techniques also require strategies to enable system control tasks. These strategies make it possible to interact with the system comfortably while controlling two hand-held objects simultaneously. The Augmented Chemical Reactions project makes intense use of such two-handed interaction tasks. Users control virtual molecules and subsets that are registered to physical markers and try to combine those by selecting and then confirming a specific bond. When a desired bond has been selected, the user needs a way to confirm that bond without letting an atom go out of position. We developed and investigated two separate methods of confirming a selected bond when both hands are already doing a two-handed symmetric interaction task. The first method is a waiting method and the second method is a back&forth motion gesture. We evaluated the two methods in a user study, showing that the first technique, holding still, outperforms the other technique.

For molecular modeling, chemical structures have to be understood and imagined both in their three-dimensional spatial extent and in their dynamic behavior. We have developed an AR-based system for tangible interaction with molecules using optical markers. When users bring several molecules close to one another, potential bonds are shown and the molecules dynamically change their 3D structure according to potential chemical reactions. A problem arises when users also need to select one such bond from of a multitude of potential bonds while already using both hands to manipulate the molecules. We present two gesture-based techniques, shake-based and proximity-based to solve this problem. We report on user tests evaluating these techniques with respect to speed, precision and user acceptance.

unchen, Germany Abstract: This paper describes an approach for increasing the understanding and ease the learning of chemistry for students by visualizing and controlling virtual models of molecules in a intuitive way. With the Help of Augmented Reality, we developed a tool with the name Augmented Chemical Reactions. This program visualizes models of molecules rendered to a camera picture at the position of special markers hold in the hands of the users. The intuitive controlling of the position and orientation of the molecules is done by moving and rotating the markers in front of a camera, so the virtual objects behave as they would have been manipulated themselves. For a better understanding of the subject of chemistry, Augmented Chemical Reactions also shows the dynamic deformation of molecules when they come close to each other. Here the users can have a better view on certain behaviours between molecules. This program has the potential to increase the understanding and ease learning chemistry because of its intuitive controlling of the 3D structure of molecules. In the same way that Augmented Chemical Reactions can help students, it also has the potential to speed up the process of designing new molecules. Scientists can inspect the created molecules and see, if they meet the spatial requirements for specific reactions. This approach prevents time consuming reactions in the laboratory to create those molecules and test them for their desired attributes.

Spatial understanding and the understanding of dynamic change in the spatial structure of molecules during a reaction is essential for designing new molecules. Knowing the physical processes in the reactions helps to speed up the designing process. To support the designer with the correct representation of the designed molecule as well as showing the dynamic behavior of the whole reacting system is the goal of our application. Our system shows the spatial deformation of the molecules at every time interval by minimizing the energy level of the molecules. The position and orientation of the molecules can be intuitively controlled by manipulating objects of the real world using Augmented Reality techniques. Our approach has the potential to speed up the design of new molecules and help students to understand the chemical processes better. Keywords—Augmented Chemical Reactions, Augmented Reality, chemistry, education.