A museum hybrid space combines physical artifacts co-located with virtual and augmented reality displays. Although the technology exists to provide museums with hybrid space, there are no empirical studies on effectiveness of the museum hybrid space in terms of learning and enjoyment. This article takes an experimental approach and measures the enjoyment and learning (dependent variables) of participants in response to selected environments (independent variables) including a traditional environment (based on photos and labels), a video-enhanced environment (based on projected video clips), and a VR-enhanced environment (based on video game). The main outcome of this article is demonstrating that the use of VR technology and the resulting hybrid space (i.e., VR-enhanced environment) results in novel museum experiences that provide greater impacts on audience in terms of learning and enjoyment.

New media and technology are changing the museum experience in the twenty-first century. One such change is that of hybrid space in museums. A museum hybrid space combines physical artifacts co-located with virtual and augmented reality displays. Although the theory and technology exist to provide museums with hybrid space, there are few efforts to put hybrid space, particularly those that utilize commercial video games, into practice. The goal of this research is to explore how a certain type of museum hybrid space, namely, a virtual reality-enhanced environment relying on a commercial video game, can support and improve audience experiences. To reach this goal, a cognitive model is applied in the design of an experimental context that creates an exhibition-like environment for the viewer-participants. In the experiment, a virtual reality-enhanced environment is compared with two environments relying on commonly used media. Results show improvements in viewer-participants' experiences in terms of cognitive and edutainment aspects. Relying on a commercial video game, the VR-enhanced environment stimulated emotions and increased engagement in viewer-participants while helping them enjoy learning. The experimental context of this research can only approximate the full real-world experience that a museum visitor would have. However, the experimental context does provide the basic elements that a viewer would expect and associate with an exhibition, such as, objects for examination, labels, and didactic supports. Results from this research can encourage further investigations of hybrid space in other environments relying on various media types.

In addition to the actual space and virtual space, there seems to be a third type that can be called hybrid space. Hybrid space borrows the power of information to empower the physical space around us using technologies such as augmented reality, virtual reality, and augmented virtuality. Hybrid space has been explored and conceptualized in the literature, but it has yet to reach its potential as an effective medium in museums. However, it seems to have quite a few advantages to be employed in the museums to attract more people, motivate a higher participation, and change the existing paradigms by reinventing museums. This article applies a qualitative content analysis to a sample of publications to conceptualize hybrid space and position it in a suggested continuum of space. Moreover, the role of technology and considerations about the museum content in a hybrid space are explored. The aim of this theoretically and technologically oriented article is to promote the professional use of the hybrid space in museums.

The number of video games that are developed based on real historical events and evidence is increasing. These history-based video games provide learning opportunities to players, but a certain type of such games—first- and third-person shooters—has not been carefully examined for their potentials. Knowing what players say about their game experience—even if the information and knowledge are inaccurate—helps researchers understand what type of learning could happen with such games. In this article, we propose a systematic approach to assessing games as learning environments, using the method of comparing the authenticity of popular history-based video games. Through a qualitative data analysis, we studied players’ comments on the web-based communication services, such as game forums, digital distribution platforms, and discussion websites. Casual players’ conversations on these websites showed that there exist several learning potentials in the games for players, including building their understanding about history and historical forces of the time, through personally relating to specific events, social artifacts, and places.

This paper takes an interdisciplinary approach to find out what makes the experience of spaces different and how can it be enhanced? Based on a literature review this paper draws on cognitive theory to provide a model for enhancing quality of spatial experiences. The model has three stages: encouraging, enabling, and enclosing. The model asserts that in every enhanced spatial experience the audience gets encouraged at the outset by a variety of strategies such as persuasion, designing for meanings, and including concepts in design. The audience must be then enabled by special means, such as immersive and interactive capabilities of the environment along with its security and safety attributes, to get involved with the spatial experience. Consequently, the experience shifts towards a cognitive level at the enclosing stage, focusing on emotion and engagement. To compose this model, at the very beginning, essential components, dimensions, and elements of experience were identified and defined. Seven selected experts were then informed and asked to decide on the priority of the experiences elements. Finally, selected elements were employed to propose the model for enhancing quality of spatial experiences in the built environment. The proposed model is then followed by an example that clarifies how the film industry could apply the model to enhance the quality of spatial experience in the built environment of a movie theatre.

We present a method for emotional musification that utilizes the musical game MUSE. We take advantage of the strong links between music and emotion to represent emotions as music. While we provide a prototype for measuring emotion using facial expression and physiological signals our sonification is not dependent on this. Rather we identify states within MUSE that elicit certain emotions and map those onto the arousal and valence spatial representation of emotion. In this way our efforts are compatible with emotion detection methods which can be mapped to arousal and valence. Because MUSE is based on states and state transitions we gain the ability to transition seamlessly from one state to another as new emotions are detected thus avoiding abrupt changes between music types.

Purpose: Muscle strengthening exercises have been shown to improve pain and function in adults with knee osteoarthritis (OA), but individual responses can vary widely. Moreover, identifying which patients will respond to an exercise intervention is an important step in developing a more efficient model of care. Recently, we have demonstrated the ability to use baseline gait kinematic and patient reported outcome (PRO) data to predict an individual with knee OA's response to a hip/core strengthening program. However, many clinicians do not have access to the 3-dimensional (3D) motion capture systems needed to collect kinematic gait data. Alternatively, wearable sensors can provide a unique, clinically accessible alternative to acquire 3D gait data. Therefore, the purpose of this study was twofold: (1) to determine if data from a wearable sensor array can replicate the predictive model developed using 3D gait kinematic data, and (2) to determine the optimal array of wearable sensors to predict post-intervention response to a 6-week hip strengthening exercise intervention for patients with knee OA. Methods: Thirty-nine knee OA patients completed a 6-week hip strengthening program and were sub-grouped as Non-Responders, Low-Responders, or High-Responders following the exercise intervention, based changes in PRO data (Knee Injury and Osteoarthritis Outcome Score (KOOS) subscales). Gait data were collected using tri-axial accelerometers placed on the lower back (L3), and the lateral thigh, shank, and dorsal foot of the most affected limb (Fig. 1). 3D linear acceleration data from each sensor were then segmented into gait cycles and time-normalized to 60 points for stance phase and 40 points for swing phase before computing an average vertical, anteroposterior, and mediolateral waveform for each subject. A principal component analysis was used as a data reduction technique within each sensor, before combining these data with KOOS parameters to create a gait and PRO based feature set for classifying treatment response in a linear discriminant analysis. The 10 × 10-fold cross validation accuracy was examined over all 15 possible combinations of sensor arrays and statistically compared using an analysis of variance with post-hoc Tukey tests (α = 0.05). Results: Responder subgroups were not significantly different in age (p = 0.55), height (p = 0.69), mass (p = 0.64), BMI (p = 0.66), or gait speed (p = 0.07). The thigh was the best sensor for classifying responder subgroups. In combination with baseline measures of pain and function, thigh sensor data was significantly better than other placements (back; p = 0.012, shank; p = 0.01, foot; p < 0.01) at correctly identifying responder subgroups (average 10-fold cross-validation accuracy = 73.3%). The best combination of multiple sensors was the back, thigh, and shank, which displayed an 83.1% classification accuracy (Fig. 2). This three sensor combination was significantly different from all other sensor combinations, except for the back and thigh (78.9%; p = 0.16). Conclusions: The current results support our previous findings using kinematic data from a 3D motion capture system and show that data from wearable sensors can accurately predict response to exercise. Moreover, results from this study suggest that the predictive model can be replicated using a limited number of wearable sensors. Similar to our previous findings, the motion at the thigh was most important for determining which individuals with knee OA will respond best to a hip/core strengthening program. Further, these findings suggest that while three sensors can achieve best results, using only two sensors placed at L3 and the lateral thigh can achieve equivalent results. Although further validation is required, this research is a significant step in developing an accessible and objective wearable sensor system to help clinicians make evidence-informed decisions regarding optimal treatment for patients with knee OA.

Background: Muscle strengthening exercises consistently demonstrate improvements in the pain and function of adults with knee osteoarthritis, but individual response rates can vary greatly. Identifying individuals who are more likely to respond is important in developing more efficient rehabilitation programs for knee osteoarthritis. Therefore, the purpose of this study was to determine if pre-intervention multi-sensor accelerometer data (e.g., back, thigh, shank, foot accelerometers) and patient reported outcome measures (e.g., pain, symptoms, function, quality of life) can retrospectively predict post-intervention response to a 6-week hip strengthening exercise intervention in a knee OA cohort.Methods: Thirty-nine adults with knee osteoarthritis completed a 6-week hip strengthening exercise intervention and were sub-grouped as Non-Responders, Low-Responders, or High-Responders following the intervention based on their change in patient reported outcome measures. Pre-intervention multi-sensor accelerometer data recorded at the back, thigh, shank, and foot and Knee Injury and Osteoarthritis Outcome Score subscale data were used as potential predictors of response in a discriminant analysis of principal components.Results: The thigh was the single best placement for classifying responder sub-groups (74.4%). Overall, the best combination of sensors was the back, thigh, and shank (81.7%), but a simplified two sensor solution using the back and thigh was not significantly different (80.0%; p = 0.27).Conclusions: While three sensors were best able to identify responders, a simplified two sensor array at the back and thigh may be the most ideal configuration to provide clinicians with an efficient and relatively unobtrusive way to use to optimize treatment.

Synchronized, audio feedback capitalizes on the innate human ability to entrain motion to a rhythmic, audio stimulus, and has been used in training athletes and treating gait disorders. For such feedback to be effective, it is necessary to close the loop to achieve mutual, bi-directional synchronization between human and machine. Although we know we can do the synchronization within an expensive, static motion capture facility, we seek low-cost, portable methods suitable for individual and outpatient use. This paper presents a phase-entrained particle filter that synchronizes to the periodic silhouettes of a gait, using only a simple camera as a sensor. Tests with eight participants show reliable synchronization, and timing errors in sonic stimulus generation less than 60ms, or a 1/32 note in a 120bpm march. While these errors are just audible, they do not confound mutual human-machine synchronization.

The aim of this study was to determine the test-retest reliability of linear acceleration waveforms collected at the low back, thigh, shank, and foot during walking, in a cohort of knee osteoarthritis patients, by applying two separate sensor attitude correction methods (static attitude correction and dynamic attitude correction). Linear acceleration data were collected on the subjects׳ most affected limb during treadmill walking on two separate days. Results reveal all attitude corrected acceleration waveforms displayed high repeatability, with coefficient of multiple determination values ranging from 0.82 to 0.99. Overall, mediolateral accelerations and the thigh sensor demonstrated the lowest reliabilities, but interaction effects revealed only mediolateral accelerations at the thigh and foot sensors were different than other axes and sensor locations. Both attitude correction methods led to improved reliability of linear acceleration waveforms. These findings suggest that while all sensor locations and axes display acceptable reliability in a clinical knee osteoarthritis population, the back and shank locations, and the vertical and anteroposterior acceleration directions, are most reliable.

The use of sound feedback to assist in learning and correcting motor skills has some innate advantages. For example, sound leaves a person's eyes free for important tasks like avoiding obstacles, and ample psychological evidence shows a connection between the synthesis and perception of motion and sound. When motion is periodic, as it is with most forms of human locomotion, the sound feedback should also be periodic, i.e., rhythmic and synchronized to a person's motion. We present a method to track an athlete performing a known movement in real-time using an RGB-D camera. We use a Kalman filter in conjunction with iterative closest point to synchronize incoming data against a known movement pattern and produce a musical rhythm. The athlete and the rhythm move in phase with one another. We call this process rhythmic sonification. We demonstrate our method by tracking and rhythmically sonifying an advanced in-line speed skating movement called double-push. Rhythmically sonified feedback provides a new opportunity to teach and reinforce this and other athletic skills.

This paper discusses a collaborative project that inquires about the possibilities of the tactile sensation in art and its ability to reestablish human sensory relationships with consumer technology. It introduces the visitor to the interaction style practiced by the artists with their medium. The described interactive sculpture predisposes the visitor to explore tactile interaction as an aesthetic experience within multimodal multisensory system. Disturbed System takes the visitor's experience with the artwork into unfamiliar sensory territory. Touching the soft silicone surface of the sculpture provides electronic feedback of embedded vibration and directional spatialized sound in an installation format. The artwork presents this sensory information in a form of unexpected assemblage of pulsing organic sculptural surfaces and emitting sound. It also places visitors in a shared interactive space, an aura of traveling sound warped by their touch of the sculpture. This shared interaction investigates relationships between nature, artifice, technology, human body and human social group behavior.

Modern surveillance systems for practical applications with diverse and mobile sensors are large, complex, and expensive. It is known that unexpected behaviors can emerge from such systems, and when these behaviors correspond to weaknesses in a surveillance system, we call them emergent vulnerabilities. Given their cost and importance to security, it is essential to test these systems for such vulnerabilities prior to deployment. To that end, we automate the testing process with multiagent systems and machine learning. However, the conventionaland most intuitive-approach is to focus the machine learning on the subject system, which leads to a high-dimensional problem that is intractable. Instead, we demonstrate in this paper that learning attacks on the system is tractable and provides a viable testing method. We demonstrate this with a series of studies in simulation and with a small-scale model system featuring elements typically found in real physical surveillance systems. Our machine learning method finds successful attacks in simulation, which we can duplicate with the physical system. The method is scalable, with the implication that it could be used to test larger, real surveillance installations.

We present the application of a vision algorithm based on Harr-like features in Bubblegrams a new mixed reality-based human-robot interaction (HRI) technique. Bubblegrams allows humans and robots working on collocated synchronous tasks to interact directly by visually augmenting their shared physical environment. Bubblegrams uses comics-like interactive graphic balloons or bubbles that appear above the robot’s body and allow intuitive interaction with the robot. Users wear light-weight mixed reality goggles that integrate displays and a camera, allowing the user to view and interact with the physical environment as well as with the virtual Bubblegrams interface linked to the robot’s body. In order to efficiently link Bubblegrams in real-time to the physical robot we implemented a vision algorithm based on Harr-like features which is the main topic of this paper. This paper briefly details the design of the Bubblegrams interface, the hardware and software we use for the current prototype, and the full details of the vision algorithm.

Traditionally, interactivity in architecture has been suppressed by its materiality. Building structures that can transform and change themselves have been the dream of many architects for centuries. With the continuous advancements in technology and the paradigm shift from mechanics to electronics, this dream is becoming reality. Today, it is possible to have building facades that can visually animate themselves, change their appearance, or even interact with their surroundings. In this paper, we introduce Architectural Chameleon Skin (ARCS), an installation that has the ability to transform static, motionless architectural surfaces into interactive and engaging skins. Swarm algorithms drive the interactivity and responsiveness of this "virtual skin". In particular, the virtual skin responds to colour, movements, and distance of surrounding objects. We provide a comprehensive description and analysis of the ARCS installation.

Traditionally, interactivity in architecture has been suppressed by its materiality. Building structures that can transform and change themselves have been the dream of many architects for centuries. With the continuous advancements in technology and the paradigm shift from mechanics to electronics, this dream is becoming reality. Today, it is possible to have building facades that can visually animate themselves, change their appearance, or even interact with their surroundings. In this paper, we introduce Architectural Chameleon Skin (ARCS), an installation that has the ability to transform static, motionless architectural surfaces into interactive and engaging skins. Swarm algorithms drive the interactivity and responsiveness of this “virtual skin”. In particular, the virtual skin responds to colour, movements, and distance of surrounding objects. We provide a comprehensive description and analysis of the ARCS installation.

A unique problem associated with the movements of a speed skating athlete inspired this practical work, looking into the question of using interactive auditory feedback to improve sporting movements and sporting movement acquisition. Presented here is a method for synchronizing the periodic movements of a subject against the movements of a model and sonifying that synchronization data in real-time. The sonic feedback is designed to convey information related to how the movements of a subject match against those of a model. A simple, inexpensive sensor system is created to capture speed skating movements and facilitate the sonification. The effectiveness of the system is demonstrated with two case studies. The first case study involves an experienced skater who had developed a significant anomaly in his technique, who uses this system to become aware of and correct his undesired movements. The second case study involves a new and inexperienced speed skating athlete who accelerates the acquisition of speed skating skills by listening and reacting to the relationship between his movements and those of a skilled speed skating athlete. While speed skating is used to demonstrate this sonic feedback technique, the algorithms can be applied to any repetitive movements.