Reconstructing the 3D facial expressions of head-mounted display (HMD) wearers is essential for natural avatar communication in virtual reality (VR). Camera-based methods achieve high fidelity but require heavy processing and raise privacy concerns. In contrast, non-imaging sensors are lightweight and privacy-preserving, but they provide only sparse features. We propose a reconstruction system that learns high-dimensional 3D facial representations from camera images during training and performs inference using only compact photo-reflective sensors embedded in the HMD. This design combines the expressiveness of camera-based supervision with the efficiency and privacy of sensor-based operation at inference time. Experimental results show that the system reconstructs 3D facial expressions from sensor data alone at inference time. Accuracy improves more from diverse HMD attachment conditions than from additional samples collected under a single attachment condition. A dedicated jaw-pose predictor and lightweight personalization with small wearer-specific datasets further reduce reconstruction error. A sensor ablation experiment further shows that sensor placement within a region is more important than regional coverage, and that glabella and nose sensors are particularly critical for reconstruction. Furthermore, we show that wear-invariant pretraining improves cross-wear robustness under one-set calibration. In this cohort, a calibration analysis identifies informative expression classes and yields a reduced-expression calibration protocol, in which users perform the first K expressions in a cohort-derived common order; with K=20, this protocol matches the full 30-expression calibration baseline.

To verify whether the Mott insulator Ca2RuO4 can be switched by applying electric-field alone, regardless of current flow, we employ metallisation using electric double-layer gating (EDLG). The resistance change due to EDLG occurs only when positive gate-voltage above +3 V is applied. The amplitude of the reduction, reaching similar to 97% of the initial value, is difficult to interpret as surface metallisation and is likely related to structural change in bulk.

Layered ruthenium oxide Ca2RuO4 is a Mott insulator and its frozen state can be melted by various stimuli such as temperature variation, chemical substitution, pressure, and current application. In this study, we have investigated the variation of the electronic structure of Ca2RuO4 with temperature and electric current by means of hard x-ray photoemission spectroscopy to gain valuable insights into the origin of the insulator-metal transition (IMT). We have observed the reduction in the insulating gap as well as the suppression of the spectral weight for the lower Hubbard band with increasing temperature and electric current density. The variation of the insulating gap well accounts for the characteristic nonlinear conductivity. Our spectroscopic results demonstrate the close relationship between the octahedral distortion of RuO6 and the IMT in Ca2RuO4. Furthermore, to clarify the origin of the observed current-induced phenomena, the temperature increase of the sample due to electric current was evaluated on the basis of the Joule self-heating model. The simulated results revealed that the heating effects have a significant influence on the gap suppression and the spectral changes under current. However, this cannot explain all of the observed spectral changes under current, implying the possibility of an intrinsic current-induced effect.

This paper presents an interactive virtual fishing system that offers continuous reel weight and line slack using a fishing rod-type device and a grounded device. We detect the rotation of the reel and the pulling force of the fishing rod with a current sensor from a hand-crank generator and a load cell, respectively. To render the reel weight, we leveraged a phenomenon of the resistive rotational force change due to the electrical resistance of a hand-crank generator. Also, our system provided the slack of the fishing line by the electromagnets connected to the grounded device. We demonstrate a tug-of-war game between an angler and a fish using our technology. The angler uses our system to catch the fish; the fish escapes from the angler by moving their legs.

When humans generate stimuli voluntarily, they perceive the stimuli more weakly than those produced by others, which is called sensory attenuation (SA). SA has been investigated in various body parts, but it is unclear whether an extended body induces SA. This study investigated the SA of audio stimuli generated by an extended body. SA was assessed using a sound comparison task in a virtual environment. We prepared robotic arms as extended bodies, and the robotic arms were controlled by facial movements. To evaluate the SA of robotic arms, we conducted two experiments. Experiment 1 investigated the SA of the robotic arms under four conditions. The results showed that robotic arms manipulated by voluntary actions attenuated audio stimuli. Experiment 2 investigated the SA of the robotic arm and innate body under five conditions. The results indicated that the innate body and robotic arm induced SA, while there were differences in the sense of agency between the innate body and robotic arm. Analysis of the results indicated three findings regarding the SA of the extended body. First, controlling the robotic arm with voluntary actions in a virtual environment attenuates the audio stimuli. Second, there were differences in the sense of agency related to SA between extended and innate bodies. Third, the SA of the robotic arm was correlated with the sense of body ownership.

An appealing direction to change the properties of strongly correlated materials is to induce nonequilibrium steady states by the application of a direct current. While access to these novel states is of high scientific interest, Joule heating due to current flow often constitutes a hurdle to identify nonthermal effects. The biggest challenge usually resides in measuring accurately the temperature of a sample subjected to direct current, and to use probes that give direct information of the material. In this work, we exploit the simultaneous measurement of electrical transport and magnetisation to probe non-equilibrium steady states in Ca2RuO4. In order to reveal non-thermal current-induced effects, we employ a simple model of Joule self-heating to remove the effects of heating and discuss the importance of temperature inhomogeneity within the sample. Our approach provides a solid basis for investigating current-induced phenomena in highly resistive materials.

As one of the techniques to recognize head-mounted display (HMD) user's facial expressions, the photo-reflective sensor (PRS) has been employed. Since the classification performance of PRS-based method is affected by rewearing an HMD and difference in facial geometry for each user, the user have to perform dataset collection for each wearing of an HMD to build a facial expression classifier. To tackle this issue, we investigate how transfer learning improve within-user and cross-user accuracy and reduce training data in the PRS-based facial expression recognition. We collected a dataset of five facial expressions (Neutral, Smile, Angry, Surprised, Sad) when participants wore the PRS-embedded HMD five times. Using the dataset, we evaluated facial expression classification accuracy using a neural network with/without fine tuning. Our result showed fine tuning improved the within-user and cross-user facial expression classification accuracy compared with non-fine-tuned classifier. Also, applying fine tuning to the classifier trained with the other participant dataset achieved higher classification accuracy than the non-fine-tuned classifier.

There is a method to recognize facial expressions of Head-Mounted Display (HMD) wearers by machine learning of reflection intensity information from photo-reflective sensors embedded into the interior of an HMD [1]. This study evaluates whether facial expression recognition accuracy can be improved by using a learning model that considers temporal changes in sensor values. We assessed whether facial expression recognition accuracy could be improved by adding the head posture data acquired from the Inertial Measurement Unit (IMU) in the HMD to the discriminator input and performing time-series learning. The experimental results showed that PRS-based facial expression recognition with time-series data was more accurate than without. The multimodal recognition using the reflection intensity and head posture data was slightly more accurate than the discrimination using only the reflection intensity information. It was especially effective for the learning condition without considering the time-series.

The resistivity of materials is a fundamental property of solids and is widely used to understand underlying physics as well as to engineer device applications. Conventional four-probe measurement is usually employed to exclude the contributions from parasitic contact resistances. Here, we evaluate the in-plane resistive anisotropy in Ca 2 RuO 4 crystals by using a rotational square four-point probe (4PP) method, which measures an angular dependence of the resistance to precisely detect the resistive anisotropy of materials. A clear sinusoidal dependence of the resistance has been observed, confirming the resistive anisotropy in this system. Finally, the resistance data is fitted with a theoretical angle dependence equation to extract the resistive anisotropy. The observed results are found to be matching with the actual resistivity of the sample.

As one of the facial expression recognition techniques for Head-Mounted Display (HMD) users, embedded photo-reflective sensors have been used. In this paper, we investigate how gaze and face directions affect facial expression recognition using the embedded photo-reflective sensors. First, we collected a dataset of five facial expressions (Neutral, Happy, Angry, Sad, Surprised) while looking in diverse directions by moving 1) the eyes and 2) the head. Using the dataset, we analyzed the effect of gaze and face directions by constructing facial expression classifiers in five ways and evaluating the classification accuracy of each classifier. The results revealed that the single classifier that learned the data for all gaze points achieved the highest classification performance. Then, we investigated which facial part was affected by the gaze and face direction. The results showed that the gaze directions affected the upper facial parts, while the face directions affected the lower facial parts. In addition, by removing the bias of facial expression reproducibility, we investigated the pure effect of gaze and face directions in three conditions. The results showed that, in terms of gaze direction, building classifiers for each direction significantly improved the classification accuracy. However, in terms of face directions, there were slight differences between the classifier conditions. Our experimental results implied that multiple classifiers corresponding to multiple gaze and face directions improved facial expression recognition accuracy, but collecting the data of the vertical movement of gaze and face is a practical solution to improving facial expression recognition accuracy.

Introduction: Incorporating an additional limb that synchronizes with multiple body parts enables the user to achieve high task accuracy and smooth movement. In this case, the visual appearance of the wearable robotic limb contributes to the sense of embodiment. Additionally, the user’s motor function changes as a result of this embodiment. However, it remains unclear how users perceive the attribution of the wearable robotic limb within the context of multiple body parts (perceptual attribution), and the impact of visual similarity in this context remains unknown.Methods: This study investigated the perceptual attribution of a virtual robotic limb by examining proprioceptive drift and the bias of visual similarity under the conditions of single body part (synchronizing with hand or foot motion only) and multiple body parts (synchronizing with average motion of hand and foot). Participants in the conducted experiment engaged in a point-to-point task using a virtual robotic limb that synchronizes with their hand and foot motions simultaneously. Furthermore, the visual appearance of the end-effector was altered to explore the influence of visual similarity.Results: The experiment revealed that only the participants’ proprioception of their foot aligned with the virtual robotic limb, while the frequency of error correction during the point-to-point task did not change across conditions. Conversely, subjective illusions of embodiment occurred for both the hand and foot. In this case, the visual appearance of the robotic limbs contributed to the correlations between hand and foot proprioceptive drift and subjective embodiment illusion, respectively.Discussion: These results suggest that proprioception is specifically attributed to the foot through motion synchronization, whereas subjective perceptions are attributed to both the hand and foot.

The fate of a Mott insulator under strong low frequency optical driving conditions is a fundamental problem in quantum many-body dynamics. Using ultrafast broadband optical spectroscopy, we measured the transient electronic structure and charge dynamics of an off-resonantly pumped Mott insulator Ca_{2}RuO_{4}. We observe coherent bandwidth renormalization and nonlinear doublon-holon pair production occurring in rapid succession within a sub-100-fs pump pulse duration. By sweeping the electric field amplitude, we demonstrate continuous bandwidth tuning and a Keldysh crossover from a multiphoton absorption to quantum tunneling dominated pair production regime. Our results provide a procedure to control coherent and nonlinear heating processes in Mott insulators, facilitating the discovery of novel out-of-equilibrium phenomena in strongly correlated systems.

The robotic limbs using a body remapping approach often follow either the user’s reference frame 1) fixed outside the user’s body (Space frame), 2) centered on a user’s torso (Torso frame), or 3) centered on a user’s head (Head frame). In this study, we investigate the effect of the reference frame on the sense of embodiment and the user’s motor control process. We asked 12 participants to perform some point-to-point tasks with the virtual robotic limbs remapped to the participants’ feet. The virtual robotic limbs tip followed the participants’ feet and each reference frame condition (Space frame, Torso frame, and Head frame). As a result, the reaction time, the movement straightness, and the movement priority were significantly high rate in the Torso frame condition. The subjective score on the sense of embodiment showed that the reference frame condition contributed to the sense of embodiment.

Ca2RuO4 is a transition-metal oxide that exhibits a Mott insulator-metal transition (IMT) concurrent with a symmetry-preserving Jahn-Teller distortion (JT) at 350 K. The coincidence of these two transitions demonstrates a high level of coupling between the electronic and structural degrees of freedom in Ca2RuO4. Using spectroscopic measurements with nanoscale spatial resolution, we interrogate the interplay of the JT and IMT through the temperature-driven transition. Then, we introduce photoexcitation with subpicosecond temporal resolution to explore the coupling of the JT and IMT via electron-hole injection under ambient conditions. Through the temperature-driven IMT, we observe phase coexistence in the form of a stripe phase existing at the domain wall between macroscopic insulating and metallic domains. Through ultrafast carrier injection, we observe the formation of midgap states via enhanced optical absorption. We propose that these midgap states become trapped by lattice polarons originating from the local perturbation of the JT.

We observed high harmonic generation in Mott-insulating Ca2RuO4, and found an empirical scaling law of high harmonic yields. A theoretical model considering the spin-charge effect and thermal fluctuation qualitatively reproduces the observed scaling law. Our observation and calculation indicate that high harmonic generation can be a new tool to investigate ultrafast non-equilibrium dynamics of the strongly correlated system.

Spatial cues play an important role in navigating people in both physical and virtual spaces. In spatial navigation, visual information with additional cues, such as haptic cues, enables effective guidance. Most haptic devices are applied to various body parts to make mechanical stimuli, while few devices stimulate a head despite the excellent sensitivity. This article presents Virtual Whiskers, a spatial directional guidance technique by cheek haptic stimulation using tiny robot arms attached to a Head-Mounted Display (HMD). The tip of the robotic arm has photo reflective sensors to detect the distance between the tip and the cheek surface. Using the robot arms, we stimulate a point on the cheek obtained by calculating an intersection between the cheek surface and the target direction. In the directional guidance experiment, we investigated how accurately participants identify the target direction provided by our guidance method. We evaluated an error between the actual target direction and the participant's pointed direction. The experimental result shows that our method achieves the average absolute directional error of 2.54° in the azimuthal plane and 6.54° in the elevation plane. We also conducted a spatial guidance experiment to evaluate task performance in a target search task. We compared the condition of visual information, visual and audio information, and visual information and cheek haptics for task completion time, System Usability Scale (SUS) score, NASA-TLX score. The averages of task completion time were M = 6.39 s, SD = 3.34 s, and M = 5.62 s, SD = 3.12 s, and M = 4.35 s, SD = 2.26 s, in visual-only condition, visual+audio condition, and visual+haptic condition, respectively. In terms of the SUS score, visual condition, visual+audio condition, and visual+haptic condition achieved M = 55.83, SD = 20.40, and M = 47.78, SD = 20.09, and M = 80.42, SD = 10.99, respectively. As for NASA-TLX score, visual condition, visual+audio condition, and visual+haptic condition resulted in M = 75.81, SD = 16.89, and M = 67.57, SD = 14.96, and M = 38.83, SD = 18.52, respectively. Statistical tests revealed significant differences in task completion time, SUS score, and NASA-TLX score between the visual and the visual+haptic condition and the visual+audio and the visual+haptic condition.

This work presents a method for identifying surgical field states using time-of-flight (ToF) sensors equipped with a surgical light. It is important to understand the surgical field state in a smart surgical room. In this study, we aimed to identify surgical field states by using 28 ToF sensors with a surgical light installed on each. In the experimental condition, we obtained a sensor dataset by changing the number of people, posture, and movement state of a person under the surgical light. The identification accuracy of the proposed system was evaluated by applying machine learning techniques. This system can be realized simply by attaching ToF sensors to the surface of an existing surgical light.

With the increasing popularity of robots such as teleoperated semi-autonomous robots, sense of agency (SoA) and sense of body ownership (SoBO) with semi-autonomous robots is becoming increasingly important. This study investigates the changes in the SoA and SoBO when the user control weight of the robot is altered in human-robot collaboration. Through an experiment, we compared the SoA and SoBO in a pick-and-place task using a robotic arm under the conditions in which the autonomous robotic arm shared and unshared the target with the participant. The results showed that the SoA and the SoBO increased with the increase of user control weight. However, the user control weight of 75% achieved a slightly higher SoA than that of 100%.

Adding force feedback to virtual reality applications enhances the immersive experience. We propose a prototype, featuring head-based multi-directional force feedback in a virtual environment. We designed the prototype by integrating four ducted fans into a head-mounted display. Our technical evaluation of the ducted fan revealed the force characteristics of the ducted fan, including presentable power, sound level, and latency. In the first part of our study, we investigated the minimum force that a user can perceive in different directions (forward/backward force; up/down/left/right rotational force). The result suggested the absolute detection threshold for each directional force. Following that, we evaluated the impact of using force feedback through an immersive flight simulation in the second part of our study. The result indicates that our technique significantly improved user enjoyment, comfort, and visual-and-tactile perception, and reduced simulator sickness in an immersive flight simulation.