This study proposes a visual training system for improving baseball hitting ability using an HMD. Conventional training methods such as tee batting can train batting skills, but they cannot train the visual functions necessary for batting. The visual functions that are closely related to hitting ability in baseball include kinetic visual acuity and eye-hand coordination, and training these functions will lead to improved hitting ability. We will construct a visual training system in a VR space that can train kinetic visual acuity and eye-hand coordination while batting using an HMD. By performing batting practice while training sports vision, both batting technique and visual function can be improved. To verify whether the system improves visual function and hitting ability, training was conducted. As a result, significant improvements were observed in the number of hits, kinetic visual acuity, and eye-hand coordination, indicating the usefulness of the proposed system.

Declining visual acuity caused by ICT devices usage is often classified as pseudomyopia, a condition resulting from excessive strain and tension in the ciliary muscles. This condition can potentially be alleviated through ciliary muscle stretching. In this study, we developed a VR game that incorporates three elements effective for ciliary muscle stretching: near-far focusing exercises, distance fixation exercises, and binocular stereoscopic vision. We conducted an experiment to evaluate the effectiveness of the developed VR game in recovering visual acuity. The experiment, which focused on young adults, demonstrated a significant recovery of visual acuity over a six-week period. These findings suggest the potential of the VR game as an effective tool for improving visual acuity through ciliary muscle stretching.

In recent years, as smartphones have become a part of life of young generation, opportunities to enter text using keyboards have decreased, and the speed of keyboard text entry has declined. However, it is still necessary for young children to use keyboards. In typing using a keyboard, the correspondence between keys and fingers is defined; however, there is no practical system for learning fingering to press keys with the correct fingers. A general typing learning software can only judge whether the correct input has been given or not and does not consider the correct fingering of the keys. If the system can evaluate fingering when learning typing, the user can concentrate on typing without having to judge the correctness of their own fingering. In previous research, there have been methods to acquire fingertip coordinates by attaching color stickers to fingers as markers or by using a distance camera, but these methods lack versatility. In this study, we used a monocular camera to acquire fingertip images on a keyboard. After obtaining images, we used MediaPipe to perform hand tracking and obtain the position of the fingertip. For each frame, the system calculates the distance between each key coordinate and each fingertip coordinate, determines that the closest finger hits the key, and feeds back the fingering information.

MR fluid has high speed responsiveness and shows a shear response in a direction perpendicular to the magnetic flux direction, when a magnetic field is applied. In this research, we propose a compact and lightweight wrist-mounted resistance force presentation device utilizing the characteristics of MR fluid. By controlling the magnetic flux density applied to the rubber tube filled with MR fluid, the resistance force is presented by increasing the rigidity of the wrist joint.

In order to represent virtual objects photorealistically in augmented reality (AR), the problem of optical consistency is important. There are several methods to achieve optical consistency using known real objects and special cameras, but they are difficult to use in AR applications. In this research, we propose an end-to-end method to convert an optically inconsistent AR image into an optically consistent AR image using a generative adversarial network (GAN). In addition, we propose a GAN that focuses on the structural edges of virtual objects in order to be able to handle different virtual object shapes. We confirmed that the GAN can generate photorealistic AR images consistent with the real world and that it is possible to generate images with versatility for virtual object shapes.

In this study, we propose a pen-type interface that uses a vibration motor to reproduces the feeling of writing on paper when writing using a stylus pen on a tablet device. Conventional pen-type interfaces have two problems. First, a delay occurs between the start of writing and the start of vibration. Second, because the frequency and amplitude of the vibration are fixed, the writing feel is not reproducible. Hence, we quantify the degree of time delay that humans can tolerate by conducting sensitivity evaluation experiments. To maintain the time delay within an acceptable range, we utilize an acceleration sensor to acquire the start of a writing action and the writing speed at the interface. This approach can reduce the system delay and ensure a fast response. Based on a subjective evaluation experiment, we confirm that the proposed method does not cause any time delays. To solve the second problem, the vibration generated in the pen when writing on paper at various speeds is measured, and the linear prediction coefficient of the autoregressive model is derived. The vibrations reproduced based on the model are transmitted to a vibrator at the interface in real time to improve the reproducibility of the writing feel. Spectrogram analysis results of the generated vibration confirm that the difference in vibration characteristics based on the writing speed can be reproduced.

In this paper, we propose a method for indexing, analyzing, and evaluating of the stroking movement based on vibration features as a measurement method that mimics the response characteristics of finger associated with active touch. We developed the tactile information acquisition system to obtain the vibration information during stroking. Moreover, the verification experiment was performed using 13 types of cloth selected from the texture sample set. As the result, the two factors (roughness and hardness), which constitute the tactile sensation of the fabric, were extracted. It was also confirmed that the frequency bands of the vibration features extracted during active touch corresponded to the frequency sensitivity characteristics of the four sensory receptors inside the skin. Furthermore, we evaluated the predicted and measured values of each factor, and confirmed that the proposed method can construct a model that accurately predicts the tactile sensation by measuring the vibration of the interaction force on the contact surface. This method realizes the tactile evaluation for a various materials and physical quantities. Then, the method will contribute to the realization of quantitative evaluation of sensibility, which is the key technology to product design based on sensibility value.

3D printing technology enables us to develop a composite shape of complex structures. This study describes the methodology of how haptic materials are developed using 3D printers. We present a material library to study haptic softness and texture perception.

The force-induced displacement distribution function (FDDF) is a physical quantity with a high degree of familiarity with product production parameters and sensibility values. In this study, we proposed FDDF (Stroke) as the texture perception by stroking FDDF implementation. Moreover, we showed that FDDF (Stroke) has a highly expressive tactile perception capability for hard materials.

In this study, to enable a VR experience with an HMD even in a space with obstacles, we developed a real-time construction system of a reality-based VR space that does not impair the atmosphere of the virtual world. In addition, we aim to construct a VR space that is easier to recognize its structure by classifying "objects that are boundaries of space" and "ordinary obstacles" using a deep learning network and superimposing virtual objects corresponding to each type of real object. We implemented a real-time construction system for a VR space that reflects the distribution of objects in a real space using two depth cameras mounted on HMD, and created guidelines and applications for using the proposed system.

Various methods are used for estimating light source informations from real objects to achieve optical consistency in augmented reality (AR), but in practice, there are difficulties in using real objects. We propose a method of achieving optical consistency without estimating the light source information, using generative adversarial networks (GANs) that input AR images without optical consistency and mask image. The generated AR images from our proposed method show the appropriate expression of drop shadows and reflections of surrounding objects.

Imparting vibration to the contact surface of an object is a method of presenting a tactile sensation of the object surface. However, it is difficult to temporally and spatially present wide-band vibrations through the direct vibration generation of a general actuator. In this study, we focus on sound waves, and propose a method to present tactile sensations by generating vibrations via irradiating sound waves of various frequencies and amplitudes on a plate. In experiments using a parametric speaker as an acoustic device, the vibration of the plate during sound wave irradiation was measured; it was confirmed that different tactile sensations could be obtained by changing the frequency and amplitude of the irradiated sound wave.

The demand for a Kansei (sensibility) value-based product design has recently increased. Accordingly, the digitalization of sensory information is needed to realize the construction of a design framework that realizes the product design based on the Kansei value. No standard framework for digitalization has particularly been presented with regard to tactile information. We develop herein a novel tactile measurement and analysis scheme constructed by fusing numerical simulation, sensing, and information processing technology to construct the digitalization framework of tactile perception. By a case study for textiles, we show the example for realizing the tactile measurement using a development scheme. This study realizes a high-accuracy estimation of tactile perception based on biomechanical information and provides key technological knowledge for constructing a framework for tactile perception digitalization. Furthermore, this study promotes the product design based on the Kansei value.

In the case of the product purchase decision making in e-commerce, the tactile sensation a user can obtain from products is restricted. Our goal is to construct a framework for tactile measurement and presentation to remove such restrictions. To realize high-accuracy tactile presentation, we analyze the mechanical characteristics of the fingertip. We measured and analyzed the interactional force of the contact face when stroking the surface of fabrics. We found a filter-bank like structure of vibration information processing on the fingertip. In addition, we propose a tactile presentation scheme based on this physiological characteristic. To evaluate the validity of the proposed scheme, we produced a tactile presentation device for fabrics and evaluated the reproducibility of tactile sensation. The results demonstrate that it is possible to represent basic material texture characteristics, such as roughness and softness. This study provides key technological knowledge for constructing a framework for tactile measurement and presentation. Furthermore, this study promotes the use of sensory information relative to tactile sensation in e-commerce, such as the visual and auditory senses.

-It is said that the form of the perceptual cross-modal integration, including visuo-tactile integration, is basically weighted average. In this study, we investigated whether the visuo-tactile information about the impression, the product of the higher-level cognitive process, integrates in this way or not. The results suggested that the visuo-tactile impression of textures could be regarded as the weighted average of the visual and tactile impression. Further, the ratio of modality weights seemed to be unique to the evaluation and it would reflect the ratio of the likelihood, or the reliability of modality in the evaluation.

SUMMARYHand motion capture is an important topic for understanding the mechanism of the hand. General hand motion capture methods using cameras or bent sensors can capture the limited action in the restricted environments. This study proposes a novel hand motion capture method without restrictions of usage environment and user’ action. The proposed method is based on the electrical contact resistance between the wrist skin and the electrode to measure the deformation of the wrist which is related to the hand motion. We fabricate the contact resistance measurement circuit consists of three electrodes and a multiplexer. The output voltage corresponding to each contact resistance of the electrode is measured with the circuit by switching the working electrode temporally. We confirm that the output voltage is related to the wrist shape and changes according to the hand posture. Furthermore, the result suggests that the finger joint angles can be estimated from the output voltage due to the correlation between them.

With the development of three-dimensional computer graphics (3DCG) technology, expressing various objects and phenomena graphically became possible. However, there is no method for quantitatively evaluating the realism of the generated CG images. In recent years, deep learning has been widely used for its image discrimination performance beyond that of human beings. We propose a deep learning-based method that enables quantitative evaluation of the realism of CG images with has high image discrimination ability. The results of implementation using convolution neural networks (CNNs) are presented.

This paper proposes The Rainbow Marker, a planar marker for estimating the direction of a light source using a structural color. A structural color is a color produced by microscopically structured surfaces that vary in appearance according to the viewpoint, the direction and the spectrum of the light source. The proposed marker contains a planar material which causes structural coloration. The direction of the light source is estimated by structural color pattern matching between an input pattern and referential color patterns. In this paper, two types of the marker were implemented, with a grating sheet and with a holographic sheet, to demonstrate that the proposed method is applicable in the field of augmented reality.