It is a challenge to develop high spatial resolution texture display devices using electromagnetic-driven methods, as the magnetic driving force sharply decreases due to the small size of the actuators. To address this challenge, we report a high spatial resolution texture display device using a novel multi-coil superposition driving method, which consists of 25 tactile units arranged in a 5*5 matrix with a spatial resolution of 2.75mm. Using the vector superposition of magnetic fields, the driving force of the target tactile unit can be effectively enhanced through the joint driving of multiple electromagnetic coils. The experimental result indicates that the upward holding force of the target tactile unit using the proposed multi-coil superposition driving method increases by 36.2% compared to the traditional single-coil driving method.
Complex robotic tasks require human collaboration to benefit from their high dexterity. Frequent human-robot interaction is mentally demanding and time-consuming. Intuitive and easy-to-use robot control interfaces reduce the negative influence on workers, especially inexperienced users. In this paper, we present CobotTouch, a novel intuitive robot control interface with fingertip haptic feedback. The proposed interface consists of a projected Graphical User Interface (GUI) on the robotic arm to control the position of the robot end-effector based on gesture recognition and a wearable haptic interface to deliver tactile feedback on the user’s fingertips. We evaluated the user’s perception of the designed tactile patterns presented by the haptic interface and the intuitiveness of the proposed system for robot control in a case study. The results revealed a high average recognition rate of 75.25% for tactile patterns. The average NASA Task Load Index (TLX) indicated small mental and temporal demands, indicating a high level of intuitiveness of CobotTouch for interaction with collaborative robots.
The main application of virtual reality (VR) is to immerse users in the three-dimensional simulation environment and experience the virtual reality world. At present, VR products and content on the market have some problems to be solved, such as location information error, dizziness and discomfort, stereo vision error, sound mismatch and so on. Moreover, most VR applications use stereo vision perception, but that alone is not enough to fully immerse users in the VR environment. For a long time, the single way of information transmission makes people over rely on the visual channel, which leads to visual information overload. Compared with the receptive visual information obtained by a single channel, haptic interaction system is more bidirectional. The employment of tactile feedback technology in VR can provide better immersion and interaction, and expand the scope of user experience. Among the four main tactile stimuli-vibrant stimulus, pressure stimulus, electric stimulus and temperature stimulus, vibrant stimulus has higher comfort, consistency and better response speed and adjustable range. Therefore in this paper, combined with specific VR scene, external vibration was applied around the eyes to provide periocular stimulation, so as to explore the role of vibration in enhancing the immersion of VR equipments. A mathematical model of human visual-haptic interaction process is established, and it is verified that the artificial neural network model has good fitting effect in simulating human visual-haptic nervous system.
This paper presents a novel haptic device, named DeltaFinger, designed to deliver the force of interaction with virtual objects by guiding the user’s finger by a wearable delta mechanism. DeltaFinger delivers a 3D force vector to the fingertip of the index finger of the user, allowing complex rendering of various virtual reality (VR) environments. The developed device is able to render linear forces up to 1.8 N in vertical projection and 0.9 N in horizontal projection without restricting the motion freedom of the remaining fingers. The experimental results showed a sufficient precision in perception of force vector with DeltaFinger (mean angular error in the perceived force vector of 0.6 rad). The proposed device potentially can be applied to VR communications, medicine, and navigation for people with vision problems.
Aerial images that can interact with the hands and fingers inducing realistic tactile sensations and behave as though they are composed of physical substances are referred to as materialized graphics. Materialized graphics provides a natural interface that humans can handle and manipulate using skills that are inherent in any human. The technology also enables us to confirm and enjoy the tactile feeling and mediates human–human communication. This paper reports the ten-year progress of materialized graphics based on airborne ultrasound tactile displays. Early symbolic demonstrations of materialized graphics are presented, and the recent technological advances in haptic rendering that improve the realism are summarized. It explains the current non-contact display covers the sensations of static pressure and thermal interaction in addition to vibratory sensations.
Vibrotactile feedback has been receiving increasing attention as an effective modality to draw the user’s attention to an urgent situation. The current study examines the effects of alert duration and envelope on perceived urgency, annoyance, and acceptance using a dual-task condition. Participants are instructed to complete a simple arithmetic task (primary) during which the vibrotactile alert (secondary) is provided via a wristband attached to the non-dominant hand. Experiment 1 investigates the effects of the alert duration and found that an alert duration of 1,950 ms significantly increases the perceived urgency and acceptance, while significantly decreasing annoyance. Experiment 2 compares three vibration envelope patterns (constant, increasing, and decreasing intensity) and found that a constant vibration intensity of 2.25g significantly increases the perceived urgency without significantly changing the perceived annoyance and acceptance. These findings inspire the development of vibrotactile alert systems.
In tele-operated human-robot collaboration, a human operator typically engages with a distant physical environment through a robotic system equipped with multiple sensors and actuators, allowing for haptic-based precise manipulation. Although these technical systems have been in use for years, the connection between multisensory perception and action in peripersonal space during tele-operations remains less understood. To delve deeper into this relationship, we examined distance perception in virtual peripersonal space. Participants wore an HTC Vive head-mounted display (HMD) featuring integrated eye-tracking (SMI) and moved a comparison object (a yellow ball) towards a target object (a blue ball) using a Geomagic Touch haptic device stylus, receiving either force feedback (‘closed-loop’) or no force feedback (‘open loop’) during the operation. They were instructed to focus on fixation points while performing the task, with SMI eye-tracking monitoring their gaze. The spatial positions of the comparison and target objects were arranged in four layouts: (i) center-to-center, (ii) center-to-peripheral (20 degrees in visual eccentricity), (iii) peripheral-to-center, and (iv) peripheral-to-peripheral. We employed seven distance levels between the objects in Experiment 1 and five distance levels in Experiment 2, using consistent methods of stimuli presentation. The findings revealed that estimation errors were significantly influenced by force feedback, spatial arrangement, and distance. Crucially, the visibility of the movement trajectory enhanced the effectiveness of tactile force feedback. Overall, this study proposes a potential guideline for human-computer ergonomic design, emphasizing the importance of force feedback for accurate targeting.
Although electromagnetic braille displays have fast response, reliable performance, and low price, they also have high power consumption and are susceptible to heat generation. For the development of a braille display, in this paper, we propose a solution based on an adaptive network-based fuzzy inference system(ANFIS). First, we analyze and optimize the driving voltage of the electromagnetic driver, the resistance value of the electromagnetic coil, the fingertip touch support force, and the operation temperature of the device, and we describe the development of a prototype of an electromagnetic braille display with a hierarchical structure. We verified experimentally that the device developed in this study can provide a fingertip touch support force of more than 150 mN, with a contact response frequency of 35.8 Hz, while maintaining a temperature of 32 ^∘ C, which is suitable for fingertip contact after a long operational period. In addition, the correct display rate of the braille characters can reach 100
Manipulating tiny objects adsorbed onto irregular objects is a typical simulation application, such as adjusting brackets in orthodontic training. The operator firstly places the bracket on tooth surface covered with adhesive, and then adjusts its posture with probe. There are two challenges in realizing the above simulation scenes with force feedback. Firstly, the tiny object can be affected by multiple forces when manipulating it, making it difficult to build the dynamic model. Secondly, interaction among the tool, the tiny object and the supporting object is complicated. In addition to embedding between the tool and the objects, embedding between the objects should also be avoided. In our previous work we constructed an orthodontic simulation system with force feedback. However, the haptic operation of adjusting brackets was not ideal, as serious penetration and motion chaos often occurred. In this paper, we analyzed various forces in the process of adjusting the bracket with probe to refine the dynamic model, and introduced viscous force to erase motion chaos to accurately control the bracket’s motion. In addition, we optimized the bracket’s posture through the shape matching constraint to ensure that the bracket can closely contact the tooth surface without embedding into it. The subjective assessment and objective assessment were implemented to validate the system. The experimental results indicated that the system allowed to precisely move the tiny object while maintaining its close contact with the supporting object.
In recent years, there has been a growing demand for technology to deliver haptic experiences related to skills remotely. This study attempts to reproduce the spatial pattern of vibrations generated at the wrist using a bracelet-type device equipped with multiple sensors and vibrators to convey haptic experiences of tool motions. First, we measured the frequency response characteristics of the propagation of vibrations applied to the fingertips to the wrist and confirmed that high frequencies above 1000 Hz could propagate to the wrist position. Then, we measured the tool and the wrist vibration at multiple points during multiple haptic movements. The relationship between the ability to discriminate between different haptic-related movements and spatial distribution reproduction is investigated by comparing the case of spatial reproduction at the wrist with a conventional tool-mounted device. The effect of the ISM method, which can modulate the waveform to a lower frequency while maintaining the original sensation, is also investigated. Results of discrimination experiments indicate that reproduction of the spatial distribution by vibration stimulation of multiple points on the wrist improves discrimination of different rotational directions and that ISM further improves discrimination of rotational directions compared to the raw signal.
We present a haptic guidance system for teleoperating a robot arm controlled by inverse kinematics. Unlike drones or vehicles, a robot arm occupies 3D space dynamically changing with its various configurations. With limited information on the remote environment and its current configuration, the remotely controlled robot arm has a higher chance of colliding with the surroundings. Consequently, users need to maintain a high level of attention, which results in fatigue during operation. In this paper, we propose a system of haptic force guidance that is robust to both the robot arm configuration and its surroundings. Our system first computes the guiding forces at multiple points in the robot arm using ray-based depth sampling. Then, haptic force feedback is generated by aggregating guiding forces using a motion-based approximate Jacobian. Our system requires minimal prior information about the environment and the robot arm. Moreover, the proposed ray-based depth sampling method is more efficient in computation time than the widely used potential field-based approach. User studies show that our system reduces the risk of collisions, as well as the mental workload during teleoperation in a virtual environment.
According to WHO statistics in 2020, there were approximately 253 million visually impaired people worldwide, including 36 million blind individuals. Blind students learn the same knowledge as ordinary students do in their schools, but they lack a lot of graphical and image information as a learning aid. With the development of electronic graphical tactile display, in this paper, a digital version of Braille textbook is proposed and designed. It not only displays the paper-based Braille texts, but also includes related tactile images and illustrations, therefore the blind students can read texts and images in pair. A serial of initial design guidelines and principles for digital Braille textbook are discussed, followed by the five typical design examples. Twelve blind students in a primary school experienced five lessons of digital Braille textbook, and most of them showed positive responses. Our findings pave the way for future work on improving the user experience of tactile interface design that support the learning by digital Braille textbook.
With the rise of the Tactile Internet (TI) over 5G networks, haptic teleoperation systems have attracted extensive attentions as one of the key use cases of the TI. For a typical teleoperation setup, a human operator (i.e. the leader) interacts with a robot (i.e. the follower) in the remote environment with haptic input/output devices, where haptic information is bilaterally exchanged between them. Because of the human-in-the-loop nature of haptic teleoperation systems, the quality of experience (QoE) becomes an important performance indicator of the system. It is well known that the performance of a teleoperation system degrades when there exists communication latencies between the leader and the follower. As a result, how to gain the maximum overall QoE for teleoperation sessions sharing the same communication network becomes a huge challenge. In the presence of different communication latencies, different control schemes are applied to stabilize the teleoperation system. Since different control schemes have different sensitivities to the communication delay, most recently a QoE-delay model was developed to reveal the QoE performance of control schemes with respect to round-trip delays. In this paper, we take full advantage of the QoE-delay model, and propose a novel reinforcement learning based scheduling algorithm for haptic communications aiming at maximizing the overall QoE of all active sessions sharing the communication network. Simulation results confirm the efficiency of the proposed scheduling algorithm.
Representation of object features can help visually impaired people better comprehend their surrounding environment. Tacton (Tactile Icon) is an effective method to extract and express information non-visually, utilizing users’ tactile perception capacities. However existing vibrotactile displays mainly place emphasis on directional guidance, and the number of representable object features is very limited. To leverage the egocentric spatial cognition habit and high tactile perception sensitivity of visually impaired users, this research proposes a user-centered vibrotactile cueing strategy to convey 30 kinds of spatial information through 30 tactons played by 4 vibrators on the back and front side of a pair of gloves. Three parameters including vibration sequence, stimulus location, and intensity are used to encode 10 typical objects located in 3 directions with 2 alert levels. User tests in both laboratory and natural settings are conducted to evaluate the validity of the strategy. The recognition accuracy of the designed tacton has reached 98.99% within a recognition time of less than 0.6s, indicating that this strategy can provide practical assistance for visually impaired users to perceive and respond to the pre-defined spatial information. The multi-parameter tactons provide possibility to encode a wide variety of spatial information by exploiting the communication capacities of the tactile channel of visually impaired users.
Extrapolation of thermal sensation (ETS) is a temperature-tuning phenomenon. Heat is felt outside the cold stimulator if given a single warm stimulus for 5 s and then both warm and cold stimulus for 0.5 s, while no such tuning is in reversed condition. The mechanism underlying ETS has not been clarified. We claim a common mechanism based on the transition of the spatiotemporal conditions among ETS, paradoxical heat sensation (PHS), and thermal grill illusion (TGI). We tested whether the addition and unmasking theory in TGI also works in ETS. Percentage hot judgment was found to increase as the temperature difference of stimuli increases, suggesting that hot sensation at extrapolation site can be the result of an addition of non-noxious warm and cold signals. On the other hand, the cold threshold was found to decrease as the distance between warm and cold stimulus increases, suggesting that the unmasking process occurs at the cold site. Our findings suggested that unmasking and addition process involve in the temperature-tuning phenomenon (ETS, TGI, and PHS) at the site with and without physical stimulation, respectively. The co-work of two processes forms a gate control model of temperature.
Haptic interaction is a fundamental approach to our perception surrounding people; at this stage, haptic interaction is mainly used in virtual reality (VR) and remote medical procedures (teleoperation). The traditional large volume, complex operation seriously affects the function of immersive. This paper develops a multi-modal sensing interactive glove system for teleoperation and VR/AR. The device is small in size and convenient to wear; we integrate temperature sensing and tactile sensing into our glove system and deliver real-time and accurate environmental signals to the wearer through the acquisition and processing of information and program-controlled regulation to better realize the fusion of virtual and real. This paper first introduces the structure and working principle of the system and analyzes its performance. Then a sensory experiment based on human hand skin was designed, and volunteers were selected to experience it.
Little is known about how perceptual modality affects the inter-manual coordination in bimanual force control tasks with accentuated- or attenuated-force production. This study examined the effects of relative phase patterns on the inter-manual coordination in a bimanual force control task with visual/vibrotactile cues. In the proposed force control task, both index fingers were required to simultaneously produce congruent accentuated- or attenuated-force pulses in the in-phase pattern, and produce incongruent ones in the anti-phase pattern. According to visual or vibrotactile cues, participants were prompted to quickly perform the bimanual force control task by pressing with their index fingers on force sensors from the same background force level. The inter-manual coordination performance was indexed by the delayed reaction time and timing difference of the index fingers. Results demonstrated that the advantage of the inter-manual coordination with vibrotactile cues in the in-phase patterns was significant over the anti-phase patterns. The inter-manual coordination of the index fingers could differentiate the in-phase patterns from the anti-phase patterns in the vibrotactile modality while the visual modality showed shorter delayed reaction time and less timing difference of the index fingers in the proposed force control task. Consequently, these findings suggested that the modality-specific effect of cueing on the advantage of the inter-manual coordination existed in the bimanual force control task with accentuated- or attenuated-force production using both index fingers simultaneously. This study would be referential for designing interactive applications leveraging inter-manual spatial coordination for in-phase and anti-phase patterns during bimanual synchronized force production.
Conventionally, correct motion in sports training and rehabilitation has been taught directly from a trainer. However, oral or gesture presentations are difficult based on what the trainer considers to be the accurate motion. We propose a motion timing presentation method using force stimulus, which has high relevance to the target motion. In this study, we developed a timing presentation training system for a baseball batter using force stimulus. The force stimulus is presented according to the pitching motion reproduced on a virtual reality (VR) system to present the timing stimulus repeatedly. The stimulus is used for presenting the timing of the initial foot step motion, which is the first phase of batting. For verification of the timing presentation effect of the training system, we measured the initial batting timing of the baseball beginner. As a result, it was found that the foot step could be started very close to the target timing using force stimulus presentation.
Motion effects are a key component to improve users' immersiveness in 4D contents and virtual reality. However, the production of motion effects is still very labor-intensive and time-consuming. In this demonstration, we present synthesis algorithms which generate motion effects by analyzing the audiovisual content of 4D ride and films. Our synthesis algorithm provide compelling multimedia experiences to viewers while greatly improving the productivity.
We propose a buttock skin stretch device that adopts a two-degree-of-freedom horizontal movement mechanism. We have confirmed that an acceleration sensation of self-motion can be induced by a buttock skin stretch device with one degree of freedom. In this paper, we propose a two-degree-of-freedom buttock skin deforming device that extends the direction of skin deformation, which was only in the left and right directions, to the forward and backward directions. We focused on the range of motion, position accuracy, and driving speed, and evaluated the performance of the device as a buttock skin-deforming device.
