Gaze has been studied to be related to basic emotions or to facial expressions in virtual spaces. However, Social emotion has been required to be expressed for the quality of interaction. This study was to develop gaze expression of social emotion in virtual avatar. Twenty-six females and twenty-six males at aged average 23.65±1.87 year participated in this experiment, and their gaze were measured during social interaction tasks such as cooperation, focus, reciprocal, and engagement. The tasks were designed to evoke positive and negative quality in each social emotion. The participants were asked to watch the partner''s face during task and their gazes to the partner’s face were analyzed at five Areas of Interest (AOI) which consists of the left and right eye, nose, mouth, and the rest of the face. The gaze has been calculated to obtain average time viewed in seconds, percentage, fixation, and revisits. The gaze according to social emotion resulted in 15 realistic expression factors (4 cooperation, 3 focus, 6 reciprocal, 2 engagement). A realistic expression system of virtual avatar reflecting gaze according to social emotion has been developed based on this expression factors. The proposed system provides basic data for creating a more realistic virtual avatar and a guide for virtual avatar services based on social emotion.

This study aims to develop a non-contact measurement technique for cardiac response that uses an infrared image of the patient's pupil. The pupil contraction rhythm is related to the autonomic balance and major organs (such as the heart) via a neural pathway. In this study, the response of the heart was determined by analyzing the pupillary rhythm based on the harmonic frequencies between them. Seventy undergraduate volunteers of both genders, (35 females and 35 males), with ages ranging between 20 and 30years (mean: 24.52±0.64years) were asked to conduct a simple conversation, perform slight movements, and experience sound stimuli to evoke arousal, relaxation, happiness, sadness, or a neutral mood in this experiment. Electrocardiograms and pupil images were measured and analyzed, and the harmonic frequencies were identified to determine the relational response. The cardiac time (heart rate (HR), beats per minute (BPM), the standard deviation of the normal-to-normal (NN) intervals (SDNN), the mean squared differences in the successive N-N intervals (rMSSD), and the percentage difference between adjacent normal interbeat (R-R) intervals>50 (pNN50)) and frequency (very low frequency (VLF), low frequency (LF), high frequency (HF), VLF/HF, and LF/HF) parameters were also observed with regard to the effects of the movement, conversation, and physiological state. The cardiac response was stable, showing less significance than the effects of the three conditions. Therefore, multi-cardiac measurements were successfully obtained from a simple, low-cost, non-invasive, and non-contact data acquisition method in this study.

The condition for the integral floating display is analyzed and evaluated, which elicits the distinguished accommodation responses to different axial locations of targets. The analysis is carried out to derive condition which helps to devise such a system. And measurements of accommodation response from four subjects are conducted to confirm the agreements with the analytical results, by using the auto-refractometer. The two sample Kolmogorov-Smirnov test is applied to evaluate the measurement results qualitatively. The robust agreements between the analysis and the empirical results are observed. This research can be applied in the optimization of augmented reality or virtual reality devices to provide effective three-dimensional depth cues and relieve viewing discomforts.

The eyes of a virtual human avatar are the fundamental means for communicating language and affective feelings in a virtual environment. For this purpose, this study evaluates user's visual feeling according to the changes pupillary responses based on the heart rate for a representing a human avatar; this is considered as a new factor for representing a realistic avatar. We construct a human avatar in which the pupillary response is delivered real-time based on the heart rate. The results can be regarded as the basis for designing a realistic human avatar system by supporting a new visual realistic representing factor.

The purpose of this study was to identify an evaluation method for 3D cognitive fatigue based on a heart–brain synchronization phenomenon known as the heartbeat evoked potential (HEP). Thirty undergraduate students (15 females) watched a video in both 2D and 3D for an hour. Because visual fatigue is related to cognitive load, the HEP was used as an indicator of communication between the heart and the brain and therefore of cognitive function; responses were compared after 2D and 3D viewing. At the standard EEG sites F3 and F4, the alpha activity of the first and second HEP components was significantly increased after 3D video viewing relative to 2D. This increase likely indicates that sensory input from 3D video requires heavy computation by the brain, stimulating heart activity. The conclusion is that the first and second HEP components are significant parameters that can quantitatively evaluate 3D visual fatigue. Further work is needed to uncover the cause of 3D visual fatigue.

가상환경에서 가상 아바타는 말과 감정을 전달하는 주요한 의사소통 수단이다. 본 연구에서는 가상 아바타의 홍채색과 동공 조절 요소를 기반으로 사용자의 시각적 실재감에 대한 반응을 평가하고자 한다. 이를 위해 홍채색과 동공 조절 요소를 적용한 가상 아바타를 모델링하였다. 32명의 피실험자를 대상으로 상기 이미지를 제시한 후, 선행연구를 기반으로 설문문항(18문항)을 구성하여 5점 척도로 보고하게 하였다. 실험결과, 동공 크기변화가 적용된 경우 높은 실감을 느끼는 것으로 나타났으며, 홍채색은 실재감에 영향을 주지 않는 것으로 나타났다. 이 결과는 새로운 시각적 실감 표현요소를 제안하고, 제안된 실감 표현요소의 타당성을 검증함으로써 실재감 있는 가상 아바타를 설계하기 위한 기초 연구로써 활용 가능할 것으로 사료된다. The virtual avatar is a principal way as media for communicating language and affective feelings in virtual environment. As similar purpose, this study evaluates user's visual feeling according to the changes of iris color and pupil size of virtual avatar which is considered as new factors for representing realistic avatar. Virtual avatars were configured by pupil accommodation and iris color(green, brown). After presenting above image to 32 participants, a questionnaire(18 items) based on previous studies was created, and reported as a 5-point scale. Experimental result showed that the case of adopting pupil accommodation induced more realistic visual feeling of subjects. This result can be regarded as a basis for designing realistic virtual avatar by confirming the effectiveness of pupil accommodation of avatar in terms of representing visual presence.

본 논문에서는 근적외선 카메라를 이용한 눈의 다양한 특징 분석을 통해 감성을 분류하는 방법에 관한 연구를 진행하였다. 제안하는 방법은 기존의 유사한 연구와 비교했을 때, 감성 분류를 위해 더 많은 눈의 특징을 사용하였고, 각 특징이 모두 유의미한 정보를 포함하고 있음을 검증하였다. 긍정-부정, 각성-이완의 상반된 감성 유발을 위해 청각 자극을 사용함으로써, 눈의 특징에 끼치는 영향을 최소화하였다. 감성 분류를 위한 특징으로써, 동공 크기, 동공 크기 변화율, 깜박임 빈도, 눈을 감은 지속시간을 사용하였으며, 이들은 근적외선 카메라 영상으로부터 자체 개발한 자동화된 처리 방법을 통해 추출된다. 분석 결과, 각성-이완 감성 유발 자극에 대해서는 동공 크기 변화율과 깜박임 빈도 특징이 유의한 차이를 보였다. 또한, 긍정-부정 감성 유발 자극에 대해에서는 눈을 감은 지속시간 특징이 유의한 차이를 보였다. 특히 동공 크기 특징은 각성-이완, 긍정-부정의 상반된 감성 자극 유발 상황에서 모두 유의한 차이가 없음을 확인할 수 있었다. In this paper, emotion classification was performed by using four ocular features extracted from near-infrared camera image. According to comparing with previous work, the proposed method used more ocular features and each feature was validated as significant one in terms of emotion classification. To minimize side effects on ocular features caused by using visual stimuli, auditory stimuli for causing two opposite emotion pairs such as "positive-negative" and "arousal-relaxation" were used. As four features for emotion classification, pupil size, pupil accommodation rate, blink frequency, and eye cloased duration were adopted which could be automatically extracted by using lab-made image processing software. At result, pupil accommodation rate and blink frequency were statistically significant features for classification arousal-relaxation. Also, eye closed duration was the most significant feature for classification positive-negative.

Most investigations into the negative effects of viewing stereoscopic 3D content on human health have addressed 3D visual fatigue and visually induced motion sickness (VIMS). Very few, however, have looked into changes in autonomic balance and heart rhythm, which are homeostatic factors that ought to be taken into consideration when assessing the overall impact of 3D video viewing on human health. In this study, 30 participants were randomly assigned to two groups: one group watching a 2D video, (2D-group) and the other watching a 3D video (3D-group). The subjects in the 3D-group showed significantly increased heart rates (HR), indicating arousal, and an increased VLF/HF (Very Low Frequency/High Frequency) ratio (a measure of autonomic balance), compared to those in the 2D-group, indicating that autonomic balance was not stable in the 3D-group. Additionally, a more disordered heart rhythm pattern and increasing heart rate (as determined by the R-peak to R-peak (RR) interval) was observed among subjects in the 3D-group compared to subjects in the 2D-group, further indicating that 3D viewing induces lasting activation of the sympathetic nervous system and interrupts autonomic balance.

The social relationship was including the interaction factor between the persons. In order to detect social relationship, it was using the unconscious response rather than conscious. The purpose of this study was to determine evaluation of social relationship using micro-movement expressed both automatic and unconscious. We used the „vibra-image‟ technology based on vision as the indicator of micro-movement. The vibra-image was consisted with 10 factors such as aggression, stress, tension/anxiety, suspect, balance, charm, energy, self-regulation, inhibition, and neuroticism. 12 subjects were participated in this experiment, assigned two groups. The one group was a friend (strong social relationship), and other was a stranger (weak social relationship). Participants were required interaction of facial expression with face to face. Following the result, the stranger group was significantly increased the factor of Aggression, Suspect, Balance of vibra-image factors, and friend group was not. We found that the tension and anxiety factor could be evaluating the social relationship. Also, the proposed method was only using webcam based on micro-movement without attaching the sensor based on physiological signal.

Previous research has indicated that viewing 3D displays may induce greater visual fatigue than viewing 2D displays. Whether viewing 3D displays can evoke measureable emotional responses, however, is uncertain. In the present study, we examined autonomic nervous system responses in subjects viewing 2D or 3D displays. Autonomic responses were quantified in each subject by heart rate, galvanic skin response, and skin temperature. Viewers of both 2D and 3D displays showed strong positive correlations with heart rate, which indicated little differences between groups. In contrast, galvanic skin response and skin temperature showed weak positive correlations with average difference between viewing 2D and 3D. We suggest that galvanic skin response and skin temperature can be used to measure and compare autonomic nervous responses in subjects viewing 2D and 3D displays.

최근 가상환경에서는 실재감 있는 인간의 얼굴 모델을 설계하기 위하여 얼굴의 형태나 근육의 움직임과 같이 단순히 일반적인 표현요소를 중심으로 모델링 되고 있다. 하지만, 이러한 얼굴 자체의 외형적 요소 이외에도 외적 요소에 의한 얼굴 부분의 변화 인자는 가상 아바타를 효과적으로 표현하기 위한 중요한 요소이다. 따라서 본 연구에서는 가상 아바타의 각막면에 비친 반사영상(eye reflection)을 기반으로 사용자의 시각적 실재감에 대한 반응을 평가한다. 실험결과, 각막면의 반사율을 크게 하여 반사영상이 뚜렷하게 나타난 경우 높은 실감을 느끼는 것으로 나타났다. 이 결과는 새로운 시각적 실감표현요소를 제안하고, 제안된 실감표현요소의 구체적인 표현 범위를 제시함으로써 사실적인 가상 아바타를 설계하기 위한 기초 연구로써 활용 가능할 것으로 사료된다. In the VnR (Virtual and Real Worlds) of recent virtual reality convergence, the modelling of realistic human face is focused on the facial appearance such as the shape of facial parts and muscle movement. However, the facial changing parameters caused by environmental factors beyond the facial appearance factors can be regarded as important ones in terms of effectively representing virtual avatar. Therefore, this study evaluates user's visual feeling response according to the opacity variation of eye reflection of virtual avatar which is considered as a new parameter for reprenting realistic avatar. Experimental result showed that more clear eye reflection induced more realistic visual feeling of subjects. This result can be regarded as a basis for designing realistic virtual avatar by supporting a new visual realistic representing factor (eye reflection) and its degree of representation (reflectance ratio).

Objective: The aim of this study was to investigate the correlation between 3D visual fatigue and physiological measures by canonical correlation analysis enabling to categorical correlation. Background: Few studies have been conducted to investigate the physiological mechanism underlying the visual fatigue caused by processing 3D information which may make the cognitive mechanism overloaded. However, even the previous studies lack validation in terms of the correlation between physiological variables and the visual fatigue. Method: 9 Female and 6 male subjects with a mean age of 22.53±2.55 voluntarily participated in this experiment. All participants were asked to report how they felt about their health sate at after viewing 3D. In addition, Low & Hybrid measurement test(Event Related Potential, Steady-state Visual Evoked Potential) and for evaluating cognitive fatigue before and after viewing 3D were performed. The physiological signal were measured with subjective fatigue evaluation before and after in watching the 3D content. For this study suggesting categorical correlation, all measures were categorized into three sets such as included Visual Fatigue set(response time, subjective evaluation), Autonomic Nervous System set(PPG frequency, PPG amplitude, HF/LF ratio), Central Nervous System set(ERP amplitude P4, O1, O2, ERP latency P4, O1, O2, SSVEP S/N ratio P4, O1, O2). Then the correlation of three variables sets, canonical correlation analysis was conducted. Results: The results showed a significant correlation between visual fatigue and physiological measures. However, different variables of visual fatigue were highly correlated to respective HF/LF ratio and to ERP latency(O2). Conclusion: Response time was highly correlated to ERP latency(O2) while the subjective evaluation was to HF/LF ratio. Application: This study may provide the most significant variables for the quantitative evaluation of visual fatigue using HF/LF ratio and ERP latency based human performance and subjective fatigue.