Creation and evaluation of a multi-sensory virtual assembly environment

The multi-modal information presentation, integrated into the virtual environment (VE), has potential for stimulating different senses, improving the user’s impression of immersion, and increasing the amount of information that is accepted and processed by the user’s perception system. The increase of the useful feedback information may reduce the user’s cognitive load, thus enhancing the user’s efficiency and performance while interacting with VEs. This paper presents our creation of a multi-sensory virtual assembly environment (VAE) and the evaluation of the effects of multi-sensory feedback on the usability. The VAE brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. The usability evaluation is in terms of its three attributes: efficiency of use, user satisfaction, and reliability. These are addressed by using task completion times (TCTs), questionnaires, and human performance error rates (HPERs), respectively. Two assembly tasks have been used to perform the experiments, using sixteen participants. The outcomes showed that the multi-sensory feedback could improve the usability. They also indicated that the integrated feedback offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback (visual or auditory) or no feedback. The participants’ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed by using a unifying human computer interaction framework. The implications, concluded from the outcomes of this work, can serve as useful guidelines for improving VE system design and implementation.