Auditory depth map representations with a sensory substitution scheme based on synthetic fluid sounds

A novel sensory substitution algorithm based on the sonification of depth maps into physically based fluid flow sounds is described. Spatial properties are extracted from depth maps and mapped into parameters of an empirical phenomenological model of bubble statistics, which manages the generation of the corresponding synthetic fluid flow sound. Following minimal training, the proposed approach was tested in a preliminary experiment with 20 normally sighted participants and compared against the well-known vOICe sensory substitution algorithm. Although the accuracy in recognizing visual sequences based on the corresponding sonification is comparable between the two systems, an overwhelming support for the fluid sounds compared to the vOICe output in terms of pleasantness was recorded. Collected data further suggests that ample margins of performance improvement are achievable following thorough training procedures.