Navigating from a depth image converted into sound

Chloé Stoll (1, 2) , Richard Palluel-Germain (2, 1) , Vincent Fristot (3) , Denis Pellerin (3) , David Alleysson (2, 1) , Christian Graff (2, 1)

1 UGA [2016-2019] - Université Grenoble Alpes [2016-2019]
2 LPNC - Laboratoire de Psychologie et NeuroCognition
3 GIPSA-AGPIG - GIPSA - Architecture, Géométrie, Perception, Images, Gestes

BACKGROUND: Common-manufactured depth sensors generate depth images humans normally obtain from their eyes and hands. Various designs converting spatial data into sound have recently been proposed, speculating on their applicability as Sensory Substitution Devices (SSD).OBJECTIVE: The aim of the present study was to test such a design as a travel aid in a navigation task in real mazes. METHODS: Our portable device (MeloSee) converted the 2-D array of a depth image into melody in real-time. Distance from the sensor was translated into sound intensity, stereo-modulated laterally, and pitch represented verticality. Twenty one blindfolded young adults each navigated along four different paths during two sessions separated by a one-week interval. In some instances, they had to deal with a dual task, i.e., to recognize a temporal pattern applied through a tactile vibrator while they navigated.RESULTS: All participants learnt how to use the system on both new paths and on those they had already navigated from. Based on travel time and errors, performance also improved from one week to the next. The dual task was achieved successfully, slightly affecting but not preventing effective navigation. CONCLUSIONS: The use of Kinect®-type sensors to implement SSDs is promising but it is restricted to indoor use and is inefficient at too short a range.

Mots clés: Depth; Sensory Substitution Device; Real-Time; Navigation; Electronic Travel Aid

Domaines: Psychologie; Sciences cognitives; Traitement du signal et de l'image [eess.SP]

Chloé Stoll, Richard Palluel-Germain, Vincent Fristot, Denis Pellerin, David Alleysson, et al.. Navigating from a depth image converted into sound. Applied Bionics and Biomechanics, 2015, 2015, pp.ID 543492. ⟨10.1155/2015/543492⟩. ⟨hal-01128384⟩