Sound-driven object recognition fostering semantic and spatial navigation for visually impaired individuals

Despite the historical emphasis on vision, there is growing recognition of the importance of incorporating other senses, such as hearing, to enrich various fields. This shift is particularly relevant for the 314 million globally facing visual impairment, which is expected to rise with an ageing population. Developing new sensory modalities, particularly in audio-based navigation, is crucial for enhancing the autonomy and mobility of Blind and Visually Impaired (BVI) individuals. This master’s thesis explores developing and assessing a sonification technique designed to enhance autonomous navigation by recognising objects and landmarks using auditory cues. The technique aims to convey semantic and spatial features of objects through non-speech sound cues. Theoretical foundations of human perception of sound and spatial cognition are examined, focusing on how these aspects influence object recognition. Additionally, advancements in object detection technologies and various sonification techniques are reviewed, laying the groundwork for a new sonification system prototype. The study validates the system through a structured methodology comprising three phases: analysing acoustic differences of selected sound cues using non-parametric statistical tests, assessing intuitive associations between object features and sounds through an online perceptual experiment, and conducting in-person navigation experiments in virtual reality environments. Results highlight significant acoustic differences among materials, a global accuracy of 0.69 ± 0.13 in the online perceptual experiment, and a mean accuracy of 0.73 ± 0.16 in virtual navigation experiments. These findings suggest potential improvements in sound associations and underscore the influence of demographic factors on sound perception.