An Audio-Based 3D Spatial Guidance AR System for Blind Users

Lecture Notes in Computer Science

We introduce CamIO, an AR app for Visually Impaired People (VIP) to reach objects in their immediate environment through real-time 3D audio guidance
Proceedings
Engineering
Computer Vision
Auditory Interface
Augmented Reality
Sensory Substitution
Authors
Affiliations

James Coughlan

Smith-Kettlewell Eye Research Institute

Brandon Biggs

Smith-Kettlewell Eye Research Institute

Marc-Aurèle Rivière

LITIS

Huiying Shen

Smith-Kettlewell Eye Research Institute

Published

September 12, 2020

Doi

10.1007/978-3-030-58796-3_55

Abstract

Augmented reality (AR) has great potential for blind users because it enables a range of applications that provide audio information about specific locations or directions in the user’s environment. For instance, the CamIO](/content/projects/CamIO) (“Camera Input-Output”) AR app makes physical objects (such as documents, maps, devices and 3D models) accessible to blind and visually impaired persons by providing real-time audio feedback in response to the location on an object that the user is touching (using an inexpensive stylus). An important feature needed by blind users of AR apps such as CamIO is a 3D spatial guidance feature that provides real-time audio feedback to help the user find a desired location on an object. We have devised a simple audio interface to provide verbal guidance towards a target of interest in 3D. The experiment we report with blind participants using this guidance interface demonstrates the feasibility of the approach and its benefit for helping users find locations of interest.

Project (CamIO) Code
Profile picture of Marc-Aurèle Rivière
Back to top

Citation

BibTeX citation:
@inproceedings{coughlan2020,
  author = {Coughlan, James and Biggs, Brandon and Rivière, Marc-Aurèle
    and Shen, Huiying},
  editor = {Klaus, Miesenberger and Roberto, Manduchi and Rodriguez
    Mario, Covarrubias and Petr, Peňáz},
  publisher = {Springer International Publishing},
  title = {An {Audio-Based} {3D} {Spatial} {Guidance} {AR} {System} for
    {Blind} {Users}},
  booktitle = {Lecture Notes in Computer Science},
  volume = {12376},
  pages = {475-484},
  date = {2020-09-12},
  url = {https://link.springer.com/chapter/10.1007/978-3-030-58796-3_55},
  doi = {10.1007/978-3-030-58796-3_55},
  isbn = {978-3-030-58795-6 978-3-030-58796-3},
  langid = {en},
  abstract = {Augmented reality (AR) has great potential for blind users
    because it enables a range of applications that provide audio
    information about specific locations or directions in the user’s
    environment. For instance, the CamIO{]}(/content/projects/CamIO)
    (“Camera Input-Output”) AR app makes physical objects (such as
    documents, maps, devices and 3D models) accessible to blind and
    visually impaired persons by providing real-time audio feedback in
    response to the location on an object that the user is touching
    (using an inexpensive stylus). An important feature needed by blind
    users of AR apps such as CamIO is a 3D spatial guidance feature that
    provides real-time audio feedback to help the user find a desired
    location on an object. We have devised a simple audio interface to
    provide verbal guidance towards a target of interest in 3D. The
    experiment we report with blind participants using this guidance
    interface demonstrates the feasibility of the approach and its
    benefit for helping users find locations of interest.}
}
For attribution, please cite this work as:
Coughlan, J., Biggs, B., Rivière, M.-A., & Shen, H. (2020). An Audio-Based 3D Spatial Guidance AR System for Blind Users. In M. Klaus, M. Roberto, C. Rodriguez Mario, & P. Petr (Eds.), Lecture Notes in Computer Science (Vol. 12376, pp. 475–484). Springer International Publishing. https://doi.org/10.1007/978-3-030-58796-3_55