The group of Dr. Leilei Gu, a researcher at the Qingyuan Research Institute affiliated with Shanghai Jiao Tong University, is conducting cutting-edge research in the development of smart sensors and technologies related to visual prosthetics and machine vision.

In this study, the researchers combine different technologies to help blind people move around more safely and autonomously. The main part of the proposed system is based on artificial intelligence (AI) techniques, which analyse what is seen by a camera embedded in glasses worn by the subjects. This AI is able to identify certain obstacles and calculate the safest routes. The information is transmitted to users using several sensory channels. On the one hand, auditory commands, e.g. ‘go ahead’ or ‘turn right’, are transmitted through a bone conduction headset. These headphones are special because they send sound through the bones of the skull, leaving the ears free to hear other important sounds in the environment around the subjects. In addition, the system also incorporates elastic artificial skins, which are placed on both wrists. These skins incorporate small devices capable of sending vibrations to the subjects to indicate the correct direction to follow, avoiding objects that may be on either side.

To test the functionality of the devices, the authors have carried out tests with small robots and with blind and low-vision people. These tests were conducted both in simulated virtual environments and in some real controlled environments. Participants showed significant improvements in their ability to move and avoid obstacles and in their ability to reach and grasp objects.

The study suggests that this combination of non-invasive technologies can help achieve more efficient navigation and provide blind people with a greater degree of independence and confidence when moving around unfamiliar environments. However, it should be noted that the study has been conducted on very few subjects and in limited and highly controlled environments. In addition, most object recognition systems based on deep learning techniques such as the one used by the authors can be significantly affected by lighting conditions. On the other hand, people who are blind or visually impaired may be reluctant to use multiple assistive devices for orientation and mobility. Users must carry multiple devices, which may be heavy or cumbersome, and users must process information from different sources quickly, which may induce cognitive overload leading to fatigue. The potential social stigma associated with the visible use of various assistive technologies must also be taken into account.

There is a need to improve the accuracy and reliability of these mobility assistance systems, to develop more intuitive interfaces and to reduce the size of the devices. Furthermore, not only the functionality of the devices needs to be considered, but also their usability and social acceptability. As research and development in this field continues to advance, multimodal technologies are expected to play an increasingly important role in improving the quality of life of blind people.