Talking Guide Dog Robot Lets Visually Impaired Users Discuss Routes and Find Water
BINGHAMTON, NY. Researchers at Binghamton University (SUNY) have unveiled a revolutionary robotic guide dog for people with visual impairments — one that can not only lead but also hold a conversation about navigation. The project was presented at the 40th AAAI Conference on Artificial Intelligence.
The main driver behind the development is a severe shortage of biological guide dogs. According to the researchers, only about 2% of Americans with visual impairments have access to a trained animal. “We want to fill this gap with technology that speaks human language,” the team said.
The system combines a large language model (LLM), speech recognition, and a route planner. Instead of just following a preset path, the robot interprets the user’s intent.
Example from the study:
User says: “I’m thirsty.”
Instead of asking “Where to?”, the robot knows the layout and offers choices:
“We can go to the kitchen or the water fountain. The kitchen requires opening one door and takes about three minutes. The fountain has no doors and takes about one minute. Where would you like to go?”
The researchers use a dual approach:
- Plan Verbalization: Before moving, the robot translates its internal calculations (distance, doors, time) into speech, giving the user an informed choice.
- Scene Verbalization: While moving, it comments on the environment in real time (“entering hallway,” “approaching door”), helping the user build a mental map.
Seven visually impaired participants (ages 40–68) tested three modes: minimal speech, scene descriptions only, and the full system (plan + scenes). The full system scored best:
- Usefulness: 4.83/5
- Ease of communication: 4.5/5
In simulations based on 77 navigation requests (from “I need the restroom” to “I want to sit and rest”), the system correctly identified destinations 94.8% of the time.
Even with heavily distorted speech (noise simulating real-world conditions — errors in every third character), accuracy dropped only about 5%. In contrast, simpler keyword-based systems failed completely.
The robot isn’t yet fully autonomous in movement, but its conversational system works reliably. Meanwhile, engineers at the University of Waterloo have unveiled a robot that helps people with dementia find lost objects by recording when and where items disappear from view.