A deaf athlete demonstrated the SureTalk app at an event in Chiba on October 15, 2024. The app, developed by telecom giant SoftBank Corp. in partnership with the University of Electro-Communications in Tokyo, is designed to translate sign language gestures into text. It uses a camera to capture the speaker’s movements, and the underlying AI interprets the gestures by recognizing skeletal motion, then selects appropriate words and adds necessary grammatical elements to form coherent text.
The developer, Shusei Komatsu, who has been hearing-impaired since birth, explained that sign language varies widely among speakers, making it essential to train the AI with numerous patterns. Each word in the database is registered using data collected from around 100 different individuals to ensure accuracy.
Currently, the app offers a database of approximately 5,000 words and has been publicly trialed since November 2020 in various municipal governments, including Tottori and Fukuoka. It has seen use at significant events such as the Tokyo Deaflympics and continues to gain traction in public spaces like airports, hotels, and hospitals, with hopes of further bridging the communication gap between the hearing impaired and those without disabilities.