Automatic captions on video calls: a must for the older adults

Abstract

In recent years, the use of video call or video conference tools has not stopped increasing, and especially due to the COVID- 19 pandemic, the use of video calls increased in the educational and work spheres, but also in the family sphere, due to the risks of contagion in face-to-face meetings. Throughout the world, many older people are affected by hearing loss. Auditory functional diversity can make it difficult to enjoy video calls. Using automatic captions might help these people, but not all video calling tools offer this functionality, and some offer it in some languages. We developed an automatic conversation captioning tool using Automatic Speech Recognition and Speech to Text, using the free software tool Coqui STT. This automatic captioning tool is independent of the video call platform used and allows older adults or anyone with auditory functional diversity to enjoy video calls in a simple way. A transparent user interface was designed for our tool that overlays the video call window, and the tool allows us to easily change the text size, color, and background settings. It is also important to remember that many older people have visual functional diversity, so they could have problems reading the texts, thus it is important that each person can adapt the text to their needs. An analysis has been carried out that includes older people to analyze the benefits of the interface, as well as some configuration preferences, and a proposal to improve the way the text is displayed on the screen. Spanish and English were tested during the investigation, but the tool allows us to easily install dozens of new languages based on models trained for Coqui STT.