Google Glass App Displays Real-Time Captions to Help the Hearing Impaired

Screenshot of the official Captioning on Glass app website (

Screenshot of the official Captioning on Glass app website (

Google Glass is a wearable device frowned upon by many because of privacy and safety concerns. However, it’s undeniable how this small piece of camera-bearing technological marvel can be greatly advantageous when used in the right ways. This story is a little late but we think it’s only proper for us to write something positive about Google Glass after the not-so-flattering allusions to the device in previous posts.

Captioning on Glass (CoG)

Called Captioning on Glass, this Google Glass app is designed to make communication easier for people who have difficulties hearing. It is a free application developed by a team at the Georgia Institute of Technology. It does what its name says it supposedly does – providing corresponding text captions for audio perceived in front of the person wearing Google Glass.

As reported on CNET, the idea for this application originated from Jim Foley, a professor at the School of Interactive Computing who had difficulty hearing. The team at the Georgia Institute of Technology addressed Foley’s need and subsequently made the app available to other users of Google Glass. In a demonstration witnessed by CNET, the app performed impressively well although some delay in the connection between the Android smartphone and Google Glass was experienced.

Companion App

This Google Glass app has a companion app for Android devices, which is already available on the Google Play store. Through this companion application, Google Glass can display real-time captions as a person speaks through the microphone no the device where the companion app is installed. Apparently, the microphone on Google Glass is not efficient enough to accurately detect the right voice to be converted into captions. Sans a companion smartphone (where the companion app is installed), Google Glass may end up captioning the voice of the Google Glass wearer.

Unfortunately, there is no version of the companion app for iOS devices yet. Likewise, there is no corresponding app for Windows phones. The developers, however, are planning to release a corresponding companion app for iOS and those who are eager to have it as soon as it’s available can visit the app’s official website or sign up for a notification.

Screenshot of the official Captioning on Glass app website (

Screenshot of the official Captioning on Glass app website (

Installation and Use

Installing the CoG app, including the companion app for the separate Android device, is very simple. It’s just like installing other typical Google Glass and Android apps. The app’s developers at Georgia Institute of Technology would like to obtain user feedback to help improve the app. Users interested in volunteering their time and effort in sending feedback can sign up through an online (Google Docs) form to get involved.

The app will not be automatically activated on Google Glass as you converse with someone. To activate it, you need to say the standard “Ok Glass” command and say “Recognize this” once the list of available commands is displayed. Once the app starts on Google Glass, it will ask you to open the companion app on the device where you had it installed. Once both apps are activated, pairing will automatically take place. You will be notified if there are problems. Pairing is through Bluetooth connection.

The smartphone or Android device where the companion app is installed has to be close enough to the person with whom you are conversing since it acts as the microphone and speech-to-text converter.

Once the app has been initiated on Google Glass, it will be saved as a live card so accessing it the next time you need it will be a simple swipe to the left of the Home Screen.

Screenshot of the official Captioning on Glass app website (

Screenshot of the official Captioning on Glass app website (

Transcription Accuracy

The developers admit that the transcription technology of this new application is less than 100%. Also, there are many factors that will affect the transcription. Ambient noises and the accent or fluency of the speaker are two of the most important factors. Overall, it is safe to assume that the transcription accuracy of the app is similar to that of Google Now or Google’s voice-based search function.

The CoG app is indeed a useful idea. However, we’re just not sure how many people gets to benefit from it. The still on Beta Google Glass is still a very expensive device at $1,500. Its availability across markets is also still problematic. Hopefully, CoG could reach more users and be available on other devices like the Lumus Light-Guide Optical Element glasses.