Music has a way of bringing people together in the same shared experience. 

But for more than 11 million Americans who are deaf or hard of hearing, live concerts don’t always offer that same sense of inclusion.

A group of seniors in the Biomedical Engineering Program (BME) at CU Boulder are working to change that by designing a set of closed caption glasses during their senior capstone design class.

 

A close look at the team's closed caption glasses in action during a live concert.

The project is sponsored by Shine Music, a nonprofit creating barrier-free live music experiences through accessibility, adaptive technology and universal design. It uses real-time audio transcription to display song lyrics directly through the lens of a pair of wearable glasses. 

The team said the device will help bring a new level of accessibility to concerts, theater, lectures and beyond.

“Deaf or hard of hearing individuals often don’t have the ability to experience the joy of live music and events as others. They are typically placed in designated areas where their focus is on an ASL interpreter instead of experiencing the show,” said software engineer Alena Tucker. “Our goal is to provide another option that allows them to be more fully engaged in the live setting.”

To do this, the group developed a mobile app that uses Apple Speech to transcribe local audio signals. The app connects to the glasses via Bluetooth, sending a live stream of captions that appear right in the user’s field of vision.

According to the team, the system operates with less than a one-second delay and achieves over 90% accuracy, allowing users to receive fast, reliable captions almost instantaneously. For deaf or hard of hearing people, that level of speed and accuracy can make the difference between a seamless, enjoyable experience and one that feels delayed or disconnected—especially in smaller venues where accessibility options are limited.

“It’s very costly for smaller venues to hire an ASL interpreter,” said software engineer Meredith Overton. “A lot of times this prevents people from buying last-minute tickets because there might not be an interpreter on-site.”

But despite impressive speed and accuracy benchmarks, the group still faced challenges that made the project difficult to perfect.

One challenge involved the varying complexities of music. Genres like rap and rock, which feature fast-paced beats and rapid lyrics, proved difficult for the device to process quickly.

Another challenge was budgetary constraints. However, their team dynamics and willingness to learn new things helped them overcome these obstacles.

“Even with all of the stress, everyone knew we were going to work hard, laugh, have a good time and make the best out of it,” said production lead and electronics engineer Maeve Bihn. “This process has been about learning as we go. It’s taught us a lot about the overall diversity of biomedical engineering.”

With the highly anticipated Engineering Expo event right around the corner, the group is working diligently to finalize their design and data. They hope to have a functioning set of closed caption glasses available for visitors to try themselves.