Columbia University

Technology Ventures

Mobile application providing environmental awareness for hearing-impaired individuals

Technology #cu12023

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Image Gallery
Daniel P.W. Ellis
Managed By
Satish Rao

The increasing prevalence of smart phone and tablet technology has stimulated a thriving sector of software development. Several health-related applications have been developed to aid consumers with a variety of conditions, from obesity to mental health. Hearing-impaired individuals may also benefit from this new technology, as the auditory sensors embedded in these products provide a reliable mechanism to receive vital auditory information. Currently, there is no application designed to assist hearing-impaired individuals in emergency situations, where the recognition of key sounds can be crucial. This technology is a smartphone application that continuously operates to identify important sounds, such as a fire alarm, and alerts the user when appropriate.

Audio classification algorithm can identify emergency sounds and provide haptic feedback

Although the currently available consumer devices provide tremendous opportunity for auditory assistance, the number of applications targeting hearing impairment is relatively small. While current applications identify the presence of very loud sounds, they cannot classify or identify the origin. This technology uses advanced classification algorithms, including mel-frequency cepstral coefficients, hidden Markov models, and support vector machines, to recognize specific sounds even within noisy environments. Haptic feedback to the user can then be modified based on the specific event that is occurring. Furthermore, unlike existing software, this application can automatically identify key sounds from a pre-existing library without requiring user input.

Lead Inventor:

Daniel P.W. Ellis, Ph.D.


  • Increased situational awareness and lifestyle improvements for hearing impaired individuals
  • Audio sensing/classification for security applications
  • Continuous baby monitoring
  • Emergency event identification


  • Highly advanced classification algorithms
  • Automatic detection of critical sounds
  • Customizable user feedback

Patent Information:

Patent Pending (US 20130070928)

Tech Ventures Reference: IR CU12023

Related Publications: