Issuu

Machine Learning–Based Wearable Assistive Device for Hearing-Impaired

Farhan V1 , Muhammed Aslam E 2 , V Sabarish 3 , Sneha P 4 , Theja Shivani P P 5 , Anuja K6

12345Graduate Student, 6Assistant Professor Department of Electronics and communication AWH Engineering College Calicut, Kerala, India

Abstract - Hearing-impaired people may find it challenging to receive the alert from the doorbell and other important sounds within the home environment. This paper proposes a Machine Learning-based IoT Doorbell with a wearable text and vibration alert system. When the person rings the bell, the ESP32 microcontroller will be activated to capture the sound using a Microphone. The sound will be converted to text using a Speech-to-Text module, which will be wirelessly transmitted to the wearable device to display the text message and vibration feedback. The proposed system will also include Machine Learning-based sound classification to detect other important sounds such as baby crying and cooker/kettle whistle. Keywords: ESP32, IoT, Machine Learning, Speech-to-Text, Wearable Device

Key Words: ESP32, IoT, Machine Learning, Speech-toText, Wearable Device

1. INTRODUCTION

Thehearing-impairedaresignificantlyaffectedintermsof communication and awareness of events in the surroundingenvironment.Thetraditionalmethodofusing doorbells and alerts is not feasible for hearing-impaired people. Failure to receive important sounds, such as conversationsfrom visitors,babycries,andcooker beeps, often causes inconvenience. With the development of IoT and ML technologies, it is possible to convert sounds into visual and tactile alerts. The proposed system integrates speech recognition and ML-based sound recognition with awearablealertsystem.Thiswillenablehearing-impaired peopletoleadindependentlives.

2. LITERATURE REVIEW

1. Yaganoglu and Köse (2018): proposed a wearable device for hearing-impaired individualstorecognizecriticalsoundsintheenvironment. The proposed system utilized a Raspberry Pi device, a microphone, and a vibration motor to classify various sounds like doorbell sounds, alarm sounds, phone ring sounds, honk sounds, dog barking sounds, and human speaking sounds. Audio fingerprinting, MFCC, and ZCR feature extraction methods have been utilized for sound classificationintheproposedsystem.Theproposedsystem achieved high accuracy in sound classification. For

instance, the proposed system achieved 99% accuracy in alarmsoundclassification,98%accuracyindoorbellsound classification, and 94% accuracy in overall sound classification. However, the proposed system performance is affected in noisy environments and in the presence of multiplesoundsources.Inaddition,theproposedresearch does not incorporate IoT-based wireless communication technology. Hence, there is a need to design an efficient wearablesystemwithhighaccuracyinsoundclassification innoisyenvironments.

2. Sharma and Kumar (2019):

Sharma and Kumar (2019): Proposed a smart doorbell using the Internet of Things (IoT) technology for the hearing-impaired population. The proposed system can notify the hearing-impaired people using the IoT technology. The proposed system proved the efficient usage of IoT technology for the hearing-impaired population in a smart home environment. The proposed system is limited to providing the functionality of a doorbell and sending the notification to the people using theirsmartphones.Theproposedsystemdoesnotconsider the detection of other sounds, such as alarms and crying kids. This indicates the need for a wearable device to efficientlydetectsoundsandnotifythepeopleinrealtime.

3. Saeed, M., Hussain, A., Khan, N., & Rehman, S. (2020):

Proposed an assistive system for the deaf and hearingimpairedpeoplethroughtheuseofIoTtechnologyinsmart homes. The proposed system utilizes sensors and LED indicatorstoalerttheusersregardingtheeventsoccurring in the smart homes. This increases the awareness of the users regarding the events occurring in the smart homes. The proposed system can be used in fixed indoor environments where the users can easily observe the visual indicators. However, the proposed system is not portable,andtheusersarenotprovidedwiththefacilityof vibrationalertthroughwearabledevices.Itcanalsonotbe usedwhentheuserisawayfromthesourceofthealert.

4. Kaur, J. & Singh, M. (2021):

Proposedawearableassistivedeviceforhearing-impaired people, which can transform sound into vibration. The proposedsystemcantransformsoundintovibrationusing amicrocontroller,soundsensor,andvibrationmotor.The proposed research demonstrated the efficiency of the proposedsystem,whichisportableandconvenientforuse

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

while increasing awareness of sound. However, the proposed system does not have IoT connectivity and intelligentsoundclassification.Itisdifficulttomonitorthe proposed system since it does not have IoT connectivity. This demonstrates the significance of integrating IoT and intelligent sound classification methods to enhance the overallefficiencyoftheproposedsystem.

5. Paidi, Haliza, Zain, and Othman (2022):

Proposed a wearable doorbell notification system using nRF24L01 wireless modules in the communication between the doorbell transmitter and the wearable receiver device. In this proposed research study, the authors proposed using Arduino Nano boards, a push button, vibration motor, LED, and Nokia 5110 LCD as wearable devices to send and receive information to and from the doorbell and wearable devices. In this proposed research study, the authors proposed testing the performance of the proposed wearable doorbell notificationsystemusingdifferentdataratesof250Kbps, 1Mbps,and2Mbpsinaresidentialhouseenvironment.In this proposed research study, the authors proposed using 250 Kbps as the most stable data rate with a response time of 0.8 ms and indoor range of up to 22 meters. However, this proposed research study does not include sound recognition, speech-to-text recognition, and IoT devices, which is a limitation of this proposed research study.

6. Jayasudha and Aadhav (2024):

Proposed an IoT-based vibrating doorbell system for hearing-impaired people using Arduino Nano and a coin vibration motor. In this proposed system, the coin vibration motor will be activated when the doorbell button is pressed. This proposed system is simple, costeffective, compact, and easy to implement in home environments. In the proposed study, it was shown that the proposed vibration-based assistive technology can be used to increase accessibility among deaf people. However,thisproposedsystemcanonlysupportdoorbell alerting function and does not support wireless communication and environmental sound detection. This impliesthatthereisaneedtodevelopanadvancedsystem thatcandetectdifferent environmental soundsignalsand sendwirelesswearabledevicenotifications.

3. SYSTEM OVERVIEW

The proposed system is a Machine Learning–Based IoTDoorbell integratedwith a wearablealertdevice designed for hearing-impaired individuals. The systemconvertsaudiosignals into readable textand vibration alerts to ensure effective communication andsafetyinsideahomeenvironment.

Theoverallsystemisdividedintotwomainunits:

DoorbellUnit

WearableAlertUnit

Both units communicate wirelessly using Wi-Fi throughESP32microcontrollers.

3.1 BLOCK DIAGRAM DESCRIPTION

Fig 1: Block Diagram

The doorbell unit consists of a push button, ESP32 microcontroller, and a microphone. The push button acts as the trigger mechanism when a visitor arrives. The microphone captures speech and environmental sounds. The ESP32 performs audio processing, speech-to-text conversion, machine learning classification, and wireless datatransmission.

The wearable unit consists of another ESP32, an OLED/TFT display, and a vibration motor. This unit receives processed data from the doorbell unit and generatesvisualandhapticalertsfortheuser.

The wireless communication between the two units ensures flexibility and portability without physical wiring constraints.

3.2 OPERATIONAL METHODOLOGY

The proposed system operates in two primary modes: visitorspeechdetectionandhouseholdsoundmonitoring. When a visitor presses the push button on the doorbell unit, the ESP32 microcontroller is activated. The connected microphone captures the visitor’s speech. The capturedaudioisprocessed andconvertedtotextusing a speech-to-textmodule.Thegeneratedtextmessageisthen transmitted wirelessly via Wi-Fi to the wearable unit. Upon receivingthe message,thewearabledevice displays theconvertedtextinrealtimeontheOLED/TFTscreen.At thesametime,thevibration motorisactivatedtoprovide haptic feedback, ensuring that the user receives immediatenotificationevenwithoutvisualattention.

In addition to visitor speech detection, the system continuously monitors household sounds. The

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

microphone captures ambient audio signals, which are processedusingfeatureextractiontechniquessuchasMel Frequency Cepstral Coefficients (MFCC). These extracted featuresarefedintoa trainedMachineLearningclassifier to identify predefined sound categories, such as baby crying or a cooker/kettle whistle. When a match is detected, the system generates a corresponding alert message and transmits it to the wearable unit. The wearable device then displays the alert message and activates vibration feedback to notify the user. This dualmode operationenhances bothcommunicationandsafety forhearing-impairedindividuals.

4. SOFTWARE INTERFACE

The software interface plays an important role in the implementation of the proposed system. It manages the interaction between the audio input, machine learning model, and the hardware components of the system. The software is responsible for creating and processing the audiodataset, extracting relevantsoundfeatures, running the trained neural network model, and generating alerts when important environmental sounds are detected. This interface ensures smooth communication between the sounddetectionsystemandthewearablealertdevice.

4.1 DATASET CREATION

Forthedevelopmentofthesoundclassificationsystem,an audio dataset containing critical household sounds was created. The dataset includes sounds such as baby crying, kettle (cooker) whistle, and background noise. These audio samples were collected from publicly available audiodatasetsandadditionalrecordingswhenrequired.

All audio files were standardized to a fixed duration and sampling rate to maintain uniformity across the dataset. Low-quality or noisy recordings were filtered out to ensure better model performance. Before feature extraction, the audio signals were normalized and converted into a consistent format. This preprocessing stephelpsreducevariationscausedbydifferentrecording environments and improves the reliability of the machine learningmodelduringtrainingandtesting.

4.2 SOFTWARE IMPLEMENTATION AND MODEL INTEGRATION

The software interface of the proposed system is responsible for processing audio signals, executing the trained machine learning model, and communicating the detectedsoundeventstothewearabledevice.Thesystem is implemented using Python, which provides powerful librariesforaudioprocessingandmachinelearning.

Audio signals captured through the microphone are processed using the Librosa library, which is used for loading audio files, preprocessing signals, and extracting Mel-Frequency Cepstral Coefficient (MFCC) features.

These features represent the spectral characteristics of soundandareusedasinputstotheneuralnetworkmodel. The classification model is developed using TensorFlow andKeras,whichprovidetoolsforbuildinganddeploying neural networks. During real-time operation, the system continuouslycapturesaudioframesfromthemicrophone, performsMFCCfeatureextraction,andfeedstheextracted featuresintothetrainedmodelforprediction.

Ifthemodeldetectsacriticalsoundsuchasbabycryingor kettle whistle, the software interface triggers an alert mechanism. The detected event is transmitted to the wearabledevice,whichnotifiestheuserthroughvibration and text-based alerts, thereby assisting hearing-impaired individuals in recognizing important environmental sounds.

5. AND IMPLEMENTATION SETUP

5.1 ESP32 Dev Board (Controller) :

 Dual-core32-bitprocessor,upto240MHz

 3.3Voperatingvoltage

 4MBFlash(typical)

 Built-inWiFi&Bluetooth

Fig 2: Flow chart

DESIGN

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

5.2 MAX Microphone Module:

 Operatingvoltage:3.3V–5V

 Analogsoundoutput

 Adjustablegain

 Highsensitivityaudiodetection

5.3 Push Buttons (Input):

 Operatingvoltage:3.3V–5V

 Momentarytactileswitch

 DigitalinputtoESP32

5.4 Vibration Motor:

 Coin/ERM(EccentricRotatingMass)vibration motor

 OperatingVoltage:3V–5VDC

 CurrentConsumption:~70–100mA

 Provides haptic (vibration) alert for sound detectionornotifications

6. Operation of the System

The proposed system is designed as a two-part wearable solution for individuals with hearing impairments. It consists of a doorbell unit and a wearable unit, both connected through Wi-Fi, allowing wireless communicationbetweenthem.

The doorbell unit includes a push button, an ESP32 microcontroller, and a MAX9814 microphone. The push button acts as a trigger when a visitor arrives. Once activated, the microphone captures the visitor’svoice,and theESP32processesthisaudiothroughseveralsteps.First, the signal is cleaned using noise reduction techniques to improve clarity. Then, it is converted into text using a speech-to-text method. After that, machine learning is applied to classify the input data. The processed informationisthensenttothewearableunitviaWi-Fi.

The wearable unit is built using an ESP32, an OLED display,avibrationmotor,anINMP441microphone,anda user control button. It receives the data from the doorbell unit and provides alerts in two forms: visual messages on the display and vibration feedback. This combination ensuresthattheusercannoticealertseasily.

The INMP441 microphone also allows the system to capture conversations, helping the user understand interactions more clearly. The user has control over enablingordisablingthisfeatureasrequired.

Inaddition,thesystemcandetectimportanthousehold soundssuchasababycryingorakettlewhistle.Whensuch sounds are identified, the device notifies the user through both text display and vibration. This makes the system usefulforimprovingsafetyandawarenessindailylife.

6.1 Prototype

Fig 3: ESP32 Dev Board (Controller)

Fig 4: MAX Microphone Module

Fig 5: Push Buttons (Input)

Fig 6: Vibration Motor

Fig 7: Wearable Unit

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

7. RESULTS

The developed system consists of a Doorbell unit and a Wearableunit

The system was tested in real life situations to check how well it can identify and capture important sounds such as baby crying and Cooker/kettle Whistle. While testing, the systemwasabletoidentifythesoundswithaccuracy.

The developed system was tested in different real-life situations to check how well it can detect and respond to important sound events. During testing, the system was abletosuccessfully identifysoundslikeababycrying,and akettlewhistlewithgoodaccuracy.

The ESP32 processed the audio signals efficiently, and the machinelearningmodelhelpedinidentifyingthesoundsin real time. Once a sound was detected, the system immediatelyalertstheuserintwoways:bydisplayinga message on the OLED screen and by providing vibration feedback.

Thewirelessconnectionbetweenthedoorbellunitandthe wearable device worked using Wi-Fi. The system responded quickly enough to be useful in real-time situations.

Overall, the system performed reliably and showed that it can effectively assist people with hearing impairments by notifying them of important sound events in their surroundings

8. CONCLUSIONS

This project shows a simple system that can detect important sounds like a baby crying or a kettle whistle. Whenthesesoundsaredetected,itgivesalertsusinglight and vibration. This is very helpful for people who cannot hear properly, as it helps them stay safe and aware. The

projectalsoshowshowmachinelearningcanbeusedwith electronicstocreateusefulreal-lifesolutions.Overall,this systemcanmakedailylifeeasierandsafer.

REFERENCES

[1] D. Kornack and P. Rakic, “Cell Proliferation without Neurogenesis in Adult Primate Neocortex,” Science, vol. 294, Dec. 2001, pp. 2127-2130, doi:10.1126/science.1065467.

[2] M. Young, The Technical Writer’s Handbook. Mill Valley,CA:UniversityScience,1989.

[3] R. Nicole, “Title of paper with only first word capitalized,”J.NameStand.Abbrev.,inpress.

[4] K.Elissa,“Titleofpaperifknown,”unpublished.

[5] Davis, S., & Mermelstein, P. (1980). Comparison of parametric representations for monosyllabic word recognition in continuously spokensentences.

BIOGRAPHIES

Final Year B tech student in the Electronics and communication Engineering Department at AWH Engineering College Calicut KeralaIndia.

Author

Final Year B tech student in the Electronics and communication Engineering Department at AWH Engineering College Calicut KeralaIndia.

Fig 8: Doorbell unit

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

Anuja, k completed her B tech in Electronics and Instrumentation under Cochin University of science and Technology. Completed hermaster'sin Power Electronics and drives under Anna University Chennai. Pursuing PhD in control systems from National Institute of technology Calicut. Her research interest includes model predictive controller, assertive devicesneuralnetwork..