GESTURE BASED ASSISTIVE COMMUNICATION SYSYTEM

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

GESTURE BASED ASSISTIVE COMMUNICATION SYSYTEM

Athira M1 , Janna mol K2,Ahammed Shaheer3 ,Ajmal Sherief4 ,Punitha V5 ,Anuja K6

1234Graduate Student, 56Assistant Professor Department of Electronics and communication AWH Engineering College Calicut, Kerala, India

Abstract - Paralysis severely limits a patient’s ability to communicate basic needs, creating a needfor reliable assistive technologies. This paper presents a Gesture-Based Assistive Communication Smart Glove designed to enable paralysis patients to convey messages through simple hand gestures. Flex sensors are used to detect finger movements, while an ESP32 microcontroller processes the sensor data and maps gestures to predefined text messages. These messages are transmitted via Wi-Fi using a web server and displayed on the caregiver’s mobile phone for immediate assistance. The system focuses on a low-cost, portable design that improves patient independence and enhances caregiver responsiveness. The proposed solution provides an effective communicationaidfor individuals with mobility impairments.

1. INTRODUCTION

Paralysis is a medical condition that results in the loss of musclefunctioninpartsofthebody,significantlyaffectinga patient’sabilitytocommunicateandperformdailyactivities independently. Individuals suffering from paralysis often face difficulty in expressing their basic needs, such as requesting food, water, or assistance, which can lead to discomfort and delayed care. Traditional communication methodsforsuchpatientsrelyheavilyonconstantcaregiver presence or complex assistive devices, which may be expensive,difficulttooperate,orinaccessibleinresourcelimitedsettings.

Advancements in embedded systems and wearable technology have enabled the development of affordable assistive devices that enhance patient independence and caregiver responsiveness. Gesture-based communication systems,inparticular,provideanintuitiveandefficientway for patients to convey messages using simple hand movements.Bydetecting fingerbendingpatternsthrough flexsensorsandprocessingthedatausingamicrocontroller, meaningfultextmessagescanbegeneratedanddeliveredto caregiversinrealtime.

This project proposes a Gesture-Based Assistive Communication Smart Glove that utilizes flex sensors to detect hand gestures and an ESP32 microcontroller to process the sensor data. The recognized gestures are mappedtopredefinedtextmessagesandtransmittedviaWiFi using a web server, allowing caregivers to view the messages on their mobile phones instantly. The system is designedtobelow-cost,portable,andeasytouse,makingit suitableforcontinuousassistanceinhomeandhealthcare environments.

Theproposedsolutionaimstoimprovethequalityoflifefor paralysis patients by enabling reliable communication, reducing dependency on caregivers, and ensuring timely assistancethroughsimpleandeffectivetechnology.

2. LITERATURE REVIEW

1.Gesture Recognition System Using Flex Sensors

Author:DeliFeng,ChengZhou,JipengHuang,GangyinLuo, andXinWu

Thepaperproposesawearablegesturerecognitionsystem thatcapturesfingerbendingandhandmovementsusingflex sensorsandaninertialsensor.Thesystemintegratesasmart glove,Bluetoothcommunication,andanAndroidapplication to convert gestures into readable outputs. An LSTM-based deep learning model with transfer learning enhances recognition accuracy and adapts to different users. Experimentalresultsshowover92–94%accuracyforboth staticanddynamicgestures,provingthesystemisreliable, portable, and useful for improving communication for hearing-impairedindividuals.

2 GSM-Based Smart Glove for Gesture Recognition and Health Monitoring

Author:K.N.Rajesh,P.Keerthana,M.Priyadharshini,andS. Vignesh

This paper presents a smart glove designed for paralysis patients that combines gesture recognition with health monitoring. Flex sensors detect hand movements to communicatepredefinedmessagesviaGSM,whileintegrated sensors monitor vital parameters such as heart rate and bodytemperature.Thesystemprovidesreal-timealertsand remotecommunication,offeringanaffordableandportable assistivesolutiontoimprovepatientsafety,independence, andcaregiversupport.

3.Head Gesture Recognition System Using Gesture Cam

Author:R.T.Bankar,S.S.Salankar

Through the use of a Gesture Cam, the system captures picturesandemployscomputervisiontechniquestoidentify head movements. The system is designed to assist individuals with disabilities who are paralyzed from head

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

movementbyusingfacialrecognitionmethodssuchasthe Viola-Jonesalgorithm.Thisinvolvestakingphotographsof anindividual'sheadandidentifyingitslocationinvarious ways when itis turned left, right or forward. Itcomprises threeprimaryparts:imagecaptureandgesturerecognition, aswellasmotiondetectionalgorithmslikeaccelerometers andtactilesensors.

4. IoT Translator for Sign Language Based on Glove

Author:S.Mathupriya,D.Roopa,M.Subashini

ThisresearchpresentsanIoT-basedsystemthattranslates signlanguagegesturesintovoiceoutputusingasmartglove. The system is designed to help people with hearing and speech disabilities communicate with others who do not understandsignlanguage.Thesmartgloveisequippedwith ADXL335 accelerometer sensors that detect hand movements and finger gestures. These sensor signals are processedusinganArduinomicrocontrollerandtransmitted throughaBluetoothmoduletoanAndroiddevice,wherethe gesturesareconvertedintovoicemessages.Theproposed systemcanrecognizemultiplegestureswithhighaccuracy andprovidesanefficient,low-costcommunicationsolution for speech-impaired individuals. It also demonstrates the potential of integrating IoT devices with assistive technologiestoimprovecommunicationandaccessibility

5. Modelling of Sign Language Smart Glove Based on Bit Equivalent Implementation Using Flex Sensor

Author:

NitinThoppeyMuralidharanetal.

Thispaperpresentsasmartglovedevelopedtohelppeople with speech and hearing impairments communicate more easilywithothers.Thegloveusesfiveflexsensorsplacedon thefingerstodetecthandmovementsandgestures.These sensor signals are processed by an Arduino Uno, which converts the gestures into text messages that can be displayedonascreen.Thesystemworksintwomodes:one for commonly used phrases and another for alphabets, allowing users to express both simple messages and completewords.Sincethesystemusesonlyflexsensors,itis simple, low-cost, and easier to implement compared to camera-basedgesturerecognitionsystems.Thissmartglove canmakecommunicationmoreconvenientandeffectivefor peoplewhorelyonsignlanguage.

3. METHODOLOGY

The developed Gesture based assistive communication systemwithheathmonitoringwasbuildusingastructured design.Inthestartingsystemrequirementswereanalyzed and suitable literatures was reviewed to find suitable technologies.Flexsensorsweredevelopedintoawearable glove to record the movement of fingers, here ESP32 was used as the main micro controller for data collection,

processing it and for wireless communication. In heath monitoringitincludethesensorsMAX30102forheartrate and SpO2 measurement and LM35 for the temperature measuring,thisbothsensorsareinterfacedwiththemain microcontroller.Thecollectingdata wereprocessedusing thepredefinedgesturerecognitionalgorithmandlinkedto corresponding messages. The web-based interface was proposed to display the detected hand gesture along with therealtimehealthparameters.Thecompletedsystemwas testedtoevaluateaccuracy,responsivenessandconsistency.

Implementationmethodologyblockdiagram.

4. SOFTWARE INTERFACE

[1].

Data set creation

A customized dataset was created to train the gesture recognitionmodelusingsignalsobtainedfromflexsensors attached to the glove. The data collection was conducted with the participation of 25 individuals who performed a predefined set of hand gestures using four fingers: index, middle, ring, and little fingers. The flex sensors placed on these fingers measure the bending of each finger and produce analog signals according to the level of finger movement. These analog signals were read by the ESP32 microcontrollerthroughitsanaloginputpinsandconverted into digital values using the built-in Analog-to-Digital Converter(ADC).Eachgesturegeneratedauniquepatternof

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

sensorreadingsdependingonthebendingpositionsofthe fingers.

Theobtainedsensorvalueswerethenmanuallyrecordedin an Excel sheet for further analysis and processing. In the dataset,eachrowrepresentsasinglegestureinstance,while the columns store the values from the four flex sensors correspondingtothefourfingers.Anadditionalcolumnwas usedtolabeleachgesture(suchasa,b,c,andd),allowing the machine learning model to associate specific sensor patternswiththeircorrespondinggestures.

Toenhancetherobustnessofthedataset,multiplesamples of each gesture were collected from all participants. This helps capture variations in hand movements and finger flexibilityamongdifferentusers.Thefinalizedandlabeled datasetwassubsequentlyusedtotrainandtesttheLSTMbased gesture recognition model, enabling the system to learn gesture patterns and accurately classify different gestures.

The effectiveness of the gesture recognition model was assessed using a confusion matrix. This matrix presents a comparison between the actual gesture classes and the gestures predicted by the model. The values along the diagonalindicatecorrectlyrecognizedgestures,whereasthe values outside the diagonal represent incorrect classifications. The results indicate that the majority of gestures were accurately identified, confirming the reliabilityoftheproposedgesturerecognitionsystem.

Thefigureshowsthetrainingandvalidationperformanceof the gesture recognition model over multiple epochs. The accuracygraphindicatesthatbothtrainingandvalidation accuracyincreasegradually,showingthatthemodellearns gesture patterns effectively. The loss graph shows a decreasing trend, indicating that the prediction error reducesduringtraining.Theseresultsdemonstratethatthe modelimprovesitsperformanceasthetrainingprogresses.

[2] Gesture Detection Using Threshold and LSTM

In the proposed system, gesture recognition is achieved usingdatacollectedfromfourflexsensorsmountedonthe index, middle, ring, and little fingers of the glove. These sensorsgenerateanalogsignalsthatvaryaccordingtothe bending of each finger. The signals are read by the ESP32 microcontrollerthroughitsanaloginputpinsandconverted intodigitalvaluesforprocessing.

One approach used for gesture identification is the threshold-based method. In this technique, a specific thresholdvalueisdefinedforeachflexsensor.Whenafinger bends,thesensoroutputincreasesandiscomparedwiththe predefined threshold. If the measured value exceeds this limit,thesystemrecognizesthecorrespondinggesture.For instance,whenthesensorvalueofthemiddlefingercrosses the threshold, the system may interpret the gesture as “Water,”whileotherfingermovementsrepresentdifferent predefinedmessages.Thismethodissimpletoimplement, computationally efficient, and suitable for real-time applications.

In addition to the threshold approach, an LSTM-based gesture recognition method is also employed. For this method,sensorreadingscollectedfrommultipleparticipants areorganizedandlabeledtoformadataset.TheLongShortTermMemory(LSTM)modelistrainedusingthisdatasetso thatitcanlearnthepatternsassociatedwithdifferentfinger movements. During system operation, real-time sensor values are provided to the trained LSTM model, which analyzes the input pattern and predicts the appropriate gesture.

Thus, the system is capable of recognizing gestures using either the threshold-based technique or the LSTM-based model. While the threshold method provides a straightforwardrule-basedsolution,theLSTMmodeloffers improvedaccuracyandbetteradaptabilitytovariationsin userhandmovements.

5. HEALTH MONITORING

The proposed system also includes a health monitoring moduletotrackimportantphysiologicalparametersofthe user.ThismoduleusessensorssuchastheMAX30102and LM35tomeasurevitalsignsinrealtime.

The MAX30102 sensor is used to monitor heart rate and bloodoxygensaturation(SpO₂).Itworksbyemittinglight throughtheskinanddetectingtheamountoflightabsorbed

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

bytheblood,whichhelpsestimatethepulserateandoxygen level. At the same time, the LM35 temperature sensor measures the body temperature and produces an analog voltageproportionaltothetemperature.

The ESP32 microcontroller continuously reads the data generated by these sensors and processes the values to determine the user’s health condition. The measured parameters are then compared with predefined normal ranges. If the values remain within the normal limits, the system continues monitoring the user's health status. However,ifanyabnormalconditionsuchasirregularheart rate,lowoxygenlevel,orhightemperatureisdetected,the systemgeneratesahealthalert.

This monitoring process runs continuously, enabling the systemtoprovidereal-timeobservationoftheuser’svital signsandensuringtimelyalertsincaseofabnormalhealth conditions.

6. DESIGN AND IMPLEMENTATION

The operation of the proposed system follows the sequenceshownintheflowchart.Theprocessbeginswhen thesystemispoweredon.Afterpoweractivation,theESP32 microcontroller is initialized to control the overall functioningofthesystem.

Oncethemicrocontrollerisready,thesensorsconnected tothesystemareinitialized.Theflexsensorsattachedtothe fingers are prepared to measure finger bending. These sensorsgenerateanalogsignalsbasedonthemovementof thefingers.

TheESP32continuouslyreadsthesensorvaluesthrough itsanaloginputpins.Aftercollectingthesensorreadings,the systemcheckswhetheragesturehasbeenperformed.Ifno gesture is detected, the system continues monitoring the sensorvalues.

When a gesture is detected, the system analyzes the sensor pattern to identify the specific gesture. After recognizingthegesture,thecorrespondingpredefinedtext messageisgeneratedandsentforcommunication. Finally,thesystemrepeatsthesameprocesscontinuouslyto detectgesturesinrealtimeandallowuserstocommunicate effectivelythroughhandmovements.

Fig 4: Flowchart of the proposed gesture-based communication and health monitoring system.

FUTURE SCOPE

Theproposedsystemcanbefurtherimprovedbyenhancing boththehardwareandsoftwarecomponents.Inthefuture, additional sensors and advanced machine learning techniques can be integrated to improve the accuracy of gesturerecognitionandsupportalargernumberofgestures. Thesystemcanalsobeexpandedtorecognizecompletesign languagesentencesinsteadofonlybasicwordsorphrases.

Another possible improvement is the development of a dedicated mobile application and a more advanced web interface for better visualization and data management. Voice output features can also be integrated so that the recognized text messages can be converted into speech, makingcommunicationevenmoreeffective.Inaddition,the device can be mademorecompactand wearable byusing smaller components and wireless modules. These improvements can make the system more reliable, userfriendly,andsuitableforreal-worldapplicationsinassisting peoplewithspeechandhearingimpairments.

7. RESULTS

The system successfully converts hand gestures into text messagesthataredisplayedonamobilephone.Whenthe userperformsagesture,theflexsensorsdetectthebending

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

of the fingers and send the corresponding signals to the microcontroller.Thesoftwareprocessesthesesignalsusing therecognitionmethodandidentifiestheintendedgesture. Afterrecognition,thesystemgeneratesthecorresponding messageandsendsittothesmartphonethroughawireless communicationmodule.

Thereceivedmessageappearsonthemobilephonescreen, allowing others to easily understand the user's communication. During testing, several gestures representingdifferentwordsandphraseswereperformed, andthecorrectmessagesweredisplayedonthephone.The resultsshowthatthesystemcaneffectivelytranslatehand gestures into readable messages, enabling easier communication for individuals with speech or hearing difficulties.

7.1 SOFTWARE OUTPUT

Resultsobtainedinwebserver:

7.2 HARDWARE OUTPUT

3. CONCLUSIONS

The proposed system provides an effective solution for gesture-based communication using flex sensors and an ESP32microcontroller.Fingermovementsaredetectedand convertedintotextmessages,allowingeasierinteractionfor users with speech or hearing difficulties. In addition, the systemmonitorsimportanthealthparameterssuchasheart rate,SpO₂,andbodytemperature.Theintegrationofgesture recognition and health monitoring in a single wearable device demonstrates a practical and reliable approach for improvingcommunicationandusersafety.

REFERENCES

[1] Nitin Thoppey Muralidharan, “Modelling of Sign Language Smart Glove Based on Bit Equivalent Implementation Using Flex Sensor”. International Journal of Research in Engineering and Technology, 2022.Handbook. Mill Valley, CA: University Science, 1989.

[2] DeliFeng,ChengZhou,JipengHuang,GangyinLuo,Xin Wu, “Design and Implementation of Gesture Recognition System Based on Flex Sensors,” IEEE SensorsJournal,Vol.23,No.24,December2023.

[3] Priya Seema Miranda A., Aneesha Acharya K., Sean Francis Dante, Shreyas Rai, Sonal J. Ail, Vinol Arun Dsouza,SomashekaraBhat,AdarshRagS., “GSM-Based Smart Glove for Gesture Recognition and Health Monitoring in Paralysis Patients,” IEEE Conference/ResearchPaper,2023.

[4] RushikeshT.Bankar,SureshS.Salankar, “HeadGesture RecognitionSystemUsingGestureCam,” Proceedings oftheFifthInternationalConferenceonCommunication SystemsandNetworkTechnologies,2015.

[5] S.Mathupriya,D.Roopa,M.Subashini, “IoTTranslator for Sign Language Based on Glove,” International Conference on Intelligent Computing and Control for EngineeringandBusinessSystems(ICCEBS),IEEE,2023.

BIOGRAPHIES

Janna mol K Final Year B tech student in the Electronics and communication Engineering Department at AWH Engineering CollegeCalicutKeralaIndia.

Athira M FinalYearBtechstudent in the Electronics and communication Engineering Department at AWH Engineering CollegeCalicutKeralaIndia.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

Ahammed saheer Final Year B techstudentintheElectronicsand communication Engineering Department at AWH Engineering CollegeCalicutKeralaIndia.

Ajmal sherief Final Year B tech student in the Electronics and communication Engineering Department at AWH Engineering CollegeCalicutKeralaIndia.

Ms. Punitha V iscurrentlyworking as an Associate Professor in the Department of Electronics and Communication Engineering. She hasseveral yearsofexperiencein teachingandresearch.

Anuja,k completed her B tech in Electronics and Instrumentation underCochinUniversityofscience and Technology, Completed her master'sinPowerElectronicsand drives under AnnaUniversity Chennai. Pursuing PhD in control systemsfromNationalInstituteof technology Calicut.Her research interestincludesmodelpredictive controller,assertivedevicesneural network.