Issuu

EYE MATE AI: A Multi-Modal Vision Assistance System for Visually Impaired and Elderly People

Aisha Shaikh1 , Saba Jalgaonkar2 , Aaliya Shaikh3 , Humera Shaikh4 , Zeba Syed5

1Aisha Shaikh Hardware & software

2Saba Jalgaonkar Hardware & Software

3Aaliya Shaikh Hardware Coding

4Humera Shaikh Hardware Coding

5Zeba Syed, Dept. of computer Engineering, Abdul Razzak Kalsekar Polytechnic, Maharashtra, India

Abstract - Visually impaired individuals encounter numerous challenges while attempting to read any materialinprintorinhandwrittenformduringtheir everydayliving.Thiscanmakeitdifficultforavisually impairedpersontobeabletoreadbooks,documents, medicine labels, sign boards, or notes and may necessitate making use of another individual for assistance, which diminishes their ability to be independentandtypicallycreatesadditionaldifficulty in accomplishing their daily activities. Presently knownmethodsforhelpingindividualswhoareblind or visually impaired include braille books and audio recordings, however, these methods may not always beavailableforuseatthetimetheyareneeded.Other assistive methods of reading are also expensive and difficult to use. Therefore, an affordable and simple method for assisting individuals who are blind or visuallyimpairedtoreadtextisnecessary.

Key Words: Assistive Technology,OCR,Image Processing, ESP32-CAM, Text-to-Speech, Computer Vision, Visually Impaired,Real-TimeTextRecognition,OpenCV,Embedded System

1.INTRODUCTION

Many people with vision impairments have challenges reading printed or hand-written text throughout the day. Books,documents,medicinepackaging,andsignagecanall be hard to read without the assistance of another person. As a result of not being able to read independently, those withvisionimpairmentshavetorelyonothersforsupport, whichhinderstheirindependenceandmakesitthatmuch harder to accomplish daily tasks independently. Braille booksandaudiobookrecordingscanassistindividualswith print disabilities, however, neither option is always availableinatimelyfashion.Electronicassistivedevicesto help people with print disabilities read can also be prohibitivelyexpensiveordifficulttouse,sothereforenot everypersonwithaprintdisabilitycanusethesedevices.

EyeMate solves this problem by providing visually impairedindividualsandotherswithprintdisabilitieswith a simple, intelligent assistive decision support system for identifying and reading text (in real-time). The EyeMate captures an image of the text using a camera module and sends the image to a computer for processing. The computer will use both Image Processing and OCR to convert the text into speech (voice output). The EyeMate can read both printed and clearly hand-written text, and thereforewouldprovideaudiooutputinreal-timetoassist visuallyimpaireduserswhentheyneedassistancereading duringtheireverydaylives.

1.1 Challenges Faced by Visually Impaired People

All over the world, printed information is available on books, posters, product labels, packaging, and official documentation. Visually impaired individuals cannot independently access information until someone helps them. They face continual dependence upon others and a lackofconfidenceduetothis.

Brailleandothertactilesystemshavemadeitpossiblefor manytouseprintedtext,butnotallprintedmaterialsexist inbraille.Audiobookshavelimiteduseastheyarenotable to provide information in real-time. As a result, visually impaired consumers continue to experience significant challengesintheirdailylives.

There is a need for an easy and quick way for visually impairedconsumerstoreadandreceiveprintedmaterials aroundthem,usinginformationfromtheirenvironment.

1.2 Limitations of Existing Assistive Devices

Manytoolsexisttohelpthosewhoareblindorhavelow vision; however, they come with several drawbacks. First, different tools can be prohibitively expensive for some users.Second,othertoolsmaynotbesuitableforsomeone who has not previously had any training/experience in operatingthem.Third,somecanonlyreadprintandcannot interprethandwrittenmaterials(e.g.,businesscards).

Also, some tools may not provide immediate responses, thereby making them less useful for many people every day.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 03 | Mar 2026 www.irjet.net p-ISSN: 2395-0072

This is why any new technology must be developed that allowsforreal-timeandlow-costdetectionofbothprinted and handwritten documents within one product that speaks back through a voice synthesizer. This is what EyeMateisabletoaccomplish.

2. System Architecture

.EyeMateassistsindividualswithvisualimpairmentsin readingprintedandhandwrittenwordsclearly.Itallowsfor easy communication between the camera module, processing unit, and audio output devices. The ultimate purposeofEyeMateistoscantheenvironmentandconvert the contents of the scanned materials into audio format, allowingthevisuallyimpairedusertohearinformation.

There are three main components that comprise the EyeMatesystem:(1)capturinganimage,(2)processingthe text within the image, and (3) outputting audio. The EyeMate system captures an image of either printed or writtentextwiththeESP32cameramodule.Oncetheimage hasbeencaptured,itiswirelesslytransmittedtoacomputer forprocessing.

The processing of the captured image uses OpenCV to enhance the image quality through greyscale conversion and removing noise. After the image has been preprocessed,OCRisemployedtoidentifythetextwithinthe captured image. The EyeMate system can recognize both printed text and importantly it also recognizes clear handwriting.

Oncethetexthasbeendetected,thetext-to-speechfunction oftheEyeMatesystem convertsthetext intoanaudio file whichisplayedbackoveraloudspeaker,thusenablingthe visuallyimpairedusertohearthetextasitisreadaloud.

2.1 Hardware and Processing Interaction

The relationship between the computer system and the hardware device is straightforward with the EyeMate System. The ESP32 CAM camera takes the picture of the environmentaroundtheuserandsendsitviawirelessdata transmission to the computer processor for processing. When the user points their camera toward an object, the ESP32CAMcamerawilltakeaphotographoftheobjectand then send the image wirelessly to the computer. The computer functions as the main processor where image processing and text recognition takes place. Image processingwillbedonebytheOpenCVlibrary,whichallows for improved image quality so that text can be easily detected.

Once the image has been processed, the OCR system will readthedetectedtextfromtheimageandsendthetextto thetext-to-speechconversionsystem,wherethetextwillbe converted to speech and output through a speaker. The hardware and software interaction provides a smooth operationoftheEyeMateSysteminreal-timeforuserswith visual impairments.

2.2 Image Processing and Text Recognition Layer

The EyeMate System consists of an Image Processing and TextRecognitionLayer.Atthislayer,capturedimagesare processed and converted to readable text. The image is processed and made available for text detection. The imagecapturedmaynotbeingoodqualitysothefirststep isatimagepreprocessingbeforedetectingthetext.

The first step in processing the image is to process the image using OpenCV techniques to change the original imageintoagrayscaleimagewhileremovingi.e.unwanted noise, so the text will appear clearer. This will help in improving the overall accuracyoftext detection. Oncethe imageispreprocessed,itisutilizedbytheOCREnginefor recognitionofcharacters.

Fig -1:WorkingofEyeMate

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 03 | Mar 2026 www.irjet.net p-ISSN: 2395-0072

After the OCR system does its job of recognizing the character in the image, the OCR converts the recognized characterstoanelectronicdigitaltextfileineitherprinted or clear handwriting. Once the text is retrieved, then it is passedtothetext-to-speechmodule.

Finally, the text-to-speech module converts the detected texttoanaudiooutputorsound.Theoutputaudiowillbe played through the speaker so the user who is visually impairedcanlistentothetextinreal-time.

3. Core modules

TheMain(Core)modulesarethekeysystemsofEyeMate thatallowfordeterminingtextandconvertingittospeech. Theyfunctioncollectivelytocollectimagedata,processthe text, and produce aural output for those who are blind or vision impaired. The EyeMate system provides a precise andeasy-to-usemeansofaccessingwheelchairviasightby eachmodulecompletingadistinctiveduty.

For instance, the Image Capture module uses the ESP32 CAM to produce the image of printed or handwritten text around the EyeMate user. Next, the Image Processing module refines the captured image using OpenCV so the printedorhandwrittentextisdisplayedclearly.Finally,the OCRmodulereviewsthetextcontainedintherefinedimage andconvertsthetextfromtheimagetoadigitaltextformat. Next,theTexttoSpeech(TTS)moduleconvertsthetextto be read to audio. Finally, the Audio Output module reproducestheaudioversionoftextthroughthespeakerto allowtheusertoheartheirtext.

The combination of the Image Capture, Image Processing, OCR,TTS,andAudioOutputmodulesprovideaquick,easy and effective way for EyeMate users to read printed or handwrittentextinstantaneously.

3.1 Image Capture Module

The Image Capture Module collects visual data in the surroundingenvironment.TheESP32CAMcanbeusedto collect images of printed or handwritten text using the ESP32 CAM as the image capture device. By aiming the cameraatabook,label,ordocument,theusercapturesan image and wirelessly transfers it to a PC. The module is criticalbecausethequalityofthecapturedimageisdirectly proportionaltotheaccuracyofthetextdetection.

3.2 Image Processing Module

This module helps increase clarity of captured images in preparation for text detection through image preprocessingwithOpenCVbyconvertingthecolorimageinto agreyscaleimage,andremovingunwantedimageartifacts. Theresultisanimprovedtextdetectionrateforeachimage as the text becomes more readable. Overall, the improvement of the performance of OCR systems and

devicesisagoalofimagepreprocessing.

3.3 OCR Text Recognition Module

Various features of OCR modules, such as recognition of both printed and clearly written text, can be useful in a variety of contexts, including character recognition and digitizationofimages.Detectionaccuracyiscloselyrelated to the quality of the input image and the preprocessing donepriortoanalysis.Oncetexthasbeendetected,output fromOCRissenttoanaudioconversionmodule.

3.4 Text to Speech Module

Using special software called a text-to-speech engine, the text written out on the screen will get converted into an audio file that can be played back. Consumers that are eitherunabletoseeorhavetroublereadingwillnowhave awaytohearwhatothershavewritten,andtheabilityto experience what visually-impaired people have been able todofordecades.Italsomakesthissystemveryhandyfor usewithapplicationswherespeediscritical.

3.5 Audio Output Module

The last component of this program is the Audio Output Module.Thismoduleplaysbackthespeechoutputfromthe program (audio) using either a set of speakers or headphones, allowing users to clearly hear whatever text has been detected by the system. This allows individuals who are visually impaired to receive information on their ownwithoutassistanceandenablesthemtousethesystem effectivelyintheirdailyactivities.

4, Methodology

The methodology of EyeMate describes how the system functionstodetermine whatisprintedtext andconvert it tospeechforpeoplewithdisabilities.Themainfunctionof EyeMateistofacilitateeaseofreadingbyusingaCameras totake a snapshot of text in the environment it is located and processed in four steps; capturing the image, preprocessing (improving the quality) of the image, using OCR to determine text present within the image, and the finaloutputfromthetext-to-speechtoaudio.

Theprocessbeginsbyimagingtheprintedor writtentext using a camera. Next, the computer takes the captured imageandprocessesittoincreasethequalityoftheimage. Aftertheimagehasbeenprocessedtoimproveitsquality, OCRwillruntodeterminethetext.Thetextthathasbeen identified by the computer is converted to audio for the user to listen (i.e., the user can read the data independently). This step-by-step process is designed to providereal-timereadingtoassistthosewhocannotseeto readindependently.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 03 | Mar 2026 www.irjet.net p-ISSN: 2395-0072

4.1 Image Capture

ThecapturedimagefromanESP32CAMwillbeastillimage takenwithacameraasinputfromauserwhowillusethis unittotakeapictureofprintedorhandwrittentext.Users willpointthecameraatanimagetext,abook,atext-printed orhandwrittenpieceofpaper,alabel,asign,orwhateveris required,andtheimagecapturedwillbesentviaawireless connection to a computer. Image quality must be good in orderforthedevicetodetectthetextwithHighAccuracy.

4.2 Image Preprocessing

Preprocessing has been completed when the image has been received. The OpenCV library performs two main preprocessing functions: creating a grayscale copy of the imageandreducinganynoisepresentintheimage.These twotasksresultinamoreviewableandreadableversionof thetext;thus,thispreprocessingincreasestheaccuracyof theOpticalCharacterRecognition(OCR)engine

4.3 Text Recognition using OCR

At this stage, the utilization of Optical Character Recognition(OCR)isemployedtoidentifycharactersfrom animage.TheOCRreadsthecharactersandconvertsthem intodigitaltext.Thesystemrecognizesbothclearprinted characters as well as clear handwriting. The recognized characterswillbesavedforthenextstep

4.4 Text to Speech Conversion

Oncethetextisrecognizedbythesystem,thesoftwarewill utilize text-to-speech capabilities to generate sound from the recognized text allowing a visually impaired user to heartheinformationratherthanreadit.

4.5 Audio Output and User Interaction

TheEyeMateprocessiscompletedbyturningtheidentified textual information into audio, which is then delivered to theuser.Oncetheopticalcharacterrecognition(OCR)part of the EyeMate has correctly identified the text from the imageithastaken.Theidentifiedtextissenttothetext-tospeech processor so that it can be synthesized into clear voice output and sent out to either a speaker or headphonesconnectedtothedevice.

The ability to generate and deliver the audible version of theidentifytextallowsthevisuallyimpairedusertolisten tothetextratherthanhavingtoreadthetext,thusmaking theEyeMateusefulineverydaysituations,suchasreading books,readingdocuments,readinglabelsandreadingsigns orbillboards.TheeyeMatedeliversthisoutputinreal-time, allowingtheusertoreceivetheoutputwithoutdelay. Theoverallinteractionoftheuserwiththedeviceissimple. The user simply points the camera at the text and the EyeMateautomaticallycaptures,processes,andgenerates

theaudibleoutput,whichmakestheEyeMateuser-friendly and usable daily. The audio output portion of EyeMate finalizes the operation of EyeMate and allows visually impairedusersgreaterindependence.

5. RESULTS AND SYSTEM PERFORMANCE

TheEyeMatetestingsystemwasusedtoassesshowwellit coulddetectandconverttextinto speech.Itcomprisedan ESP32 camera for capturing images; OpenCV to process images;OpticalCharacterRecognitionsoftware(orOCR)to detect printed or clear handwritten text; and a Text-toSpeechEngine(orTTS)toconvertthetextintoaudio.

Through testing, the EyeMate System successfully identifiedbothprintedandclearhandwrittentextandread the information aloud to the user from books, labels, or documentsthatwereavailabletotheuser.

Before capturing images, they were preprocessed by improving their quality resulting in improved clarity of image-recognized characters by the OCR engine. The convertedaudiowouldbeofgoodenoughqualitysuchthat userswouldfindtheaudioreadilyunderstandable.

TheEyeMateAutomatedHandheldSystemallowedfornear real-time utilization, meaning the user could capture text, andreceiveaudio output,withminimal delay. Thetesting resultsindicatedthattheEyeMateAutomatedSystemwill provide visually impaired people with the means to read text independently. Additionally, the EyeMate Automated System is relatively simple and low- cost to use and can

Fig -2:WorkflowofEyeMate

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 03 | Mar 2026 www.irjet.net p-ISSN: 2395-0072

therefore be utilized in many applicable situations on a dailybasis.

5.1

System Accuracy

ThelegitimacyoftheEyeMatesystemreliesheavilyonboth the quality of photos taken of documents and the environment in which those documents were produced. Testing results produced extremely accurate results for printed materials while providing reasonably accurate resultsforclearlywrittendocumentscreatedbyhand.Preprocessing of images using OpenCV helped with noise reduction and improving visibility of text, leading to increased levels of accuracy when performing OCR processing.

TheOCRenginehadveryhighlevelsofaccuracyacrossthe entirerangeofdetectedcharacterswhenthedocumentwas properly photographed. Under appropriate lighting conditions, the system produced accurate results and created speech output that matched each character recovered. The text-to-speech module provided clear audiblerepresentationsacrosstheentirerangeofdetected input characters without experiencing substantial errors. As noted above, the overall performance of the system underreasonableoperatingconditionswasreliable.

5.2 User Experience and Performance

The EyeMate System is packed with features to ensure userswithPoorVisioncanuseiteffortlessly.Userssimply have to point the camera at the text to enact the process. The EyeMate will automatically take an image of the text, process the image and verbalize the information on the image.

Theoverallprocessforuserstoreceivethereal-timeverbal outputoftextisquick.Ontopofthat,TheEyeMatecanbe used to read Work-Books, Medicine Label's, Document's, and most Sign-Board's. This Independence will allow someone with Poor Vision to do activities more easily by themself.

5.3 Why EyeMate is Different from Other Systems

Unlikecurrenttextreadingsystems,EyeMateisareal-time assistivedeviceforvisuallyimpairedusersasitwillwork with both printed and handwritten text. All currently available solutions only provide an ability to read digital text.InadditiontotheabovethisallowsEyeMatetobevery usefulinnormalday-to-daysituationswhenreadingbooks, medicine labels, notices, and signs exactly as they are locatedintoday'sworldusingacamera.

EyeMate also uses an ESP32 CAM which is a great advantageintermsofreducingcostandportability.Many assistive devices currently available in the market are pricedtoohighly,anddifficulttouse.TheEyeMatesystem

issimpletouse,andalsoisaffordable.Allimagescaptured through the use of OpenCV are enhanced for additional clarity which increases the overall accuracy of text detection. Therefore, the OCR engine can recognize text with a significantly higher degree of accuracy than the standardtextreadingapplications.

EyeMate scans text continuously, and then converts that scannedtexttospeechalmostimmediately.Therefore,the combinationofcamera,imageprocessing,OCR,andtextto speech results in a completely automated EyeMate. EyeMatehasalowpricepoint,realtimeperformanceand canreadbothhandwrittenandprintedtext.Thismakesit easier to use and more practical than many other current systemsavailabletoreadtext.

6. FUTURE SCOPE

The EyeMate technology has a bright future ahead of it; thereisgreatpotentialfortheEyeMatesolutiontocontinue evolving into a system that is more accurate, faster and easier to use for those individuals who have a vision impairment.

Currently, the EyeMate can read text that is printed or in legible handwriting; however, future generations will enable the EyeMate to read many different handwriting stylesandtypesoffonts.

Another area of development that will enhance the EyeMateistheportabilityofthesystem.TheEyeMatecan alsobemadeintoafullyportableproductsousershavethe abilitytocarryitandutilizeitintheireverydaylives.

Another exciting opportunity for the EyeMate system is that its future versions will support many different languageoptions,whichwillallowuserstoreadinavariety of regional languages. Additionally, the quality of speech generated by the EyeMate can continue to improve based upon more "natural sounding" speech output. By utilizing improved image processing and improved optical character recognition (OCR) models, the EyeMate system willprovideuserswiththeabilitytoreadinamuch more accuratemanner,eveninlowlight.

In summary, future development initiatives for the EyeMate will focus on enhancing the EyeMate to provide userswiththeabilitytoreadprintedtextinanindependent manner

6.1 Expansion to Multiple Languages

OnesignificantfutureenhancementtotheEyeMateSystem involves providing support for multiple languages. Currently,theEyeMateSystemreliesprimarilyonEnglish text;however,itisanticipatedthattheEyeMateSystemwill be trained in various regional languages at some point in the future. This functionality is essential to help users to

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 03 | Mar 2026 www.irjet.net p-ISSN: 2395-0072

readbooks,documents,andotherlabelswhichcontaintext inseverallanguages.

Addingmultilanguagesupportwillalsoimproveaccessto the EyeMate System for many different people, no matter which region they live in. By utilizing advanced Optical Character Recognition models and language processing techniques,theEyeMateSystemwillbeabletoaccurately recognizelettersfrommultiplealphabetsandconvertthem into speech. As a result, the EyeMate System will be even moreusefultomanydifferentpeople.

6.2 Portable Device and Mobile Integration

Future advancements include creating a truly portable system which will replace the computer used for processing today with a smaller, integrated device. Users willbeabletocarrythenewdevicewiththemeverywhere anduseiton-the-go.

Current systems can connect to mobile devices, allowing images captured by the system to be processed using mobile-apptechnology.Withmobileintegration,userswill experience faster result times and improved audio playback. Future enhancements may include wireless headsets, improved camera technology, and upgraded processor speeds. All of these features will provide EyeMate users with a much more functional and userfriendlysystemtouseineverydaylife.

6.3 Advanced AI and Accuracy Improvement

ToimprovetheEyematessystemforthefuture,advanced AImodelsshouldbeusedinsteadofjustthecurrentOptical Character Recognition (OCR) being used now. The use of these types of deep learning models will help to further enhancetheaccuracyandspeedofthesystem.Thesedeep learning models will also assist in detecting readable text even if the image is taken in poor lighting, blurry or has someothertypeofobstructions.

Using various OpenCV techniques, the image processing aspect of the system may also be improved in order to obtaina clearerimageforsubsequentOCR processing.By making the image clearer, it will improve how the OCR enginecanreadthecharactersonthepageorimage.Future versionsofthesystemmaybeprogrammedwithautomatic language identification so that users would not have to changethesettingstoaccesstextinotherlanguagesfrom theoriginalfile.

Through enhanced AI models and rapid processing time, Eyemates can provide a much more dependable and efficient system, as well as offer additional assistance for visuallyimpairedusersinday-to-daysituations.

CONCLUSION

In conclusion, EyeMate is an intelligent assistive system that helps visually impaired people read text (printed or handwritten) easily. The EyeMate system uses an ESP32CAMmoduletotakeapictureofthetargettext.Theimage is then processed with OpenCV to identify regions of the imagethatmaycontaintext.Next,therecognizabletext is convertedfromimagedatatoaudiousingOpticalCharacter Recognition (OCR) technology and Text-to-Speech (TTS). The EyeMate system provides a means for a user that is visuallyimpairedtounderstandtheinformationcontained inprintedtext.

In addition, unlike traditional text reading software solutions, EyeMate provides a real-time reading solution andcanreadtextfromavarietyofsourcesincludingbooks, labels,documents,andsignsdirectly without the need for assistancefromanotherperson.Furthermore,theEyeMate systemislow-cost,portable,andeasytooperate;therefore, itwouldbeidealforitsuseineverydaylife.Byusingimage preprocessing technology to process images before performing text recognition, the accuracy of the text recognitionincreases,andbyprovidinganaudiooutputof the recognized text, users have a clear auditory source of theinformationprovidedbytheprintedtext.

Theprojectfurtherdemonstratesthatassistivetechnology can provide support to visually impaired individuals so they can live independently. Future enhancements of the EyeMate system could include improved OCR models, the ability to support multiple languages, and a fully portable version of EyeMate. Overall, EyeMate represents a simple butbeneficialassistivetechnologysolutionwhichhelpsthe visuallyimpairedtohavegreateraccesstoprintedtextina moreconfidentandefficientmanner.

Insummary,EyeMateisanintelligentassistantcreatedto makereadingprintedandhandwrittentexteasierforthose who are blind or visually impaired, allowing them to complete tasks at their own pace and independently. The systemutilizesESP32-CAMtocaptureimages,Opensource Computer Vision (OpenCV) for image preprocessing, Optical Character Recognition (OCR) for text recognition, and Text to Speech technology to convert text into audio output.Bycombiningthesetechnologies,EyeMateprovides userswithawaytocomprehendprintedmaterialwithout needingtoseeitvisually,therebyallowingthemtoperform dailytaskswithgreaterease.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

REFERENCES

R. Smith, “An Overview of the Tesseract OCR Engine,” Proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR), 2007. OpenCVOrganization, OpenCV: Open-Source Computer Vision Library, https://opencv.org EspressifSystems, ESP32-CAM AI-Thinker Module Datasheet, EspressifSystems,2020. J.Redmonetal., Image Processing and Computer Vision Techniques, IEEEComputerVisionPublications. GoogleDevelopers, Text-to-SpeechAPIDocumentation, https://developers.google.com P.ViolaandM.Jones,

RapidObjectDetectionusingaBoostedCascadeofSimple Features, IEEEConferenceonComputerVision,2001. WorldHealthOrganization(WHO), AssistiveTechnologyforVisuallyImpairedPeople, https://www.who.int PythonSoftwareFoundation, PythonProgrammingLanguageDocumentation, https://www.python.org Gonzalez,R.C.,&Woods,R.E., DigitalImageProcessing,PearsonEducation,2018.

| Page1767