Driver Dormant Monitoring System to Avert Fatal Accidents Using Image Processing

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 06 | Jun 2022 www.irjet.net p ISSN: 2395 0072

Driver Dormant Monitoring System to Avert Fatal Accidents Using Image Processing

1 Associate Professor, Dept. of Electronics and Communication Engineering, Sapthagiri College of Engineering, Karnataka, India

2345 Student, Dept. of Electronics and Communication Engineering, Sapthagiri College of Engineering, Karnataka, India ***

Abstract - Drowsiness is defined as a state of sleepiness when one needs rest. This state of a person can have a severe impact on the performance of day to day tasks: lack of awareness, micro sleeps (blinks with duration over 500ms), fretfulness, lethargy, and lack of mental agility Therefore, the prototype of driver tiredness identificationtoensurethesafety of both the driver and the vehicle is developed using Raspberry pi and with the help of software such as Python, Open CV, and VNC viewer. This prototype system initially checks the alcohol levels of the driver and then works towards detecting drowsiness. After detecting the face successfully, the region of the eye is extracted by eliminating facial hair, clothes, and variedbackground. Haar facedetectionalgorithm is used for object detection, efficient in identifying and extracting the face and, real time video. Thesuccessiveframes are taken as input and the eye’s region of interest (ROI) is detected by estimating the threshold value given anddifferent levels of the alerting system are activated such as a buzzer, sprinkler, light indicator, and finally, vehicle ignition is turned off. Later a mail is sent to an immediate family member or the owner to apprise.

Key Words: Drowsiness; Raspberry pi; Open CV; VNC viewer; Haar face; Frames; Region of Interest (ROI); Threshold value; Alert system.

1. INTRODUCTION

The increase in population has led to the emergence of privatetransportation.Thisbeingthemainreasoniscausing agreaternumberofaccidentswhichinturnisthecausefor lossoflifeandproperty.Thedriver’sunalertnessismainly due to a prolonged journey without doze and relaxation. There is a fatal car accident for about 25 30 seconds. The public too has become more concerned about this issue regarding the safety and security of the vehicle under the circumstances of thieving or misfortune. The other parametersthatcontributetothisfindingofgriefaccidents include weather, traffic, road conditions, lack of driver’s safetymeasuresandmechanicalperformanceofthevehicle, etc., The psychological cause is mainly due to driver’s unawareness. This may be the kind drunkenness, fatigue, mentalcondition,anddrowsiness.Theslackeningofthese factorscaneffectivelyreducedrivers’consciousness.

Overthedecades,severaldrowsinessdetectiontechniques toidentifyfaceandeyesinreal timehavebeenformulated for monitoring driver drowsiness. This paper targets to assess the driver’s EAR (Eye Aspect Ratio) to identify the drowsiness. The raspberry pi is programmed with the pythoncode.Thealgorithmisformulatedinsuchawaythat it allows a raspberry pi camera module to be able to recognize the face and eye region efficiently. The above process is the most significant activity and a prime measuringfactorthatcanservetomeasurethedrowsiness ofthedriver.Aftersuccessfuldetectionoftheeye,itisthen analysed for drowsiness detection using the PERCLOS algorithm. The driver is further alerted for drowsiness through a different alerting system such as a buzzer, sprinkler,andLEDindicatoratdifferentlevels.Therebythe safety of the vehicle and the driver is ensured preventing lossoflife.

2. LITERATURE SURVEY

[1] Neetu Saini et.al. propose a face processing prototype system. This implementation is done through template matching,skincolormodelusingthealgorithmsofintegral images, and cascaded weak classifiers. The applications includebiometrics,imagedatabasemanagement,andvideo surveillance.

[2]SPriyadarsiniet.al.proposeacomputervisiontechnique based system for driver and road safety. This system is efficientinfunctioningand workingofall itsthreephases viz. face detection, eye extraction, and detection of drowsiness.Theproposedsystemiseffectiveandefficientas itisoflowcost.

[3] K Subhashini Spurjeon et.al. [3] propose a dedicated system for analyzing drivers’ tiredness. The cam shift algorithm is used for the tracking process. The methodologies include eye position detection, eye gaze recovery,andeyeblinkand,eyeclosureevaluation.Thisis anoutbreakofthetraditionalwayofdrowsinessdetection.

[4]JasmeenGillet.al.statesthattrafficaccidentsaremainly causedduetodrivers’sleepiness.Themethodologiesused include electroencephalogram, ECG and, eye closure capturing. The various detection technique in this system

2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

includes Lab Colour Space [LAB], thresholding fuzzy C: Means Clustering method, and Circular Hough Transform [CHT].

[5] V B Navya Kiran et.al. implement machine learning techniques and, AI technologies to monitor drivers’ sleepiness. The areas focused include driver distraction, unalertnessand,aggressivedrivingbehavior.Thedifferent methodologiesincludePerclos,Camshaft,Haartraining,and Viola Jonesalgorithm.

[6]PriyaSwaminarayanet.al.proposethetechniqueofface detectioninpixelswithelaboratedfeaturesandvariabilities provided throughout human faces. Features include pose, expression, smile, role and orientation, pores and complexation and, photo resolutions. This system is also used for computer learning, especially OpenCV. The applicationsincludeCCTVvideosecurity,human computer interface, image database search, banking security, e commerceservices,passports,andemployeeids.

[7] J Manikandan et.al. propose a face recognition system withthebenefactionofcomputervision,OpenCVwithinthe scope of cops’ investigation. The phases include identification of the face, positioning of the face and, outlining of facial capabilities. The classifiers employed includeLBPandHaar.

[8] Ipshita Chatterjee et.al. present a comprehensive and non obtrusivedriver’srobustnessdetectionsystem.Italso primarily includes alcohol sensors employed along with motiondetectionand,landmarkdetectioncomputervision techniques.

[9]Wen B Hornget.al.proposeadrivingsafetysystemby employingthemethodologiesbasedoncolormodelssuchas RGB, YCM and HSS color model that is well suited for differentiatingskinnyand,non skinnycolorsirrespectiveof shadowsandreflections.

[10]KhushbuPandeyet.al.proposeacontemporaryimage presentation technique based on contractive images. The hardwarerequirementsincludean89C52microcontroller, L293D motor driving, and LCD MAX232IC power supply underthebridgerectifier.Theadvantageistheadditionof moredatabasesandcheapercomponents.Itcanbeusedin electronicgadgetsforsecurity.

[11]FengYouet.al.isanon invasiveand,cheapmethodof identifyingdrivers’unalertnessbasedontheirbehavior.The fatigue detection algorithm is based on CNN. It is used in combination with AdaBoost and kernel correlation filter. Thisalgorithmoutperformanceinbothaccuracyandspeed, the services include an intelligent transport system and trafficsafety.

[12] Paul Viola et.al. describe a distinguished work with three contributions viz representation of a new image,

AdaBoostlearningalgorithmand,complexcascadeclassifier for computation on regions where promising objects are detected.Thissystemachieveshighcomputationalefficiency withlesstime.Italsothrowsgenericinsightsthathavewide applicationsincomputervisionandprocessingoftheimage.

[13] Souvik Das et.al. propose experimental results formulatedbyusingcomputervisionand,librariesunderthe frameworkofOpenCV.Apythonprogrammingtechniqueis usedfortrackingand,detectionofthefaceandinturnfor correctclassification.

[14]MuhammadRamzanet.al.statethatdriverdrowsiness is the main factor leading to severe injuries and, financial losses.Theimplementedsystemconsistsofanalarmtoalert the drivers if out of concentration. The research methodology comprises data acquisition, data selection, drowsiness detection techniques, dynamic template matching, analysis of mouth and, yawn, eye closure and, headposturetechniques.

3. OBJECTIVES

1. Todetectthealcoholconsumptionandifthealcoholis consumedbythedriver,thentheignitionwillnotturn on.

2. Todesignasystemtodetectdriver’sdrowsinessbased onmeasurementofthefaceandeyedetection.

3. Toimple0mentabuzzerandsprinklersystemtofurther increasethevigilanceofthedriver.

4. Further to increase the protection of the driver, the indicator is turned on followed by switching off the ignition.

5. Tomailtheimageofthedrivertotheowner/immediate familymembertoappraise.

4. METHODOLOGY

Although Viola Jonesisanoutdatedframework,itisquite exceptionalandinfluentialinreal timefacedetection.This algorithm has a slow training speed but, once trained can detect faces with enormous speed in real time. The characteristics of this algorithm include an eye positive detection rate, robustness, and practical application of detection of faces from non faces. The working of this algorithmisinsuchawaythatimagepixelsaresummedup withinrectangularareas.Thefourstagesofthisalgorithm include the selection of Haar features, integral image creation, training of AdaBoost and training of cascading classifiers.

Amethodologywhereaclassifierisframedcomprisingofa set of positive and negative photos and drilled into a classifier with the aid of machinelearning is what is Haar

Volume: 09 Issue: 06 | Jun 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page1080

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 06 | Jun 2022 www.irjet.net p ISSN: 2395 0072

cascadingalgorithm.BoththeHaarcascadeandViola Jones algorithmwereputforthby Paul Viola andMichael Jones. TheclassifiersimplementedforHaarfeatureextractionare exclusively trained for object detection. The successful detectionofthefaceandfacialexpressioninanimageisalso achieved.Boththepositiveandnegativeoftheimagearefed totheclassifierandthecharacteristicsareextracted,where each character is an individual attribute obtained by the differenceinthesummation of pixelsin a white rectangle andfromthesummationofpixelsinablackrectangle.The Haar likefeatureisanintegralimageandcanbecalculated inconstanttimeirrespectiveofitssize.

AdaBoost short for adaptive boosting is a meta algorithm used for statistical classification. It is flexible as it can be used with other learning algorithms to improve performance.Theprinciplebehindthisboostingalgorithmis tofirstbuildamodelbasedonthetrainingdataset,asecond model is built to rectify the errors present in the former model.Thisisacontinuousprocedureanditterminatesonce alltheerrorsareminimizedandacorrectpredictionofthe datasetisachieved.Boostingalgorithmcombinesmultiple weaklearnerstoreachthefinaloutputconsistingofstrong learners

5. BLOCK DIAGRAM

Fig 1:Methodologyfordrowsinessdetection

Thiscanbecategorizedandworkeduponintwoways:by evaluatingandanalyzingvariationsinphysiologicalbehavior suchaschangesinbrainwaves,heartrateandflickeringof the eye. The second path is by the analysis of physical changes,forexample,postureverificationwhetherstraight orsagging,driver’sheadinclination,andpercentageofthe eyes open/shut under varied light conditions. But, all of these methods pose threat and it is not reasonable as the electrodesthatdetectalloftheabove mentionedvariations needtobedirectlyinjected intothedriver’sbodythereby causingirritationsanddiversionstothedriver.Theprecise screeningcapabilityofelectrodesmayreducewhenexposed foralongdurationandalsoduetotheprofusesweatingof the driver. But, the proposed system for drowsiness

detectionmainlytargetsPERCLOSalsocalledthepercentage ofclosurewhichprovideserror freeandpreciseinformation on eye closure. This approach is non obtrusive and non invasiveandacompletelydriver friendlysystemworkswell irrespective of road conditions even a micro nap can be detected as per the eye threshold value given in the code. The stages of development of this system are a series of important operations such as face tracking, identification, eyeextraction,detectionofthestateoftheeyeandtestingof driverfatigue.ThePERCLOSestimationefficientlycomputes the portion of the eyes being shut/open with the average numberofframesforaparticularperiod.Andaddingto it willbetheAlcoholSensortodetectthelevelofconsumption ofalcoholifthedriverhasconsumedalcoholmorethanthe thresholdvaluethevehicleignitionisnotenabled.

6. MODULE DISCRIPTION

1. FaceDetection:Byusingawebcamthefaceiscaptured andcontinuousvideoimagesorfaceisconsidered,and this face is further divided into different frames. DifferentfeaturesbyusingtheViola Jonesalgorithmare appliedtoframes.

2. Eyedetection:Oncethefaceisdetected,thenextregion in the face is the eyes. Here, the per closure value or thresholdvalueisconsideredandthisvaluedependson theusercode.Forexample,iftheuserfixesavalueof 0.33asthethresholdinthecode,ifthedriverclosesthe eyebelowthedefinedthresholdvalue,thenthedriveris saidtobedrowsy.

3. Drowsiness detection: As mentioned in the eye detection, if the driver closes his eyes below the threshold,thenthedriverisconsideredtobedrowsy. This drowsiness is mainly due to continuous driving without taking short breaks or any mental disorders. Thisproblemcanbesolvedbyusinganalertsystem.

4. Alert system: To alert the driver, this system is implemented. In this project, the alert systems are buzzerandsprinkler.Wheneverthedriverisdrowsyi.e., when the driver closes his eyes below the threshold valuethealertsystemgetsalerted.Thebuzzerturnson forafewsecondsandifthedriverdoesnotrespond,the sprinklersprinklesthewateronthedriver’sfacewhich is acting as the next level of an alert system. Then further to increase the vigilance of the driver the indicator is turned on followed by switching off the ignition.

Here, the indicator is a signal for the vehicle behind and around,therebyindicatingthedrowsinessdetectedvehicle isabouttostop.Assoonasthevehiclestops,theimageofthe drowsydriverissenttotheimmediatefamilymemberorthe ownertoappraise.

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 06 | Jun 2022 www.irjet.net p ISSN: 2395 0072

7. SOFTWARE DESCRIPTION

1. OpenCV(Open SourceComputerVisionLibrary):Itisan open source computer vision and machine learning softwarelibrary.OpenCVwasbuilttoprovideacommon infrastructureforcomputervisionapplicationsandto acceleratetheuseofmachineperceptionincommercial products.BeingaBSD licensedproduct,OpenCVmakes iteasyforbusinessestoutilizeandmodifythecode.The libraryhasmorethan2500optimizedalgorithms,which includesacomprehensivesetofbothclassicandstate of the art computer vision and machine learning algorithms.Thesealgorithmscanbeusedtodetectand recognizefaces,identifyobjects,classifyhumanactions in videos, track camera movements, track moving objects,extract3Dmodelsofobjects,produce3Dpoint cloudsfromstereocamerasandstitchimagestogether toproduceahigh resolutionimageofanentirescene, findsimilarimagesfromanimagedatabase,removered eyes from images taken using flash, follow eye movements,recognizesceneryandestablishmarkersto overlayitwithaugmentedreality,etc.OpenCVhasmore than47thousandpeopleintheusercommunityandan estimatednumberofdownloadsexceeding18million. Thelibraryisusedextensivelybycompanies,research groups,andgovernmentalbodies.

2. Dlib:Dlibisalandmarkfacialdetectorwithpre trained models,theDlibisusedtoestimatethelocationof68 coordinates (x, y) that map the facial points on a person’s face like the image below. These points are identified from the pre trained model where the iBUG300 Wdatasetwasused.

3. Imutils: Imutils consist of a series of convenient functionsofbasicimageprocessingsuchastranslation skeletonization resizing, sorting contour colors, and edge detection and work easier with OpenCV. The shifting of an image is done along its axes. The image translation process involves shifting upwards, downward ds, or sideways directions or with the combinationsofthesedirections.

4. NumPy: NumPy stands for Numerical Python, is a standardlibraryconsistingofobjectsinanarrayandac collectionofparticularroutinesforthepreviouslyarray processing.Mathematicalandlogicaloperationscanbe performed on this array. NumPy is a python package createdintheyear2005byTravisOliphant.

5. SMTP:SimpleMailTransferProtocolisusedtosendand receiveemail.ItisusuallypairedwithIMAPorauser level application such as POP3, to handle the reclamation of messages. SMTP is an asymmetrical protocolinwhichoneserverinteractswithmanyclients anditrunsonTCP/IPlisteningtoport25.

8. FUTURE SCOPE

Thedrowsinessdetectionsystemcanbefurtherenhanced andextendedbyextractingthemouthregionofthedriverto indicate drowsiness through yawning. This work can be achieved by implementing an IR webcam that uses IR radiations to detect drowsiness. To reduce the cost of hardware, this project can be turned into a mobile application.TheRaspberryPiandPicameracanbemounted onthesunvisionofthevehicle.Afullywirelesssystemor loopcanbeachieved.

9. CONCLUSION

Drowsy driving is as destructive and life threatening as drunkdriving.Anautomaticprototypesystemisdeveloped fordrowsyconditiondetectionwhiledriving.Thealcoholic levelsofthedriverarechecked,anddifferentnotthealcohol sensors have varied ranges to verify the same. The continuousvideostreamisextracted,readanddetectedby using the Haar Cascade algorithm. Haar features include digitalimagefeaturesthatareusefulinobjectdetection.This prototypesystemisefficientindetectingdrowsinessunder variedlightconditionsandinthecasewhenthedriverwears spectacles. An inbuilt system for driver safety and car securityispresentinluxurious cars.Hencethisprototype systemcanbeinterfacedwithnormalcarsalso.

REFERENCES

1. NeetuSaini,SukhwinderKaur,HariSingh,“AReview: FaceDetectionMethodsandAlgorithm”,IJERT,Vol.2, Issue6,June2013.

2. SPriyadarsini,ChahakAgarwalD,DeshiyaNarayanM, “DriverDrowsinessDetectionSystemUsingRaspberry Pi”,IJSDR,Vol.4,Issue3,pp214 218,March2019.

3. K Shubhashini Spurgeon, Yogesh Bahindwar, “A DedicatedSystemforMonitoringofDriver’sFatigue”, IJIRSET,Vol.1,Issue2,pp256 262,December2012.

4. Jamesasmeen Gill, Chisty, “A Review: Driver DrowsinessDetectionSystem”,IJCST,Vol.3,Issue4, pp243 252,August2015.

5. VBNavyaKiran,RakshaR,AnisoorRahman,VarshaK N,NagamaniNP,“DriverDrowsinessDetection”,IJERT, Vol.8,Issue15,pp33 35,2020.

6. Priya Swaminarayan, Manoj Nath, Bipasha Mandal, “FaceDetection withMachineLearningandOpenCV Classifier”,JST,Vol.5,Issue6,pp100 106,December 2020.

7. JManikandan,SLakshmiPrathyusha,PSaiKumar,Y Jaya Chandra,Umadithya Hanuman,”Face Detection andRecognitionUsingOpenCVBasedonFisherFaces

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Algorithm”, IJRTE, Vol. 8, Issue 5, pp 1204 1208, January2020.

8. Ipshita Chatterjee, Isha, Apoorva Sharma, “Driving FitnessDetection:AHolisticApproachforPrevention ofDrowsyandDrunkDrivingUsingComputerVision Techniques”,ICSSS,2017.

9. Wen Bing Horng,Chih Yuan,YiChang,Chun Hai Fan, “DriverFatigueDetectionBasedonEyeTrackingand DynamicTempletMatching”,proceedingofthe2004 IEEE,ICNSCApril2004.

10. Khushbu Pandey, Reshma Lilani, Pooja Naik, Geetha Pol, “Human Face Recognition Using Image Processing”,IJERT,ICONIC14Conferenceproceedings.

11. FengYou,XiaolongLi,YunboGong,HaiweiWang,And HongyiLi,“AReal timeDrivingDrowsinessDetection AlgorithmwithIndividualDifferencesConsideration”, IEEE Access, Vol. 7, pp 179396 179408, December 2019.

12. Paul Viola, Michael Jones, “Rapid Object Detection usingaBoostedCascadeofSimpleFeatures”,Accepted Conference on Computer Vision and Pattern Recognition2001.

13. Souvik Das, Soumyadeep Sett, Subhojyoti Saha, “A Novel Face Detection Technique Using OpenCV”, IJRESM,Vol.4,Issue7,pp121 124,July2021.

14. MuhammadRamzan,AminaIsmail,AhsanMahmood, “A Survey on State of the Art Drowsiness Detection Techniques”, IEEE Access, Vol. 7, pp 61904 61919, May2019.

15. Prakash Jadhav, G.K. Siddesh, “Bandwidth oriented ImageCompressionusingNeuralNetworkwithASAF”, International Journal of Neural Networks and AdvancedApplications,ISSN:2313 0563,Vol5,pp25 32,2018.

16. Prakash Jadhav, G.K. Siddesh, "Neuro Fuzzy Nod RestitutionandMulti ScaleWaveletsforSuperiority Video",International Journal ofImage,Graphicsand Signal Processing, Vol.9, No.9, pp.11 17, 2017. DOI: 10.5815/jigs.2017.09.02

17. Prakash Jadhav, G.K. Siddesh, “Near Lossless Compression of Video Frames using Soft Computing TechnologiesinImmersiveMultimedia”International Journal of Soft Computing and Engineering ISSN: 2231 2307,Volume 4Issue 6,pp16 24,January2015.

18. Prakash Jadhav, Sai Eshwar K R, “Bandwidth ReductionofVideoCompressionUsingAnnforVirtual Multimedia”, International Journal of Management,

Technology And Engineering, ISSN NO: 2249 7455, VolumeIX,IssueVI,pp1830 1840,JUNE/2019.

19. PrakashJadhav,Yeshwanth,“DesignandFabrication of a Multitasking Agricultural Robot with Stairs Climbing Capacity”, International Journal of Current EngineeringandScientificResearch,Vol.5,Issue5,pp 70 73,2018.

20. Prakash Jadhav, Sharath Gowda, “Smart Pulmonary System with Doctor Appointment”, International Journal of Research in Engineering, Science and Management, Volume 3, Issue 5, pp 759 762, May 2020.

21. PrakashJadhav,ChilukuriMadhu,“FacialRecognition Based Attendance Management System Using Raspberry Pi”, International Journal of Research in Engineering, Science and Management, Volume 3, Issue 5,pp654 658,May 2020.

22. PrakashJadhav,Rima,“WirelessSensorNetworksfor DataAcquisitionandRemoteActuation”,International Journal of Research in Engineering, Science and Management,Volume 3,Issue 5,pp960 964,May 2020.

23. Prakash Jadhav, G. K. Siddesh, “Codec with Neuro Fuzzy Motion Compensation & Multi Scale Wavelets forQualityVideoFrames”,InternationalConferenceon Advanced Computing, Networking, and Informatics 2017, Recent Findings in Intelligent Computing Techniques,Vol3,pp599 606,04November2018.

BIOGRAPHIES

Dr. Prakash Jadhav is an Associate Professor in the department of Electronics and CommunicationEngineering,SCE, Bengaluru, Karnataka, India has over 19 years teaching and research experience He has publishedmorethan10technical papers in National and International conferences and journals HeisalsoaLifemember ofinISTE

Email: Pcjadhav12@gmail.com

Volume: 09 Issue: 06 | Jun 2022 www.irjet.net p ISSN: 2395 0072 © 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal |

International Research Journal of Engineering and Technology (IRJET) e ISSN: 2395 0056

Volume: 09 Issue: 06 | Jun 2022 www.irjet.net p ISSN: 2395 0072

Deepa B is a UG student of department of Electronics and CommunicationEngineering,SCE, Bengaluru,Karnataka,India.

Harshitha V is a UG student of department of Electronics and CommunicationEngineering,SCE, Bengaluru,Karnataka,India.

Email: harsithavkarkera182@gmail.com

Lavanya N is a UG student of department of Electronics and CommunicationEngineering,SCE, Bengaluru,Karnataka,India

Email: deepainchu@gmail.com Email: natarajulavanya@gmail.com

Megha S is a UG student of

department of Electronics and CommunicationEngineering,SCE, Bengaluru,Karnataka,India.

Email: meghas9155@gmail.com

2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal