Stay Awake Alert: A Driver Drowsiness Detection System with Location Tracking and Alarm

Stay Awake Alert: A Driver Drowsiness Detection System with Location Tracking and Alarm

1Principal, Department of Computer Science J D College of Engineering and Management Nagpur, India 2345UG student, Department of Computer Science J D College of Engineering and Management Nagpur, India ***

Abstract - Drowsiness is a condition when a person feels the need to fall asleep. There are many reasons that can cause drowsiness which include lack of sleep, depression, working overtime, etc. which turns out to be dangerous in the form ofroad accidents if the drowsy person is a driver driving a vehicle. Studies reveal that a person is most likely to die from drowsy driving as compared to driving while consuming alcohol or being distracted while driving. This paper focuses on a real-time low-cost system that detects drowsiness using a machine-learning approach. the driver will be monitored continuously by using a webcam. openCV is used with the haar cascade algorithm for face detection, dlib is used to detect facial landmarks, and compute EAR eye aspect ratio to detect driver drowsiness based on the threshold value. Here CNN Convolutional neural network is used for determining the state of the driver whether the driver is drowsy or not.

Key Words: Dlib, Eye Aspect Ratio, opencv, Haar cascade algorithm, convolutional neural network, Machine learning.

1. INTRODUCTION

AsperreportsfromtheCentersforDiseaseControl andPrevention[1].1outof25adultdriversfallasleepwhile driving.Accordingtothenationalsleepfoundationaround 6,400 people die yearly involving accidents caused by drowsiness.Theseaccidentsarenotonlydangerousforthe driverbutalsoforthepassengersandthepeoplewhoare using the road it can cause mental, physical, and financial damage. NHTSA National Highway Traffic Safety Administration reported that accidents related to drowsinesscausinginjuryordeathcost$109billionyearly. Thusthereisaneedoccurtodevelopasystemthatwillkeep thedriverawakewhiledriving.

There are many techniques used for developing driver drowsiness detection system. [2] Vehicle-based drowsinessdetectionsysteminthistechniquedrowsinessis detectedbyin-vehiclesensorscollectdatafordetectingthe drowsiness level of the driver through his behaviour the detectionaspectsarethesteeringwheelmovement,vehicle deviation and position, and vehicle speed. This type of system requires costly infrastructure and complex programming.Physiologicaldrowsinessdetectionsystems

usephysiologicalsignalsfromthehumanbodysuchasthe brain,eyes,andheartThissystemusessignalssuchasEEG electroencephalography signals for the brain or EMG electromyographysignalsformuscletone.Thissystemhas tobeimplementedwithwearabledeviceswhichmightmake the driver uncomfortable because of wearability issues. Because of these issues in physical and physiological techniquesinthissystembehaviouralmeasureshavebeen used for detecting driver drowsiness in the proposed research.Itdoesnotrequireanycomplexprogrammingor costlycomponentsandduetoitsnon-contactbehaviour,the driverdoesnotworryaboutwearabilityissues

.

Firstly a webcam is used for recording real-time video of the driver. the webcam is placed in front of the drivertocontinuouslycapturetheimageofthedriver.The frames are extracted from the video using OpenCV.it is a real-time computer vision library. haar cascade algorithm used to detect faces from the frames. After face detection, faciallandmarkshavebeenextractedbyusingliblibrary.the facial landmarks are then used to compute the EAR eye aspectratio..afterthisconvolutionalneuralnetworkisused for classifying the state of the driver. the EAR value is comparedwiththethresholdvaluethatistakenas0.2inthe proposed system if EAR value becomes less than the thresholdvalueItisfoundthatthedrowsinessisdetectedas eyesarefoundtobeclosed. Thenanalarmwill besent to alertthedriver.Afterthatthelocationofthedriverwillbe showntothedriver.

Yann Lecun is the director of the Facebook AI research group [3] who built the first model of a Convolutional neural network in the year 1988. As the domain of computer vision is increasing day by day it is enabling machines to view the world as humans do. The amazing advancement in computer vision is because of machinelearning,particularlywiththeconvolutionalneural networkalgorithm.Machinelearninggivesthemachinethe ability to learn and use this learning to perform various tasks.aconvolutionalneural networkisusedfordetecting and classifying objects. Therefore to build our proposed system we have used machine learning with the convolutionalneuralnetwork.

1.1 OVERVIEW

Therestofthispaperisdividedasfollows.Inthenextsection 2.Relatedwork,Thissectionrepresentspreviousworkdone forimplementingadriverdrowsinessdetectionsystem.In section3proposedmethodologiesisincludedwhichshows theproposedalgorithmbasedonmachinelearningandfacial landmarkdetectionalongwiththeaccuracy,efficiencystudy ofthegivenmodelwiththeexperimentalandanalysisin4.5 and6showsconclusionandreferences.

2. RELATED WORK

U.Shrinivasulu Reddy.et al [4] proposed a deep neural network a machinelearning method used to detect driver drowsinessbydetectingthestateoftheeye.inhissystemhe used a stacked deep convolutional neural network to examinefacialfeatures.

Mohammed .S Majdi., et al [5] developed a drive-net for detectingdriverdrowsinesshisteamcomparedthissystem withRNNrecurrentneuralnetworkandmultiplesystemsfor analysinghowhissystemachievesbetterresults.

Jin ming gua., et al [6] proposed a driver drowsiness detection system using hybrid cnn and long-short-term memory LSTM is used in this system as a time skipper to improvetheprocessofcatchingframes.itignorestheframe that does not show any slight change thus improve processingtime.

Sonekar, Shrikant V., et al [7] proposed a computational algorithmthathelpstodecidetheterminationofaproposed head algorithm using the finite state machine concept, the behavior of the attackers is identified and the finite state machine clearly gives the idea about the identification of intrusion.

researchbasedonEnhancedrouteoptimizationtechniquefor maliciousnodedetection techniqueSonekar,ShrikantV.,et al [8] proposed research in which it shows that out of thousands of paths fromsource to destination, only one is optimalandalsogivestheinformationAboutseveralnodes involved in the creation of optimal Path using Cluster and clusterheadformationalgorithm

Kyong hee lee., et al [9] proposed a study on different methodsusedforfeatureextraction.inthispaperdifferent methods are defined and compared for facial feature extraction.itshowshowtomonitoreyes,headposition,and mouthfromthefacetodetectdrowsiness.

Mohesen Babaein., et al [10] proposed logistic regression techniquesusingamachine-learningalgorithmfordetecting drowsiness. Here ECG Signal is used to monitor heart rate when a driver is driving a vehicle. The logistic regression methodisusedtoimprovethedetectiontimeandaccuracyof thesystem.thispapershowsthatbyaddingmorelayerstoa neuralnetworktheaccuracyofthesystemcanbeincreased.

Lemkaddem., etal[11]proposedastudyon amulti-modal driverdrowsiness detectionsystem.inthis papermultiple methods are used same time to detect drowsiness. A hybrid

systemwasdevelopedby combingbothphysiological behaviouralmethods.Acamerausedwithwearablewatch. ThatrecordsthePPGphotoplethysmographysignal.

3. PROPOSED MEHODOLOGY

3.1 Image Capture :Real-timeimageiscapturedbyusinga webcamera with the help of OpenCV. The inbuilt laptop’s

webcamfrontcamerawillbeusedforthispurpose.Byusing webcam,imagesfromthevideoareextractedcontinuously.

3.2 Face detection: Fordetectingtheentirefacefromthe imageViola-Jonesobjectdetectingcalculation[12]withHaar overflowclassifierhasbeenusedwithOPENCVandpython. Haar cascade classifier computes Haar highlights for identifyingthefacefrompictures.Fig-2:

afterthataccordingtothecnnpredictionitwillcomparedto athresholdvaluewhichwehavetaken0.2intheproposed system.ThevalueofEARisconstantlymonitored.whenthe EARvaluebecomeslessthanthethresholdvalueablinkis considered and the program will move for further processing..

3.3 Eye Localization: To locate eyes from the face, facial landmarksareused.Opensourcedliblibraryhasbeenused to produce 68 (x, y) coordinates that match specific facial structures.dliblibraryfunctionsareusedtodetecttheeyes inreal-time.Theselandmarksareobtainedfromtrainingthe shape_predictor. Points 37-42representthe right eyeand 43-48 represent the left eye from facial landmarks. These pointsareusedtocreatearectangularcroppedimageofthe twoeyes.

3.5 Convolutional neural network: Aconvolutionalneural networkisusedintheproposedpaperfordriverdrowsiness detection.Itwillcontinuouslyexaminethereal-timeimages extracted from video througha webcam.it will predict probabilities to calculate the stateof the driver if the set threshold value found to be less than the prediction accordingtotheEARvaluetheeyes arefoundtobeclosed. thealarmwillbestart

3.5.1 Layers of proposed cnn model

Figure4showstheplannedCNNmodel utilizedin thiswork.

3.4 Calculating Eye aspect Ratio: Theeyeblinkisdetected on the basis of eye aspect ratio.To find theEAR following formulaisused.

EAR=(��2−��6)+(��3−��5)/2(��1−��4)

The EAR isfed intothe convolutional neural network and

CNNconsistsoflayersliketheconvolutionallayer,max pooling, layer, Relu layer and fully connected layer. In the proposedsystem3convolutionallayershasbeenused.The numberofchannelsinthefirstandsecondlayer is32,andin thethirdchannelis64.Alltheseconvolutionallayershavea filtersizeof3*3.Thislayerproducesafeaturemapasaresult ofcalculatingthescalarproductbetweenthekernelandthe localregionofimages.forspeedcalculationcnnusesamax poolinglayerofsize1*1..inthislayerinputimageisdivided into different regions then on each region operations are performed.thebestfeaturehasbeenselected

In this layer which is trailed by dropout with 0.25% whichhasbeenusedtoavoidoverfitting.afterthisaflatten layerisusedtoflattentheoutputafteralltheselayersafully

connectedlayerhasbeenusedinwhichactivationsfromall thelayersoutputarecombinedwithasoftmaxfunctiontoget probability.themodelistrainedunderacceptableaccuracy

3.6 Determining the location of the driver : After the alarmwillstarttoalertthedriver.theproposedsystemwill show the location of the driver with nearby hospitals and hotels. so that the driver can pull over to takea break or stay alert. For providing this function in the system web browsermoduleisusedalongwiththedesiredaddress

4. EXPERIMENTS AND ANALYSIS

Here we have performed two types of experiments .In the firsttypeofexperimentcollecteddatasethasbeenused.To performthefirsttypeofexperimentwehaveusedadataset containing 3843 images label with EAR value. From the dataset,wehaveused2000imagesfortrainingtheproposed modeloutofwhich500imageswereclosedeyesFigure7and 8showstheaccuracyandlossvariationfor50epochs

Andtheremainingwereopeneyes.fortestingtheproposed model we have used 1000 images from the dataset out of which600weredrowsyimagesand400werenon-drowsy images.Forvalidationpurposes,wehaveused843images fromthedatasetoutofwhich500imageswereofopeneyes andtheremaining

IntheSecondtypeofexperiment,livevideoiscapturedwith thehelpofawebcamassociatedwithalaptop.anaudiofileis beingplayedintheformofanalarmtowakeupthedriver when drowsinessisdetected.Thismethodisuser-friendly andonlyneedsawebcamnoneedforanyhardware.Figure 9showstheresultofthesecondtypeofexperiment

5. CONCLUSIONS

This system plays a very important role in preventing accidentscausedbythedrowsinessofthedriver.Themain focusandconcentrationwhiledevelopingthissystemwere to develop a prototype of a driver drowsiness detection systemthatwillaccuratelymonitorthedrowsystateofthe driver. theproposed system was able to detect the facial landmarks of the driver by using a webcam and further processedbyconvolutionalneuralnetworkforclassification. Awarningisgiventothedriverwhenthedriverisdetected tobedrowsy.theproposedsystemachievedanaverageof 95%accuracy.Theproposedsystemcanbeextendedfurther fordifferentuserssuchasbikeridersordifferentdomains likeairlinesorrailways.

6. REFERENCES

1) https://www.nsc.org/road/safetytopic/f%20atigueve r#:~:text=Prevalence%20of%20Drows%%2020Drivi ng%20Crashes,fatalities%20and%20about20%20%2 C000%20injuries

2) Femilia Hardina Caryn*, Laksmita Rahadianti (2021) DriverDrowsinessDetectionBasedonDrivers’Physical Behaviours:ASystematicLiteratureReviewComputer EngineeringandApplicationsVol.10,No.3,

https://doi.org/10.18495/comengapp.v10i3.381

3) https://www.simplilearn.com/tutorials/deeplearning tutorial/convolutionalneuralnetwork#:~:text=MLExp lore%20Program,Introduction%20to%20CNN,netwo rk%20called%20LeNet%20in%201988

4) VenkataRamiReddyChirra,SrinivasuluReddyUyyala2, Venkata Krishna Kishore Kolli (2019). Deep CNN: A Machine Learning Approach for Driver Drowsiness DetectionBasedonEyeState.Internationalinformation andEngineeringtechnologyassociationIIETA https://doi.org/10.18280/ria.330609.

5) Mohammed S Majdi, Sundaresh Ram, Jonathan T Gill, Jeffrey J Rodríguez (2018) Drivenet:convolutional networkfordriverdistractiondetection.IEEESouthwest Symposium on Image Analysis and Interpretation (SSIAI).https://ieeexplore.ieee.org/abstract/document/ 8470309/

6) Jing. Guo , Herleeyandi Markoni (2018).Driver drowsinessdetectionusinghybridconvolutionalneural network and long short-term memory. Journal : Multimedia Tools and Applications https://doi.org/10.1007/s11042-018-6378-6/

7) SonekarShrikantV.,etal."Computationterminationand malicious node detectionusing finitestatemachine in mobile adhoc networks." 2020 7th International Conference on Computing for Sustainable Global Development(INDIACom).IEEE,2020.https://ieeexplore.i eee.org/abstract/document/9083710

8) Sonekar ShrikantV.,etal."Enhancedrouteoptimization techniqueanddesignofthreshold-Tformaliciousnode detectioninadhocnetworks."InternationalJournalof Information Technology 13.3(2021):857-863. https://link.springer.com/article/10.1007/s41870-021-00639-5

9) W. Kim, Hyun-Kyun Choi, Byung-Tae Jang,(2019).A StudyonFeatureExtractionMethodsUsedtoEstimatea Driver’s Level of Drowsiness. 21st International Conference on Advanced Communication Technology (ICACT)

https://doi.org/10.23919/ICACT.2019.8701928

10) Mohsen Babaeian, N. Bhardwaj, Bianca Esquivel, M. Mozumdar (2016). Real time driver drowsiness detection using a logistic-regression-based machine learning algorithm. IEEE Green Energy and Systems Conference(IGSEC)

https://doi.org/10.1109/IGESC.2016.7790075

11) A. Lemkaddem, R. Delgado-Gonzalo, Engin Türetken (2018). Multi-modal driver drowsiness detection: A feasibilitystudy.IEEEEMBSInternationalConferenceon Biomedical&HealthInformatics(BHI)

https://doi.org/10.1109/BHI.2018.8333357

12) Yi-QingWang(2014).AnAnalysisoftheViolaJonesFace Detection Algorithm. Image processing online IPOL.https://doi.org/10.5201/ipol.2014.104