Heart Health & Drowsiness Analysis of Driver for RoadSafety
Prof. Soumya S1 , Mr. Thanaya B2 , Mr. Mehfooz Ahmed3 , Mr. Vithesh4 , Mr. Swasthik5
1Asst. Professor, Dept. of CSE, YIT Moodbidri, Mangalore, Karnataka, India
2B.E Student, Dept. of CSE, YIT Moodbidri, Mangalore, Karnataka, India
3B.E Student, Dept. of CSE, YIT Moodbidri, Mangalore, Karnataka, India
4B.E Student, Dept. of CSE, YIT Moodbidri, Mangalore, Karnataka, India
5B.E Student, Dept. of CSE, YIT Moodbidri, Mangalore, Karnataka, India ***
Abstract - The "HEART HEALTH & DROWSINESS
THE " ANALYSIS FOR ROAD SAFETY" project was created exclusively to ensure the safety of those who operate motor vehicles, rest in passenger seats, ride bicycles, stroll along footpaths next to roads, and engage in various other activities. According to a survey, about 1.3 million people die yearly in traffic accidents because of road conditions, driver error, car component failure, and driver error. It is said that a vehicle accident occurs every five seconds. Another study by Stanford Law School found that 90% of car crashes are the result of human error. Fatigued drivers, who drive despite being warned not to, or who have other health problems can make mistakes. 40% of all traffic mishaps are the result of sleep deprivation. By preventing drivers who are sleep deprived or have heart issues from operating a vehicle, our device seeks to decrease the enormous number of accidents. It will guarantee fewer accidents each year due to heart disease and lack of sleep, boosting the confidence of people who cross in front of moving vehicles.
Key Words: roadsafety, drowsiness, heartrate analysis,roadaccidents
1.
INTRODUCTION
It is clear from reading studies conducted by internationalandnationalorganizationsliketheMinistry of Road Transport & Highways and Motor Vehicle Departments that accidents caused by health issues are becoming more frequent every day. As the years go by, therearemorecarsontheroad.Asthisisoccurring,the number of accidents on the road brought on by human error is rising proportionally. When a fatal accident occurs due to a basic factor like lack of sleep, it is frightening for anyone driving a motorcycle or walking nexttotheroad.Theconceptforthisstraightforwardyet effective project was born out of the need to overcome this fear of the driver and other road users. If properly implementedglobally,thisstraightforward devicecould assistinreducingthe40%to0%atsomepointinthe
future.Nationalandstatehighwaysmakeup5.04%ofall roadsinIndiaandcollectivelyaccountforalmost54%of accidents and 60% of fatalities, with the remaining 94% of Indian roadways accounting for 45% of accidents and 39%offatalities.Eachyear,businessaccidentsclaimthe lives of around 1.3 million individuals. A new 20 to 50 million people each year suffer lethal injuries, and many of them go on to have disabilities. Road traffic accidents result in huge economic losses for individuals, their families,andentirenations. Theselosseshaveanimpact on the cost of medical care, missed stipends for people who pass away or become disabled as a result of their injuries,andcaregivingcostsforrelativeswhomusttake timeofffromworkorschooltocarefortheinjured.Most countries lose three percent of their GNP due to transportationaccidents.
1.1 Objective Of Research
We plan to create a straightforward yet effective embedded system that can capture the driver's face, analyze the data from it, and determine whether the driver is sleep deprived. Additionally, this device will have aPPGsensortomonitorheartrateanddetermineifthere are any significant changes. If so, the driver will be promptedtorestortakesafetyprecautions.
2. LITERATURE REVIEW
[1] Design of Drowsiness, Heart Beat Detection System, and Alertness Indicator for Driver Safety:
Accident rates are rising and there is no safety for the driver or passengers as a result of driver error. Hence, accident rates can be reduced by continuously monitoring the driver, identifying tiredness and heart rate variations, and also warning him when he deviates from his regular situation. Motion detection forpredicting drowsiness and theR-peakdetectionalgorithmforcounting heartbeats are twoeffectivetechniques for performing this. For the sake of the
simulation, an example is used to demonstrate the driver'scondition.The insights collectedcanbeused to create an intelligent vehicle that recognizes the driver’s weariness,boostingthedriver'ssafetyandsecurity,and loweringaccidentrates.
[2] A Real-Time Heart-Rate Monitor Using Noncontact Electrocardiogram for Automotive Driver:
Contact Electrocardiogram for Automotive Driver: The designanddevelopmentofaquickandaccuratereal-time heart rate monitoring device for automobile drivers the created system used the non-contact ECG principle on a steering wheel. To provide real-time HR monitoring, the systemusedastraightforwardanalogsignalconditioning circuitandadigitalcomputingunit.
[3] A Video-based Heart Rate Monitoring System for Drivers Using Photoplethysmography Signal:
Real-Time Driver’s Drowsiness Monitoring Based on Dynamically Varying Threshold: Our face detection and recognitionalgorithmsuseeffective,highlyaccurate,and low false positive rate techniques. We have developed a conceptwherethesystemmonitorsthedrivingschedule to dynamically adjust the threshold after three hours of driving because the driver is more likely to fall asleep lateron.Byincorporatingfacerecognition,wewereable tosuccessfullyimplementthisideaandmakethesystem work for numerous drivers. The number of traffic accidents can be decreased by using our suggested approach to inform the driver and check the driver's condition.
[4] Real-Time Driver’s Drowsiness Monitoring Based on Dynamically Varying Threshold:
Our face detection and recognition algorithms use effective, highly accurate, and low false positive rate techniques. We have developed a concept where the system monitors the driving schedule to dynamically adjustthethresholdafterthreehoursofdrivingbecause the driver is more likely to fall asleep later on. By incorporating face recognition, we were able to successfully implement this idea and make the system work for numerous drivers. The number of traffic accidents can be decreased by using our suggested approach to inform the driver and check the driver's condition.
[5] IoT-based System for Heart Rate Monitoring:
IoT-based system formonitoringand controllinghuman cardiac rate is created. For data collecting, this system makesadvantageofthecapabilitiesofaheartratesensor.
The microcontroller processes data signals representing a person's heartbeat. For additional analytics and visualization, the processed data are sent to the IoTplatform. The system’s ability to sense and read theuser's heartbeat rate and transfer that data over Bluetooth to the Android mobile app (Blynk) led to thediscovery that the experimental results were accurate. The results showed that the heartbeat rate was low if it was between40 and 60, medium if it was between 60 and 100,andhighifitwas between100and150.
3. PROPOSED WORK
Drowsiness Check Module:
The system initializes essential libraries for image capturing and frame extraction and checks if the camera is properly initialized. If the camera is initialized,thesystembeginscapturingandextracting frames of the driver's face. If not, the process is aborted and an exception is logged. The driver's eye aspect ratio (EAR) is calculated and compared to a predefined threshold. If the EAR is less than the threshold,itmeansthedriver'seyesareopen,andthe system continues to operate normally. But if the EAR is greater than the threshold, it means the driver's eyes are closed, and an alarm systemis triggered to alertthedriver.Thissystemisintendedtohelpprevent accidents caused by drowsy driving. The EAR calculation involves measuring the ratio of the distance between the top and bottom eyelids to the distancebetweentheleftandrightcornersoftheeye. The system continuously captures and processes frames,updating the EAR calculation with each new frame. The alarm system can be configured to use different types ofalerts, such as a sound or a visual alert. By monitoring thedriver's eyes, the system can detect when the driver is falling asleep or distracted, allowingforpromptinterventiontopreventaccidents. The use of a camera tocapture images and frames, combined with the EAR calculation, allows for the accurate detection of driver drowsiness. Overall, this system provides a reliable and effective method for monitoring driver behavior and promoting safe driving habits. It is a useful application ofcomputer vision and machine learning techniques to real-world problems.WeareusingawebcamwithRaspberryPi4 for running the software and getting the alert when thedrivergetsdrowsy.Forthepresentprototype,The Pi is connected to a webcam instead of a camera module since it is a developmental prototype. If the systemistobeimplementedinCar,it’llbeconnectedto a camera module integrated into the Pi. Below in Figure1,Youcanseethetwodevicescoupledtogeta favorableresult.
Technical Working Principle:
The module imports several libraries, including Idlib, imutils,andOpenCV(cv2).Theselibrariesareusedforface identification, image processing, and other computer vision-relatedactivities. The facial landmark predictor file location, the minimum EAR (eye aspect ratio) value for openeyes,andthemaximumnumberofconsecutiveframes inwhichEARcanremainlessthantheminimumEARareall setafterthat.Themoduletheninitializesthefacedetector and landmark finder/predictor based on an adlib’s HOG (Histogram of Oriented Gradients). Also, it sets up the webcam stream so that photographs can be taken in realtime. Next, it defines a function called "beep" that, when called,playsanaudiofilecalled"alarm.wav."Additionally, it defines a different function called "eye aspect ratio," which accepts an eye as input and determines its EAR. LowervaluesoftheEARindicateclosedorpartiallyclosed eyes, whereas higher values indicate how much the eye is open. It then starts reading frames from the webcam feed continuallywhileinitializingacountervariablecalledEYE CLOSEDCOUNTER.Theimageisscaleddownto800pixels inwidthandmadegrayscaleforeachframebythecode.The face detector is then used to find faces in the image. The module locates the facial landmarks for each recognized face using the landmark finder and transforms them into NumPy arrays. The left and right eyes are then separated from the facial landmarks, and each eye's EAR value is calculated. The computed EAR value is then displayed on theimage,andoutlinesaredrawnaroundtheleftandright eyes. The function increases theEYECLOSED COUNTER if the estimated EAR value is less than the minimum EAR value.ThefunctionresetstheEYECLOSEDCOUNTERto0 if the EAR value is greater than or equal to the minimum EAR value.Thecodeshowstheword"Drowsiness" onthe image and activates the beep function to play an alarm soundiftheEYECLOSEDCOUNTERgoesovertheallotted frame count. Until an exception happens, this process continues. Here the frame is captured when the eye is openedandwhenitdetectsdrowsyeyesaswell.
Heart Rate Analysis:
To effectively monitor the driver's heart rate, several libraries are initialized to aid in the computation of the heartrate.Thesystemthencheckswhethertheheartrate sensor is properly initialized, and if not, an exception is raisedandloggedfordebuggingpurposes.Ifthesensoris properly initialized, the heart rate is collected from the optical heart rate sensor. The data is then processed to remove any noise that may have been captured during collection. The cleaned heart rate data is compared to a predefined threshold value that was set to identify a potentially dangerous situation. If the driver's heart rate fallsbelowthethreshold,thesystemcontinuestooperate normally, andno action is taken.However,if the heart rate exceedsthethreshold,thesystemsendssignalstotheESP, which begins controlling the throttle of the vehicle and aligning it to the left side of the road. The vehicle is eventually brought to a stop, and an SMS alert is sent to theappropriatesetofpeople.Overall,thesystemensures thatthedriver'sheartrateiscontinuallymonitored,and
if any anomalies are detected, appropriate actions are takentopreventpotentialaccidents.
4. HARDWARE DESCRIPTION
A. Requirement:
1. PPGHeartRateSensor-ForAnalyzingheartbeats
2. RaspberryPi4-Forprocessingthecode
3. Webcam–Fordrowsinesscheck
4. Buzzer/Speaker–Foralertingthedriver
5. ConnectingWires–Forconnections
B. Raspberry Pi 4: TheRaspberryPi4isasingle-board computer that is small, cheap, and powerful, with a variety of networking possibilities. It is perfect for a variety of crafts and uses, including DIY projects for hobbyists and commercial products. We use RaspberryPiasa controllingandprocessingunitfor ourmodule.
Technical Working Principle:
Themoduleimportsseveralnecessarylibraries,suchaswin sound,time,statistics,andrandom.Themoduleinitializes listsforstoringhighandlowreadingsaswellasvariables forfrequency,duration,andthresholdvalues.Thenitgoes intoanendlesslooptokeeptrackoftheheartrate.First,a timerfor10secondsofsamplingtimeandanemptylistof heartratesareinitializedwithinthe loop. Then,tosimulate arealheartratereading,itgeneratesarandomheartrate number between 0 and 160. The list of heart rates now includesthisvalue.Upuntilthe10-secondsamplingperiod has passed, the loop keeps collecting samples every 1 second. The statistics module is then usedto determine the averageheartrate.Thescriptreportsthattheheartratehas exceeded the threshold and adds the most recent heart rate measurementstothelistofthehighiftheaverageheartrate is higher than the threshold high value. When this conditionissatisfiedtwiceinarow,thewinsoundmodule emitsaprolonged,high-frequencybeep.Similarly,tothis, iftheaverageheartratefallsbelowthethresholdlowvalue, the module reports this and adds the most recent heart ratedatatothelistoflows.Whenthisconditionissatisfied twice in a row, the win sound module emits a prolonged, low-frequency beep. The scriptreports that the average heartrateiswithintherangeifitfallswithinthethreshold range. When the heart rate has returned to the normal range, the warning count variable is reset to zero. The module then waits one second before collecting the subsequent10-secondsample.
C. PPG Heart Rate Sensor: PPG heart rate sensors are frequently found in wearable gadgets like smartwatches and fitness trackers. They work by shininga light on theskinand measuringchangesin the absorption or reflection of light caused by blood flow.Weusedthissensorfortheeffectivedetectionof heartrate.
D. Webcam: A webcam is a digital camera that records video and transmits it to a computer or over the internet. We use a webcam for drowsiness detection which will capture the whole face and send it for processing.
E. Algorithm: Algorithm uses a combination of computer vision libraries including OpenCV, dlib, imutils, and scipy to detect facial landmarks and estimateeyeaspectratiofordrowsinessdetection.
F. Programming: we use Python as a programming languagetocommunicatewiththehardwaredevices, read sensor data, work with the General-Purpose Input/Output (GPIO) pins on the Raspberry Pi, and process the data which is collected from the PPG sensorandWebcamandcontroltheoutput.
5. PERFORMANCE METRIC
Drowsiness Detection Algorithm (dlib vs haar cascade)
OpenCV and Dlib provide face detection algorithms commonlyusedincomputervisionapplications.Whileboth algorithms are effective at detecting faces in images and videos,therearesomedifferencesintheirperformanceand capabilities that make Dlib a superior algorithm in many cases.
Another advantage of Dlib is its ability to detect facial landmarks, such as the eyes, nose, and mouth, with high precision. This is useful for applications such as facialrecognition and emotion detection, which require precise location of facial features. Dlib also provides a face recognition module that can be used to recognize facesfromadatabaseofknownindividuals.Thismodule uses a deep learning-based algorithm to extract facial features and match them to a database of known faces. ThisfeatureisnotavailableinOpenCV.
Finally, Dlib is an open-source library with an active developer community, which means that it is constantly being updated and improved. It is also available in several programming languages, including Python,C++,andJava.
Overall, while OpenCV is a solid face detection algorithm, Dlib's combination of accuracy, robustness, and additional featuresmakeitasuperior choiceformanycomputervision applications, particularly those that require high precision and performance.
Time Consideration of dlib and haar cascade cv2
Consideringloadingmodel
dlib:2.208890199661255
HaarCascade(cv2):2.6401994228363037
Withoutmodelloadtime
dlib:0.7304553985595703
HaarCascade:2.6089558601379395
The shape_predictor_68_face_landmarks.dat model has been evaluated in various studies, with reported average errors ranging from 2.45 to 4.4 pixels depending on the specific evaluationmetricanddatasetused.Inaddition,themodelhas been shown to achieve high precision and recall rates on the facial landmark detection task, indicating a high level of accuracy and consistency. Overall, the shape_predictor_68_face_landmarks.dat model is widely regarded as one of the most accurate and reliable facial landmarkdetectionmodelsavailable.
Ppg Sensors
WhilePPGsensorsmaynotbeasaccurateasECGsensors,they arestillavaluabletoolformonitoringheartrateinreal-world settings. PPG sensors offer several advantages over ECG sensors, including their non-invasive nature and their ability to beintegratedintowearabledevicessuchassmartwatchesand fitnesstrackers.
PPGsensorsarealsolessexpensiveandeasiertousethanECG sensors,makingthemaccessibletoawiderrangeofusers.PPG sensorscanprovidecontinuousheartratemonitoringoveran extended period of time, which can be useful for tracking changes in heart rate during physical activity or throughout the day.
Several studies have compared the performance of different facial landmark detection models, and the shape_predictor_68_face_landmarks.dat model has been showntobeamongthemostaccuratemodels.Forexample,in a comparative study by Zhang et al. (2016), the shape_predictor_68_face_landmarks.dat model achieved an average error of 3.68 pixels on the facial landmark detection task, which was found to be among the best-performing modelsinthestudy.
Additionally, PPG sensors have been shown to be effective in detecting certain health conditions such as sleep apnea and hypertension. One study found that a wrist-worn PPGsensorwas able to detect sleep apnea with an accuracy of 92%, compared to 87% for a traditionalsleepstudy.
6. ADVANTAGES
Wehavemainly2modulesinourprojectoneisHeartRate AnalysisandDrowsinessDetection.Bothmodulestakea vital part in road safety and precautions for road accidents.Itischallengingtopinpointtheprecisenumber ofaccidentscausedbysleepydrivingbecausethereareso many contributing variables, and reporting requirements differ between nations and regions. The National HighwayTraffic Safety Administration (NHTSA) intheUnitedStatesestimatesthatdrowsydrivingcauses 100,000 collisions, 71,000 injuries, and 1,550 deaths annually. The four-wheeler vehicle's drowsiness detection module can prevent thousands of collisions andsavethelivesofthedriverandpassengersbytaking preventative measures like alerting the driver with an alarm system or vibrator on the steering whenever a sleepyfaceisdetected.Anotherimportantmoduleinour system is precautionary actions based on the driver’s heart rate analysis. Heart-related problems can pose a serious risk to drivers and passengers, particularly duringlong-distanceorhigh-stressdrivingsituations.In some cases, a heart attack or other cardiac event can causethedrivertolosecontrolofthevehicle,leadingto a potentially fatal accident. However, if heart problems canbedetectedearlywhiledriving,stepscanbetakento reducetheriskofanaccidentandsavelives.WeusePPG heart rate sensors on the steering and seat belt of the driver.Thiswilleffectivelycollectthedriver’sheartrate and controls the ADAS if there is any major changes. ADASwiththehelpofAIappliesthebreaksmoothlyand stops the vehicle. Not only does it trigger the ADAS for safe deceleration, but it also alerts the emergency contactsviaSOS.
7. DISADVANTAGES
Since we have two modules in our system drowsiness detectionandheartrateanalysis.Bothhavetheirownset of advantages and disadvantages. When it comes to the Drowsiness detection module there can be many cases where the accuracy of our module would be less such as Falsealarms,whichcanannoyanddistractthedriver,are one of the major problems. Inaccurate drowsiness detection by the system could result in false alerts, giving the driver a false sense of security. Another drawback of drowsiness detection systems is that they might not be effective for all drivers due to the broad range of physiological conditions and individual sleep habits.
Installation costs. To ensure optimum performance, the system might also need routine calibration and maintenance,whichcanbeexpensiveandtime-consuming. Last but not least, a few drivers might find the system intrusive, leading to worries about data protection and sharing of personal information. And in the Heart Rate analysis module there can be limited accuracy of PPG sensors is one of their major drawbacks. The accuracy of theheartratereadingmaybeimpactedbytheirsensitivity
to skin pigmentation, ambient light, and motion artifacts. Furthermore, PPG sensors only have a small detection range, which may make them less effectivefor detecting pulse rates during vigorous exercise or otherphysical activity.
8. RESULT AND CONCLUSION
Heart-related problems can pose a serious risk to driversand passengers, which can cause the driver to lose control of the vehicle, leading to a potentially fatal accident. However, if heart problems can be detected earlywhiledriving,stepscanbetakentoreducetheriskof an accident and save lives. Sensors and monitoring systems can be used to identify heart issues while driving. The driver's heart rate might be continuously monitored, for instance,by integrating a PPG heart rate sensor into the driver's seat or steering wheel. An automated mechanism may be activated to slow down the car and bring it to a stop safely if the sensor picks up aberrant heart activity. After correctly identifying the signs of heart difficulties, the system might send an SOS to the relevant group of people, suchasemergency services or a chosen contact, inaddition to automating breaking. This could ensure that the motorist receives immediate medical care, possibly saving their life. Especially on long drives or at night, driver fatigue is a frequent contributor to car accidents. Driving while fatigued can result in decreased alertness, slowed reaction times, and poor decision-making abilities, endangering both the driver and other road users. Eye movement detection devices are used to stop accidents brought on by drowsy driving. To measure the driver's level of attention and detect eye movements, we use web cameras and sensors. An alarm can be delivered to the driver if the system notices symptoms of drowsiness, suchas extended periodsofeyeclosureoralackofeyemovement.
9.
[2] Anil Kumar CV, Mansoor Ahmad IEEE International conference on recent trends, Design of Drowsiness, Heart Beat Detection System and AlertnessIndicatorforDriver
Safety.
[3] po-Wei Huang, graduate student member, IEEE, Bing- Jhang Wu, graduate member, IEEE Journal of Biomedical and health informatics, a heart rate monitoringframeworkfor Real-world drivers using remotephotoplethysmography.
[4] Greentheonly, Youtuber, Face Tracking during Autopilot.
[5] The Hindu Times, AccidentSurvey. https://www.thehindu.com/data/data-in-2021
[6] HindustanTimes,Accidentduetoheartattack, https://www.hindustantimes.com/man-suffers-heartattack-dies-in-car-accident