International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume:09Issue:05|May2022 www.irjet.net p-ISSN: 2395-0072
Cross-platform Remote Photoplethysmography (rPPG) based Heart's
Vital Signs Monitoring
Satish Namdev Kadam1 , Prof. S. V. Bodake2
1Dept. of Computer Engineering, TSSM’s Padmabhushan Vasantdada Patil Institute of Technology, Bavdhan Pune, India satishkadam.me@gmail.com
2HOD, Dept. of Computer Engineering, TSSM’s Padmabhushan Vasantdada Patil Institute of Technology, Bavdhan Pune, India email svbpvpit@gmail.com ***
Abstract In the context of the COVID 19 outbreak, inexpensive technologies for assessing heart rate and oxygen saturation are critical for tracking symptoms and assisting in disease control. Methods for predicting heart rate and blood pressure (BP) without the use of sensor equipment have important applications in both the medical and computing fields. Smartphones are the most convenient deviceavailable to everyone today, and their cameras can be used to capture the relevant physiological data. The remote photoplethysmography (rPPG) technology can be used to measure heart rate (HR) and blood pressure (BP) utilizing videos of fingertip and real-time videos of the user taken with a smartphone camera or laptop camera. The PPG signals are collected by recording a videos from the smartphone camera while the users placing their fingers on the camera lens to extract required information. The signals then can be retrieved using minor variations in the video caused by changes in the skin's light reflection characteristics as blood flows through the finger as a result of cardiovascular activity. These color shifts are imperceptible to the naked eye, but digital cameras can detect them. It is feasible to obtain the camera based PPG signal by covering a light source and the camera sensor with a finger. As a result, our system uses the camera and flash of a smartphone as the photosensor and light source, respectively. We run a series of tests to assess the PPG biometric trait's recognition performance, including cross-session scenarios. Statistical, curve widths, frequency domain, and fiducial points based characteristics are also considered. We used principal component analysis (PCA) to minimize the amount of frequency domain features and curve width groups, and then categorized them using the support vector machine algorithm. In order to estimate the health metrics, a cross platform solution i.e. iOS application and Windows desktop application isdeveloped.
Keywords Heart rate measurement, Remote, noncontact, Camera-based, Photoplethysmography (PPG), Image Processing
1. INTRODUCTION
According to the World Health Organization (WHO), heartdisorders,suchasheartattacks,strokes,heartfailure, and heart valve abnormalities, are responsible for more than 30% of all fatalities worldwide. Because heart disorderscanbeasymptomaticandintermittent,especially in the early stages, clinicians have had a difficult time detectingthem[1] Asaresult,foroutpatientuseanddaily activities, a basic cardiac rhythm monitoring technique (that is easily available and does not require extra electrodes/sensors) is required. As smartphones become more common around the world, and smartphone cardiovascular apps are developed and utilized to track users' health, the opportunity to supply high quality smartphone cardiac monitoring technology to the medical communityemerges.
Fig.1describingmethodsforcalculatingbloodpressure (BP).
(A) Contact methods To obtain a finger blood volume pulse, you must touch your finger against the phone camera. Each frame averages the pixels in the video. A waveform as a function of time can be created by further processingandfilteringthesignal.TocomputeBP,features from the waveform are collected and fed into machine learning algorithms. The pulse transit time (PTT) can be linked to blood pressure (BP) [2], although many sensors arenecessary.
(B) Non contact method Ambient light reflected from the face is used in non contact procedures. The video is then processed to improve the signal to noise ratio of the hemoglobin signal, which is then sent into a machine learningalgorithmtodeterminebloodpressure(similarto contact methods). PTT [2] can be computed using only a
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume:09Issue:05|May2022 www.irjet.net p-ISSN: 2395-0072
single camera by measuring the difference between pulse arrivals at distinct body locations because several facial areasorbodypartscanbephotographedatthesametime.
2. RELATED WORK
The physiological foundations of PPG based Heart Rate Monitoring are presented in this part, as well as several measuring methods and related study. Many previous researchhaveshownthatcamera basedapproachescanbe used to remotely monitor cardiovascular activity without contacting the measurement location [3], [4] [6]. In terms of the following aspects, camera based vital sign monitoring systems outperform traditional methods: 1) The measurement is done without the use of any attached or wired sensors, and it is non contact; 2) It does not necessitate the presence of trained personnel; 3) It allows for less intrusive, continuous monitoring for usage in a widerrangeofcircumstances,aswellaseasiercollectionof rich,usefulhealth relateddata
The camera based health monitoring method has been tested in a number of settings, including ICUs, stressful work environments, the home environment, and space exploration. [6]. Ibrahim et. al. [4] developed an HR extractionapproachbasedonblindsourceseparation(BSS) from recorded videos. Photoplethysmography (PPG) is a techniqueformeasuringbloodvolumechangesinresponse to cardiac activityat specific body sites, such as the finger, earlobe,andface[5].
In hospitals, the PPG approach has been used to consistently assess SpO2 and HR. Recent PPG research has shown that with the right extraction techniques, PPG may beusedtomonitorBP[3],RR[4],andHRV[6].Asshownin Fig. 1, there are two types of PPG techniques now in use: transmission mode PPG and reflectance mode PPG. Figure 2 depicts a morphological representation of PPG coupled with the associated electrocardiogram (ECG). In the same way as ECG is a dependable method for monitoring heart activity,PPGhassimilarpropertiesforprospectiveuses.
3. PROPOSED SYSTEM & ALGORITHMS
We compute the mean of the pixel wise luma component from the pixels in each video frame to get the signal from therawvideo(seefig4),sothatifFisavideocomposedby a sequence of frames {f1,...,fm}, then the signal originating fromFis:
Fig.3PPGextractiontoestimationprocessingflow
S={Y(f1),...,Y(fm)},where
InEquation1,iandjiterateoverthepixelsoftheimageand thesuperscripts(r)indicatetheconsideredRGBchannelof theframe,eitherredgreenorblue.Thechannelcoefficients ofEquation1aretakenfromtheITU RBT.601standard.
VideoProperties Values
Framewidth 360
Frameheight 240
Datarate 1517kbps
Totalbitrate 1517kbps Framerate 240frames/sec
Table1: InputVideoProperties
Fig2.ECGandPPGmorphologicalrepresentation
Fig4 Meanofthepixel wiselumacomponent
Signal Preprocessing We employ the following preprocessing methods to eliminate noise after extracting thesignalfromavideo.Toeliminatetrendsfromthesignal,
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume:09Issue:05|May2022 www.irjet.net p-ISSN: 2395-0072
we first compute and subtract the signal's rolling average fromthesignalitself Slighthandorfingermovementscan often result in signal noise that bypasses our filtering system [8]. As a result, we create a set of individual beat quality criteria with the purpose of removing noisy beats from further processing.Forthe rolling average, weutilize a 1 second window size. Then, to reduce high frequency noise,weemployalow passfilterwithacutofffrequencyof 4Hz(240beatsperminute)[4].
Fig5.SignalPreprocessing Rollingaverage
Fiducial Points Detection Following the extraction of individual beats,welook forfiducial pointsinthesignalto extractfeatures.
Fig6 FiducialPointsDetectionPPGsignal
We pay special attention to three points: the systolic peak, the dichotic notch, and the diastolic peak. These pointscanbeeasilydetectedinlow noisesignalsbylooking for maximums and minimums in the signal and its first derivative (see Figure 6 and 7). However, we discovered that this approach must account for noisy signals, so we createda more robustalgorithm to detect them that relies onbestguesses[8].
Fig7.Fducialpointbasedfeatures
Feature Extraction Statistical,curve widths,frequency domain, and fiducial points based characteristics are also considered. To make features independent ofa user's bpm atthetimeofcapture,were samplebeatsafixedsampling rateof1,000Hzandnormalizethemsothattheamplitude valuessitinthesamerangeasthephysiologicalfeatures[0, 1]. Features are calculated on a beat by beat basis, with eachbeatresultinginasample.
To minimize the number of features in the frequency domain and the curve width groups, we apply principal component analysis (PCA) [1]. We fit a PCA with 100 components for the frequency features and keep only the firstncomponents,whichdescribe99percentofthespace variance. We do the same thing with the curve width features,butweuseaPCAmodelwith15components.We discovered that the frequency group requires approximately 5 components (depending on the dataset), whilethebreadthgrouprequiresonly9.Ontheremaining features, we mix two alternative strategies for feature selection. We first compute the pairwise correlation coefficient between features, and then eliminate one feature from the data set at random if the correlation coefficientbetweenthemisrf1,f2>.95fora pairoffeature distributions(f1,f2).
We stratified the data at random and trained two distinct classifiers, a support vector machine (SVM) with a radial basis function kernel and gradient boosted trees (GBT). Before feeding the data into the SVM classifier, we employ a conventional scaler to normalize it, and we only utilizethetrainingparttofitthescaler.
It was confirmed that the rPPG obtained with the smartphonecameraanddisplayallowsforbettercontrolof the emitted light, hence enhancing the acquired signal quality.Inordertobroadenscopeandverifytheresultsof proposedmethodologythecrossplatformsystemiscreated to measure the heart rate and Blood pressure based on windows desktop application and iOS application for mobile user. The activity of building software products or services for several platforms or software environments is known as cross platform development, which we have achievedbybuildingsamesolutionondifferentapplication environmentlikewindows,iOSorAndroid.
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume:09Issue:05|May2022 www.irjet.net p-ISSN: 2395-0072
4. DATA SETS & RESULTS
When employing an aggregate window size > 10, we discovered that single sample EER is above 15%, but quickly drops to less than 5%. We discovered that SVM outperforms the other models, with an EER of 1% at an aggregation window size of 20. Over the course of 4 5 capture sessions per user, we acquire a dataset of PPG readingsfromagroupof6users.Wecreateadataanalysis pipeline and run a series of tests to assess recognition performance using a set of features collected from individualheartbeats.
In terms of heart rate measurement, our equipment performed admirably. In comparison to the present smartphone software, it was pretty accurate. We assumed linear regression in the case of blood pressure. This approximation,ontheotherhand,yieldsapromisingresult. Following table (Table 2) shows the actual heart rate, the heartratemeasuredwithactualdevices,andtheheartrate assessed with the Instant Heart Rate application, all based on10samples.
Subject Actual(device) Measured(app)
a1 a2 a3 a4 m1 m2 m3 m4
1 88 82 80 85 88 86 85 88
2 82 80 78 75 80 80 78 79
3 73 77 74 85 73 78 74 80
4 88 87 85 75 88 87 82 75
5 106 101 100 98 106 102 100 98
6 72 67 74 80 72 67 74 80
7 75 79 80 74 75 79 80 74
8 89 93 91 99 89 93 91 99
9 92 88 87 94 92 88 87 94
10 76 77 79 73 78 77 76 73
Table2: Heartratemeasurementwithcrossplatform applications
5. CONCLUSION
In this paper, we present a method that uses a smartphonecamera and a laptop camera toassess HR and BP using rPPG. In iOS framework, the rPPG signal is captured by recording a video from the camera while the user rests their finger on top of the lens, and in Windows, we used recorded videos to get measurements with a 92 percent accuracy. The signal is extracted based on subtle variationsinthevideocausedbychangesintheskin'slight absorption characteristics as blood flows through the finger.
Future Work: We discovered that a number of parameters had a significant impact on signal fidelity. Initially, we discovered that the warmth of the fingertip altersbloodflow,resultinginslightlyvariedmeasurements. Second, breathing has an impact on the signal: inhaling causes faster heartbeats than exhaling. We will have to consider these parameters to make any platform specific application to leverage the benefits of the methodology Performance drops dramatically in the cross session case samples which will also be considered as part of future work.
REFERENCES
Fig8.MeasurementofVitalSignsonSmartphone
Wediscoveredthatwhenenoughsamplesaregatheredfor adecision,wecanattainidenticalerrorratesaslowas8% Hence overall accuracy considering above methodology is 92%.
[1] Jain M, Deb S, Subramanyam AV. Face video based touchless blood pressure and heart rate estimation. ConferenceProceedings:2016IEEE18thInternational Workshop on Multimedia Signal Processing (MMSP). (2016)1 5.doi:10.1109/MMSP.2016.7813389
[2] Hassan, M.A., Malik, A.S., Fofi, D., Saad, N. and Meriaudeau,F.,2017.Novelhealthmonitoringmethod usinganRGBcamera.Biomedicalopticsexpress,8(11), pp.4838 4854.
International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume:09Issue:05|May2022 www.irjet.net p-ISSN: 2395-0072
[3] Chao, T.F., Lip, G.Y., Liu, C.J., Lin, Y.J., Chang, S.L., Lo, L.W.,Hu,Y.F.,Tuan,T.C.,Liao,J.N.,Chung,F.P.andChen, T.J., 2018. Relationship of aging and incident comorbidities to stroke risk in patients with atrial fibrillation. Journal of the American College of Cardiology,71(2),pp.122 132.
[4] Sharifi, I., Goudarzi, S., & Khodabakhshi, M. B. ”A novel dynamical approach in continuous cuffless blood pressure estimation based on ECG and PPG signals,” ArtificialIntelligenceinMedicine,2018.
[5] Ibrahim,N.,Tomari,R.,Zakaria,W.N.W.,&Othman,N., ”Analysis of Non invasive Video Based Heart Rate Monitoring System obtained from Various Distances and Different Facial Spot,” Journal of Physics: ConferenceSeries,vol.1049,No.1,pp.012003,2018.
[6] Tania Pereira, Nate Tran, Kais Gadhoumi, Michele M. Pelter, Duc H. Do, Randall J. Lee, Rene Colorado, Karl Meisel, and XiaoHu. Photoplethysmography based atrial fibrillation detection: a review. npj Digital Medicine,3,2020.
[7] Jian Liu Yingying Chen Jerry Cheng Jiadi Yu Tian ming Zhao,YanWang.Trueheart:Continuousauthentication on wrist worn wearable susing ppg based biometrics. In Proceedings of IEEE International Conference on Computer Communications (IEEE INFOCOM 2020), 2020.
[8] A.Raposo,H.P.daSilvaandJ.Sanches,"Camera based Photoplethysmography(cbPPG)usingsmartphonerear and frontal cameras: an experimental study," 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021,pp.7091 7094,doi:10.1109/EMBC46164.2021.
9630847