Remote User Authentication using blink mechanism - ‘Iblink’ with Machine Learning

Remote User Authentication using blink mechanism - ‘Iblink’ with Machine Learning

123Final Year Students, Department of CSE, Global Academy of Technology, Bangalore, Karnataka – 560098, India

4 Associate Professor, Department of CSE, Global Academy of Technology, Bangalore,Karnataka–560098,India

Abstract - There are many security requirements for commonterminalauthenticationsystems.Oneamongthemis that they are simple, rapid, and safe. Users must authenticate themselvesusingtraditionalknowledge-basedtechniqueslike passwordswhilebeingconstantlyofferedwithauthentication strategies. However, these techniques are risky since they might be seen by unintended observers who utilize surveillance techniques like shoulder-surfing, which involves watching someone write a password, to acquire user authentication data. PINs are frequently used for user verification in a variety of applications, such as managing money at ATMs, authenticating electronic devices, unlocking mobile devices, and accessing doors. Although when PIN authentication is used, as it is in banking systems and gate access control, perfect identity verification is still challenging to accomplish. The researchers developed a three-layered security architecture that prevents shoulder surfing by requiring users to input their password by blinking their eyes attheappropriatesymbolsintheproperorderHands-freePIN authentication using eye blinks On the other hand, PIN input methods leave no physical traces and offer a more secure alternative to typing passwords. The three stages of this method are facial recognition, eye-blink verification, and a one-time password. By combining all of these layers, the systemissetupwithasecure architecturethatguardsagainst risks like thermal tracking and shoulder surfing. Our framework completely resists attacks like thermal tracking and shoulder surfing because there is no physical password entry.

Key Words: HAAR Cascade Classifier, Local Binary Pattern Histogram, Machine Learning, Shoulder Surfing, Password Authentication, Thermal Tracking.

1. INTRODUCTION

People must constantly authenticate themselves using conventional knowledge-based techniques like passwords, hence one of the security requirements for typical terminal authentication systems is that they be simple,quick,andsecure.Unfortunately,thesetechniques are risky because they could be observed by malicious watchers who utilize surveillance methods like shouldersurfing watching someone type a password on a keyboard to gather user authentication information. Inadequate interactions between systems and users also cause security problems. The researchers came up with a

three-layered security architecture to safeguard PIN numbers as a result. By blinking their eyes at the proper symbols in the proper order to enter the password, users make themselves immune to shoulder surfing. PINs, or personalidentificationnumbers,arefrequentlyemployedin anumberofapplications,suchasATMmoneymanagement, electronic transaction verification, personal device unlocking, and door access. Authentication is always required, even when PIN authentication is used, as in banking systems and gateway management an obstacle. AccordingtoEuropeanATMSecurity,therewere26%more ATM fraud assaults in 2016 than there were in 2015. PIN inputissubjecttopasswordassaultslikeshouldersurfing and thermal surveillance since an authorized user must enter the code in an open or public place. We made the decision to develop a real-time eye blink-based password authentication solution in order to counter risks from thermaltrackingandshouldersurfing.Toprotecttheuser authentication system from shoulder surfing assaults. To fendoffkeyboardthermaltrackingattempts.Toofferthree levels of security for user authentication. This approach makes it simpler and more secure for people who are physically challenged to access their account information. We may utilize this technique for user authentication in ATMs,Gmail,etc.

2. LITERATURE SURVEY

[1] Authors “Vani, A, Gowhar A R, Jeevika G S, SangeethaD,andVallabhMahale”haveimplementedReal Time Eye Based Password Authentication. While determining whether an eye is open or closed, the HAAR Cascade classifier uses a technique called eye flicker identification with the help of the calculation of the histogram of situated gradients (HOG).Advantage of this system is shorter secret words because there is less likelihoodofflickering,whichleadstolongersecretwords and more security. The based on biometrics highlights Validationframeworksdon'tprovideappropriatesecurity; a protectedandactiveframeworkisclearlyneeded.

[2] Authors “Sen, Udit, Vaibhav Bhawlkar, Vinay Yadav, Kavita Namdev, and Satyam Shrivastava“ have implemented I SPEAK–SMART TYPING WITH BLINK. The softwaredetectseyeblinkingandconvertsthemotioninto textusingComputerVisionandHAAR-Cascades.Thispaper willbeabletotranslateeyeblinksintomeaningfultextand

recitethewordsaloudtomakeiteasierforthosewhoare paralysedandthosewhohavespeechand motorfunction problemstocommunicate.Thisisjustasmallprojecttohelp personswithparalysisandothercommunicationchallenges communicatebetterandmoreaffordably.

[3] Authors “Rusanu, O. A., L. Cristea, M. C. Luculescu,P.A.Cotfas,andD.T.Cotfas”haveimplemented Virtual keyboard based on a brain-computer interface In this research, a LabVIEW application was developed that aimstoassistpersons with impairmentsby givingthem a reliable way to type using a virtual keyboard that can be usedbyblinkingtheeyes.TheDivideandConquerparadigm serve as the foundation for its operation. The switch command is connected to the transition (alternative highlighting) between rows, half rows, or keys and is represented by a single eye blink (characters). Two eye blinksaretiedtothechoosecommandandareconnectedto thepreviousdiscussion'salternateselectionoftheobjects. Thetextfieldcanalsobeusedtoenteracharacter.Itlessens evesandshouldersurfingthedevelopmentofacomparable mobilevirtualkeyboardpoweredbyeyeblinkswillbethe maingoaloffutureresearch.

[4]Authors“Attiah,AfraaZ.,andEnasF.Khairullah” have implemented Eye-Blink Detection System for Virtual Keyboard.Onthebasisofthewell-known"68points"ofthe facedetectionmethod,thisisdone. Withthistechnique, a characterisenteredbyblinkingtheeye,muchlikepressing the"Enter"keyonakeyboard.Thesystemwascreatedwith disabled individuals in mind. The outcomes demonstrate highlevelsofusersatisfactionanddemonstratethesystem's value.FurtherimprovementsincludeaddingArabicsupport, makingitavailableasamobileapplication,andaddingvoice assistance, a night mode, shortcuts, and graphics to the virtualkeyboard.

[5] Authors “Chakraborty, Partha, Dipa Roy, Z. R. Zahid, and Saifur Rahman” have implemented Eye gaze controlled virtual keyboard The system employs the HoG descriptorbyadheringtodlibinOpenCVtodetectfaces.The OrientedGradientsHistogram(HOG) ItisbasedonLinear SVMandHoGcharacteristics.Thetechnologyprovedhighly effectiveatdetectingeyesandeyeblinkingthroughglasses, anditevenworksforpersonswhoarewearingeyeglasses. Thesuggestedsysteminthispapergivesthosepeoplewho lack any other physical abilities other than eye movement anewdimensionintheirlives.

3. METHODOLOGY

Thefollowingalgorithmsareused:

 HAARCascadeClassifier

 LocalBinaryPatternHistogram(LBPH)

3.1 HAAR Cascade Face Detection:

TheHAARCascadeobjectidentificationmethod,whichwas introduced in the 2001 publication "Rapid Object Identification using a Boosted Cascade of Basic Characteristics," was based on characteristics created by Paul Viola and Michael Jones. A function that is a cascade classifier is developed by utilising a lot of positive and negativepicturesinthismachine-learningtechnique.Thenit isusedtolocateobjectsinotherimages.

TheAlgorithmconsistsof4phasesthatis:

1. SelectingaHAARfeature

2. ComponentPictureCreation

3. TrainingofAdaboost

4. CascadingClassifiers

It can be trained to recognise virtually any object, even though it is best known for its capacity to recognise faces and body parts in photos. As an example, consider face detection.Themethodfirstneedsalotofpositivepictures with faces and a lot of negative pictures without faces in ordertotraintheclassifier,Afterthisthefeatureextraction shouldbedone.TheHAARFeaturesmustbegatheredinthe first phase. An adjacent rectangular region in a detection windowistakenintoaccountbyaHAARfeature,whichthen calculatesthedifferencebetweenthesummationofthepixel intensitiesineachsector.

SteponeistoobtaintheHAARAttributes.AHAARfeature analysesneighboringrectilinearareasinadetectionwindow at a certain location, and it determines the discrepancy betweenthesesumsandthepixelintensitiesineacharea.To createthis,integralimageswereused.Nonetheless,thevast majority of the factors we calculated are unimportant. Examinetheillustrationbelow.Thetoprowdemonstrates two positive traits. It appears that the first characteristic pickedisthattheareaaroundtheeyesisfrequentlydarker than the area around the forehead and cheekbones. The

second characteristic was chosen because the irises are darker than the nasal bridge. Nevertheless, applying the identicalwindowstothecheeks,oranyotherarea,doesn't matter

•Atruepositiveoccurswhenapositivesampleisaccurately detected.

Alowrateoffalsenegatives,thecascademustperformwell for each phase. If a stage wrongly classifies an object as negativeandyouareunabletofixtheerror,theclassification processisinterrupted.Yet,everystephasasizableriskof falsepositives.Althoughanonobjectismistakenlymarked as affirmative via the detector, the error can still be corrected. With an increase in the total false positive rate andtotalrealpositiveratedecreaseasthenumberofsteps increases.

3.2 Local Binary Pattern Histogram (LBPH):

Let's look at the facial recognition scenario. To train the classifier,alargenumberofpositivephotoswithfacesanda largenumberofnegativeimageswithoutfacesarerequired. Then,wemustinfercharacteristicsfromit.

Cascade Classifier:

Thecascadeclassifierhasnumerouslevels,andeachlevel hasa groupofweak students.Decisionstumps,whichare simple classifiers, are the weak learners. Each stage of training uses the "boosting" method. Using a weighted average of the underachievers' classifications, A highly accurateclassifiercanbetrainedwiththeuseof boosting. The area that the sliding window's present location designates is positively labelled or negatively by the classifier at each stage. Finding anything is indicated by a positive number, whereas finding nothing is shown by a negativenumber.Thedetectorcompletescategorisingthis regionifthelabelisfalse,thenmovesontothenextlocation. Theregionispromotedtothenextlevelbytheclassifierif the label is positive. If the last step qualifies the area as positive,thedetectorindicatesanobjectthatwasidentified atthelocationwithrespecttothepresentwindow.Due to the arrangement of the phases, negative samples are not accepted as soon as practical. The supposition is that the majorityofwindowsdon'tdisplaythedesiredobject.True positives, however, are uncommon and need to be looked into.

• False positives happen when a negative sample is incorrectly categorized as positive, and false negatives happenwhenapositivesampleisincorrectlycategorizedas negative.

Thisalgorithmwasproposedintheyear2006.Itcan bediscoveredonalocalbinaryoperator thattheuserhas access to. Due to its strong discriminative powers and computationalsimplicity,itiscommonlyemployedinfacial recognition.

Todothis,thefollowingproceduresmustbetaken:

1. Creatingadataset

2. Acquiringfaces

3. Extractingfeatures

4. Classifyingthedata.

AnelementofOpenCVistheLBPHalgorithm.

Let'ssayanimagewithNxMdimensionsisconsidered.Itis dividedintomanyregionsof thesamewidth andheight, givingeachzoneadimensionofmxm.

Foreachlocation,alocalbinaryoperatorisemployed.The LBPoperatorisspecifiedina3by3window.

Theintensityofthecenterpixel(Xc,Yc)inthisimageis"Ic,” theintensityoftheneighboringpixelis"In."Thisfunction compares a pixel to its eight nearest neighbors using the medianpixelvalueasthethreshold.

Thefollowingneighborhoodsarethusidentified:

The neighbor value will be set to 1 if it is greater than or equaltothecentralvalue,elseitisputto0.Itissetto0if not.

Figure7:Assigningarbitraryvaluestoneighbors

(Xc,Yc)arethe given pairofcoordinates,the neighboring pairposition(Xp,Yp),whichbelongstoP,andisdetermined asfollows:whereRistheradiusofcircleandPisthecount ofsamplepoints.

Usingthreshold,themxnblock'sdecimalrepresentationis changedtobinary Subsequently,itwasshownthatafixed neighborhood cannot encode information with scale variations. The algorithm was enhanced to incorporate variousradiusandneighborcounts,anditisnowreferredto asLBP.

Ifapoint'slocationonthecircledoesnotmatchtheimage's coordinates, it is often interpolated using bilinear interpolation:

After an histogram of each region is extracted, all these histograms are concatenated to create a new and bigger histogramwhichrepresentstheimagewhichisknownasa feature vector of image. The monotonic grey scale modificationscannotharmtheLBPoperator.

Thegoalistoarrangeanynumberofneighborsinastraight lineonacircleofcustomizableradius.

Following the production of LBP values, the region's histogramisconstructedbytallyingthenumberofnearby LBPvaluesthataresimilar.

Then, the image with histogram that is closest to the histogramisreturnedaftercomparingthehistogramsofthe testingimageandthedatabase'simages.(Severalmethods canbeusedforthis.Tonameafew,Euclideandistance,chisquareetc.)

Bycontrastingthetestingimage'sfeatureswiththosefrom the dataset, the Euclidean distance is found out. The matchingrateisfoundbythesmallestdistancebetweenthe originalandimages.

interacts with the front end of the project first. The user registersonthepageinthefirststepiftheydonotalready have an account; otherwise, they sign in to their account. Afterloggingin,theuser'sfaceisdetectedusingtheLBPH approach.TheOTPisverifiedwhenthefacehasbeenfound. Inthis,arandomone-timepasswordmadeupofnumbersis generatedbythesystemandsenttotheregisteredmobile number.Theuserissuccessfullysignedintothepageonce thisOTPhasbeensuccessfullyentered.

AnIDoftheimagefromthedatabaseisgivenasoutputonly ifrecognitionofthetestimagetakesplace.

LBPHmodulecanrecognizethefrontpartandsidepartof faces, and it is unaffected with changes in light set up, makingitmostrobust.

4. DESIGN AND DEVELOPMENT PROCESS

Themodel'sstructure,behavior,andprocessaredescribed in the sequence diagram. With our project, a user first engageswiththeproject'sfrontend.Theuserchecksinto hisaccountinthefirststepifanaccounthasalreadybeen createdandregistersonthepageotherwise.Afterloggingin, the user's face is detected using the local Binary Pattern Histogram (LBPH) approach. It will keep the user's credentialsinthedatabaseaftertheuser'sfaceisrecognized and it captures the frames to extract eye moments, which aidsinsubsequentlogins.Verificationiscompleted,and a conclusionisdrawn.

Level0:

Figure 10:SystemArchitecture

Asystem'sconceptualmodel,alsoreferredtoasitssystem architecture, determines the system's structure, behavior, and other attributes. With respect to our project, a user

Theproject'soverallprocessisdescribedinLevel0.Users' eye blinksarebeingusedasinput.Here,theprotection of useraccountinformationfromshouldersurfingassaultsis done.Forthisprocess,thesystemwilluseOpenCV

Level1:

Thenproject’sfirststepisdescribedinLevel1.Thesystem willreadtheuser'scredentialsasinputandinturn,thiswill generateapassword.

Level2:

Theproject'sfinal stepisdescribedatLevel:2.Users'eye blinks and a randomly generated password are used as input.Thesystemwillwatchtheuser'seyeblinkandverify theiridentity.

4. UseCaseDiagram:

According to the use-case diagram, after the user launchestheprogramme,thesystemwilltakeapictureof the user's eyes and preprocess it. The computer will recognizetheuser'sface,keeptrackofeyeblinks,andthen generate a pin by blinking a set of characters. The user is authenticatedandcancheckwhethertheyareauthorizedto loginatthefinalstageofverification.

5. RESULTS AND DISCUSSION

Thethreestagesoftheauthenticationlayer,suchasfacial recognition,eyeblinkdetection,andOTPgeneration,Atthe end results should be more precise and efficient, after consulting the few papers that are stated in the literature review.

6. CONCLUSION

Using a camera-based eye-blinking mechanism and the aforementioned methodologies, a special application for eyelidblink-basedPINdetectionhasbeendeveloped.Aninedigitkeypadhasbeenusedtosuccessfullytestthesystem anditmaybemadetoacceptpasswordcombinationsthat combine characters and numbers. The user's eye blink stabilityhasanimpactontheaccuracyofthepinsidentified, thereforethismustbeconsidered.

Nowadays,real-timeeye-blinkcomputationsandrecording arecompletedbeforePINidentification.

ACKNOWLEDGEMENT

We would like to thank our HOD Dr.Kumaraswamy S,andGuideDr.KanagavalliR,Dept.of Computer Science and Engineering, Global Academy of technology,Bangalorefortheirguidanceandsupport.

REFERENCES

[1] DVani, A, Gowhar A R, Jeevika G S, Sangeetha D, and VallabhMahale.2022.“RealTimeEyeBasedPassword Authentication”. Perspectives in Communication, Embedded-Systems and Signal-Processing - PiCES,June, 20-22. http://picesjournal.com/ojs/index.php/pices/article/view/356

[2] Sen, Udit, Vaibhav Bhawlkar, Vinay Yadav, Kavita Namdev, and Satyam Shrivastava. "I SPEAK–SMART TYPINGWITHBLINK."

[3] Attiah, Afraa Z., and Enas F. Khairullah. "Eye-Blink Detection System for Virtual Keyboard." In 2021 NationalComputingCollegesConference(NCCC),pp.1-6. IEEE,2021.

[4] Rusanu,O.A.,L.Cristea,M.C.Luculescu,P.A.Cotfas,and D. T. Cotfas. "Virtual keyboard based on a braincomputerinterface."In IOPConference Series:Materials Science and Engineering,vol.514,no.1,p.012020.IOP Publishing,2019.

[5] Chakraborty, Partha, Dipa Roy, Z. R. Zahid, and Saifur Rahman."Eyegazecontrolledvirtualkeyboard." Int. J. Rec. Technol. Eng.(IJRTE) 8,no.4(2019):3264-3269.