AUTOMATED PROCTORING SYSTEM
1Jay Mayekar, Dept of Information Technology Engineering, Atharva College of Engineering
2Shubham Pal, Dept of Information Technology Engineering, Atharva College of Engineering
3Aditya Pandey, Dept of Information Technology Engineering, Atharva College of Engineering
4Bikram Pani, Dept of Information Technology Engineering, Atharva College of Engineering
5Prof Preeti Mishra, Dept of Information Technology Engineering, Atharva College of Engineering Maharashtra, India ***
Abstract - Remote learning has grown over the years majorly due to the pandemic. Learninghastransferredtoapps like Google Meet, Microsoft Teams, Zoom, and others. The practical knowledge gained by students deteriorated due to online learning. Students started attending lectures only for the sake of attending them. The growth factor of students declined which should have affected their grades overall however, the results were a total surprise. The majority of the students outperformed their average score. This is a result of the fact that institutions lacked a proper way to perform an organized evaluation. To tackle this problem many schools and colleges came up with their own reasonable approaches. Some universities implemented remote proctoring which involved a manual proctor keeping watch on all student activities. In spite of this students still manage to cheat and score more marks through unfair means. This just isn’t enough to tackle such an important problem. A system that can assist in analyzing unfair tactics used by students was much needed.
In this paper, we have developed an automated proctoring website utilizing various technologies like computer vision, object detection, etc. It includes various measures to prevent the use of unfair means that students may use during their examinations. It comprises features like eye gaze tracking, mouth open detection, object detection and identification, head position detection, and face detection. Here we present techniques and tools through which the proctor need not be present throughout the exam. This is based on neural networks and machine learning All of this combinedcreates a smart system that can detect any malpractice that may occur during their tests. This system hopes to make the world of examination, supervision, and proctoring function smoother and more effectively. This research will reveal waystoprevent cheating in online assessments using technologies like computer vision to provide proctoring and monitor multiple students at a time.
Key Words: Remote Learning, Automated proctoring system, Computer vision, Object Detection, Open Mouth detection, Online Tests, Online Exam Proctoring, Examinations.
1. PROBLEM STATEMENT
Theaccessibilityofonlinetestsprovidesvariousadvantages however,italsointroducesanewsetofchallengesforthe actualassessmentofthetests.Manyfactorscomeintoplay when conducting an examination online. A classroom test can be conducted smoothly with the help of a physical supervisorasitallowssimultaneousmonitoringofstudents inarestrictedenvironment.Onlinetestsontheotherhand make it difficult as students do not share any physical environmentwiththeproctor.
Thus,wegottheideaofcreatinganAIsystemthatmonitors thestudentduringtheirtests.Thissystemshouldalsoholda log of the malpractices to keep a track of a student’s behavior.Theselogscanbemanuallyverifiedlatertoinspect anystudent’scasefurther.
2. INTRODUCTION
Thepandemicledtotheshiftingofacademicstoanonline modewhichledtothereinventionofsystemsaidingonline education. This poses a major challenge not only from a learning point of view but also from the perspective of examinations. Conducting examinations without any wrongdoingisabigchallengeforinstitutions.Thenumberof internet users in our country has doubled over the past 6 years.Thishasbeenaboonformanyinstitutions,students, andotherlearningplatforms.Thisfacilitatedinstitutionsto conductexaminationsonline,bringingtheconceptofonline proctoringtotheacademiclevel.
The main goal of a proctoring system is to allow the invigilatorstoproctorremotely.Manyinstitutionsadopted thissetofmanualonlineproctoringbutthisledtoalmostno improvementintheprocessofstoppingmalpractices.
A good online proctoring system must include all the featuresthatmaybeneededbyaproctorduringtheoffline modeofexaminations.Itshouldbeabletodetectmovement, sound,andeyemovementandgivereasonablewarningsto thestudentsincaseofanymisconduct
3. LITERATURE REVIEW
In[1]publishedbyAsepHadianS.GandYoanesBandung, followedauniqueapproach.Averylargedatasetofimages isusedtotrainandidentifytheuserinlowlightandgeneral scenarios. It helped in understanding the dynamics while analyzinganimage.Itwasnotdesignedforfirst-timesetup foranonlineexamination.
In[2]publishedbySanjanaYadavandArchanaSingh,which usedcomputervisionforinformationextractionforobject detection.Theimageischeckedwithamatchingalgorithm using the methods such as re-scaling, filtration, and binarization.Chamferdistancetransformation.
In [3] published by A.T. Awaghade, D.A. Bombe, T. R. Deshmukh, and K. D. Takwane proposed that all the contributions measure and gauge the assortment of occasions,practices,andexamplesordinarilyconnectedwith cheating.
In[4],thefocusbyAimanKiunisonfrauddetectioninvideo recordings of examinations using Convolutional Neural Networks(CNN),wherebyimageclassificationmodelswere built using Rectified activation units (RAU), which in turn displayedfantasticresultsfor big size data sets. Interface, videoprocessing,andframecategorizationwereallpartof theirsystem.Theinterfacefeedsthefootageofthestudents taking the test into a pipeline consisting of several algorithms.Theenormousrecordingwouldbereducedtoa smallnumberofminimalisticframes,andseveralduplicate orsimilar-lookingframeswouldberemoved.Theframesare thensentintoapipeline,wheretheyareusedtotrainCNNs torecognizeobjectsinthesecondpartofthepipeline.
In [5], The work given by N.L Clarke and P. Dowland proposesarealisticstrategytopermitremoteandelectronic proctoring during student examinations. The technique entails using transparent recognition to provide nondisruptive and permanent identification of the student’s identityduringthetest-takingprocess.Amodelisbuilt,and an evaluation of the technology of the generated platform demonstratesthemethod’seffectiveness.
In[6],Afull-fledgedOnlineProctoringSystemwasmadeto conduct assessments. The type of technology, approach, problematicreasoning,andaccuracywerethekeypointsfor ourlearning. Thisworkputourprojecttoatesttoperform better and work more smoothly. The data representation with the various models provided valuable references to understandallthethoughtprocessesforthefinalbundlingof ourmodules.
4. PROPOSED SYSTEM
Theproposedsystemwillbethebestmodeltomimicallthe elementsconsideredduringanofflineclassroomassessment.
Factors like movement, eye movement, whispering, and using other devices which are the prime symptoms of somebody possibly cheating come into the picture, so we proposedevelopingacomprehensivesystemwithnumerous detectionand validation mechanisms capable ofdetecting anymalpractices.
Studentsfirstwillbeaskedtoregisteronaportalforthefirst timewheretheywillentertheirpersonaldetails,idcard,and a picture will be taken. This picture will be saved in the databaseanditwillbelaterusedtoverifythembeforethe exameliminatinganychancesofimpersonation.
4.1. GAZE TRACKING
Theexaminee’sgazeshallbetrackedthroughouttheexam. Gazetrackinghelpsusdeducewheretheexamineeistrying tolookiftheexaminee’sgazeseemstobemovingconstantly orseems to be steady forfixed durationsover sometime. The probability that the examinee is reading the answer fromsomewhereincreases.
4.2. HEAD POSE ESTIMATION
Thisisproposedtofindwheretheuserislooking.Thiscan beverybeneficialtodetectiftheuseristryingtocheatby looking at some additional display or devices. The DNN Model of OpenCV comes in clutch here with very high accuracy.
4.3. MOUTH OPENING DETECTION
Thiswasproposedinthissystemtocheckiftheexaminee opens his/her mouth to say something during the exam. Hereitusesthedlibfacialkeypointsandforthistask,the examineeisrequiredtositstraightandthedistancebetween thekeypoints(5outerpairsand3innerpairs)ischecked for 100 frames. If the examinee opens his mouth the distances of the points increase and if the increase in distanceismorethanafixedvalueforatleastthreeouter pairsandtwoinnerpairsthenthevectorisgenerated.
4.4. MOBILE & OBJECT DETECTION
The use of other devices is strictly not permitted during examinations as everything is accessible through such devices which makes it a lot easier for the examinees to cheat.Thesystem usesthe YOLOv3 model whichhaspretrainedobjectsclassifiedinitscorrelations.Anydetectionof suchobjectsgivesawarningtotheexamineeandmultiple detectionsmayresultintheterminationofthetest.
5. SYSTEM DESIGN
5.1. COMPUTER VISION
Objectdetectionandclassificationaredoneusingcomputer vision.Mapsandmotionestimationareotheruseforit. It enablesthecomputertorecognizeandunderstandtheitems initsenvironmentanduseMLmodelstoinferaparticular result.
Amachinelearningsystemmayautomaticallylearnabout the interpretation of visual input with the use of preprogrammed computational frameworks. Convolutional neuralnetworksbreakdownimagesintosmallerchunksto helpmachinelearninganddeeplearningmodelsunderstand them. It employs tags, then uses the tertiary function to producesuggestions,conductsconvolutions,thenassesses the accuracy of those recommendations after each cycle. Overall,thisgivesitahuman-likeabilitytoperceiveimages.
5.2. YOLO WEIGHTS MODEL v3
SantoshDivvala,JosephRedmon,andRosshavesuggested thedeeplearningarchitectureknownasYOLO.Itsexcellent precisionandabilitytoruninreal-timeorbeusedforrealtimeapplicationsmakeitverypopular.TheYOLOmethod "justlooksonce,"oronlyrequiresoneforwardpropagation pass through the network, to make predictions from the inputimage.
Previousdetectionsystemscarryoutthedetectionprocess usinglocalizersorclassifiers.Themodelisthenusedtoalter thescaleandlocationofapicture.Theimage'shigh-scoring areas are considered for detection. The YOLO algorithm follows an entirely different methodology. A single neural networkisusedbythealgorithmtoprocesstheentirefull image. Thisnetworkthendividesthatpicture.
range of programming options in particular for computer vision.
TheOpenCVBasedDNNFaceDetectionisbeingusedinthis proctoring system. It's a Caffe model with a single action detectorinitscoreandResNet10infrastructuretobackit up.Afterversion3.3,itmadeitsdebutintheOpenCVDeep Neural Network module. A comparison of multiple face recognition models has revealed very small variations in accuracy,withtheDNNFaceDetectionModelthat'salready been embedded into a popular OpenCV library not even takingsecondplace.
5.4. DLIB
Dlib is a cutting-edge C++ toolkit that includes machine learningtechniquesandtoolsfordevelopingsophisticated software to address real-world issues. Although it is less well-known than alternatives like OpenCV, its trained modelsperformmorefasterthankstoitshigheraccuracy.It includesafaciallandmarksmodelthataidsinthedetection ofanumberoffactors,includingheadposeestimation,face swapping, face alignment, gaze tracking, and other parameters.Thismodelhas68facialdetectionpoints.
5.3. OpenCV
The OpenCV Library is an open-source library of programmingcapabilities,primarilyforreal-timecomputer vision,createdbyIntel.Itisusingsomeofthemostadvanced technology,includingDeepLearning,aswellasoffersawide
II: Facial Landmark Detection
6. COMPARATIVE ANAYLSIS
6.1. YOLO MODEL
Weusedinstancesegmentationwhichcanbedistinguishing between different labels and able to distinguish between multiple objects of that label. We used YOLOv3 already trained model to classify 80 objects. It uses Darknet-53 which has 53 convolutional layers and is 1.5 times faster thanpreviousversions.
III: YOLO vs Other Models
Apart from the other models, YOLO itself has multiple versionsthathavebeenreleasedovertheyears.TheYOLO v1 model uses GooleNet which is superior to VGG16 and madeitpopularduringitsrelease.ThenextreleaseofYOLO usesDarkNet-19asitsprimarynetworkwhichbecamevery popularforobjectdetectionpurposes.DarkNetmaintained its popularity as it proved to be much faster than its competitors.TheYOLOv3modelusesDarkNet-53i.e.Ithas 53convolutionallayerswhichimproveditsobjectdetection capabilitiesmassively.Moreover,theYOLOv3modelusesa residualblock,unlikeitspreviousversionwhichusesanchor boxes.
The further releases of the YOLO model which are the YOLOv4 and YOLOv5 included the DarkNet approach but lackedanyinnovation.TheYOLOv4modelmajorlyfocused onobjectcomparisonwithlittletonoimprovementinobject detectionwhereastheYOLOv5modeldidimproveitsobject detectioncapabilities,especiallyforsmallerobjectsbutits mainfocuswasprovidingflexibilitytothemodelsize.Hence forthepurposeofproctoringYOLOv3seemedtobethemost suitableapproach.
6.2. OpenCV DNN
Aproctoringsystemshouldbeabletocontinuouslyverifyif theexamineeisthesamepersonheclaimstobe.Thereare variousmethodsforcontinuoususerverification,weused facialrecognitioninoursystem.Thismodelwasputtotest againstitsfellowmodelswiththepurposeoffacialdetection.
Figure IV: Open CV DNN vs Other Models
7. TESTING PROCEDURE
Thetestwindowopensinfull-screenmode,whichdisables anyattemptsatswitchingtabsorswitchingwindows.The test terminates if there is excessive switching of tabs or windows.Theexamineewillattemptthetestwhilehis/her webcamwillbeonfortheentiredurationofthetest.Objects likemobilephones,laptops,calculatorsetc.ifdetectedthen the examinee will be given a warning. Similarly, gazetracking will also be conducted all the time and if any misconduct is observed the examinee will be given a warning.Ifanyofthesewarningsstackupto3times,thetest willbeterminatedandtheexamineewillberoutedbackto thehomescreen.
8. RESULT AND DISCUSSION
The proposed automated proctoring system gives decent resultsinalltheproposedmodelssuchasEyeGazeTracking, MouthOpenorCloseDetection,ObjectDetection,HeadPose Estimation,etc.Thesystemissufficientandholdstruetothe purposeofproctoring.Thesystemissimpleandconvenient to use from the perspective of the test taker, as it only requires two inexpensive items i.e., cameras and a microphone.Finally,amanualproctorwillbeprovidedwith thelogsincaseofanyfailureormistakefromtheproctoring system.
Theeyetrackingmoduleupdateseveryhalfasecondgiving the direction of the eye gaze as the output. This helps the system to detect even the slightest eye movements. This frequency could be changed later on depending on the seriousnessoftheexaminationandtheexaminee’ssystem.
InObjectdetection,theperson,andobjectslike,thebed,the mobile,notebook,i.e.,eachobjectintheframeonthecamera is detected. Major objects include a person and a mobile phone. The accuracy for person detection came out to be 99.91%andthatformobilephonedetectionis97.08%.
9. CONCLUSION
In this paper, we have proposed and implemented an automated proctoring system using computer vision techniques.Thesystemhelpsinconductingexaminationsby fair means and hence, maintains its integrity. This study demonstrateshowtoavoidcheatinginonlineexaminations by employing semi-automated proctoring based on vision and audio capabilities, as well as monitoring several students
The system provides promising environment for any organizationtoconducttheirexamination.Italsoprovides user-friendlyinterfacewhichmakeiteasierfortheexaminee to give their exams with comfort. Furthermore, a manual
assistancesteamcouldalsobesetupperexaminationwith thehelpoftheexamconductingorganizationtosmoothen outthewholeexaminationprocess.
10. FUTURE WORK
It would be more efficient if the system could include YOLOv7 model or its versions as it has sustainable improvement in its object detection capabilities but it requireshighercomputationcostwhichmakesitdifficultto implement.Vision-basedcapabilitieslikeID-cardverification canalsobeincludedinthesystem.Theproctoringplatform couldbeexpandedtomultilingualplatformsandthesystem could also provide support for different multilingual examinations.
ACKNOWLEDGMENT
This research work was performed under the guidance of Prof.PreetiMishra.Thankstoherguidance,thepublication hasaccomplisheditsaimofservingtheintendedpurpose.
The authors would like to thank Atharva College of Engineeringforprovidingallofthenecessaryresources.The authorsappreciateallthehelptheygotfromthementioned peers.
REFERENCES
[1] “A Design of Continuous User Verification for Online Exam Proctoring on M-Learning”, Hadian S. G. Asep; Yoanes Bandung, 2019 International Conference on ElectricalEngineeringandInformatics(ICEEI),9-10July 2019.
[2] AnImageMatchingandObjectRecognitionSystemusing Webcam Robot”, Sanjana Yadav; Archana Singh, 2016 FourthInternationalConferenceonParallel,Distributed andGridComputing(PDGC),22-24Dec.2016.
[3] “OnlineExamProctoringSystem”,A.T.Awaghade,D.A. Bombe,T.R.Deshmukh,K.D.Takawane,International Journal of Advance Engineering and Research Development(IJAERD)“E.T.C.W”,January-2017.
[4] “Fraud detection in video recordings of exams using ConvolutionalNeuralNetworks”,AimanKuin,University ofAmsterdam,June20,2018.
[5] e-Invigilator:Abiometric-basedsupervisionsystemfor e- Assessments”, N.L Clarke, P. Dowland, S.M. Furnell International Conference on Information Society (iSociety2013),24-26June2013.
[6] “Smart AI-based Online Proctoring System”, Neil Malhotra, Ram Suri, Puru Verma, Rajesh Kumar, IEEE,April-2022
[7] Kimberly K. Hollister, Mark L. Berenson Proctored Versus non-proctored Online Exams: Studying the ImpactofExamenvironmentonStudentPerformance, DecisionSciences,Volume7,Issue1January2009,Pages 271-294
[8] ExaminingOnlineCollegeCyberCheatingMethodsand PreventionMeasuresJamesMotenJr.,AlexFitterer,Elise Brazier,Jonathan.
[9] “YOLO3Real-TimeObjectDetection”byJosephRedmon, andAliFarhadipublishedbyCornellUniversityon April 8,2018.
[10] “Facial landmarks with dlib, OpenCV, and Python” by AdrianRosebrockonApr3,2017.
[11] ExaminingtheEffectofProctoringonOnlineTestScores - Helaine M. Alessio, Nancy Malay, Karsten Maurer, A. JohnBailer,andBethRubin