SentifyAI – An AI Emotion Detector

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 03 | Mar 2026 www.irjet.net p-ISSN: 2395-0072

SentifyAI – An AI Emotion Detector

Afifa Shaikh1 , Sufiya Ansari2 , Iqra Essani3 , Noorusabah Sayed4

1,2,3Department of Information Technology, M. H. Saboo Siddik Polytechnic, Mumbai, India

4Head of Department, Department of Artificial Intelligence, M. H. Saboo Siddik Polytechnic, Mumbai, India

Abstract - This paper presents SentifyAI, a web-based system for real-time emotion detection using facial expressions. The application uses OpenCV for face detection and a Convolutional Neural Network (CNN) model to classify emotions from live webcam video. The system detects faces, predicts emotions with confidence scores, and displays the results with emojis in real time. SentifyAI improves humancomputer interaction and has potential applications in healthcare, education, and customer service. The system is designedtobesimple,fast,anduser-friendlyforreal-timeuse. It demonstrates the practical use of artificial intelligence in building emotion-aware and interactive systems.

Key Words: Emotion Detection, Deep Learning, Convolutional Neural Network (CNN), OpenCV, Facial Expression Recognition, Human-Computer Interaction, WebApplication.

1. INTRODUCTION

Human emotions are fundamental to communication, decision-making, and social interaction. In traditional computing paradigms, machines operate as passive tools, responding only to explicit commands and remaining oblivioustotheuser'saffectivestate.Thislackofemotional intelligencecreatesasterileinteractionenvironmentwhere themachinecannotadaptitsresponsesbasedonwhether the user is frustrated, confused, happy, or engaged. As technologybecomesincreasinglyintegratedintodailylifein healthcare,education,entertainment,andcustomerservicethe demand for emotionally aware systems has grown significantly.

The field ofAffective Computing, pioneered by Rosalind Picard, aims to give machines the ability to recognize, interpret, and simulate human emotions. Recent advancementsinArtificialIntelligence(AI),particularlyin DeepLearningandComputerVision,havemadeitpossible to automatically analyze facial expressions, which are a primary channel for expressing emotions. Ekman's foundationalworkonuniversalfacialexpressionsidentified sixbasicemotions:happiness,sadness,anger,fear,surprise, and disgust, which serve as the core categories for most recognitionsystems.

This paper introducesSentifyAI, a web application developed to address the challenge of real-time emotion detection.Thesystemutilizesastandardwebcamtocapture a video stream. It employs OpenCV for efficient face detectionandpre-processing,isolatingtheregionofinterest

(theface)fromeachframe.Thispre-processedfacialimage is then passed through a trained Convolutional Neural Network(CNN)model,whichclassifiestheexpressioninto oneofthepredefinedemotioncategories.

The detected emotion and its associated confidence score arethendisplayedbacktotheuseronthevideofeed,often accompaniedbyarepresentativeemojiforenhanceduser experience.Thisprojectdemonstratesacompletepipeline from image acquisition to real-time affective feedback, showcasingthepotentialofintegratingdeeplearningmodels intoaccessibleweb-basedplatforms.

2. PROBLEM STATEMENT

Traditionalcomputersystemsandinterfacesareunableto effectively perceive and interpret human emotions. This limitation reduces the quality of human-computer interactionandmakescommunicationlessnaturalandless personalized.

Theabsenceofemotionalawarenesscreatesagapbetween humansandmachines,resultingincommunicationthatlacks empathyandresponsiveness.Asaresult,usersmayfeelless engaged and understood when interacting with digital systems.

In important applications such as teletherapy, online education,andautomatedcustomersupport,theinabilityto recognize a user's emotional state can lead to misunderstandings and reduced service quality. This can ultimatelyresultinlowerusersatisfaction.

Therefore,thereisastrongneedforanintelligentautomated systemthatcandetectandanalyzehumanemotionsthrough facialexpressionsinrealtime.Suchsystemscanhelpcreate

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 03 | Mar 2026 www.irjet.net p-ISSN: 2395-0072

more adaptive, interactive, and user-friendly digital environments.

3. LITERATURE SURVEY

Thedomainofemotionrecognitionhasevolvedsignificantly over the past decades, transitioning from psychological studiestocomplexcomputationalmodels.

a.PsychologicalFoundations:Thegroundworkforautomatic emotion recognition was laid by psychologists like Paul Ekman, who conducted cross-cultural studies and establishedthatcertainbasicemotions(happiness,sadness, anger,fear,surprise,disgust)areuniversallyexpressedand recognizedthroughspecificfacialconfigurations,knownas ActionUnitsintheFacialActionCodingSystem(FACS).

b.Traditional Machine Learning Approaches:Early computational approaches relied on handcrafted feature extractiontechniquessuchasLocalBinaryPatterns(LBP), Histogram of Oriented Gradients (HOG), or Gabor filters. These features were then fed into classifiers like Support VectorMachines(SVM)orRandomForests.Whileeffective toa degree, thesemethods weresensitive tovariations in lighting, pose, and occlusion and required careful feature engineering.

c. DeepLearningRevolution:TheadventofDeepLearning, particularly Convolutional Neural Networks (CNNs), revolutionized the field. CNNs can automatically learn hierarchical features directly from raw pixel data, eliminating the need for manual feature extraction. Architectures like VGGNet, ResNet, and custom smaller modelshavebeensuccessfullyappliedtofacialexpression recognitiondatasetssuchasFER-2013,CK+,andAffectNet, achievingstate-of-the-artaccuracy

d. Existing Systems:Current commercial systems like Microsoft Azure Face API and Amazon Recognition offer emotiondetectionaspartofbroaderfacialanalysisservices. However, they are often cloud-based, introducing latency and privacy concerns. SentifyAI differentiates itself by aimingforalightweight,real-time,andpotentiallyprivacypreserving (if run locally) web broader facial analysis services.However,theyareoftencloud-based,introducing latencyandprivacyconcerns.

4. METHODOLOGY

ThedevelopmentofSentifyAIfollowedasystematicpipeline, integratingcomputervisionanddeeplearningforreal-time performance.

a. Data Acquisition and Preprocessing: Acrucialstepfor anydeeplearningmodel isthedata.Thesystemutilizesa publicly available dataset for training, such as FER-2013 (Facial Expression Recognition 2013), which contains grayscale images of faces labelled with seven emotion

categories (Angry, Disgust, Fear, Happy, Sad, Surprise, Neutral).Preprocessinginvolvesconvertingframesfromthe webcamtograyscale,detectingthefaceusingOpenCV'sHaar Cascadeclassifier,croppingthefaceregion,andresizingitto the dimensions expected by the CNN model (e.g., 48x48 pixels).

b. Model Architecture (CNN): The core of SentifyAI is a ConvolutionalNeuralNetwork.Thearchitectureconsistsof:

ConvolutionalLayers:Multiplelayerswithsmallfilters(e.g., 3x3) to extract features like edges, textures, and complex facial patterns. ReLU (Rectified Linear Unit) activation is usedtointroducenon-linearity.

Pooling Layers:Max-pooling layers follow convolutional layers to reduce the spatial dimensions, decrease computational load, and make feature detection more robust.

Fully Connected Layers:After several convolutional and pooling layers, the feature maps are flattened and passed through one or more dense layers to perform high-level reasoning.

Output Layer:A final dense layer with 7 neurons (for 7 emotions)andaSoftMaxactivationfunction,whichoutputsa probabilitydistributionovertheemotionclasses.

c. Model Training: The CNN model is trained on the preprocessed dataset using a categorical cross-entropy loss functionandanoptimizerlikeAdam.Thegoalistominimize lossandmaximizeclassificationaccuracyonavalidationset.

d. Real-Time Inference and Integration: The trained model is saved and loaded into the main application. The applicationcapturesvideoframesfromthewebcamusing OpenCV. For each frame, face detection is performed. For everydetectedface,theregionispre-processedandfedinto the CNN model. The model predicts the emotion and its confidencescore. OpenCVis usedtodrawa boundingbox around the face and display the emotion label, confidence score,andacorrespondingemojiontheframeinreal-time.

5. SYSTEM ARCHITECTURE

The architecture of SentifyAI follows a modular pipeline, processing data sequentially from input to output. Input Module (Webcam) captures real-time video feed. Face Detection Module (OpenCV) employs a pre-trained Haar Cascadeclassifiertolocatefaceswithineachframe.Theface region is cropped. Preprocessing Module converts the cropped faceto grayscale(if notalready),resizesittothe targetdimensions(e.g.,48x48),andnormalizespixelvalues. EmotionClassificationModule(TrainedCNNModel) takes the preprocessed face image as input and outputs a probability for each emotion class. The class with the highest probability is selected. Visualization Module (OpenCV)drawstheresults(boundingbox,emotionlabel,

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 03 | Mar 2026 www.irjet.net p-ISSN: 2395-0072

confidence, emoji) onto the original video frame. Display Moduleshowstheprocessedvideostreamtotheuserviathe webapplicationinterface.

6. SYSTEM REQUIREMENTS

Software Requirements:

OperatingSystem:Windows,Linux,ormacOS ProgrammingLanguage:Python3.x

Libraries:OpenCV,TensorFlow/Keras,Flask,NumPy Frontend:HTML,CSS,JavaScript Development Environment:VS Code / PyCharm / Jupyter

Notebook

Hardware Requirements:

Processor:Intel Core i5 / AMD Ryzen 5 or higher (for smoothreal-timeprocessing)

RAM:Minimum8GB(16GBrecommended)

Camera:StandardUSBWebcam(720porhigher) GPU (Recommended):NVIDIA GPU with CUDA support to acceleratemodelinference

Storage:Minimum2GBfreespaceforcodeandlibraries

7. OUTPUT & RESULTS

Theprimaryoutputofthesystemisareal-time,annotated videostream.Theexpectedoutputsinclude:

Emotion Label:Text overlay displaying the predicted emotion (e.g., "Happy", "Sad"). Confidence Score:A percentage value indicating the model's certainty in its prediction. Visualization:A representative emoji correspondingtothedetectedemotiondisplayednexttothe labelforintuitiveunderstanding. BoundingBox:Arectangle drawnaroundthedetectedface.

8. CONCLUSION

SentifyAIsuccessfullydemonstratesthefeasibilityofarealtime,web-basedemotiondetectionsystemusingaccessible technologieslikeOpenCVandCNNs.Theprojectachievesits primary objective of bridgingthe gap inhuman-computer interaction by enabling machines to perceive and display humanaffectivestates.Thissystemservesasafoundational model for integrating emotional intelligence into various applications.

REFERENCES

[1] M. M. Hassan, G. Muhammad, and M. S. Hossain, “AffectiveComputinginHuman–ComputerInteraction: A Review,” IEEE Access, vol. 7, pp. 117393–117406, 2019.

[2] S.LiandW.Deng,“DeepFacialExpressionRecognition: ASurvey,” IEEE Trans. Affective Comput.,vol.13,no. 3, pp. 1195–1215, 2022 https://doi.org/10.1109/TAFFC.2020.2981446

[3] S.Wang,Z.Liu,Y.Lv,Y.Zhu,andF.Wu, “ASurveyon Facial Expression Recognition: From Traditional to Deep Learning Methods,” IEEE Trans. Affective Comput.,vol.10,no.3,pp.394–409,2019.

[4] Mollahosseini,B.Hasani,andM.H.Mahoor,“AffectNet: ADatabaseforFacialExpression,Valence,andArousal Computing in the Wild,” IEEE Trans. Affective Comput., vol. 10, no. 1, pp. 18–31, 2019. https://doi.org/10.1109/TAFFC.2017.2740923

[5] E. Barsoum, C. Zhang, C. C. Ferrer, and Z. Zhang, “Training Deep Networks for Facial Expression Recognition withCrowd-SourcedLabel Distribution,” Proc.ICMI,2016.

[6] J. Chen, Z. Chen, Z. Chi, and H. Fu, “Facial Expression Recognition in the Wild Using Improved CNN,” IEEE Access,vol.8,pp.190652–190660,2020.

[7] M. Minaee, M. Minaei, and A. Abdolrashidi, “DeepEmotion: Facial Expression Recognition Using AttentionalConvolutionalNetwork,” Sensors,vol.21, no.9,2021.https://doi.org/10.3390/s21093116

[8] S. S. S. R. Kumar and P. V. Arun, “Real-Time Facial Emotion Recognition Using Deep Learning,” Electronics, vol. 8, no. 10, 2019. https://doi.org/10.3390/electronics8101178

[9] X. Li, J. Chen, D. Zhao, and J. Du, “Lightweight Deep Learning Model for Real-Time Facial Expression Recognition,” Applied Sciences,vol.11,no.21,2021. https://doi.org/10.3390/app11219278

2026, IRJET | Impact Factor value: 8.315 | ISO 9001:2008 Certified Journal | Page95