International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 10 Issue: 04 | Apr 2023
p-ISSN: 2395-0072
www.irjet.net
STUDENT TEACHER INTERACTION ANALYSIS WITH EMOTION RECOGNITION FROM VIDEO AND AUDIO INPUT: RESEARCH Rajat Dubey1, Vedant Juikar2, Roshani Surwade3, Prof. Rasika Shintre4 1,2,3B.E. Student, Computer Department, 4Project Guide, Smt. Indira Gandhi College of Engineering Navi Mumbai, Maharashtra, India
---------------------------------------------------------------------***--------------------------------------------------------------------
Abstract - Emotions are significant because they are
expressions. Real-world classrooms allow for in-person interactions. In order to accomplish this, many schools have regularly scheduled chat rooms with audio and video conferencing interactions. In these spaces, students may easily connect with one another and the instructor just as they would in a traditional classroom. Through this technique, the teacher will be able to identify the student's facial expressions or spoken expressions of satisfaction with me as they occur throughout interactions with the student. The fundamental tenet is that teachers must be able to read students' minds and observe their facial expressions.
essential to the learning process. Emotions, behaviour, and thoughts are intimately connected in such a way that the sum of these three factors determines how we act and what choices we make. The selection of a database, identifying numerous speech-related variables, and making an appropriate classification model choice are the three key hurdles in emotion recognition. In order to understand how these emotional states relate to students' and teachers' comprehension, it is important to first define the facial physical behaviours that are associated with various emotional states. The usefulness of facial expression and voice recognition between a teacher and student in a classroom was first examined in this study.
We analyzed at whether the students' facial action units conveyed their emotions in relation to comprehension. The primary hypothesis of the first step of this study proposed that students frequently use nonverbal communication in the classroom, and that this nonverbal communication takes the form of emotion recognition from video and audio. This, in turn, enables lecturers to gauge the level of understanding of the students.
Key words: Speech Emotion Recognition, Facial Emotion Recognition, JAVA, Node.js, Computer Neural Network, Recurrent Neural Network, MFCC, Support Vector Machine
1. INTRODUCTION In artificial intelligence, it is common practice to take real time photos, videos, or audios of people in order to analyze their facial and verbal expressions minutely. Because there is very little facial muscle twisting, it is difficult for machines to recognize emotions, which leads to inconsistent results. The contact between teachers and students is the most important component of any classroom setting. In interactions between teachers and pupils, the impact brought about by facial expressions and voice is very strong.
The models that are available include Voice Emotion Recognition, which recognizes emotions through speech but is unable to produce results from video input. Face Emotion Recognition uses video to identify emotions, however it is unable to do so with audio input, and no analysis report is produced.
2. LITERATURE SURVEY We provide an overview of some current research in the fields of speech emotion recognition (SER) and facial expression recognition (FER), as well as some FER systems utilized in classrooms as the foundation for teacherstudent interaction, because the face plays a significant role in the expression and perception of emotions. A system that allowed them to collect audio recordings of the emotions of irritation, happiness, and melancholy together with their STE, pitch, and MFCC coefficients. Only the three basic emotions—anger, happiness, and sadness—were identified. Using feature vectors as input, the multi-class Support vector machine (SVM) generates a model for each emotion. Deep Belief Networks (DBNs) have an accuracy rate that is roughly 5% greater than Support Vector Machines (SVMs) and Artificial Neural Networks (ANNs) when compared to them. The results
Fig. 1 Basic Emotion Recognition The key sources of information for figuring out a person's interior feelings in humans are speech and facial
© 2023, IRJET
|
Impact Factor value: 8.226
|
ISO 9001:2008 Certified Journal
|
Page 736