International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 13 Issue: 02 | Feb 2026
p-ISSN: 2395-0072
www.irjet.net
A Multimodal AI Mental Health Companion Using Voice and Facial Emotion Analysis Ojasvita Akojwar, Niketa Tembhare, Tanvi Wankhade, Vaishnavi Bodele, Prof. Abhilasha Borkar Department of Computer Engineering Cummins College of Engineering for Women Rashtrasant Tukadoji Maharaj Nagpur University Nagpur, Maharashtra, India ---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - The prevalence of mental health illnesses is
many current systems rely mostly on text-based interaction, which limits their ability to effectively record the user’s emotional state.
rising globally, but there is still a lack of prompt, ongoing, and individualized mental health support. The development of intelligent systems that can assist people through continuous emotional evaluation and interactive communication has been made possible by recent advancements in artificial intelligence. In this research, a multimodal AI-based mental health companion is proposed that uses both voice signals and facial expressions to analyze users’ emotional states. The suggested system performs real-time emotion detection by combining machine learning models, audio processing techniques, and face emotion identification. Combining various input modalities improves overall user engagement and interactivity while increasing the accuracy of emotion recognition. The suggested system is successful in recognizing emotional states and providing sympathetic reactions, according to experimental evaluation. In addition to traditional treatment procedures, the developed system is meant to serve as an easily available and user-friendly mental health support option.
This research suggests a multimodal AI-based mental health companion that makes use of both voice and face emotion analysis in order to overcome these constraints. The technology can more precisely identify emotional states and offer sympathetic reactions in real time by combining speech processing methods with facial expression detection. The suggested method preserves usability and accessibility while improving user involvement and emotional comprehension. This strategy is intended to be a useful tool for early emotional support and to supplement conventional mental health treatment techniques.
2. EASE OF USE Because users may come from a variety of technical back grounds and emotional situations, ease of use is crucial to the efficacy of mental health support systems. The user centric architecture of the suggested system guarantees easy and straightforward interaction. Natural voice input and facial expressions allow users to interact with the system without the need for complicated user interfaces or a lot of manual input.
Index Terms—Artificial Intelligence, Mental Health Assistance, Speech Signal Analysis, Facial Expression Recognition, Emotion Recognition, and Multimodal AI Systems
1. INTRODUCTION
Real-time feedback and responses are provided by the system, allowing for smooth communication with no discernible lags. The application is suited for frequent usage due to its minimal setup requirements and automated processing, which further improve accessibility. The suggested approach promotes user engagement and prolonged involvement by emphasizing comfort and simplicity.
Stress, anxiety, depression, and other psychological prob lems have significantly increased across all age groups, making mental health a major global concern. Global health surveys state that many people do not receive timely mental health care because of things including societal stigma, expensive therapy, and a shortage of mental health specialists. These difficulties underscore the necessity for readily available, reasonably priced, and ongoing mental health support services.
A. Maintaining the Integrity of the Specifications To guarantee dependable and moral operation, it is crucial to preserve the integrity of system specifications. The suggested system continuously processes user data via secure and verified pipelines in accordance with predetermined design and performance standards. To ensure accuracy and consistency, standardized datasets are used for both training and evaluation of emotion detection algorithms.
The creation of intelligent systems that can comprehend and react to human emotions has been made possible by recent developments in artificial intelligence (AI) and human–computer interaction. Through conversational interfaces, AI-based mental health companions seek to help people express their emotions and cope with emotional suffering. However, although emotions are frequently expressed by voice tone and facial expressions,
© 2026, IRJET
|
Impact Factor value: 8.315
|
ISO 9001:2008 Certified Journal
|
Page 538