International Research Journal of Engineering and Technology (IRJET) Volume: 12 Issue: 11 | Nov 2025
www.irjet.net
e-ISSN: 2395-0056 p-ISSN: 2395-0072
TalkLens – Integrated Vision and Sign Language Recognition for Inclusive Human Interaction 1st Dr. Sanjay Malode
2nd Ganesh Dhere
3rd Gouri Kashettiwar
Artificial Intelligence and Data Science K.D.K. College of Engineering Nagpur, India
Artificial Intelligence and Data Science K.D.K. College of Engineering Nagpur, India
Artificial Intelligence and Data Science K.D.K. College of Engineering Nagpur, India
4th Rhushikesh Ugemuge
5th Vaishnavi Bele
6th Yarmika Narad
Artificial Intelligence and Data Science K.D.K. College of Engineering Nagpur, India
Artificial Intelligence and Data Science K.D.K. College of Engineering Nagpur, India
Artificial Intelligence and Data Science K.D.K. College of Engineering Nagpur, India
--------------------------------------------------------------------***-----------------------------------------------------------------------Abstract—Communication barriers greatly impact Keywords—Vision-based object detection, Sign the independence, social engagement, and overall quality of life for individuals with visual, speech, or hearing challenges. This review looks at the TalkLens framework, a new solution aimed at overcoming these barriers. It integrates real-time vision-based object detection, sign language recognition, and natural language processing with empathetic artificial intelligence. The framework uses technologies like TensorFlow SSD MobileNet for object detection, MiniGPT-4 for understanding vision and language, Sentence-BERT for processing text, and natural text-to-speech synthesis to enable smooth communication. By combining these elements, TalkLens can interpret visual and textual inputs accurately and turn them into meaningful outputs while responding empathetically to users' needs. The system not only improves accessibility but also encourages social inclusion by allowing users to interact confidently in various real-world situations. Additionally, TalkLens tackles the issues faced by traditional assistive technologies by offering real-time, adaptable, and smart support that fosters active participation in educational, professional, and social settings. This review critically evaluates the design, methods, and possible applications of TalkLens, emphasizing its role in promoting independence, enhancing communication effectiveness, and contributing to a more inclusive society. By bringing together recent advancements in vision-language AI, sign language processing, and empathetic interaction systems, this study highlights the potential of TalkLens to create meaningful and inclusive human-computer interactions. © 2025, IRJET
|
Impact Factor value: 8.315
language recognition, Empathetic AI, Natural language processing, Text-to-speech synthesis, Inclusive communication.
I.INTRODUCTION Communication is a fundamental part of human life. It allows people to share emotions, exchange ideas, and form relationships. However, for millions worldwide who are visually, hearing, or speech-impaired, communication can be challenging. These sensory impairments create barriers that limit participation in education, jobs, and social activities. The World Health Organization (WHO) states that over 1 billion people have some form of disability, with nearly 430 million living with disabling hearing loss. Even with the growth of assistive technologies, a gap still exists between available solutions and the complex communication needs of differently-abled users. Traditional assistive systems, such as text-to-speech devices, Braille tools, or sign language translators, usually focus on just one aspect of disability. Some systems only help with visual impairment by turning text or images into speech. Others are designed only to interpret sign language gestures. While these tools can be effective on their own, they often do not support inclusive, two-way communication involving multiple disabilities. This divide forces users to rely on others or limits them to specific situations, which can lower their independence and confidence. The growth of artificial intelligence (AI) has created new possibilities for closing these gaps. Deep learning models now allow for real-time visual understanding, natural |
ISO 9001:2008 Certified Journal
|
Page 828