International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 11 Issue: 04 | Apr 2024
p-ISSN: 2395-0072
www.irjet.net
"Visual Speech Translation for Sign Language" Tushar Surwade1, Vrushali Wankhede2, suraj Menon 3, Dipti Mondal4, Ritesh Kakade5, Sanket Pawar6 1,2,3,4,5,6Computer Engineering Department, Keystone School of Engineering,
Pune, Maharashtra, India. --------------------------------------------------------------------***--------------------------------------------------------------------Abstract: 1. INTRODUCTION: Sign language is fundamental for communication within the deaf and hard of hearing community. Nonetheless, there remains a significant communication barrier between sign language users and non-signers. This study introduces a deep learning-based framework for instantaneous sign language recognition, with the objective of bridging this divide. By harnessing advancements in computer vision and natural language processing, our framework translates sign language gestures captured via camera into spoken language. We tackle challenges including gesture diversity and environmental influences to establish a dependable and effective system. Our goal is to elevate accessibility and inclusivity for individuals dependent on sign language.
The deaf and hard of hearing community faces significant challenges due to a communication gap between sign language users and non-signers, which hinders effective interaction and accessibility. In response to this issue, our research introduces an innovative deep learning-based framework for real-time sign language recognition, aiming to overcome this communication barrier. Our approach capitalizes on recent advancements in computer vision and natural language processing to translate sign language gestures captured by a camera into spoken language. We leverage convolutional neural networks (CNNs) for robust hand gesture detection and tracking, enabling accurate recognition of dynamic hand movements and configurations. Additionally, recurrent neural networks (RNNs) or transformer models are employed to capture temporal dependencies and extract meaningful features from gesture sequences.
2. SYSTEM ARCHITECHTURE The sign language recognition system comprises several interconnected modules designed to seamlessly translate sign language gestures into spoken language. The architecture consists of three main components: Input Module, Deep Learning Model, and Output Module.
To assess the effectiveness of our proposed framework, we conduct extensive experiments using publicly available sign language datasets, encompassing both isolated and continuous sign sequences. Through comparative analysis with existing methods, we demonstrate superior accuracy and real-time processing capabilities of our model. Furthermore, we evaluate the generalization of our approach across various sign languages and adaptability to variations in lighting conditions, background clutter, and signer demographics.
1.
Our results underscore promising outcomes in real-world scenarios, highlighting the potential of our approach to enhance accessibility and facilitate seamless communication between sign language users and nonsigners. This research contributes significantly to the advancement of assistive technologies and sets the stage for future developments in sign language recognition systems.
2.
Key Words: Visual speech translation, deep learning, computer vision, natural language processing, gesture recognition, hand tracking, real-time translation, accessibility
© 2024, IRJET
|
Impact Factor value: 8.226
Input Module:
The Input Module captures sign language gestures using a camera or other input devices.
Preprocessing techniques, such as noise reduction and normalization, are applied to enhance the quality of the input data.
Feature extraction methods may be utilized to effectively represent the spatial and temporal characteristics of the gestures.
Deep Learning Model:
|
The Deep Learning Model serves as the core component responsible for recognizing and interpreting sign language gestures.
ISO 9001:2008 Certified Journal
|
Page 2505