International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 11 Issue: 03 | Mar 2024
p-ISSN: 2395-0072
www.irjet.net
SIGN TO SIGN TRANSLATION USING MEDIAPIPE U.V. Sai Madhumita1, Meghna Malla2, Ponnada Nikheel Deo3, S. Venkat Sai4, Dr. Mrudula Owk5 1,2,3,4 Student, Dept. Of Computer Science Engineering, GITAM Institute of Technology (GIT), GITAM University, AP,
INDIA
5Assistant Professor, Dept. Of Computer Science Engineering, GITAM Institute of Technology (GIT), GITAM
University, AP, INDIA ---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - Communication is the cornerstone of human
popular sign language, and ISL- our local sign language. This initiative aims to improve cross-cultural communication among sign language users worldwide.
interaction, enabling individuals to express thoughts, emotions, and intentions. Whether verbal or non-verbal, language plays a pivotal role in fostering understanding and connection. However, the diversity of sign languages, such as American Sign Language (ASL) and Indian Sign Language (ISL), presents unique challenges for crosscultural communication. To address this, our work aims to bridge the gap between ASL and ISL through innovative technology solutions. The model takes ISL gestures as input. Accurate translation between these languages is enabled by training the model on comprehensive datasets of ASL and ISL gestures. The model involves Machine Learning to capture data and create datasets (using OpenCV and Mediapipe), Deep Learning models to recognize the dynamic gestures (RNN architectures to deal with Sequential data), and Analysis tools such as NLTK (VADER for Sentiment Analysis). The model is trained on LSTM, LSTM+GRU, and GRU neural network architectures. Upon training, the network with the highest accuracy is used to implement the final gesture detection. The final result is given in ASL gestures (in GIF format) along with the tonality of the subject matter. We have noted from training and testing the model that the GRU model shows promising accuracy for the three models, followed by LSTM+GRU and LSTM.
1.1 American Sign Language (ASL) ASL, a visual language predominantly employed by Deaf, Mute, and hard-of-hearing communities in the United States and parts of Canada, possesses its grammar and vocabulary. Communication in ASL involves the use of hand shapes, facial expressions, and body movements. Beyond North America, various ASL dialects are utilized in several West African and Southeast Asian countries. Originating nearly two centuries ago as a fusion of local sign languages and the French Sign Language (LSF), ASL exhibits a close relationship with LSF. ASL comprises 26 gestures corresponding to each letter in the alphabet and distinct signs for numbers 1-10. While precise statistics are elusive, an estimated half a million people in the US use ASL as their natural language.
Keywords: American Sign language (ASL), Gesture recognition, Indian Sign language (ISL), Machine learning, Mediapipe and Sign language.
1.INTRODUCTION Communication is a fundamental aspect of human life, enabling interaction and meeting basic needs. From making requests to sharing problems, various tasks require communication. While verbal communication is the primary mode for hearing individuals, non-hearing communities have developed regional sign languages. Sign language involves hand gestures and facial expressions representing words or phrases. Despite its local efficiency, global communication between sign language users faces challenges due to regional variations. Many deaf individuals are not taught to read; written languages differ by location. To address this gap, we have proposed a system that converts one language's sign language gesture to another. The focus was mainly on ASL – the most
© 2024, IRJET
|
Impact Factor value: 8.226
Fig -1: ASL alphabet gestures
1.2 Indian Sign Language (ISL) ISL is the sign language used in India since it is a country of linguistic riches and countless regional dialects. This language is gestural and is expressed using the hands and face with origins in the early 20th century. Indo-Pakistani Sign Language is used by about 15 million deaf persons in the South Asian subcontinent. Therefore, ISL, like ASL, has unique signs for each letter of the alphabet. Legal recognition of the sign language in India shows the use and application of this language in the country’s communicational growth process. ASL and ISL are vital
|
ISO 9001:2008 Certified Journal
|
Page 1319