International Research Journal of Engineering and Technology (IRJET) Volume: 11 Issue: 05 | May 2024
www.irjet.net
e-ISSN: 2395-0056 p-ISSN: 2395-0072
Developing a system for converting sign language to text Umesh kumar1, Ishu singh2, Ritik chauhan3, Raman baliyan4, Harsh tyagi5 1Assistant professor, Dept. of computer science and engineering (artificial intelligence and machine learning),
Meerut Institute Of Engineering And Technology, Meerut, Uttar Pradesh ,India 2345B.tech Student, Dept. of computer science and engineering (artificial intelligence and machine learning),
Meerut Institute Of Engineering And Technology, Meerut , Uttar Pradesh, India ---------------------------------------------------------------------***-------------------------------------------------------------------Amidst its multifaceted capabilities, one of the primary Abstract - This project focuses on the development of
objectives of this system is the accurate detection and interpretation of an extensive range of hand signs, encompassing not only the 26 letters of the English alphabet but also the recognition of the backslash symbol, a crucial component for seamless textual communication. By harnessing the power of CNN, the system demonstrates an unprecedented accuracy rate exceeding 99%, enabling the precise translation of intricate hand gestures into their corresponding textual representations.
a Hand Sign Language to Text and Speech Conversion system using Convolutional Neural Networks (CNN). With an achieved accuracy of over 99%, the model accurately translates hand signs, including the 26 alphabets and the backslash character, to their corresponding text characters. The system utilizes the OpenCV library for image processing and gesture recognition, and the Keras library for the implementation of the CNN model. The process involves capturing real-time video input of hand gestures, preprocessing the images, and making predictions using the trained CNN model. The system is equipped with a Graphical User Interface (GUI) to display the captured video and the recognized hand sign, along with options for users to choose suggested words or clear the recognized sentence. Additionally, the system enables users to listen to the recognized sentence through text-to-speech functionality. The effectiveness and accuracy of the proposed system were evaluated through extensive testing, demonstrating its potential for real-world applications.
The core architecture of the system integrates the robust OpenCV library for intricate image processing and gesture recognition, coupled with the flexible Keras library, serving as the backbone for the streamlined implementation of the CNN model. The comprehensive workflow of the system encompasses real-time video input capturing, sophisticated image preprocessing, and informed predictions based on the robust CNN model and using Mediapipe for recognisition of various points, reflecting a harmonious blend of cutting-edge technology and usercentric design. Furthermore, the system is equipped with a highly intuitive Graphical User Interface (GUI) that showcases the captured video feed and the recognized hand sign, providing users with a seamless experience to interact with the system effortlessly. Users are presented with an array of options, including the ability to select suggested words or effortlessly clear the recognized sentence, fostering an environment of interactive and dynamic communication. Additionally, the integration of text-tospeech functionality empowers users to not only visualize but also audibly comprehend the recognized sentence, enhancing the overall accessibility and user experience.
Keywords: CNN, Text to Speech, GUI, OpenCV, Suggested Words, Real Time,Mediapipe
1.INTRODUCTION In the contemporary era of rapid technological advancements, the quest for innovative solutions that foster seamless communication for individuals with diverse linguistic abilities remains a pivotal focal point. Within this context, the development of a Hand Sign Language to Text and Speech Conversion system using Mediapipe advanced Convolutional Neural Networks (CNN) represents a significant stride towards inclusivity and accessibility. This groundbreaking system stands as a testament to the fusion of state-of-the-art image processing, machine learning methodologies, and intuitive user interfaces, all converging to bridge the gap between conventional spoken language and the intricate nuances of sign language.
© 2024, IRJET
|
Impact Factor value: 8.226
Through rigorous and extensive testing, the efficacy and precision of the proposed system have been extensively validated, underscoring its immense potential for realworld applications across a diverse spectrum of contexts. By facilitating the seamless conversion of intricate hand gestures into coherent textual and auditory output, this system paves the way for enhanced communication and inclusivity, catering to the diverse needs of individuals with
|
ISO 9001:2008 Certified Journal
| Page 1100