International Research Journal of Engineering and Technology (IRJET) Volume: 09 Issue: 04 | Apr 2022
www.irjet.net
e-ISSN: 2395-0056 p-ISSN: 2395-0072
Real Time Sign Language Detection Sahil Rawal1, Dhara Sampat2, Priyank Sanghvi3 1,2,3Students,
Department of Electronics and Telecommunication Engineering, K. J. Somaiya Institute of Engineering and Information Technology, Mumbai, Maharashtra, India. ---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - Throughout the long term, correspondence has
sensor based sign language recognition and Vision-based sign language recognition. Sensor based sign language recognition uses designs such as the robotic arm with a sensor, smart glove, golden glove for the conversion of ASL Sign language to speech. But the issue is that many people do not use it. Also, one must spend money to purchase such a glove, which is not easily available.
played an indispensable job in return of data and sentiments in one's day to day existence. Sign language is the main medium through which deaf and mute individuals can interact with rest of the world through various hand motions. With the advances in machine learning, it is possible to detect sign language in real time. We have utilized the OpenCv python library, Tensorflow Object Detection pipeline and transfer learning to train a deep learning model that detects sign languages in Real time.
2. LITERATURE SURVEY ASL recognition is not a new computer vision problem. Over the past two decades, researchers have used classifiers from a variety of categories that we grouped roughly into linear classifiers, neural networks and Bayesian networks. The first approach in relation to sign language recognition was by Bergh in 2011 [2]. Haar wavelets and database searching were employed to build a hand gesture recognition system. Although this system gives good results, it only considers six classes of gestures. Many types of research have been carried out on different sign languages from different countries. For example, a BSL recognition model, which understands finger-spelled signs from a video, was built [3]. As Initial, a histogram of gradients (HOG) was used to recognize letters, and then, the system used hidden Markov models (HMM) to recognize words. In another paper, a system was built to recognize sentences made of 3-5 words. Each word ought to be one of 19 signs in their thesaurus. Hidden Markov models have also been used on extracted features [4]. In 2011, a real time American Sign Language recognition model was proposed utilizing Gabor filter and random forest [5]. A dataset of color and depth images for 24 different alphabets was created. An accuracy of 75% was achieved utilizing both color and complexity images, and 69% using depth images only. Depth images were only used due to changes in the illumination and differences in the skin pigment. In 2013, a multilayered random forest was also used to build a real time ASL model. The system recognizes signs through applying random forest classifiers to the combined angle vector. An accuracy of 90% was achieved by testing one of the training images, and an accuracy of 70% was achieved for a new image. An American Sign Language alphabet recognition system was first built by localizing hand joint gestures using a hierarchical mode seeking and random forest method. An accuracy of 87% was achieved for the training, and accuracy of 57% when testing new images. In 2013, the Karhunen-Loeve Transform was used to classify gesture images of one hand into 10 classes [6]. These were
Key Words: American Sign Language (ASL), Sign Language Detection, Convolution, Neural Network (CNN), Transfer learning, Tensorflow, OpenCV
1. INTRODUCTION The disabled are the main users of sign language, and just a few others, such as families, campaigners, and teachers, comprehend it. The natural cue is a manual (hand-handed) expression agreed upon by the user (conventionally), recognised as limited in a certain group (esoteric), and utilised by a deaf person as a substitute for words (as opposed to body language). A formal gesture is a cue that is established consciously and has the same language structure as the spoken language of the society. More than 360 million of world population suffers from hearing and speech impairments. Sign language detection is a project implementation for designing a model in which web camera is used for capturing images of hand gestures which is done by open cv. After capturing images, labelling of images are required and then pre trained model SSD Mobile net v2 is used for sign recognition. Thus, an effective path of communication is developed between deaf and normal audience. Three steps must be completed in real time to solve our problem: 1. Obtaining footage of the user signing is step one (input). 2. Classifying each frame in the video to a sign. 3. Reconstructing and displaying the most likely Sign from classification scores (output). People with hearing impairments are left behind in online conferences, office sessions, schools. They usually use basic text chat to converse — a method less than optimal. With the growing adoption of telehealth, deaf people need to be able to communicate naturally with their healthcare network, colleagues and peers regardless of whether the second person knows sign language. Being able to achieve a uniform sign language translation machine is not a simple task, however, there are two common methods used to address this problem namely
© 2022, IRJET
|
Impact Factor value: 7.529
|
ISO 9001:2008 Certified Journal
|
Page 3540