International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 10 Issue: 08 | August 2023
p-ISSN: 2395-0072
www.irjet.net
Design and Development of Motion Based Continuous Sign Language Detection Vijayaraghavan R, Hrishi S Kakol, Mithun TP Student, Dept. of ETE, R V College of Engineering, Bengaluru, Karnataka Student, Dept. of ETE, R V College of Engineering, Bengaluru, Karnataka Assistant Professor, Dept. of ETE, R V College of Engineering, Bengaluru, Karnataka ---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - Sign language, a structured form of hand
standard database is another appealing aspect. ISL is different from sign languages spoken in other countries in terms of syntax, phonology, morphology, and grammar. The Rehabilitation Council of India authorised the teaching materials, ISL grammar, ISL teaching programmes, ISL teacher training courses, and ISL teacher training in 2002. There hasn't been much study done on ISL recognition since the language was just recently established and because tutorials on ISL gestures weren't readily available. Indian Sign Language (ISL) is more dependent on both hands than American Sign Language (ASL), making an ISL recognition system more complicated. The impetus for designing such a useful application sprang from the fact that it would be extremely beneficial for socially assisting individuals as well as raising social awareness. . Systems for sign language development have been created using a variety of methods. These may be broadly divided into sensor-based systems and vision-based systems. Data was collected from various sources and processed in a similar manner in both procedures. The algorithms for extracting indications from photos varied.
gestures incorporating visual motions and signals, is utilised as a communication mechanism to assist the deaf and speechimpaired communities in their daily interactions. Many previous works make use of simpler algorithms rather than efficient ones like MediaPipe holistic to extract features. Also the use of pre-existing data set can limit the accuracy as opposed to developing a custom based one. Real time feed is captured from the webcam and preprocessed by excluding the user’s facial features and enhancing only the hands. It is userfriendly because no extra hardware is used. This image is then passed through several layers of a Convolutional Neural Network(CNN) with the usage of an advanced pooling layer, for detection of signs. Gesture recognition is executed with the use of MediaPipe Holistic whose major functionality is detecting feature points of hands and face of the user. The extracted feature points are used to detect the gestures with the help of a Hidden Markov Model (HMM) and Long ShortTerm Memory (LSTM) model. Using this methodology, we achieve an epoch categorical accuracy of 91%. With the help of this system, the communication gap between the hearingand speech-impaired and the general public is meant to be closed.
Hand shape and hand motions will be retrieved for sign language. As a result, hand characteristics are crucial in hand identification. Fingertips, knuckles, and the palm's centre will be sensed. Various soft computing-based algorithms for gesture detection, such as neural networks, hidden markov models, and long short-term memory (LSTM) models, will be employed using the dataset built particularly for ISL. Fig 1 represents all the signs that are in the ISL.
Key Words: ISL, CNN, HMM, MediaPipe Holistic, Image Processing, Machine Learning, Sign Language.
1. INTRODUCTION The international federation of the deaf estimates that over 300 sign languages are used by 70 million deaf individuals worldwide. The deaf and speech-impaired community uses sign language as a structured form of hand gestures incorporating visual motions and signals to aid with daily contact. Recognition of sign languages would aid in lowering social obstacles for sign language users. Using this technology, the speech and hearing impaired community can communicate with the rest of the world. Like spoken language, sign language is not universal and has its own regional variations. Some of the most widely used sign languages worldwide are American Sign Language(ASL), British Sign Language (BSL), Indian Sign Language (ISL), etc. Since most ASL signs aremade with a single hand and are therefore simpler, the majority of studies in this field focus on ASL recognition. The fact that ASL already has a usable
© 2023, IRJET
|
Impact Factor value: 8.226
Fig-1: Signs in the Indian Sign Language
|
ISO 9001:2008 Certified Journal
|
Page 563