Issuu

https://doi.org/10.22214/ijraset.2022.41578

April 2022

A Survey Paper on Sign Language Recognition Nabah Inamdar1, Zaid Inamdar2, Sohel sheikh3, Prof. Sunil A. Sushir4 1, 2, 3, 4

Department of Computer Engineering, International Institute of information Technology, Pune, India

Abstract: Sign language recognition system is important nowadays. With the development of these systems, there will be easy communication between speech impaired people and normal people. The focus of developing sign language recognition is to create a communication bridge between speech impaired people and normal people for conveying information. This paper presents a survey on recent technology of Sign language recognition both static and dynamic. It shows all methods that were used for Sign language recognition in the different research papers. Keywords: Image processing, noise removal, feature, data extraction, ASL and ISL. I. INTRODUCTION The idea of sign language interpretation is to develop a technique that can identify predefined sign gestures/movements and use them to transfer information. In sign language translator hand gesture recognition method is used, a camera captures the human body movements and communicates the data to a computer that uses the gestures as input to give output as text or speech. The idea of developing a sign language recognition procedure is to develop an interaction between gifted people and normal people and the recognized signs are used to communicate meaningful information. The principal components of a sign gesture recognition process are data acquisition, hand localization, hand characteristics recognition, and gesture identification. The gesture is an indication of physical behavior or emotional expression. It consists of body gestures and hand gestures. It divides into two categories: static gesture and dynamic gesture. Static hand gesture recognition is performed without any additional devices. A person commands the machine using his bare hands, the person’s hand’s images are captured. and analyzed to determine the meaning of hand gestures. To recognize static gestures a common classifier or template match-er is used. A dynamic movement is supposed as a route between the first phase and the last phase. Dynamic gesture recognition characterizes hand movements. It has four features: - velocity, shape, location, and orientation. The use of this system in areas/places like:hospital, malls, bus, and railway ticket counters, etc. II. LITERATURE REVIEW Sharma[1]In this paper data set was compiled using stationary signs from a pre-existing, open-source GitHub repository. This paper uses the CNN Algorithm which makes use of Keras’s deep learning library, offering high accuracy. The model accuracy ranges of 85-95% for the training data-set and in the range of 75-85% for the validation dataset.Jinsu and Rajesh[2]Developed a sign language translation system that could translate Indian sign language to English, Malayalam. The system converts sign language with the help of a flex sensor attached to a glove, Arduino UNO with microcontroller and output displayed on android phone with the help of Bluetooth HC05. The system had an accuracy of 90 percent. Fernandes[3]Developed recognition system for ISL and BSL which consists of both hardware and software approaches. hardware approach consists of gloves with flex sensors and accelerometer, system accuracy was not good. later they developed a software model consisting of code to capture an image; a custom data-set was created by performing preprocessing on an image such as RGB image to the grayscale image than into inverse binary image by using a suitable threshold value. Used CNN algorithm for predicting and achieved an accuracy of 99.98 percentage. Truong, Yang and Tran [4]created a system for ASL (American sign language) to text and speech .firstly in this system pre-processing was done to find feature vector of hand posture image like noise reduction grayscale conversion and then equalize the image histogram later used Haar-like classifier to recognize asl and text output was given .then after they used SAPI 5.3 to convert text into speech. The system had an accuracy of 98.7%. KarayOlan[5]Developed system for sign language recognition˜ using Multi-layer Perceptron Neural Network. Marcel Static Hand Posture Database was used. Two classifiers were used in the system the neural network of Raw Features Classifier and Histogram Features Classifier. Then network was trained with Back-propagation Algorithm to recognize which letter is given for each classifier. The system gave 70% and 85% accuracy rates from respectively Raw Features Classifier and Histogram Features Classifier. Mahesh[6]Developed a system for static sign images of ISL. After performing preprocessing on the images, the sign was recognized by using the LDA algorithm. This system only recognize alphabets of ISL. Kumar, Thankachan and dominic [7] created the system that can take both static and dynamic input. Then features are extracted using Ada-boost, HSV. Later trained the system using an SVM classifier to predict signs.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 |

1696

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538 Volume 10 Issue IV Apr 2022- Available at www.ijraset.com The classification model was built using Java machine learning library JAVA -ML. Used two Classification models Zernike feature vector and trajectory feature vector. Output was in the text. Mariappan[8] created real-time sign language translation system for ISL. In the system, the region of interest are identified and signs are tracked using the segmentation feature of OpenCV. The training and prediction of signs are performed by Fuzzy C means clustering machine learning algorithm (FCM) with the accuracy of 75%. Dixit and Jalal[9] This paper presents a methodology that recognizes the Indian sign language ISL and translates it into normal text. This system uses Combinational parameters of Hu invariant moment and structural shape descriptors are created to form a new feature vector to recognize sign. A multi-class Support Vector Machine (MSVM) is used for training and recognizing signs of ISL.Accuracy was 96 percent.. Kartik, Tejas[10]Developed a software-based Indian sign language recognition system that used the image recognition concept. In this system face detection and elimination were done by using a histogram of oriented gradients followed by an SVM classifier later the system uses an HMM chain for each pose and KNN model to classify each hand poses. The system had an average accuracy of 97.2 percent Priyanka, thilagavathi[11] Their system was a real-time recognition system. The system used a Canny edge detector for feature extraction. After segmentation and hand tracking, the latter ANN model/ architecture is used for recognizing the sign and displaying the corresponding text. The whole system was implemented using MatLab 2014 version Chen and Zang[12] Implemented sign language recognition system with Kinect sensor launch to by Microsoft. This sensor can capture the color depth and skeleton frames. In this system Kinect sensor was used to capture signs, later HOG and SVM algorithms way used to recognize hand shape features. The recognition rate was 89.6 percent. Mayand, Piyush[13] Developed a system for recognizing American sign language. Preprocessing of the MNIST the data set is done. A model was developed using a combination of convolutional, Max pooling, flattering layers in Keras, mainly the model depended on CNN. The accuracy of the model was 99.6 percent. Their data set didn’t include J and z alphabets. Roa, Kishore and Sastry[14] The paper proposed a sign recognition system that is based on CNN architectures. Mainly they trained the system three times with different data-set sizes. A stochastic Technique was also used. This paper also suggested that CNN models are much better than SLR models. Huang, Zhou [15] Developed a 3D CNN model for sign language recognition. 3Dconvolutional neural network (CNN) extracts discriminative spatial-temporal features from a raw video stream. To boost the performance, multi-channel video streams, including color information, depth clue, and body joint positions, are used as input to the 3D CNN to integrate color, depth, and trajectory information. They use a multilayer perceptron classifier to classify these feature representations. For comparison, they evaluated both 3D CNN and GMM-HMM on the same dataset. Accuracy for GMM-HMM is 90.8 percent and for CNN is 94.2 percent. III. Technology/ Hardware

Author

TABLE Description This paper makes use of stationary dataset. This system uses CNN algo with Keras deep learning library. Accurarcy of this system is 85-90 percent

Sharma

CNN with Keras library

Jinsu and Rajesh

Flex sensor with Arduino UNO

Fernandes

Flex sensor, accelerometer CNN Algo.

They proposed two System for ISL and BSL. Hardware approach with flex sensors and accelerometer Software approach with CNN .software system with best accuracy

Truong, Yang and Tran

Haar-like classifier, SAPI 5.3

They developed system for ASL. Used Haar-like classifier for sign recognition that gave output in text, later used SAPI 5.3 to convert text to speech.

KarayOlan˜

MLPs, BPNN

Developed system for sign language recognition using MLP. BPNN is used to train the system

Mahesh

LDA

They proposed system for ISL that uses flex sensor attached to glove with Arduino UNO

Proposed for Indian sign language recognition with help of LDA algorithm .

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 |

1697

SVM classifier, JAVA-ML

Developed a system that can take static and dynamic input. system was trained using SVM classifier to predict sign. The classification was built in JAVA-ML library

FCM

Proposed real-time sign language translation system for ISL that uses FCM for predicting sign. System Accuracy 75%

Dixit and Jalal

MSVM

This paper presents a methodology that recognizes ISL and translates it into normal text. System uses MSVM for prediction with accuracy of 96 %.

Kartik, Tejas

HMM, KNN

Proposed a system for recognizing ISL that uses HMM and KNN model for prediction.system accuracy is 97.2 %.

Priyanka, thilagavathi

ANN Algo. MATLAB

They proposed System for ISL with use of ANN model/architecture. system was implemented in MATLAB.

Chen and Zang

Kinect sensor, HOG and SVM algo.

the proposed system uses Kinect sensor for capturing signs, later with using of Hog and SVM algorithm signs are predicted with rate of 89%

Mayand, Piyush

Keras, CNN

The system depend on CNN architecture for ASL recognition with an accuracy of 99.6%.data set didn’t contain J and alphabets.

Roa, Kishore and Sastry

CNN

The system depend on CNN architecture for sign recognition. This Paper also proposed that CNN systems are better that SLR models.

GMMHMM, CNN

Developed a system for sign Recognition with use of GMM-HMM and CNN algorithm. Accuracy for HMM-GMM is 90.8% and for CNN model is 94.2%

Kumar, Thankachan and dominic

Mariappan

Huang, Zhou

IV. CONCLUSION The Sign language recognition system is developed to bridge the communication gaps between speech-impaired people and normal people. In this survey, we briefly discussed various systems. The System consists of two types of hardware and software in which software system is much cost-effective, accurate. As in hardware systems they used flex sensor, Kinect sensor, etc. A software system can be implemented by using various algorithms/architectures like SLR, SVM, GMM-HMM, FCM, ANN, CNN, etc. In the software system, we found that LDA and CNN algorithms had the highest recognition rate and the CNN algorithm was used the most. Most systems used sign language alphabets not actual words or sentences. REFERENCES [1] [2]

Aishwarya Sharma, Dr. Siba Panda, and Prof. Saurav Verma. ” Sign Language to Speech Translation” 11th ICCCNT 2020. Jinsu Kunjumon, Dr. Rajesh Kannan Megalingam. ” Hand Gesture Recognition System For Translating Indian Sign Language Into Text And Speech” Second International Conference on Smart Systems and Inventive Technology (ICSSIT) 2019. [3] Lance Fernandes, Prathamesh Dalvi, Akash Junnarkar and Professor Manisha Bansode” Convolutional Neural Network based Bidirectional Sign Language Translation System ” Proceedings of the Third International Conference on Smart Systems and Inventive Technology (ICSSIT 2020) [4] Vi .N.T. Truong, Chuan-Kai Yang and Quoc-Viet Tran. ” A Translator for American Sign Language to Text and Speech “ IEEE 5th Global Conference on Consumer Electronics 2016. [5] Tulay Karay¨ Olan,˜ Ozkan Kilic¸ ” SIGN LANGUAGE RECOGNITION¨ USING “ Fifth International Conference on Intelligent Computing and Control Systems (ICICCS) 2021. [6] Mahesh Kumar N B. ” Conversion of Sign Language into Text” International Journal of Applied Engineering Research 2018. [7] Anup Kumar, Karun Thankachan and Mevin M. Dominic ” Sign Language Recognition “3rd InCI Conf. on Recent Advances in Information Technology RAIT-2016. [8] Muthu Mariappan H, Dr Gomathi V ” Real-Time Recognition of Indian Sign Language “Second International Conference on Computational Intelligence in Data Science (ICCIDS-2019). [9] Karishma Dixit, Anand Singh Jalal ” Automatic Indian Sign Language Recognition System “ 3rd IEEE International Advance Computing Conference (IACC) 2013 [10] Kartik Shenoy, Tejas Dastane, Varun Rao, Devendra Vyavaharkar” Realtime Indian Sign Language (ISL) Recognition “9th ICCCNT 2018.

1698

International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538 Volume 10 Issue IV Apr 2022- Available at www.ijraset.com [11] Priyanka C Pankajakshan, Thilagavathi B” Sign Language Recognition System “IEEE Sponsored 2nd International Conference on Innovations in Information Embedded and Communication Systems 2015. [12] Yuqian Chen, Wenhui Zhang ” Research and Implementation of Sign Language Recognition Method Based on Kinect “ 2nd IEEE International Conference on Computer and Communications 2016. [13] Mayand Kumar, Piyush Gupta, Rahul Kumar Jha, Aman Bhatia, Khushi Jha and Bickey Kumar Shah ” SIGN LANGUAGE ALPHABET RECOGNITION USING CONVOLUTION NEURAL NETWORK “ Fifth International Conference on Intelligent Computing and Control Systems (ICICCS) 2021. [14] G.Anantha Rao, K.Syamala, P.V.V.Kishore , A.S.C.S.Sastry” Deep Convolutional Neural Networks for Sign Language Recognition “ Conference on Signal Processing And Communication Engineering Systems (SPACES) 2018 [15] Jie Huang, Wengang Zhou, Houqiang Li, and Weiping Li. ” SIGN LANGUAGE RECOGNITION USING 3D CONVOLUTIONAL NEURAL NETWORKS “ IEEE 5th Global Conference on Consumer Electronics 2019.

1699