A Deep Learning Framework for Real-Time Sign Language Recognition Based on Transfer Learning

A Deep Learning Framework for Real-Time Sign Language Recognition Based on Transfer Learning
  IJETT-book-cover           
  
© 2022 by IJETT Journal
Volume-70 Issue-6
Year of Publication : 2022
Authors : Vijeeta Patil, Sujatha C, Shridhar Allagi, Balachandra Chikkoppa
DOI : 10.14445/22315381/IJETT-V70I6P204

How to Cite?

Vijeeta Patil, C. Sujatha, Shridhar Allagi, Balachandra Chikkoppa, "A Deep Learning Framework for Real-Time Sign Language Recognition Based on Transfer Learning," International Journal of Engineering Trends and Technology, vol. 70, no. 6, pp. 32-41, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I6P204

Abstract
Hearing Impairment is common among all the groups irrespective of gender, location, and genes. The cause of it may vary according to the geographical and biological aspects. However, for the betterment of humankind, the solution to this is obvious, either through medical or with the help of technology. Sign Language recognition is a worldwide concern across the globe. The use of technology has a scope in aiding the necessary help in recognition of sign language. The major challenge lies in detecting and understanding signs, as the language differs across the various geographical regions, and there are no specific rules for understanding them. Hence, this research article uses a transfer learning algorithm with TensorFlow object detection to recognize the sign language. The proposed model has achieved an accuracy of around 97.87% for different types of sentences used in the experimentation. The main advantage of the proposed model is that it is feasible to use different sign languages, such as American sign language and any regional sign language. The system is helpful to the deaf and dumb community's society and encourages such people's upliftment.

Keywords
Machine Learning, Transfer Learning, Sign Language, Object Recognition, and American Sign Language.

Reference
[1] V. A. & R. R, A Deep Convolutional Neural Network Approach for Static Hand Gesture Recognition, Procedia Computer Science. 171 (2020) 2353–2361. https://doi.org/10.1016/j.procs.2020.04.255
[2] Sharma P & Anand R. S, A Comprehensive Evaluation of Deep Models and Optimizers for Indian Sign Language Recognition, Graphics and Visual Computing. 5 (2021) 200032. https://doi.org/10.1016/j.gvc.2021.200032.
[3] Wadhawan A & Kumar P, Deep Learning-Based Sign Language Recognition System for Static Signs, Neural Computing and Applications. 32(12) (2020) 7957–7968. https://doi.org/10.1007/s00521-019-04691-y.
[4] Kolivand H, Joudaki S, Sunar M. S & Tully D, A New Framework for Sign Language Alphabet Hand Posture Recognition Using Geometrical Features through Artificial Neural Network Part 1, Neural Computing and Applications. 33(10) (2020) 4945–4963. https://doi.org/10.1007/s00521-020-05279-7.
[5] Ahuja R, Jain D, Sachdeva D, Garg A & Rajput C, Convolutional Neural Network Based American Sign Language Static Hand Gesture Recognition, International Journal of Ambient Computing and Intelligence. 10(3) (2019) 60–73. https://doi.org/10.4018/ijaci.2019070104
[6] Mohammed A. A. Q, Lv J, Islam M. S & Sang Y, Multi-Model Ensemble Gesture Recognition Network for High-Accuracy Dynamic Hand Gesture Recognition, Journal of Ambient Intelligence and Humanized Computing. (2022). https://doi.org/10.1007/s12652-021- 03546-6
[7] Ghotkar A. S & K. Kharate D. G, Study of Vision-Based Hand Gesture Recognition Using Indian Sign Language, International Journal on Smart Sensing and Intelligent Systems. 7(1) (2014) 96–115. https://doi.org/10.21307/ijssis-2017-647.
[8] Abdulhussein A & Raheem F, Hand Gesture Recognition of Static Letters American Sign Language ASL Using Deep Learning, Engineering and Technology Journal. 38(6) (2020) 926–937. https://doi.org/10.30684/etj.v38i6a.533.
[9] Al-Hammadi M, Muhammad G, Abdul W, Alsulaiman M, Bencherif M. A, Alrayes T. S, Mathkour H & Mekhtiche M. A, Deep Learning-Based Approach for Sign Language Gesture Recognition with Efficient Hand Gesture Representation, IEEE Access. 8 (2020) 192527–192542. https://doi.org/10.1109/access.2020.3032140.
[10] Gangrade J & Bharti J, Vision-based Hand Gesture Recognition for Indian Sign Language Using Convolution Neural Network, IETE Journal of Research. (2020) 1–10. https://doi.org/10.1080/0338342772063.2020.18.
[11] Ghotkar A & Kharate G, Interpretation Algorithm for Indian Sign Language Using Microsoft Kinect Sensor, JouDynamic Hand Gesture Recognition for Sign Words and Novel Sentence International of Pattern Recognition Research. 10(1) (2015) 24–38. https://doi.org/10.13176/11.626.
[12] Joudaki S & Rehman A, Dynamic Hand Gesture Recognition of Sign Language Using Geometric Features Learning, International Journal of Computational Vision and Robotics. 12(1) (2022) 1. https://doi.org/10.1504/ijcvr.2022.119239
[13] M. Darwish S, M. Madbouly M & B. Khorsheed M, Hand Gesture Recognition for Sign Language: A New Higher Order Fuzzy HMM Approach Hand Gesture Recognition for Sign Language: A New Higher Order Fuzzy HMM Approach, International Journal of Engineering and Technology. 8(3) (2016) 157–164. https://doi.org/10.7763/ijet.2016.v6.877.
[14] Sharma S & Singh S, Vision-Based Hand Gesture Recognition Using Deep Learning for the Interpretation of Sign Language, Expert Systems with Applications. 182 (2021) 115657. https://doi.org/10.1016/j.eswa.2021.115657.
[15] [Online]. Available: https://www.populationu.com/india-population.