Document Type : Review Paper
Authors
Department of MCA, School of Computer Science and IT, Jain (deemed-to-be) University, Bengaluru, India.
Abstract
The move to standardize Indian Sign Language has created an opportunity for researchers to focus on solving local problems, to increase its reach. In this paper, a survey and assessment of the techniques applied to the recognition and conversion of Indian Sign Language are performed. An overview of the techniques used in sign language recognition for Indian Sign Language is provided to understand the status of research in this field. Following this, a comparison of techniques aimed at rendering a more detailed picture of the research results is presented. The challenges faced by researchers, the limitations of current techniques, and the need for improved research in this area are highlighted. With the intent of spurring more in-depth research, key areas within the approaches and techniques in need of improvement are summarized.
Keywords
- Indian sign language
- Sign language recognition
- Gesture recognition
- Convolutional neural network
- Multilayer perceptron
- Neuro-fuzzy system
Main Subjects
- Broumi, S., Dey, A., Talea, M., Bakali, A., Smarandache, F., Nagarajan, D., ... & Kumar, R. (2019). Shortest path problem using Bellman algorithm under neutrosophic environment. Complex & intelligent systems, 5(4), 409-416. https://doi.org/10.1007/s40747-019-0101-8
- Kumar, R., Dey, A., Broumi, S., & Smarandache, F. (2020). A study of neutrosophic shortest path problem. In Neutrosophic graph theory and algorithms(pp. 148-179). IGI Global. DOI: 4018/978-1-7998-1313-2.ch006
- Kumar, R., Edalatpanah, S. A., Jha, S., Broumi, S., Singh, R., & Dey, A. (2019). A multi objective programming approach to solve integer valued neutrosophic shortest path problems. Neutrosophic sets and systems, 24, 34-49. DOI: 5281/zenodo.2595968
- Kumar, R., Edalatpanah, S. A., Jha, S., & Singh, R. (2019). A novel approach to solve gaussian valued neutrosophic shortest path problems. International journal of engineering and advanced technology, 8(3), 347-353.
- Kumar, R., Edaltpanah, S. A., Jha, S., Broumi, S., & Dey, A. (2018). Neutrosophic shortest path problem. Neutrosophic sets and systems, 23, 5-15.
- Pratihar, J., Kumar, R., Dey, A., & Broumi, S. (2020). Transportation problem in neutrosophic environment. In Neutrosophic graph theory and algorithms(pp. 180-212). IGI Global. DOI: 4018/978-1-7998-1313-2.ch007
- Kumar, R., Edalatpanah, S. A., Jha, S., & Singh, R. (2019). A Pythagorean fuzzy approach to the transportation problem. Complex & intelligent systems, 5(2), 255-263. https://doi.org/10.1007/s40747-019-0108-1
- Pratihar, J., Kumar, R., Edalatpanah, S. A., & Dey, A. (2021). Modified Vogel’s approximation method for transportation problem under uncertain environment. Complex & intelligent systems, 7(1), 29-40. https://doi.org/10.1007/s40747-020-00153-4
- Gayen, S., Jha, S., Singh, M., & Kumar, R. (2019). On a generalized notion of anti-fuzzy subgroup and some characterizations. International journal of engineering and advanced technology, 8(3), 385-390.
- Gayen, S., Smarandache, F., Jha, S., & Kumar, R. (2020). Interval-valued neutrosophic subgroup based on interval-valued triple t-norm. In Neutrosophic sets in decision analysis and operations research(pp. 215-243). IGI Global. DOI: 4018/978-1-7998-2555-5.ch010
- Gayen, S., Smarandache, F., Jha, S., Singh, M. K., Broumi, S., Kumar, R. (2019). Introduction to plithogenic subgroup. In Neutrosophic graph theory and algorithms (pp 213-259). IGI Global.
- Gayen, S., Smarandache, F., Jha, S., Singh, M. K., Broumi, S., & Kumar, R. (2020). Soft subring theory under interval-valued neutrosophic environment, Neutrosophic sets and systems, 36, 193-219.
- Gayen, S., Smarandache, F., Jha, S., & Kumar, R. (2020). Introduction to interval-valued neutrosophic subring. Neutrosophic sets and systems, 36, 220-245.
- Gayen, S., Smarandache, F., Jha, S., Singh, M. K., Broumi, S., & Kumar, R. (2020). Introduction to plithogenic hypersoft subgroup. Neutrosophic sets and systems, 33, 208-233.
- Kumar, R., Edalatpanah, S. A., & Mohapatra, H. (2020). Note on “Optimal path selection approach for fuzzy reliable shortest path problem”. Journal of intelligent & fuzzy systems, 39(5), 7653-7656. DOI:3233/JIFS-200923
- Kumar, R., Jha, S., & Singh, R. (2020). A different approach for solving the shortest path problem under mixed fuzzy environment. International journal of fuzzy system applications (IJFSA), 9(2), 132-161. DOI: 4018/IJFSA.2020040106
- Kumar, R., Jha, S., & Singh, R. (2017). Shortest path problem in network with type-2 triangular fuzzy arc length. Journal of applied research on industrial engineering, 4(1), 1-7. DOI: 22105/jarie.2017.48858
- Kumar, R., Edalatpanah, S. A., Jha, S., Gayen, S., & Singh, R. (2019). Shortest path problems using fuzzy weighted arc length. International journal of innovative technology and exploring engineering, 8(6), 724-731.
- Kumar, R., Edalatpanah, S. A., Gayen, S., & Broum, S. (2021). Answer Note “A novel method for solving the fully neutrosophic linear programming problems: suggested modifications”. Neutrosophic sets and systems, 39(1), 148-152.
- Mohapatra, H., Panda, S., Rath, A., Edalatpanah, S., & Kumar, R. (2020). A tutorial on powershell pipeline and its loopholes. International journal of emerging trends in engineering research, 8(4), 975-982.
- Mohapatra, H., Rath, S., Panda, S., & Kumar, R. (2020). Handling of man-in-the-middle attack in wsn through intrusion detection system. International journal of emerging trends in engineering research, 8(5), 1503-1510.
- Mohapatra, H., Debnath, S., & Rath, A. K. (2019). Energy management in wireless sensor network through EB-LEACH. EasyChair Preprint. https://easychair.org/publications/preprint_open/tf5s
- Mohapatra, H., Rath, A. K., Landge, P. B., Bhise, D. H. I. R. A. J., Panda, S., & Gayen, S. A. (2020). A comparative analysis of clustering protocols of wireless sensor network. International journal of mechanical and production engineering research and development (IJMPERD) ISSN (P), 10(3), 8372-8385.
- Mohapatra, H., & Rath, A. K. (2020). Survey on fault tolerance-based clustering evolution in WSN. IET networks, 9(4), 145-155.
- Mohapatra, H., Debnath, S., Rath, A. K., Landge, P. B., Gayen, S., & Kumar, R. (2020). An efficient energy saving scheme through sorting technique for wireless sensor network. International journal, 8(8), 4278-4286.
- Mohapatra, H., & Rath, A. K. (2021). Fault tolerance in WSN through uniform load distribution function. International journal of sensors wireless communications and control, 11(4), 385-394. DOI: https://doi.org/10.2174/2210327910999200525164954
- Mohapatra, H., & Rath, A. K. (2019). Fault tolerance through energy balanced cluster formation (EBCF) in WSN. In Smart innovations in communication and computational sciences(pp. 313-321). Springer, Singapore. https://doi.org/10.1007/978-981-13-2414-7_29
- Mohapatra, H., & Rath, A. K. (2019). Fault tolerance in WSN through PE-LEACH protocol. IET wireless sensor systems, 9(6), 358-365.
- Mohapatra, H. (2018). C programming: practice. Independently published.
- Mohapatra, H., & Rath, A. K. (2020). Fundamentals of software engineering: designed to provide an insight into the software engineering concepts. BPB Publications.
- Mahajan Mangesh, A., Navale Ganesh D., Sharma Hemant P., Nikam K., & Shaikh Riyaj R. (2015). HCR (English) using neural network. Internation journal of advance research and innovative ideas in education, 1(4), 379-385.
- Panda, M., Pradhan, P., Mohapatra, H., & Barpanda, N. K. (2019). Fault tolerant routing in heterogeneous environment. International journal of scientific & technology research, 8(8), 1009-1013.
- Nirgude, V., Mahapatra, H., & Shivarkar, S. (2017). Face recognition system using principal component analysis & linear discriminant analysis method simultaneously with 3d morphable model and neural network BPNN method. Global journal of advanced engineering technologies and sciences, 4(1), 1-6.
- Mohapatra, H., & Rath, A. K. (2020, October). Nub less sensor based smart water tap for preventing water loss at public stand posts. 2020 IEEE microwave theory and techniques in wireless communications (MTTW)(Vol. 1, pp. 145-150). IEEE. DOI: 1109/MTTW51045.2020.9244926
- Mohapatra, H., & Rath, A. K. (2020). IoT-based smart water. IOT technologies in smart-cities: from sensors to big data, security and trust, 63. https://digital- library.theiet.org/content/books/10.1049/pbce128e_ch3
- Mohapatra, H. (2020). Offline drone instrumentalized ambulance for emergency situations. International journal of robotics and automation (IJRA), 9(4), 251-255. DOI: 11591/ijra.v9i4.pp251-255
- Mohapatra, H., & Rath, A. K. (2019). Detection and avoidance of water loss through municipality taps in India by using smart taps and ICT. IET wireless sensor systems, 9(6), 447-457.
- Panda, H., Mohapatra, H., & Rath, A. K. (2020). WSN-based water channelization: an approach of smart water. Smart cities—opportunities and challenges, 58, 157-166.
- Rout, S. S., Mohapatra, H., Nayak, R. K., Tripathy, R., Bhise, D., Patil, S. P., & Rath, A. K. (2020). Smart water solution for monitoring of water usage based on weather condition. International journal of emerging trends in engineering research, 8(9), 5314 – 5320.
- World Health Organization (WHO). Deafness and hearing loss. https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss
- Ethnologue Languages of the Word (n.d.). Sign language. Retrieved from https://www.ethnologue.com/subgroups/sign-language
- (n.d.). Pseudo family: sign language. Retrieved from https://glottolog.org/resource/languoid/id/sign1238
- Tamura, S., & Kawasaki, S. (1988). Recognition of sign language motion images. Pattern recognition, 21(4), 343-353.
- Tamura, S., & Kawasaki, S. (1988). Recognition of sign language motion images. Pattern recognition, 21(4), 343-353. https://doi.org/10.1016/0031-3203(88)90048-9
- Murakami, K., & Taguchi, H. (1991, March). Gesture recognition using recurrent neural networks. Proceedings of the SIGCHI conference on Human factors in computing systems(pp. 237-242). https://doi.org/10.1145/108844.108900
- Charayaphan, C., & Marble, A. E. (1992). Image processing system for interpreting motion in American Sign Language. Journal of biomedical engineering, 14(5), 419-425. https://doi.org/10.1016/0141-5425(92)90088-3
- Starner, T. E. (1995). Visual recognition of American sign language using hidden markov models. Massachusetts inst of tech Cambridge dept of brain and cognitive sciences.
- Lee, C. S., Bien, Z., Park, G. T., Jang, W., Kim, J. S., & Kim, S. K. (1997, July). Real-time recognition system of Korean sign language based on elementary components. Proceedings of 6th international fuzzy systems conference(Vol. 3, pp. 1463-1468). IEEE. DOI: 1109/FUZZY.1997.619759
- Sweeney, G. J., & Downton, A. C. (1997, July). Sign language recognition using a cheremic architecture. 1997 sixth international conference on image processing and its Applications(Vol. 2, pp. 483-486). IET. DOI: 1049/cp:19970940
- Braffort, A. (1996, April). A gesture recognition architecture for sign language. Proceedings of the second annual ACM conference on Assistive technologies(pp. 102-109). https://doi.org/10.1145/228347.228364
- The World Bank (n.d.). Population, Total. Retrieved October 2, 2020 from https://data.worldbank.org/indicator/%20SP.POP.TOTL?%20locations=IN
- Office of the Registrar General & Census Commissioner India. (n.d.). Retrieved October 2, 2020 from https://censusindia.gov.in/2011-common/AboutUs2017.html
- Vasishta, M., Woodward, J., & Wilson, K. (1978). Sign language in India: regional variation within the deaf population. Indian journal of applied linguistics, 4(2), 66-74.
- Aboh, E. O., Pfau, R., & Zeshan, U. (2005). When a wh-word is not a wh-word: the case of Indian Sign Language. In GLOW Asia 2005(pp. 11-43). De Gruyter Mouton. https://www.researchgate.net/publication/254913349_When_a_Wh-word_is_not_a_Wh-word_The_Case_of_Indian_Sign_Language
- Bhatnagar, G. V (2020). Activists hope standardisation of Indian sign language will pave way for inclusivity. Retrieved from https://thewire.in/rights/indian-sign-language-standardisation-inclusivity-rights
- Futane, P. R., & Dharaskar, R. V. (2012, December). Video gestures identification and recognition using fourier descriptor and general fuzzy minmax neural network for subset of indian sign language. 2012 12th international conference on hybrid intelligent systems (HIS)(pp. 525-530). IEEE. DOI: 1109/HIS.2012.6421389
- Woll, B., Sutton-Spence, R. L., & Elton, F. (2001). Multilingualism: the global approach to sign languages. In The sociolinguistics of sign languages(pp. 8-32). Cambridge University Press.
- Nandy, A., Prasad, J. S., Mondal, S., Chakraborty, P., & Nandi, G. C. (2010, March). Recognition of isolated Indian sign language gesture in real time. International conference on business administration and information processing(pp. 102-107). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12214-9_18
- Nandy, A., Mondal, S., Prasad, J. S., Chakraborty, P., & Nandi, G. C. (2010, September). Recognizing & interpreting Indian sign language gesture for human robot interaction. 2010 international conference on computer and communication technology (ICCCT)(pp. 712-717). IEEE. DOI: 1109/ICCCT.2010.5640434
- Fujitsu (n.d.). Fujitsu Malaysia presents the humanoid robot HOAP-2 Show at ACM2004. Retrieved from https://www.fujitsu.com/my/about/resources/news/press-releases/2004/fms-20040902.html
- Rekha, J., Bhattacharya, J., & Majumder, S. (2011, December). Shape, texture and local movement hand gesture features for Indian sign language recognition. 3rd international conference on trendz in information sciences & computing (TISC2011)(pp. 30-35). IEEE. DOI: 1109/TISC.2011.6169079
- Lilha, H., & Shivmurthy, D. (2011, December). Analysis of pixel level features in recognition of real life dual-handed sign language data set. 2011 international conference on recent trends in information systems(pp. 246-251). IEEE. DOI: 1109/ReTIS.2011.6146876
- Gupta, B., Shukla, P., & Mittal, A. (2016, January). K-nearest correlated neighbor classification for Indian sign language gesture recognition using feature fusion. 2016 International conference on computer communication and informatics (ICCCI)(pp. 1-5). IEEE. DOI: 1109/ICCCI.2016.7479951
- Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1), 21-27. DOI:1109/TIT.1967.1053964
- Dalal, N., & Triggs, B. (2005, June). Histograms of oriented gradients for human detection. 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05)(Vol. 1, pp. 886-893). IEEE. DOI: 1109/CVPR.2005.177
- Lowe, D. G. (1999, September). Object recognition from local scale-invariant features. Proceedings of the seventh IEEE international conference on computer vision(Vol. 2, pp. 1150-1157). IEEE. DOI: 1109/ICCV.1999.790410
- Naglot, D., & Kulkarni, M. (2016, August). ANN based Indian Sign Language numerals recognition using the leap motion controller. 2016 international conference on inventive computation technologies (ICICT)(Vol. 2, pp. 1-6). IEEE. DOI: 1109/INVENTIVE.2016.7824830
- Ruck, D. W., Rogers, S. K., Kabrisky, M., Maybeck, P. S., & Oxley, M. E. (1992). Comparative analysis of backpropagation and the extended Kalman filter for training multilayer perceptrons. IEEE transactions on pattern analysis & machine intelligence, 14(06), 686-691. https://ieeexplore.ieee.org/document/141559
- Intwala, N., Banerjee, A., & Gala, N. (2019, March). Indian sign language converter using convolutional neural networks. 2019 IEEE 5th international conference for convergence in technology (I2CT)(pp. 1-5). IEEE. DOI: 1109/I2CT45611.2019.9033667
- LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324. DOI:1109/5.726791
- Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... & Adam, H. (2017). Mobilenets: efficient convolutional neural networks for mobile vision applications. ArXiv. arXiv:1704.04861
- Abraham, E., Nayak, A., & Iqbal, A. (2019, October). Real-Time Translation of Indian Sign Language using LSTM. 2019 global conference for advancement in technology (GCAT)(pp. 1-5). IEEE. DOI: 1109/GCAT47503.2019.8978343
- Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780. DOI:1162/neco.1997.9.8.1735
- Singha, J., & Das, K. (2013). Indian sign language recognition using Eigen value weighted Euclidean distance based classification technique. International journal of advanced computer science and applications, 4(2), 188-195. DOI: 14569/IJACSA.2013.040228
- Singha, J., & Das, K. (2013). Recognition of Indian sign language in live video. International journal of computer applications, 70 (19), 17-22.
- Bhavsar, H., & Trivedi, J. (2019, December). Hand gesture recognition for Indian sign language using skin color detection and correlation-coefficient algorithm with neuro-fuzzy approach. 2019 international conference on advances in computing, communication and control (ICAC3)(pp. 1-5). IEEE. DOI: 1109/ICAC347590.2019.9036832
- Berenji, H. R. (1992). A reinforcement learning—based architecture for fuzzy logic control. International journal of approximate reasoning, 6(2), 267-292. https://doi.org/10.1016/0888-613X(92)90020-Z
- Li, D., Rodriguez, C., Yu, X., & Li, H. (2020). Word-level deep sign language recognition from video: a new large-scale dataset and methods comparison. Proceedings of the IEEE/CVF winter conference on applications of computer vision(pp. 1448-1458). 1109/WACV45572.2020.9093512
- Albanie, S., Varol, G., Momeni, L., Afouras, T., Chung, J. S., Fox, N., & Zisserman, A. (2020, August). BSL-1K: scaling up co-articulated sign language recognition using mouthing cues. European conference on computer vision(pp. 35-53). Springer, Cham. https://doi.org/10.1007/978-3-030-58621-8_3
- Latif, G., Mohammad, N., Alghazo, J., AlKhalaf, R., & AlKhalaf, R. (2019). ArASL: Arabic alphabets sign language dataset. Data in brief, 23, 103777. https://doi.org/10.1016/j.dib.2019.103777
- Passos, B. T., Fernandes, A. M. R., Comunello, E. (n.d.). Brazilian sign language alphabet. Retrieved from https://data.mendeley.com/datasets/k4gs3bmx5k/5
- Yang, S., Jung, S., Kang, H., & Kim, C. (2020, January). The Korean sign language dataset for action recognition. International conference on multimedia modeling(pp. 532-542). Springer, Cham. https://doi.org/10.1007/978-3-030-37731-1_43
- Mangamuri, L. S. T., Jain, L., & Sharmay, A. (2019, September). Two hand Indian sign language dataset for benchmarking classification models of machine learning. 2019 International conference on issues and challenges in intelligent computing techniques (ICICT)(Vol. 1, pp. 1-5). IEEE. DOI: 1109/ICICT46931.2019.8977713