Attention-based Sequence to Sequence Learning for Machine Translation of Low Resourced Indic Languages – A case of Sanskrit to Hindi

Attention-based Sequence to Sequence Learning for Machine Translation of Low Resourced Indic Languages – A case of Sanskrit to Hindi

  IJETT-book-cover           
  
© 2021 by IJETT Journal
Volume-69 Issue-9
Year of Publication : 2021
Authors : Vishvajit Bakarola, Jitendra Nasriwala
DOI :  10.14445/22315381/IJETT-V69I9P227

How to Cite?

Vishvajit Bakarola, Jitendra Nasriwala, "Attention-based Sequence to Sequence Learning for Machine Translation of Low Resourced Indic Languages – A case of Sanskrit to Hindi," International Journal of Engineering Trends and Technology, vol. 69, no. 9, pp. 230-235, 2021. Crossref, https://doi.org/10.14445/22315381/IJETT-V69I9P227

Abstract
Deep Learning techniques are powerful in mimicking humans in a particular set of problems. They have achieved a remarkable performance in complex learning tasks. Deep learning inspired Neural Machine Translation (NMT) is a proficient technique that outperforms traditional machine translation. Performing machine-aided translation on Indic languages has always been a challenging task considering their rich and diverse grammar. The neural machine translation has shown quality results compared to the traditional machine translation approaches. The fully automatic machine translation becomes problematic when it comes to low-resourced languages, especially with Sanskrit. This paper presents attention mechanism-based neural machine translation by selectively focusing on a particular part of language sentences during translation. The work shows the construction of Sanskrit to Hindi bilingual parallel corpus with nearly 10K samples and having 178,000 tokens. The neural translation model equipped with an attention mechanism has been trained on Sanskrit to Hindi parallel corpus. The approach has shown the significance of attention mechanisms to overcome long-term dependencies, primarily associated with low resources Indic languages. The paper shows the attention plots on testing data to demonstrate the alignment between source and translated words. For the evaluation of the translated sentences, manual score-based human evaluation and automatic evaluation metric-based techniques have been adopted. The attention mechanism-based neural translation hasachieved88% accuracy in human evaluation and aBLEU score of 0.92 on Sanskrit to Hindi translation.

Keywords
Attention Mechanism, Low-resourced languages, Neural Machine Translation, Sanskrit, Sequence to Sequence Learning.

Reference
[1] D. Jitendra Nasriwala and V. Bakarola, Computational Representation of Paninian Rules of Sanskrit Grammar for Dictionary-Independent Machine Translation, 1046 (2019).
[2] A. C. Woolner, Introduction to Prakrit.pdf. University of the Panjab, Lahore, (1917).
[3] P. Kiparsky, On the Architecture of Panini`s Grammar," in Sanskrit Computational Linguistics: First and Second International Symposia Rocquencourt, France, October, (2007) 29-31. Providence, RI, USA, May 15-17, 2008 Revised Selected and Invited Papers, Berlin, Heidelberg: Springer-Verlag, (2009) 33–94.
[4] B. Panchal, V. Bakrola, and D. Dabhi, An Efficient Approach of Knowledge Representation Using Paninian Rules of Sanskrit Grammar BT - Recent Findings in Intelligent Computing Techniques, (2018) 199–206.
[5] V. Mishra and R. B. Mishra, English to Sanskrit Machine Translation System: A Rule-Based Approach, Int. J. Adv. Intell. Paradigm., 4(2) (2012) 168-184. doi: 10.1504/IJAIP.2012.048144.
[6] P. Bahadur, A. Jain, and D. S. Chauhan, Architecture of English to Sanskrit machine translation, IntelliSys, 2015 - Proc. 2015 SAI Intell. Syst. Conf, (2015) 616-624. doi: 10.1109/IntelliSys.2015.7361204.
[7] P. Koehn, Statistical Machine Translation. Cambridge University Press, (2010).
[8] P. D. Mane and A. Hirve, Study of Various Approaches in Machine Translation for Sanskrit Language,2 (2013) 383–387.
[9] U. S. T. Tanveer Siddiqui, Natural Language Processing and Information Retrieval. Oxford University Press, (2015).
[10] P. R. V. Veda, Computer Processing of Sanskrit, C-DAC, Pune, (1992).
[11] H. S. Sreedeepa and S. M. Idicula, Interlingua based Sanskrit-English machine translation, Proc. IEEE Int. Conf. Circuit, Power Comput. Technol. ICCPCT, (2017) doi: 10.1109/ICCPCT.2017.8074251.
[12] M. Singh, R. Kumar, and I. Chana, Corpus-based Machine Translation System with Deep Neural Corpus-based Machine Translation System with Deep Neural Network for Sanskrit to Hindi Translation Network for Sanskrit to Hindi Translation, Procedia Comput. Sci., 167 (2020). doi: 10.1016/j.procs.2020.03.306, 2534-2544.
[13] N. Koul and S. S. Manvi., A proposed model for neural machine translation of Sanskrit into English, Int. J. Inf. Technol., (2019). doi: 10.1007/s41870-019-00340-8.
[14] A. Shewalkar, D. Nyavanandi, and S. A. Ludwig, Performance Evaluation of Deep Neural Networks Applied to Speech Recognition: RNN, LSTM, and GRU, J. Artif. Intell. Soft Comput. Res., 9(4) 235-245. doi: https://doi.org/10.2478/jaiscr-2019-0006.
[15] Y. Hu, A. Huber, and S.-C. Liu, Overcoming the vanishing gradient problem in plain recurrent networks. (2018) [Online]. Available: https://openreview.net/forum?id=Hyp3i2xRb.
[16] I. Sutskever, O. Vinyals, and Q. V. Le, Sequence to sequence learning with neural networks, Adv. Neural Inf. Process. Syst., 4 (2014) 3104–3112.
[17] M. T. Luong, H. Pham, and C. D. Manning, Effective approaches to attention-based neural machine translation, Conf. Proc. - EMNLP 2015 Conf. Empir. Methods Nat. Lang. Process., (2015) doi: 10.18653/v1/d15-116.
[18] D. Bahdanau, K. H. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate, 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., (2015) 1-15.
[19] D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, (2017).
[20] Rashi Kumar, Piyush Jha, and Vineet Sahula, An Augmented Translation Technique for Low Resource Language Pair: Sanskrit to Hindi Translation. In Proceedings of 2019 2nd International Conference on Algorithms, Computing and Artificial Intelligence (ACAI’19). Sanya, China, (2019). https://doi.org/10.1145/3377713.3377774.
[21] Kishore Papineni, Salim Roukos, Todd Ward and Wei-Jing Zhu, BLEU – A Method for Automatic Evaluation of Machine Translation, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia, July (2002) 311-318.