Citation :

Hirkani Pathak, Milind Penurkar, Aatish Daryapurkar, "Robust Cross-Lingual Legal Question Pair Similarity Analysis for English-Marathi," International Journal of Engineering Trends and Technology (IJETT), vol. 74, no. 10, pp. 250-265, 2026. Crossref, https://doi.org/10.14445/22315381/IJETT-V74I10P118

Abstract

Cross-lingual legal assistants in India process user queries in both English and Marathi while adhering to deployment speed requirements. A single model for cross-lingual and lightweight matching of legal question pairs is proposed, which consists of three modules: (i) contextualized semantics via a multilingual Transformer encoder, (ii) language-independent legal anchoring signals in the form of statutes/sections/articles in Latin and Devanagari scripts, and (iii) a sentence-embedding branch learned via knowledge distillation from teacher models. The model is trained using a joint classification and ranking loss in combination with a citation format invariance regularizer. The model’s performance is tested using a deployment-faithful protocol in which the decision threshold is tuned on English validation and then kept fixed for cross-lingual evaluation. On the English benchmark set comprising 1,233 pairs, the model achieves an F1 score of 0.964 and an AUC score of 0.996 in 1.24 ms per pair. An LLM classifier achieves a higher F1 score of 0.986, but at a slower rate of 14.6 ms per pair. For the cross-lingual setting where a fixed English-selected threshold is used, the approach achieves an F1 value of 0.935 for English and 0.852 for Marathi (ΔF1 = 0.083), while keeping a consistent rank quality (MRR ≈ 0.82). Given a small amount of labeled samples of Marathi, the F1-score for Marathi rises to 0.867, with ΔF1 equaling 0.055. This result implies that the use of multilingual semantics along with light-weight legal anchoring leads to solid cross-lingual performance in real-world legal information retrieval and question-answering applications.

Keywords

English-Marathi, Legal citation normalization, Legal question pair similarity, Multilingual transformers, Semantic similarity.

References

1. Jacob Devlin et al., “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Minneapolis, Minnesota, vol. 1, pp. 4171-4186, 2019.
   [CrossRef] [Google Scholar] [Publisher Link]
2. Nils Reimers, and Iryna Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9^th International Joint Conference on Natural Language Processing, Association for Computational Linguistics, Hong Kong, China, pp. 3982-3992, 2019.
   [CrossRef] [Google Scholar] [Publisher Link]
3. Ilias Chalkidis et al., “LEGAL-BERT: The Muppets Straight Out of Law School,” Findings of the Association for Computational Linguistics: EMNLP 2020, Association for Computational Linguistics, pp. 2898-2904, 2020.
   [CrossRef] [Google Scholar] [Publisher Link]
4. Shounak Paul et al., “Pre-Trained Language Models for the Legal Domain: A Case Study on Indian Law,” arXiv preprint arXiv, pp. 1-10, 2022.
   [CrossRef] [Google Scholar] [Publisher Link]
5. M. Gatford et al., “Okapi at TREC-3,” Proceedings of the Third Text REtrieval Conference, Published in NIST Special Publication 500-225, 1994.
   [Publisher Link]
6. Rodrigo Nogueira, and Kyunghyun Cho, “Passage Re-ranking with BERT,” arXiv preprint arXiv, pp. 1-5, 2019.
   [CrossRef] [Google Scholar] [Publisher Link]
7. Alexis Conneau et al., “Unsupervised Cross-Lingual Representation Learning at Scale,” Proceedings of the 58^th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, pp. 8440-8451, 2020.
   [CrossRef] [Google Scholar] [Publisher Link]
8. Fangxiaoyu Feng et al., “Language-Agnostic BERT Sentence Embedding,” Proceedings of the 60^th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Dublin, Ireland, pp. 878-891, 2022.
   [CrossRef] [Google Scholar] [Publisher Link]
9. Liang Wang et al., “Multilingual E5 Text Embeddings: A Technical Report,” arXiv preprint arXiv, pp. 1-6, 2024.
   [CrossRef] [Google Scholar] [Publisher Link]
10. Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant, “SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking,” Proceedings of the 44^th International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery, New York, NY, United States, pp. 2288-2292, 2021.
    [CrossRef] [Google Scholar] [Publisher Link]
11. Jay Gala et al., “Indic Trans2: Towards High-Quality and Accessible Machine Translation Models for All 22 Scheduled Indian Languages,” arXiv preprint arXiv, pp. 1-95, 2023.
    [CrossRef] [Google Scholar] [Publisher Link]
12. Binyuan Hui et al., “Qwen2.5 Technical Report,” arXiv preprint arXiv, pp. 32, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
13. Tim Dettmers et al., “QLoRA: Efficient Finetuning of Quantized LLMs,” Advances in Neural Information Processing Systems, vol. 36, pp. 10088-10115, 2023.
    [CrossRef] [Google Scholar] [Publisher Link]
14. Hugo Mentzingen, Nuno António, and Fernando Bacao, “Effectiveness in Retrieving Legal Precedents: Exploring Text Summarization and Cutting-Edge Language Models Toward a Cost-Efficient Approach,” Artificial Intelligence and Law, pp. 1-21, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
15. Yunting Zhang et al., “Kalt: Generating Adversarial Explainable Chinese Legal Texts,” Machine Learning, vol. 113, no. 9, pp. 6341-6371, 2024.
    [CrossRef] [Google Scholar] [Publisher Link]
16. Yunting Zhang et al., “Chinese Legal Adversarial Text Generation based on Interpretable Perturbation Strategies,” World Wide Web, vol. 28, no. 2, pp. 1-23, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
17. Aniket Deroy, Kripabandhu Ghosh, and Saptarshi Ghosh, “Investigating Legal Question Generation using Large Language Models,” Artificial Intelligence and Law, vol. 34, no. 3, pp. 775-813, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
18. Gaspar Dugac, and Tilmann Altwicker, “Classifying Legal Interpretations using Large Language Models,” Artificial Intelligence and Law, pp. 1-19, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
19. Priyanka Prabhakar, and Peeta Basa Pati, “Enhancing Indian Legal Judgment Classification with Embeddings, Feature Selection, and Ensemble Strategies,” Artificial Intelligence and Law, vol. 34, no. 2, pp. 523-564, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
20. Ilias Chalkidis et al., “LexGLUE: A Benchmark Dataset for Legal Language Understanding in English,” Proceedings of the 60^th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Dublin, Ireland, pp. 4310-4330. 2022.
    [CrossRef] [Google Scholar] [Publisher Link]
21. Yoshinobu Kano et al., “COLIEE-2018: Evaluation of the Competition on Legal Information Extraction and Entailment,” JSAI International Symposium on Artificial Intelligence, Yokohama, Japan, vol. 1, pp. 177-192, 2019.
    [CrossRef] [Google Scholar] [Publisher Link]  
22. Abbas Brashi, and Muhammad M.M. Abdel Latif, “Teaching Arabic-English Legal Translation using Model Texts: A Mixed-Method Study,” Humanities and Social Sciences Communications, vol. 11, no. 1, pp. 1-8, 2024.
    [CrossRef] [Google Scholar] [Publisher Link]
23. Omar Khattab, and Matei Zaharia, “ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction Over BERT,” Proceedings of the 43^rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 39-48, 2020.
    [CrossRef] [Google Scholar] [Publisher Link]
24. Vladimir Karpukhin et al., “Dense Passage Retrieval for Open-Domain Question Answering,” Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 6769-6781, 2020.
    [CrossRef] [Google Scholar] [Publisher Link]
25. Kaitao Song et al., “MPNet: Masked and Permuted Pre-Training for Language Understanding,” Advances in Neural Information Processing Systems, vol. 33, pp. 16857-16867, 2020.
    [Google Scholar] [Publisher Link]
26. Wenhui Wang et al., “MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers,” Advances in Neural Information Processing Systems, vol. 33, pp. 5776-5788, 2020.
    [Google Scholar] [Publisher Link]
27. Kevin Clark et al., “ELECTRA: Pre-Training Text Encoders as Discriminators Rather than Generators,” arXiv preprint arXiv, pp. 1-18, 2020.
    [CrossRef] [Google Scholar] [Publisher Link]
28. Iz Beltagy, Matthew E. Peters, and Arman Cohan, “Longformer: The Long-Document Transformer,” arXiv preprint arXiv, pp. 1-17, 2020.
    [CrossRef] [Google Scholar] [Publisher Link]
29. Qihui Zhao, “Legal Judgment Prediction Via Legal Knowledge Extraction and Fusion,” Journal of King Saud University Computer and Information Sciences, vol. 37, no. 3, pp. 1-16, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
30. Alan Khoja et al., “Automated Consistency Analysis for Legal Contracts,” Artificial Intelligence and Law, vol. 34, no. 3, pp. 881-922, 2026.
    [CrossRef] [Google Scholar] [Publisher Link]
31. Supreme Court of India, YouTube Channel, 1950. [Online]. Available: https://www.youtube.com/@SupremeCourtofIndia-1950
32. Ilias Chalkidis et al., “Extreme Multi-Label Legal Text Classification: A Case Study in EU Legislation,” Proceedings of the Natural Legal Language Processing Workshop 2019, Association for Computational Linguistics, Minneapolis, Minnesota, pp. 78-87, 2019.
    [CrossRef] [Google Scholar] [Publisher Link]