Enhancing Credit Card Registration Form Processing with Fine-Tuned Transformer-Based OCR Models
Enhancing Credit Card Registration Form Processing with Fine-Tuned Transformer-Based OCR Models |
||
 |
 |
|
| © 2025 by IJETT Journal | ||
| Volume-73 Issue-6 |
||
| Year of Publication : 2025 | ||
| Author : Rafi Surya, Amalia Zahra | ||
| DOI : 10.14445/22315381/IJETT-V73I6P123 | ||
How to Cite?
Rafi Surya, Amalia Zahra, "Enhancing Credit Card Registration Form Processing with Fine-Tuned Transformer-Based OCR Models," International Journal of Engineering Trends and Technology, vol. 73, no. 6, pp.281-291, 2025. Crossref, https://doi.org/10.14445/22315381/IJETT-V73I6P123
Abstract
The widespread adoption of credit cards has significantly advanced universal payment methods. As the popularity of credit cards continues to grow, so does the number of applications that need to be processed. Conventional application processes rely on manual data entry from physical forms, resulting in significant inefficiencies. Therefore, an optimized registration scheme is needed to solve this problem. Optical Character Recognition (OCR) is a potential method to solve this problem. Automating data entry with OCR makes the registration process faster and easier since the user input is minimal. This paper reports experiments on fine-tuning a transformer-based OCR model, TrOCR, for credit card application forms. The enhanced model is tested on the IAM dataset and later on real forms, i.e., credit card application forms from Bank XYZ. The results from the experiments on 50 credit card registration forms are gathered in Table. The model that achieved the lowest CER of 0.3620 was TrOCR_Model_4, which was trained with data pre-processing and tuning beam search parameters. The results indicate that handwritten text is correctly recognized by this modified TrOCR model, with relatively low CER rates, which allows for a more efficient credit card application process.
Keywords
Optical Character Recognition, Transformer-based OCR, Credit card registration forms, Handwritten text recognition, Character Error Rate.
References
[1] Bank Indonesia, Jumlah Pengguna Kartu Kredit di Indonesia, 2024. [Online]. Available: https://www.bi.go.id/id/statistik/ekonomi-keuangan/spip/Pages/SPIPSeptember-2023.aspx
[2] Bank Indonesia, Regional Credit Card/KK, Indonesia, 2024. [Online]. Available: https://www.bi.go.id/id/statistik/ekonomi-keuangan/spip/Documents/TABEL_5d.pdf#search=kartu%20kredit
[3] Ashish Vaswani et al., “Attention is All You Need,” arXiv Preprint, pp. 1-15, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Minghao Li et al., “TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models,” arXiv Preprint, pp. 1-10, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Daniel Hernandez Diaz et al., “Rethinking Text Line Recognition Models,” arXiv Preprint, pp. 1-11, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Bhavesh Kumar, “ViT Cane: Visual Assistant for the Visually Impaired,” arXiv Preprint, pp. 1-4, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Jacob Devlin et al., “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding,” arXiv Preprint, pp. 1-16, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Cheng Shuiqi, Guo Yanliang, and Chen Yue, “A Bank Card Number Recognition Method Based on Convolutional Neural Network,” International Conference on Communications, Information System and Computer Engineering, Beijing, China, pp. 826-830, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Yuanxue Xin, Pengfei Shi, and Song Han, “An Automatic Location and Recognition Method for Bank Card Number,” Proceedings of the 2019 International Conference on Robotics, Intelligent Control and Artificial Intelligence, Shanghai China, pp. 728-732, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Gege Sun, and Fucheng You, “Bank Card Number Recognition System based on Deep Learning,” Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer Engineering, Xiamen China, pp. 745-749, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Mukesh Jha et al., “Automation of Cheque Transaction using Deep Learning and Optical Character Recognition,” International Conference on Smart Systems and Inventive Technology, Tirunelveli, India, pp. 309-312, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Zhijie Lin et al., “SNRNet: A Deep Learning-Based Network for Banknote Serial Number Recognition,” Neural Process Letters, vol. 52, no. 2, pp. 1415-1426, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[13] William Watson, and Bo Liu, “Financial Table Extraction in Image Documents,” Proceedings of the First ACM International Conference on AI in Finance, New York, pp. 1-8, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Firhan Maulana Rusli, Kevin Akbar Adhiguna, and Hendy Irawan, “Indonesian ID Card Extractor Using Optical Character Recognition and Natural Language Post-Processing,” arXiv Preprint, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Emilda Zhang, Vincent Ardyan Putra, and Gede Putra Kusuma, “Improving Optical Character Recognition Accuracy for Indonesia Identification Card Using Generative Adversarial Network,” Journal of Theoretical and Applied Information Technology, vol. 100, no. 8, pp. 2424-2437, 2022.
[Google Scholar] [Publisher Link]
[16] Xin Ma, and Wei Qi Yan, “Banknote Serial Number Recognition Using Deep Learning,” Multimedia Tools and Applications, vol. 80, no. 12, pp. 18445-18459, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Shriansh Srivastava et al., “Optical Character Recognition on Bank Cheques Using 2D Convolution Neural Network,” Applications of Artificial Intelligence Techniques in Engineering, vol. 697, pp. 589-596, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[18] R. Parthiban, R. Ezhilarasi, and D. Saravanan, “Optical Character Recognition for English Handwritten Text Using Recurrent Neural Network,” International Conference on System, Computation, Automation and Networking, Pondicherry, India, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Adith Narayan, and Raja Muthalagu, “Image Character Recognition using Convolutional Neural Networks,” Seventh International conference on Bio Signals, Images, and Instrumentation, Chennai, India, pp. 1-5, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Yefan Gao et al., “Bank Card Number Recognition System based on Deep Learning,” Proceedings of the 2019 International Conference on Artificial Intelligence and Advanced Manufacturing, Dublin, Ireland, pp. 1-6, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[21] U.-V. Marti, and H. Bunke, “The IAM-Database: An English Sentence Database for Offline Handwriting Recognition,” International Journal on Document Analysis and Recognition, vol. 5, no. 1, pp. 39-46, 2002.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Hugo Touvron et al., “Training Data-Efficient Image Transformers & Distillation through Attention,” arXiv Preprint, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Yinhan Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” arXiv Preprint, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Da Chang, and Yu Li, “Mixed Text Recognition with Efficient Parameter Fine-Tuning and Transformer,” arXiv Preprint, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Felix Stahlberg, and Bill Byrne, “On NMT Search Errors and Model Errors: Cat Got Your Tongue?,” arXiv Preprint, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Philipp Koehn, and Rebecca Knowles, “Six Challenges for Neural Machine Translation,” arXiv Preprint, 2017.
[CrossRef] [Google Scholar] [Publisher Link]