Data-Driven Insights for Mobile Banking App Improvement: A Sentiment Analysis and Topic Modelling Approach for SimobiPlus User Reviews
Data-Driven Insights for Mobile Banking App Improvement: A Sentiment Analysis and Topic Modelling Approach for SimobiPlus User Reviews |
||
 |
 |
|
| © 2024 by IJETT Journal | ||
| Volume-72 Issue-6 |
||
| Year of Publication : 2024 | ||
| Author : Edwina, Tuga Mauritsius |
||
| DOI : 10.14445/22315381/IJETT-V72I6P132 | ||
How to Cite?
Edwina, Tuga Mauritsius, "Data-Driven Insights for Mobile Banking App Improvement: A Sentiment Analysis and Topic Modelling Approach for SimobiPlus User Reviews," International Journal of Engineering Trends and Technology, vol. 72, no. 6, pp. 347-360, 2024. Crossref, https://doi.org/10.14445/22315381/IJETT-V72I6P132
Abstract
This study presents a comprehensive analysis of user reviews for the SimobiPlus mobile banking application in Indonesia. By leveraging state-of-the-art natural language processing techniques, including IndoBERT embeddings and machine learning classifiers (SVM, Naïve Bayes, KNN, Random Forest, Logistic Regression), we perform multi-dimensional sentiment analysis and topic modelling on a dataset of over 7,000 user reviews. Our approach classifies reviews based on sentiment (positive/negative), information type (bug report, feature request, user experience, ratings), objectives (app-related, company-related), and emotions (anger, joy, disgust, etc.). We also extract key topics and issues discussed in the reviews using Latent Dirichlet Allocation (LDA). The results demonstrate the effectiveness of SVM with hyperparameter tuning for sentiment classification (91% accuracy) and identify several recurring themes in user feedback, such as login/update errors, transaction failures, and requests for new features. Notably, we find that stemming has minimal impact on classification performance for this Indonesian language dataset. Our findings provide actionable insights for developers and managers to prioritize app improvements and enhance the overall user experience of mobile banking services. This study contributes to the growing body of research on data-driven user feedback analysis and offers practical recommendations for digital banking innovation in emerging markets.
Keywords
Mobile banking, Sentiment analysis, Text mining, Topic modelling.
References
[1] Payment System Statistics and Financial Market Infrastructure (SPIP), Bank Indonesia. [Online]. Available: https://www.bi.go.id/id/statistik/ekonomi-keuangan/ssp/transaksi-delivery-channel.aspx.
[2] Salah Al-Hagree, and Ghaleb Al-Gaphari, “Arabic Sentiment Analysis Based Machine Learning for Measuring User Satisfaction with Banking Services’ Mobile Applications: Comparative Study,” 2022 2nd International Conference on Emerging Smart Technologies and Applications, Ibb, Yemen, pp. 1-4, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Yudo Ekanata, and Indra Budi, “Mobile Application Review Classification for the Indonesian Language Using Machine Learning Approach,” 2018 4th International Conference on Computer and Technology Applications, Istanbul, Turkey, pp. 117-121, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Said A. Salloum et al., “A Survey of Text Mining in Social Media: Facebook and Twitter Perspectives,” Advances in Science, Technology and Engineering Systems Journal, vol. 2, no. 1, pp. 127-133, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Rachmawan Adi Laksono et al., “Sentiment Analysis of Restaurant Customer Reviews on TripAdvisor Using Naïve Bayes,” 2019 12th International Conference on Information & Communication Technology and System, Surabaya, Indonesia, pp. 49-54, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Yuni Handayani, Alvin Rinaldy Hakim, and Muljono, “Sentiment Analysis of Bank BNI User Comments Using the Support Vector Machine Method,” 2020 International Seminar on Application for Technology of Information and Communication, Semarang, Indonesia, pp. 202-207, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Bramanthyo Andrian et al., “Sentiment Analysis on Customer Satisfaction of Digital Banking in Indonesia,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 3, pp. 466-473, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Kusnawi Kusnawi, Majid Rahardi, and Van Daarten Pandiangan, “Sentiment Analysis of Neobank Digital Banking Using Support Vector Machine Algorithm in Indonesia,” International Journal on Informatics Visualization, vol. 7, no. 2, pp. 377-383, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[9] M. Richard, Pengguna Mobile Banking Bank Sinarmas (BSIM) Melonjak 55 Persen Tahun Ini, Finansial, 2021. [Online]. Available: https://finansial.bisnis.com/read/20210823/90/1432719/pengguna-mobile-banking-bank-sinarmas-bsim-melonjak-55-persen-tahun-ini
[10] Paul Ekman, and Daniel Cordaro, “What is Meant by Calling Emotions Basic,” Emotion Review, vol. 3, no. 4, pp. 364-370, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Hao Fei et al., “Latent Emotion Memory for Multi-Label Emotion Classification,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 5, pp. 7692-7699, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Walid Maalej, and Hadeer Nabil, “Bug Report, Feature Request, or Simply Praise? on Automatically Classifying App Reviews,” 2015 IEEE 23rd International Requirements Engineering Conference, Ottawa, ON, Canada, pp. 116-125, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Rein Rachman Putra, Monika Evelin Johan, and Emil Robert Kaburuan, “A Naïve Bayes Sentiment Analysis for Fintech Mobile Application User Review in Indonesia,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 8, no. 5, pp. 1856-1860, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Moh. Nasrul Aziz et al., “Sentiment Analysis and Topic Modelling for Identification of Government Service Satisfaction,” 2018 5th International Conference on Information Technology, Computer, and Electrical Engineering, Semarang, Indonesia, pp. 125-130, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Gede Rizky Gustisa Wisnu et al., “Sentiment Analysis and Topic Modelling of 2018 Central Java Gubernatorial Election Using Twitter Data,” 2020 International Workshop on Big Data and Information Security, Depok, Indonesia, pp. 35-40, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Rajesh Prabhakar Kaila, and A.V. Krishna Prasad, “Informational Flow on Twitter – Corona Virus Outbreak – Topic Modelling Approach,” International Journal of Advanced Research in Engineering and Technology, vol. 11, no. 3, pp. 128-134, 2020.
[Google Scholar] [Publisher Link]
[17] Carlos Tam, and Tiago Oliveira, “Does Culture Influence M-Banking Use and Individual Performance?,” Information and Management, vol. 56, no. 3, pp. 356-363, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Nurazzah Abd Rahman, Seri Dahlia Idrus, and Noor Latiffah Adam, “Classification of Customer Feedbacks Using Sentiment Analysis towards Mobile Banking Applications,” IAES International Journal of Artificial Intelligence, vol. 11, no. 4, pp. 1579-1587, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Eman Alsagour, Lubna Alhenki, and Mohammed Al-Dhelaan, “Different Word Representation for Text Classification: A Comparative Study,” 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications, Abu Dhabi, United Arab Emirates, pp. 1-2, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Chenrui Lu et al., “A Document Analysis of Peak Carbon Emissions and Carbon Neutrality Policies Based on a PMC Index Model in China,” International Journal of Environmental Research and Public Health, vol. 19, no. 15, pp. 1-16, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Hilman Wisnu, Muhammad Afif, and Yova Ruldevyani, “Sentiment Analysis on Customer Satisfaction of Digital Payment in Indonesia: A Comparative Study Using KNN and Naïve Bayes,” Journal of Physics: Conference Series, vol. 1444, no. 1, pp. 1-10, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Sakshi Ranjan, and Subhankar Mishra, “Comparative Sentiment Analysis of App Reviews,” 2020 11th International Conference on Computing, Communication and Networking Technologies, Kharagpur, India, pp. 1-7, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Jacob Devlin et al., “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding,” Proceedings of NAACL-HLT, Minneapolis, Minnesota, pp. 4171-4186, 2019.
[Google Scholar] [Publisher Link]
[24] Muchammad Naseer, Muhamad Asvial, and Riri Fitri Sari, “An Empirical Comparison of BERT, RoBERTa, and Electra for Fact Verification,” 2021 International Conference on Artificial Intelligence in Information and Communication, Jeju Island, Korea (South), pp. 241-246, 2021.
[CrossRef] [Google Scholar] [Publisher Link] [25] Anwar Hussen Wadud et al., “Deep-BERT: Transfer Learning for Classifying Multilingual Offensive Texts on Social Media,” Computer Systems Science and Engineering, vol. 44, no. 2, pp. 1775-1791, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Saeid Pourroostaei Ardakani et al., “A Data-Driven Affective Text Classification Analysis,” 2021 20th IEEE International Conference on Machine Learning and Applications, Pasadena, CA, USA, pp. 199-204, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[27] Sani Muhamad Isa, Gary Nico, and Mikhael Permana, “Indobert for Indonesian Fake News Detection,” ICIC Express Letters, vol. 16, no. 3, pp. 289-297, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[28] Sebastian Kopera et al., “Interdisciplinarity in Tech Startups Development - Case Study of ‘Unistartapp’ Project,” Foundations of Management, vol. 10, no. 1, pp. 23-32, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[29] Jose Ramon Saura, Ana Reyes-Menendez, and Cesar Alvarez-Alonso, “Do Online Comments Affect Environmental Management? Identifying Factors Related to Environmental Management and Sustainability of Hotels,” Sustainability, vol. 10, no. 9, pp. 1-20, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[30] Jose Ramon Saura, Pedro Palos-Sanchez, and Antonio Grilo, “Detecting Indicators for Startup Business Success: Sentiment Analysis Using Text Data Mining,” Sustainability, vol. 11, no. 3, pp. 1-14, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[31] Jafar Alzubi, Anand Nayyar, and Akshi Kumar, “Machine Learning from Theory to Algorithms: An Overview,” Journal of Physics: Conference Series, vol. 1142, pp. 1-15, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[32] Majesty Eksa Permana et al., “Sentiment Analysis and Topic Detection of Mobile Banking Application Review,” 2020 5th International Conference on Informatics and Computing, Gorontalo, Indonesia, pp. 1-6, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[33] Vinod Kumar Chauhan, Kalpana Dahiya, and Anuj Sharma, “Problem Formulations and Solvers in Linear SVM: A Review,” Artificial Intelligence Review, vol. 52, pp. 803-855, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[34] Sally S. Tinkle et al., “An Outcome Evaluation of the National Institutes of Health Director’s New Innovator Award Program for Fiscal Years 2007-2009,” Science & Technology Policy Institute, 2007.
[Google Scholar]
[35] Sri Handika Utami, Anton Ade Purnama, and Achmad Nizar Hidayanto, “Fintech Lending in Indonesia: A Sentiment Analysis, Topic Modelling, and Social Network Analysis Using Twitter Data,” International Journal of Applied Engineering and Technology, vol. 4, no. 1, pp. 50-56, 2022.
[Google Scholar] [Publisher Link]
[36] Mohammed Bahja, “Identifying Patient Experience from Online Resources via Sentiment Analysis and Topic Modelling Approaches,” Thirty Ninth International Conference on Information Systems, San Francisco, pp. 1-9, 2018.
[Google Scholar] [Publisher Link]
[37] Abdullahi Sidow Osman, “Data Mining Techniques: Review,” Data Science and Networking, vol. 2, no. 1, pp. 1-4, 2019. [Google Scholar] [Publisher Link] [38] Christoph Schröer, Felix Kruse, and Jorge Marx Gómez, “A Systematic Literature Review on Applying CRISP-DM Process Model,” Procedia Computer Science, vol. 181, pp. 526-534, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[39] Aditya Wiha Pradana, and Mardhiya Hayaty, “The Effect of Stemming and Removal of Stopwords on the Accuracy of Sentiment Analysis on Indonesian-language Texts,” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, vol. 4, no. 4, pp. 375-380, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[40] Ezz El-Din Hemdan, Marwa A. Shouman, and Mohamed Esmail Karar, “COVIDX-Net: A Framework of Deep Learning Classifiers to Diagnose COVID-19 in X-Ray Images,” arXiv, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[41] Paul Mooijman et al., “The Effects of Data Balancing Approaches: A Case Study,” Applied Soft Computing, vol. 132, pp. 1-32, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[42] Der-Jang Chi, and Zong-De Shen, “Using Hybrid Artificial Intelligence and Machine Learning Technologies for Sustainability in Going-Concern Prediction,” Sustainability, vol. 14, no. 3, pp. 1-18, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[43] Rindu Hafil Muhammadi, Tri Ginanjar Laksana, and Amalia Beladinna Arifa, “Combination of Support Vector Machine and Lexicon-Based Algorithm in Twitter Sentiment Analysis,” Khazanah Informatika: Journal of Computer Science and Informatics, vol. 8, no. 1, pp. 59-71, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[44] W.P. Ramadhan, S.T.M.T. Astri Novianty, and S.T.M.T. Casi Setianingsih, “Sentiment Analysis Using Multinomial Logistic Regression,” 2017 International Conference on Control, Electronics, Renewable Energy and Communications, Yogyakarta, Indonesia, pp. 46-49, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[45] Elshrif Elmurngi, and Abdelouahed Gherbi, “Detecting Fake Reviews through Sentiment Analysis Using Machine Learning Techniques,” DATA ANALYTICS 2017: The Sixth International Conference on Data Analytics Detecting, pp. 65-72, 2017.
[Google Scholar]
[46] Thanh-Nam Doan, and Tuan-Anh Hoang, “Benchmarking Neural Topic Models: An Empirical Study,” Findings of the Association for Computational Linguistics: ACL-IJCNLP, pp. 4363-4368, 2021.
[CrossRef] [Google Scholar] [Publisher Link]