A Bi-LSTM and GRU Hybrid Neural Network with BERT Feature Extraction for Amazon Textual Review Analysis

A Bi-LSTM and GRU Hybrid Neural Network with BERT Feature Extraction for Amazon Textual Review Analysis
  IJETT-book-cover           
  
© 2022 by IJETT Journal
Volume-70 Issue-5
Year of Publication : 2022
Authors : Madhuri V. Joseph
DOI :  10.14445/22315381/IJETT-V70I5P216

How to Cite?

Madhuri V. Joseph, "A BiLSTM and GRU Hybrid Neural Network with BERT Feature Extraction for Amazon Textual Review Analysis," International Journal of Engineering Trends and Technology, vol. 70, no. 5, pp. 1-8, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I5P216

Abstract
Nowadays, businesses move towards digital platforms for their product promotion and to improve their overall profit margin. Customer reviews determine the purchase decision of the specified products in the e-commerce system in this digital world. In this case, reviewing products before buying is the common scenario in this current world. It will help the buyers to buy quality products at affordable prices. On this basis, it is necessary to implement deep learning techniques to analyze the sentimental tweets of customers based on their product ratings. The study planned to propose an enhanced BERT algorithm in feature extraction on sentiment analysis with a large data set and hybridized deep Bi-LSTM-GRU neural network for the classification process.

Consequently, amazon products` customer review datasets are employed for the analysis. The review about mobile phones is retrieved from amazon for the sentiment analysis. Initially, the data was preprocessed to increase the accuracy and performance of the classifier. Further, the feature extraction is done to reduce many data into accurate ones with BERT (Bidirectional Encoder Representations from Transformers) algorithm. It isn`t easy to evaluate the review sentiments without efficient classification. After this process, the study analyses the performance of the deep Bi-LSTM-GRU neural network method with the existing method. Finally, the study concluded that the proposed algorithm achieved 97.87, 98.36, 98.89, and 98.47, respectively, based on the accuracy level, frequency, precision, and recall measures. These performance measures are higher than the existing algorithms developed in the studies. The study achieved more accuracy and efficiency through the proposed deep Bi-LSTM-GRU neural network method in sentiment analysis on mobile phone reviews in the amazon e-commerce system.

Keywords
Sentiment analysis, Feature extraction, BERT, Bi-LSTM, GRU neural network.

Reference
[1] A. S. AlQahtani, Product Sentiment Analysis for Amazon Reviews, International Journal of Computer Science & Information Technology (IJCSIT) , 13 (2021).
[2] H. S. Sharaf Al-deen, Z. Zeng, R. Al-Sabri, and A. Hekmat, An Improved Model for Analyzing Textual Sentiment Based on a Deep Neural Network Using Multi-Head Attention Mechanism, Applied System Innovation, 4 (2021) 85.
[3] D. Mane, P. Srivastava, A. Jain, and D. Chouhan, Assessing Travellers Perception Levels towards Select Online Travel Agents (TA) in India using Netnography and Text Mining, International Journal of Engineering Trends and Technology, 68 (2020).
[4] B. Subba and S. Kumari, A heterogeneous stacking ensemble based sentiment analysis framework using multiple word embeddings, Computational Intelligence, (2021).
[5] S. K. Prabhakar, H. Rajaguru, and D.-O. Won, Performance Analysis of Hybrid Deep Learning Models with Attention Mechanism Positioning and Focal Loss for Text Classification, Scientific Programming, 2021 (2021).
[6] S. Kavatagi and V. Adimule, Bi-GRU Model with Stacked Embedding for Sentiment Analysis: A Case Study, in the Internet of Things, Artificial Intelligence and Blockchain Technology, ed: Springer, (2021) 259-275.
[7] S. Muthukumaran and P. Suresh, Text analysis for product reviews for sentiment analysis using NLP methods, Int. J. Eng. Trends Technol, 47 (2017) 474-480.
[8] H. Xu, B. Liu, L. Shu, and P. S. Yu, BERT post-training for review reading comprehension and aspect-based sentiment analysis, arXiv preprint arXiv:1904.02232, (2019).
[9] X. Han and L. Wang, A novel document-level relation extraction method based on BERT and entity information, IEEE Access, 8 (2020) 96912-96919.
[10] G. Pang, K. Lu, X. Zhu, J. He, Z. Mo, Z. Peng, et al., Aspect-Level Sentiment Analysis Approach via BERT and Aspect Feature Location Model, Wireless Communications and Mobile Computing, 2021 (2021).
[11] Y. Peng, T. Xiao, and H. Yuan, Cooperative gating network based on a single BERT encoder for aspect term sentiment analysis, Applied Intelligence, (2021) 1-13.
[12] X. Li, X. Fu, G. Xu, Y. Yang, J. Wang, L. Jin, et al., Enhancing BERT representation with context-aware embedding for aspect-based sentiment analysis, IEEE Access, 5 (2020) 46868-46876.
[13] G. More, H. Behara, and A. M. Suresha, Sentiment Analysis on Amazon Product Reviews with Stacked Neural Networks, (2020).
[14] C. Du, H. Sun, J. Wang, Q. Qi, and J. Liao, Adversarial and domain-aware BERT for cross-domain sentiment analysis, in Proceedings of the 58th annual meeting of the Association for Computational Linguistics, (2020) 4019-4028.
[15] K. Baktha and B. Tripathy, Investigation of recurrent neural networks in the field of sentiment analysis, in International Conference on Communication and Signal Processing (ICCSP), (2017) 2047-2050.
[16] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, LSTM: A search space odyssey, IEEE transactions on neural networks and learning systems, 28 (2016) 2222-2232, 2016.
[17] B. Athiwaratkun and J. W. Stokes, Malware classification with LSTM and GRU language models and a character-level CNN, in 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), (2017) 2482-2486.
[18] Ö. Yildirim, A novel wavelet sequence based on deep bidirectional LSTM network model for ECG signal classification, Computers in biology and medicine, 96 (2018) 189-202.
[19] O. Habimana, Y. Li, R. Li, X. Gu, and Y. Peng, A Multi-Task Learning Approach to Improve Sentiment Analysis with Explicit Recommendation, in 2020 International Joint Conference on Neural Networks (IJCNN), (2020) 1-8.
[20] C. N. Dang, M. N. Moreno-García, and F. De la Prieta, Using Hybrid Deep Learning Models of Sentiment Analysis and Item Genres in Recommender Systems for Streaming Services, Electronics, 10 (2021) 2459.
[21] A. Oshri, S. M. Kogan, J. A. Kwon, K. Wickrama, L. Vanderbroek, A. A. Palmer, et al., Impulsivity as a mechanism linking child abuse and neglect with substance use in adolescence and adulthood, Development and psychopathology, 30 (2018) 417-435.
[22] S. Sachin, A. Tripathi, N. Mahajan, S. Aggarwal, and P. Nagrath, Sentiment analysis using gated recurrent neural networks, SN Computer Science, 1 (2020) 1-13.
[23] N. Pathik and P. Shukla, An efficient sentiment analysis using topic model based optimized recurrent neural network, Int. J. Smart Sensing Intell. Syst., 11 (2021) 1-12.
[24] B. Bansal and S. Srivastava, Aspect context aware sentiment classification of online consumer reviews, Information Discovery and Delivery, (2020).
[25] S. A. Aljuhani and N. S. Alghamdi, A comparison of sentiment analysis methods on Amazon reviews of Mobile Phones, Int. J. Adv. Comput. Sci. Appl, 10 (2019) 608-617.
[26] M. Shaheen, S. M. Awan, N. Hussain, and Z. A. Gondal, Sentiment analysis on mobile phone reviews using supervised learning techniques, International Journal of Modern Education & Computer Science, 11 (2019)