Citation :

Ayesha Siddiqua, H C Nagaraj, "Unified Binary Aspect-Based Sentiment Analysis with Transformer Models: A Cross-Domain Evaluation of Models," International Journal of Engineering Trends and Technology (IJETT), vol. 74, no. 9, pp. 50-62, 2026. Crossref, https://doi.org/10.14445/22315381/IJETT-V74I9P105

Abstract

Aspect-Based Sentiment Analysis (ABSA) is a Fine-grained Sentiment Classification task that aims to detect the sentiment polarities expressed towards specific aspects in the text. ABSA has also become prevalent in applications of product feedback mining, brand monitoring and recommendation systems due to the growth of user-generated feedback spread on social media posts. The conventional forms of deep learning, like ATAE-LSTM and TD-LSTM, have worked sufficiently but have constraints because of shallow context perception and the inability to provide long-term dependencies. The examination performs and benchmarks the utilization of current transformer-based structures, Bidirectional Encoder Representations from Transformers (BERT), Robustly Optimized BERT Pretraining Approach (RoBERTa)and Decoding-enhanced BERT with Disentangled Attention (DeBERTa) to binary ABSA classification, i.e., distinguishing between positive and negative sentiments but ignoring neutral labels. The approach will be to encode aspect terms by using special tokens and fine-tuning models using benchmark data such as SemEval (Laptop & Restaurant) as well as the Multi-modal Aspect-based Sentiment Analysis (MAMS) dataset. In addition, a comparative literature survey is made among the recent models, including multimodal advanced structures, which confuse text and visual information. Benchmarking experimental testing (SemEval-Laptop, SemEval-Restaurant, and MAMS) demonstrates that DeBERTa-base had an accuracy of 92.45 and an F1-score of 92.44, compared with RoBERTa-base with 91.89 accuracy and BERT-base with 91.05 accuracy.

Keywords

ABSA, BERT, RoBERTa, DeBERTa, Transformer models, Sentiment classification, MAMS dataset, SemEval, Disentangled attention, Relative positional encoding, Pre-trained language models, Multimodal sentiment analysis.

References

[1] Yequan Wang et al., “Attention-based LSTM for Aspect-Level Sentiment Classification,” Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Austin, Texas, pp. 606-615, 2016.
[CrossRef] [Google Scholar] [Publisher Link] 

[2] Duyu Tang, Bing Qin, and Ting Liu, “Aspect Level Sentiment Classification with Deep Memory Network,” Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Austin, Texas, pp. 214-224, 2016.
[CrossRef] [Google Scholar] [Publisher Link] 

[3] Xin Li et al., “Transformation Networks for Target-Oriented Sentiment Classification,” Proceedings of the 56^th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Melbourne, Australia, vol. 1, pp. 946-956, 2018.
[CrossRef] [Google Scholar] [Publisher Link]

[4] Maria Mihaela Truşcǎ et al., “A Hybrid Approach for Aspect-Based Sentiment Analysis Using Deep Contextual Word Embeddings and Hierarchical Attention,” International Conference on Web Engineering, Helsinki, Finland, vol. 1, pp. 365-380, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[5] C. Lohith, and H. Chandramouli, “Fine-Grained Sentiment Analysis of Restaurant Reviews using Latent Dirichlet Allocation and Hybrid Ensembles,” International Journal Environmental Science, vol. 11, no. 20s, pp. 2468-2475, 2025.
[Publisher Link]

[6] Mandar Joshi et al., “SpanBERT: Improving Pre-Training by Representing and Predicting Spans,” Transactions of the Association for Computational Linguistics, vol. 8, pp. 64-77, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[7] Kai Wang et al., “Relational Graph Attention Network for Aspect-based Sentiment Analysis,” Proceedings of the 58^th Annual Meeting of the Association for Computational Linguistics, pp. 3229-3238, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[8] Bocheng Miao, and Changbo Xu, “Aspect-Level Multimodal Sentiment Analysis Model based on Multi-Scale Feature Extraction,” Scientific Reports, vol. 15, no. 1, pp. 1-11, 2025.
[CrossRef] [Google Scholar] [Publisher Link]

[9] Meilin Jin et al., “Aspect based Sentiment Analysis on Multimodal Data: A Transformer and Low-Rank Fusion Approach,” 2024 4^th International Conference on Computer Communication and Artificial Intelligence (CCAI), Xi'an, China, pp. 332-338, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[10] Shunjie Wang, Guoyong Cai, and Guangrui Lv, “Aspect-Level Multimodal Sentiment Analysis based on Co-Attention Fusion,” International Journal of Data Science and Analytics, vol. 20, no. 2, pp. 903-916, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[11] Wencong Wang, Junjie Chen, and Hao Fan, “A Two-Stage Framework for Multimodal Aspect-based Sentiment Analysis using Cross-modal Translation,” Proceedings of the 2024 International Conference on Image Processing, Multimedia Technology and Maching Learning, Association for Computing Machinery, New York, NY, United States, pp. 210-217, 2025. [CrossRef] [Google Scholar] [Publisher Link]

[12] Zefang Zhao et al., “Fine-Grained Cross-Modal Graph Convolution for Multimodal Aspect-Oriented Sentiment Analysis,” 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Honolulu, Oahu, HI, USA, pp. 4214-4219, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[13] Amit Chauhan, Aman Sharma, and Rajni Mohana, “A Transformer Model for end-to-end Image and Text Aspect-based Sentiment Analysis,” 2023 Seventh International Conference on Image Information Processing (ICIIP), Solan, India, pp. 277-282, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[14] Xiaoran Hu, and Masayuki Yamamura, “Hierarchical Fusion Network with Enhanced Knowledge and Contrastive Learning for Multimodal Aspect-based Sentiment Analysis,” Sensors, vol. 23, no. 17, pp. 1-16, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[15] Biqing Zeng et al., “Aspect-Aware Semantic Feature Enhanced Networks for Multimodal Aspect-based Sentiment Analysis,” The Journal of Supercomputing, vol. 81, no. 1, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[16] Jianfei Yu, Kai Chen, and Rui Xia, “Hierarchical Interactive Multimodal Transformer for Aspect-based Multimodal Sentiment Analysis,” IEEE Transactions on Affective Computing, vol. 14, no. 3, pp. 1966-1978, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[17] Tianyu Zhao, Ling-ang Meng, and Dawei Song, “Multimodal Aspect-based Sentiment Analysis: A Survey of Tasks, Methods, Challenges and Future Directions,” Information Fusion, vol. 112, 2024.
[Google Scholar]

[18] Ashok Kumar Jayaraman et al., “Aspect Category and Sentiment Polarity Detection using a Permutation Language-based Transformer Model,” Procedia Computer Science, vol. 258, pp. 4179-4189, 2025.
[Google Scholar]

[19] Hai Ha Do et al., “Deep Learning for Aspect-based Sentiment Analysis: A Comparative Review,” Expert Systems with Applications, vol. 118, pp. 272-299, 2019.
[Google Scholar]

[20] Komang Wahyu Trisna, and Huang Jin Jie, “Deep Learning Approach for Aspect-based Sentiment Classification: A Comparative Review,” Applied Artificial Intelligence, vol. 36, no. 1, pp. 1-38, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[21] Jiachang Sun, and Fuxian Zhu, “Multilayer Interactive Attention Bottleneck Transformer for Aspect-based Multimodal Sentiment Analysis,” Multimedia Systems, vol. 31, no. 1, 2025.
[CrossRef] [Google Scholar] [Publisher Link]

[22] Shezheng Song, and Shan Zhao, “Whether Current Large Language Models is Suitable for Multimodal Aspect-based Sentiment Analysis?,” Proceedings of the 2024 2^nd International Conference on Advances in Artificial Intelligence and Applications, Association for Computing Machinery, New York, NY, United States, pp. 125-130, 2025.
[CrossRef] [Google Scholar] [Publisher Link]

[23] Gagandeep Kaur, and Amit Sharma, “A Deep Learning-based Model using Hybrid Feature Extraction Approach for Consumer Sentiment Analysis,” Journal of Big Data, vol. 10, no. 1, pp. 1-23, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[24] Yujian Cai et al., “Multimodal Sentiment Analysis based on Multi-Layer Feature Fusion and Multi-Task Learning,” Scientific Reports, vol. 15, no. 1, pp. 1-21, 2025.
[CrossRef] [Google Scholar] [Publisher Link]

[25] Aminu Musa et al., “HauBERT: A Transformer Model for Aspect-based Sentiment Analysis of HAUSA-Language Movie Reviews,” Engineering Proceedings, vol. 87, no. 1, pp. 1-18, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[26] Md Saef Ullah Miah et al., “A Multimodal Approach to Cross-Lingual Sentiment Analysis with Ensemble of Transformer and LLM,” Scientific Reports, vol. 14, no. 1, pp. 1-18, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[27] Amira Samy Talaat, “Sentiment Analysis Classification System using Hybrid BERT Models,” Journal of Big Data, vol. 10, no. 1, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[28] Xiaokang Gong et al., “Text Sentiment Analysis based on Transformer and Augmentation,” Frontiers in Psychology, vol. 13, pp. 1-9, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[29] C. Lohith et al., “Aspect Oriented Sentiment Analysis on Customer Reviews on Restaurant using the LDA and BERT Method,” SN Computer Science, vol. 4, no. 4, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[30] Lingling Zi, Xiangkai Pan, and Xin Cong, “MFSC: A Multimodal Aspect-Level Sentiment Classification Framework with Multi-Image Gate and Fusion Networks,” Electronics, vol. 13, no. 12, pp. 1-15, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[31] Isidoros Perikos, and Athanasios Diamantopoulos, “Explainable Aspect-based Sentiment Analysis using Transformer Models,” Big Data and Cognitive Computing, vol. 8, no. 11, pp. 1-30, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[32] Ananya Pandey, and Dinesh Kumar Vishwakarma, “Aspect-based Multimodal Sentiment Analysis Via Employing Visual-to-Emotional-Caption Translation Network using Visual-Caption Pairs,” Language Resources and Evaluation, vol. 59, no. 3, pp. 2945-2972, 2025.
[CrossRef] [Google Scholar] [Publisher Link]

[33] kiloGrand, ABSA-Dataset, Kaggle, 2014. [Online]. Available: https://www.kaggle.com/datasets/kilogrand/absa-dataset/data

[34] Hung Phanmj, MAMS-for-ABSA, Kaggle. [Online]. Available: https://www.kaggle.com/datasets/hungphanmj/mams-for-absa

[35] ayeshanmit/Cross-Domain_Transformers-ABSA, GitHub, 2026. [Online]. Available: https://github.com/ayeshanmit/Cross-Domain_Transformers-ABSA