Comparative Evaluation of Statistical Tools in Different ChatGPT Iterations
Comparative Evaluation of Statistical Tools in Different ChatGPT Iterations |
||
 |
 |
|
| © 2025 by IJETT Journal | ||
| Volume-73 Issue-4 |
||
| Year of Publication : 2025 | ||
| Author : Pauly Awad, Maya Grigolia, Soraia Oueida, Nour Mostafa |
||
| DOI : 10.14445/22315381/IJETT-V73I4P117 | ||
How to Cite?
Pauly Awad, Maya Grigolia, Soraia Oueida, Nour Mostafa, "Comparative Evaluation of Statistical Tools in Different ChatGPT Iterations," International Journal of Engineering Trends and Technology, vol. 73, no. 4, pp.180-190, 2025. Crossref, https://doi.org/10.14445/22315381/IJETT-V73I4P117
Abstract
ChatGPT, a variant of the Generative Pre-trained Transformer (GPT) models, has emerged as an essential tool for processing, summarizing, and deriving insights from vast textual datasets. This paper delves into the multifaceted role of ChatGPT in data analysis, highlighting its functionalities, challenges, and potential; unlike traditional statistical software that demands specialized expertise and often lacks interpretive capabilities, AI-powered generative tools offer a novel approach, empowering non-experts to execute statistical models and extract results accompanied by interpretations and conclusions. Our investigation evaluates the output accuracy across different iterations of ChatGPT (3.5, 4.0, and the dedicated data analyst tool) for standard statistical inferences, uncovering notable disparities that underscore the necessity of professional validation for reliability. Our findings indicate that while the 3.5 version may exhibit numerous errors and omissions, necessitating thorough scrutiny and debugging by statisticians, version 4.0 demonstrates improved accuracy in most instances. However, the results obtained through the Data Analyst tool exhibit the highest level of nuance and correctness.
Keywords
Artificial Intelligence, Natural Language Processing, ChatGPT, Data Analysis, Statistics.
References
[1] Introducing ChatGPT, OpenAI, 2022. [Online]. Available: https://openai.com/blog/chatgpt/ [2] Tom B. Brown et al., “Language Models Are Few-Shot Learners,” arXiv, pp. 1-75, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Fiona Fui-Hoon Nah et al., “Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration,” Journal of Information Technology Case and Application Research, vol. 25, no. 3, pp. 277-304, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Miles Brundage et al., “The Malicious use of Artificial Intelligence: Forecasting, Prevention, and Mitigation,” arXiv, pp. 1-101, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Emily M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, Canada, pp. 610-623, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Bernd Carsten Stahl, and Damian Eke, “The Ethics of ChatGPT-Exploring the Ethical Issues of an Emerging Technology,” International Journal of Information Management, vol. 74, pp. 1-14, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Chung Kwan Lo, “What is the Impact of ChatGPT on Education? A Rapid Review of the Literature,” Education Sciences, vol. 13, no. 4, pp. 1-15, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Partha Pratim Ray, “ChatGPT: A Comprehensive Review on Background, Applications, Key Challenges, Bias, Ethics, Limitations and Future Scope,” Internet of Things and Cyber-Physical Systems, vol. 3, pp. 121-154, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Yeen Huang et al., “Evaluating ChatGPT-4.0’s Data Analytic Proficiency in Epidemiological Studies: A Comparative Analysis with SAS, SPSS, and R,” Journal of Global Health, vol. 14, pp. 1-10, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Yiheng Liu et al., “Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models,” Meta-Radiology, vol. 1, no. 2, pp. 1-14, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Julio Christian Young, and Makoto Shishido, “Investigating OpenAI’s ChatGPT Potentials in Generating Chatbot's Dialogue for English as a Foreign Language Learning,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 6, pp. 65-72, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Muhammad Usman Hadi et al., “A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage,” Authorea Preprints, pp. 1-56, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Anna R. McAlister, Saleem Alhabash, and Jing Yang, “Artificial Intelligence and ChatGPT: Exploring Current and Potential Future Roles in Marketing Education,” Journal of Marketing Communications, vol. 30, no. 2, pp. 166-187, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Moatsum Alawida et al., “A Comprehensive Study of ChatGPT: Advancements, Limitations, and Ethical Considerations in Natural Language Processing and Cybersecurity,” Information, vol. 14, no. 8, pp. 1-23, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Anam Nazir, and Ze Wang, “A Comprehensive Survey of ChatGPT: Advancements, Applications, Prospects, and Challenges,” Meta-Radiology, vol. 1, no. 2, pp. 1-12, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Mert Şen, Şevval Nur Şen, and Tuğrul Gökmen Şahin, “A New Era for Data Analysis in Qualitative Research: ChatGPT!,” Shanlax International Journal of Education, vol. 11, pp. 1-15, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Yixun Xing, “Exploring the use of ChatGPT in Learning and Instructing Statistics and Data Analytics,” Teaching Statistics, vol. 46, no. 2, pp. 95-104, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Staphord Bengesi et al., “Advancements in Generative Ai: A Comprehensive Review of Gans, GPT, Autoencoders, Diffusion Model, and Transformers,” IEEE Access, vol. 12, pp. 69812-69837, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Xuanming Zhang et al., “GrounDialog: A Dataset for Repair and Grounding in Task-Oriented Spoken Dialogues for Language Learning,” Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 300-314, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Carlo Perrotta, Natural Language Generation and the Automation of Pedagogical Communication, World Yearbook of Education 2024, Taylor & Francis Group, pp. 54-69, 2023.
[Google Scholar] [Publisher Link]
[21] Isha Kondurkar, Akanksha Raj, and D. Lakshmi, Modern Applications with a Focus on Training ChatGPT and GPT Models: Exploring Generative AI and NLP, Advanced Applications of Generative AI and Natural Language Processing Models, IGI Global, pp. 186-227, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Rich Caruana et al., “Intelligible Models for Healthcare: Predicting Pneumonia Risk and Hospital 30-Day Readmission,” Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, pp. 1721-1730, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Tiziano Labruna et al., “Unraveling ChatGPT: A Critical Analysis of AI-Generated Goal-Oriented Dialogues and Annotations,” International Conference of the Italian Association for Artificial Intelligence, Rome, Italy, pp. 151-171, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Richard Brath, and Craig Hagerman, “Automated Insights on Visualizations with Natural Language Generation,” 25th International Conference Information Visualisation, Sydney, Australia, pp. 278-284, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Ivonne Nuñez et al., “Designing A Comprehensive and Flexible Architecture to Improve Energy Efficiency and Decision-Making in Managing Energy Consumption and Production in Panama,” Applied Sciences, vol. 13, no. 9, pp. 1-20, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Atlas of Sustainable Development Goals, World Bank Group, 2023. [Online]. Available: https://datatopics.worldbank.org/sdgatlas?lang=en
[27] Stefano A. Bini, “Artificial Intelligence, Machine Learning, Deep Learning, and Cognitive Computing: What Do These Terms Mean and How Will They Impact Health Care?” The Journal of Arthroplasty, vol. 33, no. 8, pp. 2358-2361, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[28] John D. Kelleher, and Brendan Tierney, Data science, MIT press, 2018.
[Google Scholar] [Publisher Link]