Citation :

Priyanka Chaudhari, Geetanjali Kale, "Explainable Human Action Intelligence for Reliable and Transparent Video-based Recognition," International Journal of Engineering Trends and Technology (IJETT), vol. 74, no. 10, pp. 308-318, 2026. Crossref, https://doi.org/10.14445/22315381/IJETT-V74I10P122

Abstract

The human action recognition discipline has been boosted by the swift developments in Artificial Intelligence to a substantial degree, making it a pillar of the current computer vision studies. The current research contends with essential needs in society, since it uses a self-acquisition paradigm in order to enhance the understanding of intricate physical exercise routines, hence, enabling independent studies. The modern deep-learning models are more likely to promote predictive performance at the loss of interpretability, reliability, and trustworthiness. Their shortcomings are a hindrance in real world application where accountability and open-mindedness are essential. Explainable Human Action Intelligence (EHAI) framework, which seamlessly integrates state-of-the-art deep-feature extraction with thoroughly based explainability modules, which will contribute to putting trust in artificial intelligence. The proposed framework focuses on improving both the recognition accuracy and interpretability of human action recognition. ResNet-50 is refined using the UCF101 dataset, which comprises 101 human action categories. Gradient weighted Class Activation Mapping approach (Grad-CAM) is incorporated to localize and visualize salient regions within video frames, providing insight into the visual evidence influencing the model’s predictions. Transfer learning is employed to adapt ResNet-50 to the target action recognition task, resulting in an accuracy of 95.4%. The explainable AI methods allow generating visual explanations that underline the areas that are the most appropriate to the model predictions and promote transparency, reliability, and trust. As a result, the state-of-the art framework is not only better in current scans of human action recognition, but also improves model accountability significantly, which additionally promotes the clarity of AI-based resolutions.

Keywords

Explainable Artificial Intelligence (XAI), Human Action Recognition, Deep learning, Model explainability, Trustworthy AI.

References

1. Sajid Ali et al., “Explainable Artificial Intelligence (XAI): What We Know and What is Left to Attain Trustworthy Artificial Intelligence,” Information Fusion, vol. 99, pp. 1-52, 2023.
   [CrossRef] [Google Scholar] [Publisher Link]
2. Sadjad Asghari-Esfeden, Mario Sznaier, and Octavia Camps, “Dynamic Motion Representation for Human Action Recognition,” 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1234-1243, 2023.
   [CrossRef] [Google Scholar] [Publisher Link]
3. Aditya Chattopadhay et al., “Grad-CAM++: Generalized Gradient-based Visual Explanations for Deep Convolutional Networks,” 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA, pp. 839-847, 2018.
   [CrossRef] [Google Scholar] [Publisher Link]
4. Ahmet Çınar, Muhammed Yıldırım, and Yeşim Eroğlu, “Classification of Pneumonia Cell Images using Improved ResNet50 Model,” Signal Processing, vol. 38, no. 1, pp. 165-173, 2021.
   [CrossRef] [Google Scholar] [Publisher Link]
5. Devleena Das et al., “Explainable Activity Recognition for Smart Home Systems,” ACM Transactions on Interactive Intelligent Systems, vol. 13, no. 2, pp. 1-25, 2023.
   [CrossRef] [Google Scholar] [Publisher Link]
6. Finale Doshi-Velez, and Been Kim, “Towards a Rigorous Science of Interpretable Machine Learning,” arXiv preprint arXiv, pp. 1-13, 2017.
   [CrossRef] [Google Scholar] [Publisher Link]
7. Filip Karlo Došilović, Mario Brčić, and Nikica Hlupić, “Explainable Artificial Intelligence: A Survey,” 2018 41^st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 210-215, 2018.
   [CrossRef] [Google Scholar] [Publisher Link] 
8. M. Beatrice Fazi et al., “Beyond Human: Deep Learning, Explainability and Representation,” Theory, Culture and Society, vol. 38, no. 7-8, pp. 55-77, 2021.
   [CrossRef] [Google Scholar] [Publisher Link]    
9. Riccardo Guidotti et al., “A Survey of Methods for Explaining Black Box Models,” ACM Computing Surveys, vol. 51, no. 5, pp. 1-42, 2018.
   [CrossRef] [Google Scholar] [Publisher Link]
10. Xi Guo et al., “Learning Video Representations of Human Motion from Synthetic Data,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5678-5687, 2023.
    [CrossRef] [Google Scholar] [Publisher Link]
11. Kaiming He et al., “Deep Residual Learning for Image Recognition,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
    [Google Scholar] [Publisher Link]
12. Kaiming He et al., “Identity Mappings in Deep Residual Networks,” European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, pp. 630-645, 2016.
    [CrossRef] [Google Scholar] [Publisher Link]
13. Liam Hile et al., “Explaining Motion Relevance for Activity Recognition in Video Deep Learning Models,” arXiv preprint arXiv, pp. 1-9, 2020.
    [CrossRef] [Google Scholar] [Publisher Link]
14. Narit Hnoohom, Sakorn Mekruksavanich, and Anuchit Jitpattanakul, “An Efficient ResNetSE Architecture for Smoking Activity Recognition from Smartwatch,” Intelligent Automation and Soft Computing, vol. 35, no. 1, pp. 1245-1259, 2023.
    [CrossRef] [Google Scholar] [Publisher Link]
15. Shuaidong Ji, “Multi-Classification of Human Action based on ResNet,” Highlights in Science, Engineering and Technology, vol. 23, pp. 1-6, 2022.
    [CrossRef] [Publisher Link]  
16. Yixue Lin et al., “Human Action Recognition Algorithm based on Improved ResNet and Skeletal Keypoints in Single Image,” Mathematical Problems in Engineering, vol. 2020, no. 1, pp. 1-10, 2020.
    [CrossRef] [Google Scholar] [Publisher Link]
17. Guoyang Liu et al., “Human Attention-Guided Explainable Artificial Intelligence for Computer Vision Models,” Neural Networks, vol. 177, pp. 1-18, 2023.
    [CrossRef] [Google Scholar] [Publisher Link]
18. Tim Miller, “Explanation in Artificial Intelligence: Insights from the Social Sciences,” Artificial Intelligence, vol. 267, pp. 1-38, 2019.
    [CrossRef] [Google Scholar] [Publisher Link]  
19. Saumitra Mishra et al., “A Survey on the Robustness of Feature Importance and Counterfactual Explanations,” arXiv preprint arXiv, pp. 1-8, 2021.
    [CrossRef] [Google Scholar] [Publisher Link]
20. Mahsan Nourani et al., “Don't Explain without Verifying Veracity: An Evaluation of Explainable AI with Video Activity Recognition,” arXiv preprint arXiv, pp. 1-26, 2020.
    [CrossRef] [Google Scholar] [Publisher Link] 
21. Nashwan Adnan Othman, and Ilhan Aydin, “Development of a Novel Lightweight CNN Model for Classification of Human Actions in UAV-Captured Videos,” Drones, vol. 7, no. 3, pp. 1-19, 2023.
    [CrossRef] [Google Scholar] [Publisher Link]      
22. Rui Qian et al., “Spatiotemporal Contrastive Video Representation Learning,” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, pp. 6964-6974. 2021.
    [CrossRef] [Google Scholar] [Publisher Link]
23. Endang Sri Rahayu et al., “Human Activity Classification using Deep Learning based on 3D Motion Feature,” Machine Learning with Applications, vol. 12, pp. 1-10, 2023.
    [CrossRef] [Google Scholar] [Publisher Link]
24. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin, “Why should I trust you? Explaining the Predictions of any Classifier,” Proceedings of the 22^nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135-1144, 2016.
    [CrossRef] [Google Scholar] [Publisher Link]
25. Chiradeep Roy et al., “Explainable Activity Recognition in videos: Lessons Learned,” Applied AI Letters, vol. 2, no. 4, pp. 1-19, 2021.
    [CrossRef] [Google Scholar] [Publisher Link]
26. Avinab Saha et al., “Exploring Explainability in Video Action Recognition,” arXiv preprint arXiv, pp. 1-6, 2024.
    [CrossRef] [Google Scholar] [Publisher Link]
27. Ramprasaath R. Selvaraju, “Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization,” Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 618-626, 2017.
    [Google Scholar] [Publisher Link]
28. Thilo Spinner et al., “explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning,” IEEE Transactions on Visualization and Computer Graphics, vol. 26, no. 1, pp. 1064-1074, 2020.
    [CrossRef] [Google Scholar] [Publisher Link]
29. Amin Ullah et al., “Efficient Activity Recognition using Lightweight CNN and DS-GRU Network for Surveillance Applications,” Applied Soft Computing, vol. 103, pp. 1-13, 2021.
    [CrossRef] [Google Scholar] [Publisher Link]
30. Yunbo Wang et al., “Spatiotemporal Pyramid Network for Video Action Recognition,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8987-8996, 2019.
    [CrossRef] [Google Scholar] [Publisher Link]
31. Walt Woods, Jack Chen, and Christof Teuscher, “Adversarial Explanations for Understanding Image Classification Decisions and Improved Neural Network Robustness,” Nature Machine Intelligence, vol. 1, no. 11, pp. 508-516, 2019.
    [CrossRef] [Google Scholar] [Publisher Link]  
32. Tao Zhuo et al., “Explainable Video Action Reasoning via Prior Knowledge and State Transitions,” Proceedings of the ACM International Conference on Multimedia, pp. 521-529, 2019.
    [CrossRef] [Google Scholar] [Publisher Link]
33. K. Vanitha et al., “Enhanced Tuberculosis Detection using Vision Transformers and Explainable AI with a Grad-CAM Approach on Chest X-Rays,” BMC Medical Imaging, vol. 25, no. 1, pp. 1-16, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
34. Antonio Lavecchia et al., “Explainable Artificial Intelligence in drug Discovery: Bridging Predictive Power and Mechanistic Insight,” WIREs Computational Molecular Science, vol. 15, no. 5, pp. 1-21, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
35. Mohammad Rasras et al., “Effects of Different Attention Mechanisms Applied on 3D Models in Video Classification,” International Conference on Computational Science and its Applications, Istanbul, Türkiye, pp. 347-363, 2025.
    [CrossRef] [Google Scholar] [Publisher Link]
36. Pascal Benschop et al., “Motion Representations for Privacy-Aware Cross-Domain Action Recognition,” Frontiers in Imaging, vol. 5, pp. 1-13, 2026.
    [CrossRef] [Google Scholar] [Publisher Link]