Citation :

Ganta Raju, GS Naveen Kumar, "Intelligent Crowd Analysis: A Learning-Based Approach for Detection and Behavior Analysis in Surveillance Videos," International Journal of Engineering Trends and Technology (IJETT), vol. 74, no. 7, pp. 493-521, 2026. Crossref, https://doi.org/10.14445/22315381/IJETT-V74I7P130

Abstract

Crowd monitoring and analysis is a recent initiative that brings public safety as a result (especially if your business is in a crowded area) This work present a novel Intelligent Crowd Analysis (ICA) system in this paper that takes advantage of a Modified YOLOv4-tiny object detection model with Non-Maximum Suppression (NMS), Deep SORT object tracking and a Crowd Monitoring and Behavior Analysis (CMBA)module that is designed by us. The system enables the detection of real-time violations in social distancing, entry in restricted areas, and abnormal crowd behavior. In order to tackle these challenges, such as overlapping objects, dense crowds, and dynamic conditions, some domain-specific changes, such as setting anchor boxes manually and adding attention to the split detector networks, have been made. Experiments show that the system is effective and robust in different surveillance situations. When comparing the results of the Modified YOLOv4 with the state-of-the-art models like YOLOv3, Faster R-CNN, SSD, and FairMOT, the Modified YOLOv4 achieved the best precision (96.87%), recall (95.31%), F1-score (96.08%), and accuracy (97.46%) within no time while the real-time video/frame or image processing was performed through deep learning on individual vehicle counting providing better performance of results over these methods [153]. The system runs with an average of 25 FPS; thus, real-time capabilities are guaranteed. In addition to these totals, the performance achieved a 92% detection accuracy, identified situations of social distance violations, Restricted Area Violations, and abnormal motion with highly Reliant Identification. This proves that this intelligent surveillance system operates in very complex environments and in real-time, which makes it suitable for public safety, smart cities, and event management-related applications. The ICA system demonstrates improved performance in crowd analysis and monitoring.

Keywords

Crowd Detection, Behavior Analysis, Public Surveillance, Learning-Based Methodology, Intelligent Video Analysis.

References

[1] G. Sreenu, and M.A. Saleem Durai, “Intelligent Video Surveillance: A Review Through Deep Learning Techniques for Crowd Analysis,” Journal of Big Data, vol. 6, no. 1, pp. 1-27, 2019.
[CrossRef] [Google Scholar] [Publisher Link]   

[2] Francisco Luque Sánchez et al., “Revisiting Crowd Behavior Analysis Through Deep Learning: Taxonomy, Anomaly Detection, Crowd Emotions, Datasets, Opportunities, and Prospects,” Information Fusion, vol. 64, pp. 318-335, 2020.
[CrossRef] [Google Scholar] [Publisher Link]   

[3] Fariba Rezaei, and Mehran Yazdi, “Real-Time Crowd Behavior Recognition in Surveillance Videos based on Deep Learning Methods,” Journal of Real-Time Image Processing, vol. 18, no. 5, pp. 1669-1679, 2021.
[CrossRef] [Google Scholar] [Publisher Link]   

[4] Khosro Rezaee et al., “A Survey on Deep Learning-based Real-Time Crowd Anomaly Detection for Secure Distributed Video Surveillance,” Personal and Ubiquitous Computing, vol. 28, no. 1, pp. 135-151, 2021.
[CrossRef] [Google Scholar] [Publisher Link]     

[5] Karishma Pawar, and Vahida Attar, “Application of Deep Learning for Crowd Anomaly Detection from Surveillance Videos,” 2021 11^th International Conference on Cloud Computing, Data Science and Engineering (Confluence), Noida, India, pp. 506-511, 2021.
[CrossRef] [Google Scholar] [Publisher Link]    

[6] Mounir Bendali-Braham et al., “Recent Trends in Crowd Analysis: A Review,” Machine Learning with Applications, vol. 4, pp. 1-30, 2021.
[CrossRef] [Google Scholar] [Publisher Link]    

[7] C.V. Amrutha, C. Jyotsna, and J. Amudha, “Deep Learning Approach for Suspicious Activity Detection from Surveillance Video,” 2020 2^nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), Bangalore, India, pp. 335-339, 2020.
[CrossRef] [Google Scholar] [Publisher Link]    

[8] Heyfa Ammar, and Asma Cherif, “Deeprod: A Deep Learning Approach for Real-Time and Online Detection of a Panic Behavior in Human Crowds,” Machine Vision and Applications, vol. 32, no. 3, 2021.
[CrossRef] [Google Scholar] [Publisher Link]    

[9] Tanu Gupta, Vimala Nunavath, and Sudip Roy, “CrowdVAS-Net: A Deep-CNN based Framework to Detect Abnormal Crowd-Motion Behavior in Videos for Predicting Crowd Disaster,” 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy, pp. 2877-2882, 2019.
[CrossRef] [Google Scholar] [Publisher Link]      

[10] Pierre Bour, Emile Cribelier, and Vasileios Argyriou, Crowd Behavior Analysis from Fixed and Moving Cameras, Multimodal Behavior Analysis in the Wild, Acadamic Press, pp. 289-322, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[11] M. Sami Zitouni, Andrzej Sluzek, and Harish Bhaskar, “Visual Analysis of Socio-Cognitive Crowd Behaviors for Surveillance: A Survey and Categorization of Trends and Methods,” Engineering Applications of Artificial Intelligence, vol. 82, pp. 294-312, 2019.
[CrossRef] [Google Scholar] [Publisher Link]      

[12] Franjo Matkovic, Darijan Marčetic, and Slobodan Ribaric, “Abnormal Crowd Behaviour Recognition in Surveillance Videos,” 2019 15^th International Conference on Signal-Image Technology and Internet-based Systems (SITIS), Sorrento, Italy, pp. 428-435, 2019.
[CrossRef] [Google Scholar] [Publisher Link]        

[13] Navneet Nayan, Sitanshu Sekhar Sahu, and Sanjeet Kumar, “Detecting Anomalous Crowd Behavior using Correlation Analysis of Optical Flow,” Signal, Image and Video Processing, vol. 13, no. 6, pp. 1233-1241, 2019.
[CrossRef] [Google Scholar] [Publisher Link]        

[14] Cem Direkoglu, “Abnormal Crowd Behavior Detection using Motion Information Images and Convolutional Neural Networks,” IEEE Access, vol. 8, pp. 80408-80416, 2020.
[CrossRef] [Google Scholar] [Publisher Link]        

[15] Kang Hao Cheong et al., “Practical Automated Video Analytics for Crowd Monitoring and Counting,” IEEE Access, vol. 7, pp. 183252-183261, 2019.
[CrossRef] [Google Scholar] [Publisher Link]        

[16] Dushyant Kumar Singh et al., “Human Crowd Detection for City Wide Surveillance,” Procedia Computer Science, vol. 171, pp. 350-359, 2020.
[CrossRef] [Google Scholar] [Publisher Link]        

[17] M. Sami Zitouni, Andrzej Sluzek, and Harish Bhaskar, “Towards Understanding Socio-Cognitive Behaviors of Crowds from Visual Surveillance Data,” Multimedia Tools and Applications, vol. 79, no. 3-4, pp. 1781-1799, 2019.
[CrossRef] [Google Scholar] [Publisher Link]        

[18] Rashmiranjan Nayak, Umesh Chandra Pati, and Santos Kumar Das, “A Comprehensive Review on Deep Learning-based Methods for Video Anomaly Detection,” Image and Vision Computing, vol. 106, 2021.
[CrossRef] [Google Scholar] [Publisher Link]        

[19] Piyush Juyal, and Sachin Sharma, “Locating People in Real-World for Assisting Crowd Behaviour Analysis using SSD and Deep SORT Algorithm,” 2021 Sixth International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), Chennai, India, pp. 350-353, 2021.
[CrossRef] [Google Scholar] [Publisher Link]        

[20] Kuldeep Singh et al., “Crowd Anomaly Detection using Aggregation of Ensembles of Fine-Tuned ConvNets,” Neurocomputing, vol. 371, pp. 188-198, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[21] Aiswarya Mohan, Meghavi Choksi, and Mukesh A. Zaveri, “Anomaly and Activity Recognition using Machine Learning Approach for Video based Surveillance,” 2019 10^th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India, pp. 1-6, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[22] Gaurav Tripathi, Kuldeep Singh, and Dinesh Kumar Vishwakarma, “Crowd Emotion Analysis using 2D ConvNets,” 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India, pp. 969-974, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[23] Anitha Ramchandran, and Arun Kumar Sangaiah, “Unsupervised deep Learning System for Local Anomaly Event Detection in Crowded Scenes,” Multimedia Tools and Applications, vol. 79, no. 47-48, pp. 35275-35295, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[24] Yan Hu, “Design and Implementation of Abnormal Behavior Detection based on Deep Intelligent Analysis Algorithms in Massive Video Surveillance,” Journal of Grid Computing, vol. 18, no. 2, pp. 227-237, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[25] Ali M. Al-Shaery et al., “In-Depth Survey to Detect, Monitor and Manage Crowd,” IEEE Access, vol. 8, pp. 209008-209019, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[26] Abdulaziz Salamah Aljaloud, and Habib Ullah, “IA-SSLM: Irregularity-Aware Semi-Supervised Deep Learning Model for Analyzing Unusual Events in Crowds,” IEEE Access, vol. 9, pp. 73327-73334, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[27] Elizabeth B. Varghese, and Sabu M. Thampi, “Application of Cognitive Computing for Smart Crowd Management,” IT Professional, vol. 22, no. 4, pp. 43-50, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[28] Sohail Salim et al., “Crowd Detection and Tracking in Surveillance Video Sequences,” 2019 IEEE International Conference on Smart Instrumentation, Measurement and Application (ICSIMA), Kuala Lumpur, Malaysia, pp. 1-6, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[29] Yan Fu, Tao Liu, and Ou Ye, “Abnormal Activity Recognition based on Deep Learning in Crowd,” 2019 11^th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), Hangzhou, China, pp. 301-304, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[30] Zirgham Ilyas et al., “A Hybrid Deep Network based Approach for Crowd Anomaly Detection,” Multimedia Tools and Applications, vol. 80, no. 16, pp. 24053-24067, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[31] Shuqiang Guo et al., “An Analysis Method of Crowd Abnormal Behavior for Video Service Robot,” IEEE Access, vol. 7, pp. 169577-169585, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[32] Ebrahim Najafi Kajabad, and Sergey V. Ivanov, “People Detection and Finding Attractive Areas by the use of Movement Detection Analysis and Deep Learning Approach,” Procedia Computer Science, vol. 156, pp. 327-337, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[33] Omar Elharrouss, Noor Almaadeed, and Somaya Al-Maadeed, “A Review of Video Surveillance Systems,” Journal of Visual Communication and Image Representation, vol. 77, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[34] Habib Ullah et al., “Multi-Feature-based Crowd Video Modeling for Visual Event Detection,” Multimedia Systems, vol. 27, no. 4, pp. 589-597, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[35] Xuguang Zhang et al., “Crowd Emotion Evaluation based on Fuzzy Inference of Arousal and Valence,” Neurocomputing, vol. 445, pp. 194-205, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[36] Xinlei Wei et al., “A Very Deep Two-stream Network for Crowd Type Recognition,” Neurocomputing, vol. 396, pp. 522-533, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[37] Xianzhi Li et al., “Data Fusion for Intelligent Crowd Monitoring and Management Systems: A Survey,” IEEE Access, vol. 9, pp. 47069-47083, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[38] V.X. Gong et al., “Counting People in the Crowd using Social Media Images for Crowd Management in City Events,” Transportation, vol. 48, no. 6, pp. 3085-3119, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[39] Zhenzhen Yao et al., “Learning Crowd Behavior from Real Data: A Residual Network Method for Crowd Simulation,” Neurocomputing, vol. 404, pp. 173-185, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[40] Junyu Gao, Yuan Yuan, and Qi Wang, “Feature-Aware Adaptation and Density Alignment for Crowd Counting in Video Surveillance,” IEEE Transactions on Cybernetics, vol. 51, no. 10, pp. 4822-4833, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[41] Irina V. Pustokhina et al., “An Automated Deep Learning based Anomaly Detection in Pedestrian Walkways for Vulnerable Road users Safety,” Safety Science, vol. 142, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[42] Xingjiao Wu et al., “Fast Video Crowd Counting with a Temporal Aware Network,” Neurocomputing, vol. 403, pp. 13-20, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[43] Nian Chi Tay et al., “Abnormal Behavior Recognition using CNN-LSTM with Attention Mechanism,” 2019 1^st International Conference on Electrical, Control and Instrumentation Engineering (ICECIE), Kuala Lumpur, Malaysia, pp. 1-5, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[44] Khosro Rezaee et al., “Smart Visual Sensing for Overcrowding in COVID-19 Infected Cities using Modified Deep Transfer Learning,” IEEE Transactions on Industrial Informatics, vol. 19, no. 1, pp. 813-820, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[45] Maha Hamdan Alotibi et al., “CNN-based Crowd Counting through IoT: Application for Saudi Public Places,” Procedia Computer Science, vol. 163, pp. 134-144, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[46] Qingyang Zhang et al., “Edge Video Analytics for Public Safety: A Review,” IEEE, vol. 107, no. 8, pp.1675-1696, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[47] Shuheng Lin et al., “Social MIL: Interaction-Aware for Crowd Anomaly Detection,” 2019 16^th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, Taiwan, pp. 1-8, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[48] Bo Li et al., “Approaches on Crowd Counting and Density Estimation: A Review,” Pattern Analysis and Applications, vol. 24, no. 3, pp. 853-874, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[49] Xing Hu et al., “A Weakly Supervised Framework for Abnormal Behavior Detection and Localization in Crowded Scenes,” Neurocomputing, vol. 383, pp. 270-281, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[50] Wei Lin et al., “Learning to Detect Anomaly Events in Crowd Scenes from Synthetic Data,” Neurocomputing, vol. 436, pp. 248-259, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[51] Peter Kok-Yiu Wong et al., “Recognition of Pedestrian Trajectories and Attributes with Computer Vision and Deep Learning Techniques,” Advanced Engineering Informatics, vol. 49, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[52] Monica Gruosso, Nicola Capece, and Ugo Erra, “Human Segmentation in Surveillance Video with Deep Learning,” Multimedia Tools and Applications, vol. 80, no. 1, pp. 1175-1199, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[53] Yesin Sahraoui et al., “DeepDist: A Deep-Learning-based IoV Framework for Real-Time Objects and Distance Violation Detection,” IEEE Internet of Things Magazine, vol. 3, no. 3, pp. 30-34, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[54] Haoyue Bai, Jiageng Mao, and S.-H. Gary Chan, “A Survey on Deep Learning-based Single Image Crowd Counting: Network Design, Loss Function and Supervisory Signal,” Neurocomputing, vol. 508, pp. 1-18, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[55] Haopeng Li et al., “Video Crowd Localization with Multifocus Gaussian Neighborhood Attention and a Large-Scale Benchmark,” IEEE Transactions on Image Processing, vol. 31, pp. 6032-6047, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[56] Qi Wang et al., “NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 6, pp. 2141-2149, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[57] Virginia Negri et al., “Image-based Social Sensing: Combining AI and the Crowd to Mine Policy-Adherence Indicators from Twitter,” 2021 IEEE/ACM 43^rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS), Madrid, ES, pp. 92-101, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[58] Tehreem Qasim, and Naeem Bhatti, “A Hybrid Swarm Intelligence based Approach for Abnormal Event Detection in Crowded Environments,” Pattern Recognition Letters, vol. 128, pp. 220-225, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[59] Sabrina Aberkane, and Mohamed Elarbi, “Deep Reinforcement Learning for Real-world Anomaly Detection in Surveillance Videos,” 2019 6^th International Conference on Image and Signal Processing and their Applications (ISPA), Mostaganem, Algeria, pp. 1-5, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[60] Badri Narayan Subudhi et al., “Big Data Analytics for Video Surveillance,” Multimedia Tools and Applications, vol. 78, no. 18, pp. 26129-26162, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[61] Lur Tze Hsien, and Indriyati Atmosukarto, “Video Analytics in Train Cabin using Deep Learning,” 2019 4^th International Conference on Intelligent Transportation Engineering (ICITE), Singapore, pp. 94-98, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[62] Giovanna Castellano et al., “Crowd Detection in Aerial Images using Spatial Graphs and Fully-Convolutional Neural Networks,” IEEE Access, vol. 8, pp. 64534-64544, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[63] Rongyong Zhao et al., “Image-based Crowd Stability Analysis using Improved Multi-Column Convolutional Neural Network,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 5480-5489, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[64] Reem Alotaibi et al., “Performance Comparison and Analysis for Large-Scale Crowd Counting based on Convolutional Neural Networks,” IEEE Access, vol. 8, pp. 204425-204432, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[65] Muhammad Attique Khan et al., “A Deep Survey on Supervised Learning based Human Detection and Activity Classification Methods,” Multimedia Tools and Applications, vol. 80, no. 18, pp. 27867-27923, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[66] Valerio Nogueira et al., “RetailNet: A Deep Learning Approach for People Counting and Hot Spots Detection in Retail Stores,” 2019 32^nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Rio de Janeiro, Brazil, pp. 155-162, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[67] Mohammed Sultan Mohammed et al., “Motion Pattern-based Scene Classification using Adaptive Synthetic Oversampling and Fully Connected Deep Neural Network,” IEEE Access, vol. 11, pp. 119659-119675, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[68] Saleh Basalamah et al., “Deep Learning Framework for Congestion Detection at Public Places Via Learning from Synthetic Data,” Journal of King Saud University - Computer and Information Sciences, vol. 35, no. 1, pp. 102-114, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[69] Ruchika, and Ravindra Kumar Purwar, “Crowd Density Estimation using Hough Circle Transform for Video Surveillance,” 2019 6^th International Conference on Signal Processing and Integrated Networks (SPIN), pp. 442-447, 2019.
[Google Scholar]

[70] Virender Singh, Swati Singh, and Pooja Gupta, “Real-Time Anomaly Recognition Through CCTV using Neural Networks,” Procedia Computer Science, vol. 173, pp. 254-263, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[71] Muhammad Zia-ul-Rehman et al., “AI and IoT-based Frameworks for Real-Time Crowd Monitoring and Security,” Annual Methodological Archive Research Review, vol. 3, no. 5, pp. 292-299, 2025.
[Google Scholar]

[72] Ayman A. Alharbi, “DeepCAMS: A Deep Learning Approach for Real-Time Crowd Monitoring and Suspicious Behavior Detection using Spatial-Temporal Analysis,” Engineering, Technology and Applied Science Research, vol. 15, no. 4, pp. 26113-26119, 2025.
[CrossRef] [Google Scholar] [Publisher Link]

[73] Sarah Altowairqi et al., “Efficient Crowd Anomaly Detection using C3D-LSTM Networks with Enhanced Attention Mechanisms,” Array, vol. 29, pp. 1-14, 2026.
[CrossRef] [Google Scholar] [Publisher Link]

[74] P. Siva et al., “Smart Surveillance Systems using YOLOv8: A Scalable Approach for Crowd and Threat Detection,” International Journal of Recent Advances in Engineering and Technology, vol. 14, no. 1, pp. 51-62, 2024.
[Google Scholar]

[75] Rabia Nasir et al., “An Enhanced Framework for Real-Time Dense Crowd Abnormal Behavior Detection using YOLOv8,” Artificial Intelligence Review, vol. 58, no. 7, pp. 1-50, 2025.
[CrossRef] [Google Scholar] [Publisher Link]

[76] Zhijie Xu et al., “Passenger Flow Prediction of Scenic Spot using a GCN–RNN Model,” Sustainability, vol. 14, no. 6, pp. 1-14, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[77] J. Anvar Shathik et al., “Smart Vision Systems for Public Safety: Real-Time Crowd Monitoring and Anomaly Detection in Urban Spaces using Deep Learning and Edge Computing,” International Journal of Applied Mathematics, vol. 389, no. 6s, pp. 720-743, 2025.
[CrossRef] [Google Scholar] [Publisher Link]

[78] Zarif Bin Akhtar, “Beyond Perception: A Comprehensive Investigation into the Advancements, Challenges and Ethical Dimensions of AI and Computer Vision,” Real-World AI Systems, vol. 1, no. 1, pp. 1-27, 2025.
[CrossRef] [Google Scholar] [Publisher Link]