Validation of Speed Limit Sign Recognition System in Virtual Environments: A Simulation-Based Approach

Validation of Speed Limit Sign Recognition System in Virtual Environments: A Simulation-Based Approach
  IJETT-book-cover           
  
© 2024 by IJETT Journal
Volume-72 Issue-9
Year of Publication : 2024
Author : Sangjoong Kim, Dongha Shim
DOI : 10.14445/22315381/IJETT-V72I9P137

How to Cite?
Sangjoong Kim, Dongha Shim, "Validation of Speed Limit Sign Recognition System in Virtual Environments: A Simulation-Based Approach," International Journal of Engineering Trends and Technology, vol. 72, no. 9, pp. 405-413, 2024. Crossref, https://doi.org/10.14445/22315381/IJETT-V72I9P137

Abstract
This paper presents a comprehensive approach to validate a speed limit sign recognition system, which is one of the Advanced Driving Assistance Systems in the virtual environment. Its necessity and advantages are emphasized for enhancing automotive safety and efficiency. The recognition of speed limit signs is highlighted as crucial to improving the functionality of driver assistance systems and the development of autonomous vehicles. However, the testing of these recognition systems using actual vehicles can be identified as entailing latent risks in addition to being time-consuming and financially demanding. To address these challenges, a simulation-based validation method is proposed, eliminating the hazards and reducing both the time and financial costs associated with real-world testing. Most of the research has used simulation techniques to test or enhance the performance of the algorithm that comprises the system, but usually neglecting the reliability of the simulation-based validation themselves. This paper focuses on validating the system in simulation and the reliability of the virtual environments through experiments. Finally, the effectiveness of simulation-based validation is elicited. The recognition system used in this paper is based on the You Only Look Once (YOLO) algorithm, renowned for object detection tasks. A diverse set of virtual data to mimic a wide range of real-world scenarios has been used to test the system. This paper presents a detailed comparison between the outcomes derived from tests conducted with real data and those obtained from virtual environment simulations. The results suggest that simulation-based validation can be a possible method for assessing speed limit sign recognition systems, with performance closely matching that in real-world conditions.

Keywords
Advanced Driving Assistance System (ADAS), Speed Limit Sign Recognition (SLSR), Object detection, You Only Look Once (YOLO), Digital twin.

References

[1] Jakkree Srinonchat, "Efficient Detection of Speed Limit Signs within Obscure Environment," IET 3rd International Conference on Wireless, Mobile and Multimedia Networks (ICWMNN 2010), Beijing, pp. 311-314, 2010.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Yogesh Valeja et al., "Traffic Sign Detection Using Clara and Yolo in Python," 2021 7th International Conference on Advanced Computing and Communication Systems, Coimbatore, India, pp. 367-371, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Christian Birchler et al., "Cost-Effective Simulation-Based Test Selection in Self-Driving Cars Software with SDC-Scissor," 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), Honolulu, HI, USA, pp. 164-168, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Yue Kang, Hang Yin, and Christian Berger, "Test Your Self-Driving Algorithm: An Overview of Publicly Available Driving Datasets and Virtual Testing Environments," IEEE Transactions on Intelligent Vehicles, vol. 4, no. 2, pp. 171-185, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Joseph Redmon, and Ali Farhadi, “YOLO9000: Better, Faster, Stronger,” 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 6517-6525, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Tafreed Ahmed et al., "The YOLOv8 Edge: Harnessing Custom Datasets for Superior Real-Time Detection," 2023 18th International Conference on Emerging Technologies, Peshawar, Pakistan, pp. 38-43, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Saurabh Pujar et al., "Invited: Automated Code Generation for Information Technology Tasks in YAML through Large Language Models," 2023 60th ACM/IEEE Design Automation Conference, San Francisco, CA, USA, pp. 1-4, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Everything you need to build and deploy computer vision models, Roboflow. [Online]. Available: https://roboflow.com
[9] Yang, Speed Limit Detection Computer Vision Project, Roboflow. [Online]. Available: https://universe.roboflow.com/yang-ygcec/speed-limit-detection
[10] Kanjar De, and Marius Pedersen, “Impact of Colour on Robustness of Deep Neural Networks,” 2021 IEEE/CVF International Conference on Computer Vision Workshops, Montreal, BC, Canada, pp. 21-30, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Performance Metrics Deep Dive, Ultralytics. [Online]. Available: https://docs.ultralytics.com/guides/yolo-performance-metrics/
[12] Tsung-Yi Lin et al., “Microsoft Coco: Common Objects in Context,” Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, pp. 740-755, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Ekaba Bisong, Google Colaboratory, Building Machine Learning and Deep Learning Models on Google Cloud Platform, Apress, Berkeley, CA, pp. 59-64, 2019. [CrossRef] [Google Scholar] [Publisher Link]
[14] Jesse Jon Davis, and Mark Harlan Goadrich, “The Relationship between Precision-Recall and ROC Curves,” Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh Pennsylvania USA, pp. 233-240, 2006.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Stéphan Clémençon, and Nicolas Vayatis, “Nonparametric Estimation of the Precision-Recall Curve,” Proceedings of the 26th Annual International Conference on Machine Learning, Montreal Quebec Canada, pp. 185-192, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Kendrick Boyd, Kevin H. Eng, and C. David Page, “Area under the Precision-Recall Curve: Point Estimates and Confidence Intervals,” Machine Learning and Knowledge Discovery in Databases: European Conference, Prague, pp. 451-466, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Jiahui Yu et al., “UnitBox: An Advanced Object Detection Network,” Proceedings of the 24th ACM International Conference on Multimedia, Amsterdam The Netherlands, pp. 516-520, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Yue Wu et al., “Rethinking Classification and Localization for Object Detection,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 10186-10195, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Siemens Digital Industries Software, Siemens. [Online]. Available: https://www.sw.siemens.com/en-US/