Unlocking Long-Term Temporal Patterns: TCAE for Anomaly Detection in Multivariate Time Series Data
Unlocking Long-Term Temporal Patterns: TCAE for Anomaly Detection in Multivariate Time Series Data |
||
 |
 |
|
| © 2024 by IJETT Journal | ||
| Volume-72 Issue-9 |
||
| Year of Publication : 2024 | ||
| Author : Sangeeta Oswal, Subhash Shinde, Vijayalakshmi M |
||
| DOI : 10.14445/22315381/IJETT-V72I9P123 | ||
How to Cite?
Sangeeta Oswal, Subhash Shinde, Vijayalakshmi M, "Unlocking Long-Term Temporal Patterns: TCAE for Anomaly Detection in Multivariate Time Series Data," International Journal of Engineering Trends and Technology, vol. 72, no. 9, pp. 283-296, 2024. Crossref, https://doi.org/10.14445/22315381/IJETT-V72I9P123
Abstract
Anomaly detection in multivariate time series is extremely important in modern industrial systems to mitigate attacks and minimize system downtime. Anomalies are often seen as subtle deviations from established normal patterns. The challenge of learning extended temporal patterns in time series remains unresolved, hindering effective anomaly detection. To address the above challenge in this study, a novel approach, termed Temporal Convolutional Auto Encoder (TCAE), is introduced. TCAE utilizes Temporal Convolution Networks (TCN) and employs casual convolutions and dilations to effectively simulate long-term dependency in sequential data, taking advantage of its temporality and large fields. The autoencoder is trained on normal operations to learn the temporal dependencies present in the input time series. Two anomaly detection strategies employing the Local Outlier Factor (LOF) and thresholding are investigated. A supervised grid search technique is employed to determine the threshold, optimizing the model's performance. The thresholding technique demonstrates a performance improvement of over 20% when compared to the average performance of other baseline models.
Keywords
Anomaly detection, Auto encoder, Deep learning, Local Outlier Factor, Multivariant Time Series, Temporal Convolution Network.
References
[1] Varun Chandola, Arindam Banerjee, and Vipin Kumar, “Anomaly Detection: A Survey,” ACM Computing Surveys (CSUR), vol. 41, no. 3, pp. 1-58, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Yuan Luo et al., “Deep Learning-Based Anomaly Detection in Cyber-Physical Systems: Progress and Opportunities,” ACM Computing Surveys (CSUR),” vol. 54, no. 5, pp. 1-36, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Raghavendra Chalapathy, and Sanjay Chawla, “Deep Learning for Anomaly Detection: A Survey,” arXiv, pp. 1-50, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Charu C. Aggarwal, “An Introduction to Outlier Analysis,” Outlier Analysis, Springer International Publishing, pp. 1-34, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Tram Truong-Huu et al., “An Empirical Study on Unsupervised Network Anomaly Detection Using Generative Adversarial Networks,” SPAI '20: Proceedings of the 1st ACM Workshop on Security and Privacy on Artificial Intelligence, pp. 20-29, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Julien Audibert et al., “Usad: Unsupervised Anomaly Detection on Multivariate Time Series,” KDD- 20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3395-3404, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Bo Zong et al., “Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection,” International Conference on Learning Representations, pp. 1-19, 2018.
[Google Scholar] [Publisher Link]
[8] Kyle Hundman et al., “Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding,” KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 387-395, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Chuxu Zhang et al., “A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data,” AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, vol. 33, pp. 1409-1416, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Eoin Brophy et al., “Generative Adversarial Networks in Time Series: A Survey and Taxonomy,” ACM Computing Surveys, vol. 55, no. 10, pp. 1-31, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Markus M. Breunig et al., “LOF: Identifying Density-Based Local Outliers,” Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, vol. 29, no. 2, pp. 93-104, 2000.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Simon D. Duque Anton, Sapna Sinha, and Hans Dieter Schotten, “Anomaly-Based Intrusion Detection in Industrial Data with SVM and Random Forests,” 2019 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia, pp. 1-6, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Asrul H. Yaacob et al., “Arima Based Network Anomaly Detection,” 2010 Second International Conference on Communication Software and Networks, Singapore, pp. 205-209, 2010.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Muhammad Saad et al., “Tackling Imputation Across Time Series Models Using Deep Learning and Ensemble Learning,” 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Toronto, ON, Canada, pp. 3084-3090, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Junyoung Chung et al., “Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling,” arXiv, pp. 1-9, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Pankaj Malhotra et al., “LSTM-Based Encoder-Decoder for Multi-Sensor Anomaly Detection,” arXiv, pp. 1-5, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Zhiwei Ji, Jiaheng Gong, and Jiarui Feng, “A Novel Deep Learning Approach for Anomaly Detection of Time Series Data,” Scientific Programming, vol. 2021, no. 1, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Daehyung Park, Yuuna Hoshi, and Charles C. Kemp, “A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder,” IEEE Robotics and Automation Letters, vol 3, no. 3, pp. 1544-1551, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Ya Su et al., “Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network,” KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2828-2837, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Xuanhao Chen et al., “Daemon: Unsupervised Anomaly Detection and Interpretation for Multivariate Time Series,” 2021 IEEE 37th International Conference on Data Engineering (ICDE), Chania, Greece, pp. 2225-2230, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Alexander Geiger et al., “TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks,” 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, pp. 33-43, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Dan Li et al., “MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks,” Artificial Neural Networks and Machine Learning – ICANN 2019: Text and Time Series, pp. 703-716, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Md Abul Bashar, and Richi Nayak, “TAnoGAN: Time Series Anomaly Detection with Generative Adversarial Networks,” 2020 IEEE Symposium Series on Computational Intelligence (SSCI), Canberra, ACT, Australia, pp. 1778-1785, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Katrina Chen, Mingbin Feng, and Tony S. Wirjanto, “Multivariate Time Series Anomaly Detection via Dynamic Graph Forecasting,” arXiv, pp. 1-11, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Ailin Deng, and Bryan Hooi, “Graph Neural Network-Based Anomaly Detection in Multivariate Time Series,” Proceedings of the AAAI Technical Track on Data Mining and Knowledge Management, vol. 35, no. 5, pp. 4027-4035, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Di Ge et al., “An enhanced Spatio-Temporal Constraints Network for Anomaly Detection in Multivariate Time Series,” Knowledge-Based Systems, vol. 283, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[27] Hang Zhao et al., “Multivariate Time-Series Anomaly Detection via Graph Attention Network,” 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy, pp. 841-850, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[28] Liwen Zhou, Qingkui Zeng, and Bo Li, “Hybrid Anomaly Detection via Multihead Dynamic Graph Attention Networks for Multivariate Time Series,” IEEE Access, vol. 10, pp. 40967-40978, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[29] Peihai Zhao, Xiaoyan Chang, and Mimi Wang, “A Novel Multivariate Time-Series Anomaly Detection Approach Using an Unsupervised Deep Neural Network,” IEEE Access, vol. 9, pp. 109025-109041, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[30] Markus Thill et al., “Temporal Convolutional Autoencoder for Unsupervised Anomaly Detection in Time Series,” Applied Soft Computing, vol. 112, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[31] Yangdong He, and Jiabao Zhao, “Temporal Convolutional Networks for Anomaly Detection in Time Series,” Journal of Physics: Conference Series, vol. 1213, no. 4, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[32] Shreshth Tuli, Giuliano Casale, and Nicholas R. Jennings, “TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data,” arXiv, pp. 1-15, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[33] Jina Kim, Hyeongwon Kang, and Pilsung Kang, “Time-Series Anomaly Detection with Stacked Transformer Representations and 1D Convolutional Network,” Engineering Applications of Artificial Intelligence, vol. 120, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[34] Ling-rui Yu, Qiu-hong Lu, and Yang Xue, “DTAAD: Dual TCN-Attention Networks for Anomaly Detection in Multivariate Time Series Data,” Knowledge-Based Systems, vol. 295, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[35] Fanyu Zeng et al., “Multivariate Time Series Anomaly Detection with Adversarial Transformer Architecture in the Internet of Things,” Future Generation Computer Systems, vol. 144, pp. 244-55, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[36] Alban Siffer et al., “Anomaly Detection in Streams with Extreme Value Theory,” KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1067-1075, 2047.
[CrossRef] [Google Scholar] [Publisher Link]
[37] Guangrun Wu, and Wenliang Qiu, “Threshold Selection for POT Framework in the Extreme Vehicle Loads Analysis Based on Multiple Criteria,” Shock and Vibration, vol. 2018, no. 1, pp. 1-9, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[38] Qingsong Wen et al., “Transformers in Time Series: A Survey,” arXiv, pp. 1-9, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[39] Shaojie Bai, J. Zico Kolter, and Vladlen Koltun, “An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling,” arXiv, pp. 1-14, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[40] Zhiguo Ding, and Minrui Fei, “An Anomaly Detection Approach Based on Isolation Forest Algorithm for Streaming Data Using Sliding Window,” IFAC Proceedings Volumes, vol. 46, no. 20, pp. 12-17, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[41] Jonathan Goh et al., “A Dataset to Support Research in the Design of Secure Water Treatment Systems,” Critical Information Infrastructures Security, Conference Paper, pp. 88-99, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[42] Aditya P. Mathur, and Nils Ole Tippenhauer, “SWaT: A Water Treatment Testbed for Research and Training on ICS Security,” 2016 International Workshop on Cyber-Physical Systems for Smart Water Networks (CySWater), Vienna, Austria, pp. 31-36, 2016.
[CrossRef] [Google Scholar] [Publisher Link]