A Parallel Fusion RNN-LSTM Approach to Classify Mental Stress using EEG Data

A Parallel Fusion RNN-LSTM Approach to Classify Mental Stress using EEG Data

© 2022 by IJETT Journal
Volume-70 Issue-10
Year of Publication : 2022
Authors : Megha V. Gupta, Shubhangi L. Vaikole
DOI : 10.14445/22315381/IJETT-V70I10P228

How to Cite?

Megha V. Gupta, Shubhangi L. Vaikole, "A Parallel Fusion RNN-LSTM Approach to Classify Mental Stress using EEG Data," International Journal of Engineering Trends and Technology, vol. 70, no. 10, pp. 285-297, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I10P228

Preventing and physiological healing problems need early stress diagnosis and a participant's predisposition to operate healthily under stress. Traditional methods of evaluating anxiety levels, such as interviewing the person and having to ask strain-based queries to develop a better understanding of their situation and observing facial gestures - individuals under stress respond by changing their brows, pupils dilating, or one’s flashing strobe percentage could differentiate - are limited even though they may overlook stress episodes. Electroencephalogram (EEG) is a newly created physiological measure that has the potential to be utilized as a stress gauge in everyday life. It is due to the commercial availability of EEG headsets for studying brain activity conveniently and cost-effectively. This investigation used machine learning methods to classify stress status using resting-state EEG signal recordings. The method was tested using a dataset from the MathWorks® EEGLAB toolbox, and a dataset of 20 patients was constructed using a questionnaire and Neurosky's Mindwave EEG headset. For stress detection, a support vector machine (SVM), recurrent neural network (RNN), long short-term memory (LSTM), and a novel technique based on a parallel fusion of RNN-LSTM are used. The results of the MATLAB simulations show that the proposed technique is faster and more accurate than other machine-learning approaches. The proposed technique has a 95% accuracy rate, up to a 15% improvement over other results.

Stress detection, EEG signals, machine learning, EEGLAB toolbox, Meurosky's Mindwave EEG headset, MATLAB.

[1] R. Katmah, F. Al-Shargie, U. Tariq, F. Babiloni, F. Al-Mughairbi, and H. Al-Nashash, “A Review on Mental Stress Assessment Methods using EEG Signals,” Sensors, MDPI AG, vol. 21, no. 15, 2021. Doi: 10.3390/s21155043.
[2] G. Giannakakis, G. Dimitris, G. Katerina, S. Olympia, R. Alexandros, and T. Manolis, “Review on Psychological Stress Detection Using Biosignals,” IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 440–460, 2022. Doi: 10.1109/TAFFC.2019.2927337.
[3] D. B. O’connor, J. F. Thayer, and K. Vedhara, “Stress and Health: A Review of Psychobiological Processes,” Annual Review of Psychology, vol. 1, no. 72, pp. 663–688, 2021. Doi: 10.1146/annurev-psych-062520.
[4] A. R. Subhani, W. Mumtaz, M. N. B. M. Saad, N. Kamel, and A. S. Malik, “Machine Learning Framework for the Detection of Mental Stress at Multiple Levels,” IEEE Access, vol. 5, pp. 13545–13556, 2017. Doi: 10.1109/ACCESS.2017.2723622.
[5] G. Jun and S. K. G., “EEG-based Stress Level Identification,” in 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3270–3274, 2016. Doi: 10.1109/SMC.2016.7844738.
[6] P. Lahane, A. Vaidya, C. Umale, S. Shirude, and A. Raut, “Real Time System to Detect Human Stress Using EEG Signals,” International Journal of Innovative Research in Computer and Communication Engineering An ISO, vol. 3297, 2007. Doi: 10.15680/IJIRCCE.2016.
[7] H. Jebelli, S. Hwang, and S. H. Lee, “EEG-Based Workers’ Stress Recognition at Construction Sites,” Automation in Construction, vol. 93, pp. 315–324, 2018. Doi: 10.1016/j.autcon.2018.05.027.
[8] M. Zanetti et al., “Multilevel Assessment of Mental Stress via Network Physiology Paradigm using Consumer Wearable Devices,” Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 4, pp. 4409–4418, 2021. Doi: 10.1007/s12652-019- 01571-0.
[9] X. Hou, Y. Liu, O. Sourina, Y. R. E. Tan, L. Wang, and W. Mueller-Wittig, “EEG Based Stress Monitoring,” in Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015, pp. 3110–3115, 2016. Doi: 10.1109/SMC.2015.540.
[10] C. Vuppalapati, M. S. Khan, N. Raghu, P. Veluru, and S. Khursheed, “A System to Detect Mental Stress Using Machine Learning and Mobile Development,” Proceedings - International Conference on Machine Learning and Cybernetics, vol. 1, no. 7, pp. 161– 166, 2018. Doi: 10.1109/ICMLC.2018.8527004.
[11] G. Giorgos, G. Dimitris, and T. Manolis, “Detection of Stress/Anxiety State from EEG Features during Video Watching,” Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 7, no. 1, pp. 6037–6041, 2015. Doi: 10.1109/EMBC.2015.7319767.
[12] G. Pallavi and P. A. N, “Novel Approach for Stress Recognition using EEG Signal by SVM Classifier,” in 2017 International Conference on Computing Methodologies and Communication (ICCMC), pp. 967–971, 2018. Doi: 10.1109/ICCMC.2017.8282611.
[13] P. Lahane and M. Thirugnanam, “A Novel Approach for Analyzing Human Emotions Based on Electroencephalography (EEG),” in International Conference on Innovations in Power and Advanced Computing Technologies [i-PACT2017], pp. 1–6, 2017. Doi: 10.1109/IPACT.2017.8245056.
[14] X. Zhang, S. A. Huettel, O. A. Mullette-Gillman, H. Guo, and L. Wang, “Exploring Common Changes After Acute Mental Stress and Acute Tryptophan Depletion: Resting-State FMRI Studies,” Journal of Psychiatric Research, vol. 113, pp. 172–180, 2019. Doi: 10.1016/j.jpsychires.2019.03.025.
[15] F. Al-Shargie, T. B. Tang, and M. Kiguchi, “Stress Assessment Based on Decision Fusion of EEG and fNIRS Signals,” IEEE Access, vol. 5, pp. 19889–19896, 2017. Doi: 10.1109/ACCESS.2017.2754325.
[16] F. Al-shargie, T. B. Tang, N. Badruddin, and M. Kiguchi, “Towards Multilevel Mental Stress Assessment using SVM With ECOC: An EEG Approach,” Medical and Biological Engineering and Computing, vol. 56, no. 1, pp. 125–136, 2018. Doi: 10.1007/s11517-017-1733-8.
[17] N. M. Ehrhardt, J. Fietz, J. Kopf-Beck, N. Kappelmann, and A. K. Brem, “Separating EEG Correlates of Stress: Cognitive Effort, Time Pressure, And Social-Evaluative Threat,” European Journal of Neuroscience, 2021. Doi: 10.1111/ejn.15211.
[18] S. Keshmiri, “Conditional Entropy: A Potential Digital Marker for Stress,” Entropy, vol. 23, no. 3, pp. 1-14, 2021. Doi: 10.3390/e23030286.
[19] E. Alyan et al., “Frontal Electroencephalogram Alpha Asymmetry during Mental Stress Related to Workplace Noise,” Sensors, vol. 21, no. 6, pp. 1–12, 2021. Doi: 10.3390/s21061968.
[20] J. Chae, S. Hwang, W. Seo, and Y. Kang, “Relationship Between Rework of Engineering Drawing Tasks and Stress Level Measured From Physiological Signals,” Automation in Construction, vol. 124, 2021. Doi: 10.1016/j.autcon.2021.103560.
[21] F. Al-Shargie, U. Tariq, F. Babiloni, and H. Al-Nashash, “Cognitive Vigilance Enhancement Using Audio Stimulation of Pure Tone at 250 Hz,” IEEE Access, vol. 9, pp. 22955–22970, 2021. Doi: 10.1109/ACCESS.2021.3054785.
[22] P. Boonyakitanont, A. Lek-uthai, K. Chomtho, and J. Songsiri, “A Review of Feature Extraction and Performance Evaluation in Epileptic Seizure Detection Using EEG,” Biomedical Signal Processing and Control, Elsevier Ltd, vol. 57, 2020. Doi: 10.1016/j.bspc.2019.101702.
[23] X. Jiang, G. bin Bian, and Z. Tian, “Removal of Artifacts from EEG Signals: A Review,” Sensors (Switzerland), MDPI AG, vol. 19, no. 5, 2019. Doi: 10.3390/s19050987.
[24] W. Mumtaz, S. Rasheed, and A. Irfan, “Review of Challenges Associated with the EEG Artifact Removal Methods,” Biomedical Signal Processing and Control, vol. 68, 2021. Doi: 10.1016/j.bspc.2021.102741.
[25] K. J. Friston, “Functional and Effective Connectivity: A Review,” Brain Connectivity, vol. 1, no. 1, pp. 13-36, 2011. Doi: 10.1089/brain.2011.0008.
[26] L. Xia, A. S. Malik, and A. R. Subhani, “A Physiological Signal-Based Method for Early Mental-Stress Detection,” Biomedical Signal Processing and Control, vol. 46, pp. 18-32, 2018. Doi: 10.1016/j.bspc.2018.06.004.
[27] S. M. U. Saeed, S. M. Anwar, M. Majid, M. Awais, and M. Alnowami, “Selection of Neural Oscillatory Features for Human Stress Classification with Single Channel EEG Headset,” BioMed Research International, vol. 2018, 2018. Doi: 10.1155/2018/1049257.
[28] I. Dziembowska, P. Izdebski, A. Rasmus, J. Brudny, M. Grzelczak, and P. Cysewski, “Effects of Heart Rate Variability Biofeedback on EEG Alpha Asymmetry and Anxiety Symptoms in Male Athletes: A Pilot Study,” Applied Psychophysiology Biofeedback, vol. 41, no. 2, pp. 141–150, 2016. Doi: 10.1007/s10484-015-9319-4.
[29] N. Barraza, S. Moro, M. Ferreyra, and A. de la Peña, “Mutual Information and Sensitivity Analysis for Feature Selection in Customer Targeting: A Comparative Study,” Journal of Information Science, vol. 45, no. 1, pp. 53–67, 2019. Doi: 10.1177/0165551518770967.
[30] R. A. Movahed, G. P. Jahromi, S. Shahyad, and G. H. Meftahi, “A Major Depressive Disorder Classification Framework Based on EEG Signals using Statistical, Spectral, Wavelet, Functional Connectivity, and Nonlinear Analysis,” Journal of Neuroscience Methods, vol. 358, 2021. Doi: 10.1016/j.jneumeth.2021.109209.
[31] H. Peng, Y. Gao, and X. Mao, “The Roles of Sensory Function and Cognitive Load in Age Differences in Inhibition: Evidence from the Stroop Task,” Psychology and Aging, vol. 32, no. 1, pp. 42–50, 2017. Doi: 10.1037/pag0000149.
[32] R. Khosrowabadi, “Stress and Perception of Emotional Stimuli: Long-Term Stress Rewiring the Brain,” Basic and Clinical Neuroscience, vol. 9, no. 2, pp. 107–120, 2018. Doi: 10.29252/nirp.bcn.9.2.107.
[33] F. M. Al-Shargie, O. Hassanin, U. Tariq, and H. Al-Nashash, “EEG-Based Semantic Vigilance Level Classification Using Directed Connectivity Patterns and Graph Theory Analysis,” IEEE Access, vol. 8, pp. 115941–115956, 2020. Doi: 10.1109/ACCESS.2020.3004504.
[34] F. Al-Shargie, U. Tariq, O. Hassanin, H. Mir, F. Babiloni, and H. Al-Nashash, “Brain Connectivity Analysis Under Semantic Vigilance and Enhanced Mental States,” Brain Sciences, vol. 9, no. 12, 2019. Doi: 10.3390/brainsci9120363.
[35] A. Arsalan, M. Majid, A. R. Butt, and S. M. Anwar, “Classification of Perceived Mental Stress using a Commercially Available EEG Headband,” IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 6, pp. 2257–2264, 2019. Doi: 10.1109/JBHI.2019.2926407.
[36] O. Attallah, “An Effective Mental Stress State Detection and Evaluation System using Minimum Number of Frontal Brain Electrodes,” Diagnostics, vol. 10, no. 5, 2020. Doi: 10.3390/diagnostics10050292.
[37] M. J. Hasan and J. M. Kim, “A Hybrid Feature Pool Based Emotional Stress State Detection Algorithm Using EEG Signals,” Brain Sciences, vol. 9, no. 12, 2019. Doi: 10.3390/brainsci9120376.
[38] D. Shon, K. Im, J. H. Park, D. S. Lim, B. Jang, and J. M. Kim, “Emotional Stress State Detection using Genetic Algorithm-Based Feature Selection on EEG Signals,” International Journal of Environmental Research and Public Health, vol. 15, no. 11, 2018. Doi: 10.3390/ijerph15112461.
[39] S. M. U. Saeed, S. M. Anwar, H. Khalid, M. Majid, and U. Bagci, “EEG-based Classification of Long-Term Stress Using Psychological Labeling,” Sensors (Switzerland), vol. 20, no. 7, 2020. Doi: 10.3390/s20071886.
[40] J. Minguillon, E. Perez, M. A. Lopez-Gordo, F. Pelayo, and M. J. Sanchez-Carrion, “Portable System for Real-Time Detection of Stress Level,” Sensors (Switzerland), vol. 18, no. 8, 2018. Doi: 10.3390/s18082504.
[41] S. Lotfan, S. Shahyad, R. Khosrowabadi, A. Mohammadi, and B. Hatef, “Support Vector Machine Classification of Brain States Exposed to Social Stress Test Using EEG-Based Brain Network Measures,” Biocybernetics and Biomedical Engineering, vol. 39, no. 1, pp. 199–213, 2019. Doi: 10.1016/j.bbe.2018.10.008.
[42] Z. Halim and M. Rehan, “On Identification of Driving-Induced Stress using Electroencephalogram Signals: A Framework Based on Wearable Safety-Critical Scheme and Machine Learning,” Information Fusion, vol. 53, pp. 66–79, 2020. Doi: 10.1016/j.inffus.2019.06.006.
[43] J. W. Ahn, Y. Ku, and H. C. Kim, “A Novel Wearable EEG and ECG Recording System for Stress Assessment,” Sensors (Switzerland), vol. 19, no. 9, 2019. Doi: 10.3390/s19091991.
[44] K. Masood and M. A. Alghamdi, “Modeling Mental Stress Using a Deep Learning Framework,” IEEE Access, vol. 7, pp. 68446– 68454, 2019. Doi: 10.1109/ACCESS.2019.2917718.
[45] K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, “When Is ‘Nearest Neighbor’ Meaningful?,” Springer Lecture Notes in Computer Science, vol. 1540, pp. 217–235, 1999. Doi: https://doi.org/10.1007/3-540-49257-7_15.
[46] D. Devi, S. Sophia, A. Athithya Janani, and M. Karpagam, “Brain Wave-Based Cognitive State Prediction for Monitoring Health Care Conditions,” Materials Today: Proceedings, 2020. Doi: 10.1016/j.matpr.2020.09.616.
[47] N. Murali Krishna et al., “An Efficient Mixture Model Approach in Brain-Machine Interface Systems for Extracting the Psychological Status of Mentally Impaired Persons Using EEG Signals,” IEEE Access, vol. 7, pp. 77905–77914, 2019. Doi: 10.1109/ACCESS.2019.2922047.
[48] F. Al-Shargie, U. Tariq, M. Alex, H. Mir, and H. Al-Nashash, “Emotion Recognition Based on Fusion of Local Cortical Activations and Dynamic Functional Networks Connectivity: An EEG Study,” IEEE Access, vol. 7, pp. 143550–143562, 2019. Doi: 10.1109/ACCESS.2019.2944008.
[49] A. Delorme and S. Makeig, “EEGLAB: An Open-Source Toolbox for Analysis of Single-Trial EEG Dynamics Including Independent Component Analysis,” Journal of Neuroscience Methods, vol. 134, no. 1, pp. 9–21, 2004. Doi: 10.1016/j.jneumeth.2003.10.009.
[50] S. Koelstra et al., “DEAP: A Database for Emotion Analysis Using Physiological Signals,” IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18–31, 2012. Doi: 10.1109/T-AFFC.2011.15