Deployment of Deep Reinforcement Learning and Market Sentiment Aware Strategies in Automated Stock Market Prediction

Deployment of Deep Reinforcement Learning and Market Sentiment Aware Strategies in Automated Stock Market Prediction
  IJETT-book-cover           
  
© 2022 by IJETT Journal
Volume-70 Issue-1
Year of Publication : 2022
Authors : Keerthi Sagiraju, Prof. Shashi Mogalla
DOI :  10.14445/22315381/IJETT-V70I1P205

How to Cite?

Keerthi Sagiraju, Prof. Shashi Mogalla, "Deployment of Deep Reinforcement Learning and Market Sentiment Aware Strategies in Automated Stock Market Prediction," International Journal of Engineering Trends and Technology, vol. 70, no. 3, pp. 37-47, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I1P205

Abstract
Tracking and responding to the dynamic stock market to maximize profit within a risk-controlled framework to meet the investment objective continues to be challenging. In addition to this, it is understood that market sentiment also plays a pivotal role in investment decisions. The deep reinforcement learning concept shows promising outcomes on stock market prices by training an intelligent agent. The stock market investments and returns can be predicted using Reinforcement learning with historical data and agent-based training in the given environment. In the proposed approach, a deep reinforcement learning agent is trained that uses historical stock and market sentiment consisting of Dow Jones and S&P 500 and get the resultant strategy in trading with Auto Encoder and LSTM along with four algorithms, namely Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG) and Deep Q-Learning (DQN). These are compared using the financial metrics, Sharpe ratio, Sortino ratio, max drawdown, cumulative return, annual volatility, and annualized investment return.

Keywords
Deep reinforcement learning (DRL), Stock Market Trading, Sentiment Analysis, Auto Encoder(AE), Long Short-Term Memory(LSTM).

Reference
[1] Kartika MaulidaHindrayani; Tresna Maulana Fahrudin; R. PrismahardiAji; Eristya Maya Safitri, Indonesian Stock Price Prediction including Covid19 Era Using Decision Tree Regression, 3rd International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), (2020).
[2] JinlongRuan, Wei Wu, Jiebo Luo, Stock Price Prediction Under Anomalous Circumstances, IEEE International Conference on Big Data (Big Data), (2020).
[3] Christopher Chou; Junho Park; Eric Chou, Predicting Stock Closing Price After COVID-19 Based on Sentiment Analysis and LSTM, IEEE, (2021).
[4] Jae Won idee, STOCK PRICE PREDICTION USING REINFORCEMENT LEARNING, IEEE, (2001).
[5] Kunal Pahwa; Neha Agarwal, Stock Market Analysis using Supervised Machine Learning, International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), (2019).
[6] ZaharaddeenKaramiLawal; HayatiYassin; Rufai Yusuf Zakari, Stock Market Prediction using Supervised Machine Learning Techniques: An Overview, IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE), (2020).
[7] Jae Won Lee, Stock price prediction using reinforcement learning, IEEE International Symposium on Industrial Electronics Proceedings (Cat. No.01TH8570), (2001).
[8] V François-Lavet, P Henderson, R Islam. An introduction through deep reinforcement learningarXiv preprint arXi, (2018). - arxiv.org[PDF] arxiv.org.(
[9] PrahladKoratamaddi, Karan Wadhwani, Mridul Gupta a, Sriram G. Sanjeevi, Market sentiment-aware deep reinforcement learning approach for stock portfolio allocation, Engineering Science and Technology, an International Journal, (2021).
[10] Li, Y., Ni, P., & Chang, V. Application of deep reinforcement learning in stock trading strategies and stock forecasting. Computing, (2019) (1-18).
[11] Carta, S., Corriga, A., Ferreira, A., Podda, A. S., &Recupero, D. R. A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning., Applied Intelligence, 51(2) (2021) 889- 905.
[12] Rundo, F. Deep LSTM with reinforcement learning layer for financial trend prediction in FX high-frequency trading systems. Applied Sciences, 9 (20) (2019) 4460.
[13] Long, W., Lu, Z., & Cui, L. Deep learning-based feature engineering for stock price movement prediction. Knowledge-Based Systems, 164 (2019) 163-173).
[14] Dang, Q. V. Reinforcement learning in stock trading. In International Conference on Computer Science, Applied Mathematics and Applications (2019) 311-322. Springer, Cham.
[15] Jia, W. U., Chen, W. A. N. G., Xiong, L., &Hongyong, S. U. N. Quantitative trading on the stock market based on deep reinforcement learning. In 2019 International Joint Conference on Neural Networks (IJCNN), IEEE, (2019) (1-8).
[16] Huotari, T., Savolainen, J., &Collan, M. Deep reinforcement learning agent for S&P 500 stock selection. Axioms, 9(4) (2020) 130.
[17] M. Baker, J. Wurgler Investor sentiment in the stock market J. Econ. Perspect., 21 (2007) 129-152. 10.1257/jep.21.2.12.
[18] N. Oliveira, P. Cortez, N. Areal, The impact of microblogging data for stock market prediction: Using Twitter to predict returns, volatility, trading volume, and survey sentiment indices, Expert Syst. Appl. 73 (2017) 125–144. https:// doi.org/10.1016/j.eswa.2016.12.036.
[19] Wei Bao, Jun Yue, Yulei Rao, A deep learning framework for financial time series using stacked autoencoders and long short-term memory, (2017).
[20] C.J. Hutto, E. Gilbert, VADER: A parsimonious rule-based model for sentiment analysis of social media text, in E. Adar, P. Resnick (Eds.), Proc. 8th Int. Conf. Weblogs Soc. Media, ICWSM 2014, Ann Arbor, (2004).
[21] J Patel, S Shah, P Thakkar, K Kotecha, Predicting stock market index using a fusion of machine learning techniques, Expert Systems with Applications, (2015). Elsevier
[22] V Mnih, AP Badia, M Mirza, Graves Asynchronous methods for deep reinforcement learning The 33rd International Conference on Machine Learning, 02 (2016).[PDF] mlr.press
[23] J. Bollen, H. Mao, X. Zeng, Twitter mood predicts the stock market, J. Comput. Sci., 1-8. 10.1016/j.jocs.2010.12.007,2011.
[24] Mehrnaz Faraz, Hamid Khaloozadeh, Milad Abbasi, Stock Market Prediction-by-Prediction Based on Autoencoder Long Short-Term Memory Networks, Conference Paper • December, (2020). DOI: 10.1109/ICEE50131.2020.9261055.
[25] M Van Otterlo, M Wiering Reinforcement learning & Markov decision processes Reinforcement learning, (2012) – Springer[PDF] rug.nl
[26] S Carta, Ferreira, AS Podda, DR Recupero, Multi-DQN: An ensemble of Deep Q-learning agents for stock market forecasting unicamp. br
[27] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, Proximal policy optimization algorithms, arXiv:1707.06347, 07 (2017).
[28] T Théate, D Ernst An application of deep reinforcement learning through algorithmic trading[PDF] arxiv.org
[29] B. Yang, Z.-J. Gong, and W. Yang, Stock market index prediction using deep neural network ensemble, in 2017 36th Chinese Control Conference (CCC). IEEE, (2017) 3882–3887.
[30] F.N. Ribeiro, M. Araújo, P. Gonçalves, M. André Gonçalves, F. Benevenuto, SentiBench – a benchmark comparison of state-of-thepractice sentiment analysis methods, EPJ Data Sci., 5 (2016) 1–29. https://doi.org/10.1140/epjds/ s13688-016-0085-1.