Transformer Based Knowledge Graph Construction in Adverse Drug Reactions Prediction from Social Media Reviews

Transformer Based Knowledge Graph Construction in Adverse Drug Reactions Prediction from Social Media Reviews
  IJETT-book-cover           
  
© 2022 by IJETT Journal
Volume-70 Issue-10
Year of Publication : 2022
Authors : Arijit Dey, Jitendra Nath Shrivastava, Chandan Kumar
DOI : 10.14445/22315381/IJETT-V70I10P239

How to Cite?

Arijit Dey, Jitendra Nath Shrivastava, Chandan Kumar, "Transformer Based Knowledge Graph Construction in Adverse Drug Reactions Prediction from Social Media Reviews," International Journal of Engineering Trends and Technology, vol. 70, no. 10, pp. 402-407, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I10P239

Abstract
Adverse Drug Reaction (ADR) prediction is an essential research topic in the field of pharmacovigilance. It seeks the attention of many researchers nowadays. Reporting of ADR in social media is increasing as patients can directly share their opinions through various online forums, blogs, Twitter, etc. Several deep neural network models have been introduced to detect the presence of ADR in the data collected from social media and sometimes attain a promising outcome while sometimes not up to the mark. Recently knowledge graph embedding was introduced in the prediction of ADR with the most familiar Word2Vec approach in Natural Language Processing (NLP). However, Word2Vec suffers from converting large textual data and the context of data as this approach uses two common methods: Continuous Bag of Words and Skip-gram. This article proposes a new Transformer-Based Knowledge Graph to overwhelm the difficulties of a large corpus and the manifoldness of the words. This proposed model deals with building a knowledge graph from the token found in transformer models and passes it to the binary classifier to predict the presence or absence of ADR in a drug with an accuracy of 91%.

Keywords
Adverse Drug Reaction (ADR), Transformer, Knowledge Graph, Word2Vec, Natural Language Processing (NLP).

Reference
[1] Abdelaziz I, Fokoue A, Hassanzadeh O, Zhang P, and Sadoghi M, “Large-Scale Structural and Textual Similarity-Based Mining of Knowledge Graph to Predict Drug–Drug Interactions,” Journal of Web Semantics, vol. 44, pp. 104-117, 2017.
[2] AlBadani B, Shi R, Dong J, Al-Sabri R, and Moctard O. B, “Transformer-Based Graph Convolutional Network for Sentiment Analysis,” Applied Sciences, vol. 12, no. 3, pp. 1316, 2022.
[3] Allison M, “Reinventing Clinical Trials,” Nature Biotechnology, vol. 30, no. 1, pp. 41-49, 2012.
[4] Arijit D, J., N., S., Chandan K, and Subhadip C, “Adverse Drug Reactions Extraction from Social Media: A Systematic Review,” Grenze ID: 01.GIJET.8.1.11, 2022.
[5] Bahdanau D, Cho K, and Bengio Y, “Neural Machine Translation by Jointly Learning to Align and Translate,” arXiv preprint arXiv:1409.0473, 2014.
[6] Biseda B, and Mo K, “Enhancing Pharmacovigilance with Drug Reviews and Social Media,” arXiv preprint arXiv:2004.08731, 2020.
[7] Bouvy J. C, De Bruin M. L, and Koopmanschap M. A, “Epidemiology of Adverse Drug Reactions in Europe: A Review of Recent Observational Studies,” Drug Safety, vol. 38, no. 5, pp. 437-453, 2015.
[8] Chen X, Jia S, and Xiang Y, “A review: Knowledge Reasoning Over Knowledge Graph,” Expert Systems with Applications, vol. 141 pp. 112948, 2020.
[9] Cocos A, Fiks A. G, and Masino A. J, “Deep Learning for Pharmacovigilance: Recurrent Neural Network Architectures for Labeling Adverse Drug Reactions in Twitter Posts,” Journal of the American Medical Informatics Association, vol. 24, no. 4, pp. 813-821, 2017.
[10] Devlin J, Chang M.W, Lee K, and Toutanova K, “Bert: Pre-training of Deep Bidirectional Trans- Formers for Language Understanding,” arXiv preprint arXiv:1810.04805, 2018.
[11] Edwards I. R, and Aronson J. K, “Adverse Drug Reactions: Definitions, Diagnosis, and Management,” The Lancet, vol. 356, no. 9237, pp. 1255-1259, 2000.
[12] Habimana O, Li Y, Li R, Gu X, and Yu G, “Sentiment Analysis using Deep Learning Approaches: An Overview,” Science China Information Sciences, vol. 63, no. 1, pp. 1-36, 2020.
[13] Harnoune A, Rhanoui M, Mikram M, Yousfi S, Elkaimbillah Z, and El Asri B, “Bert Based Clinical Knowledge Extraction for Biomedical Knowledge Graph Construction and Analysis,” Computer Methods and Programs in Biomedicine Update, vol. 1, pp. 100042, 2021.
[14] Hirohara M, Saito Y, Koda Y, Sato K, and Sakakibara Y, “Convolutional Neural Network Based on Smiles Representation of Compounds for Detecting Chemical Motif,” BMC Bioinformatics, vol. 19, no. 19, pp. 83-94, 2018.
[15] Hogan A, Blomqvist E, Cochez M, d’Amato C, Melo G. D, Gutierrez C, Kirrane S, Gayo J. E. L, Navigli R, Neumaier S, et al., “Knowledge Graphs,” Synthesis Lectures on Data, Semantics, and Knowledge, vol. 12, no. 2, pp. 1-257, 2021.
[16] Huang J.Y, Lee W.P, and Lee K.D, “Predicting Adverse Drug Reactions from Social Media Posts: Data Balance, Feature Selection and Deep Learning,” In Healthcare, MDPI , vol. 10, pp. 618, 2022.
[17] Ji S, Pan S, Cambria E, Marttinen P, and Philip S. Y, “A Survey on Knowledge Graphs: Representation, Acquisition, and Applications,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 2, pp. 494-514, 2021.
[18] Lardon J, Abdellaoui R, Bellet F, Asfari H, Souvignet J, Texier N, Jaulent M.C, Beyens M.N, Burgun A, Bousquet C, et al., “Adverse Drug Reaction Identification and Extraction in Social Media: A Scoping Review,” Journal of Medical Internet Research, vol. 17, no. 7, pp. e4304, 2015.
[19] Rotmensch M, Halpern Y, Tlimat A, Horng S, and Sontag D, “Learning a Health Knowledge Graph from Electronic Medical Records,” Scientific Reports, vol. 7, no. 1, pp. 1-11, 2017.
[20] Li F, Zhang M, Fu G, and Ji D, “A Neural Joint Model for Entity and Relation Extraction from Biomedical Text,” BMC Bioinformatics, vol. 18, no. 1, pp. 1-11, 2017.
[21] Li J, Zheng S, Chen B, Butte A. J, Swamidass S. J, and Lu Z, “A Survey of Current Trends in Computational Drug Repositioning,” Briefings in Bioinformatics, vol. 17, no. 1, pp. 2-12, 2016.
[22] Michael Allen, “1804 Python Healthcare,” 2020.
[23] Moon C, Jin C, Dong X, Abrar S, Zheng W, Chirkova R. Y, and Tropsha A, “Learning Drug-Disease-Target Embedding (DDTE) from Knowledge Graphs to Inform Drug Repurposing Hypotheses,” Journal of Biomedical Informatics, vol. 119, pp. 103838, 2021.
[24] Naseem U, Razzak I, Khan S. K, and Prasad M, “A Comprehensive Survey on Word Representation Models: From Classical to State-of-the-Art Word Representation Language Models,” Transactions on Asian and Low-Resource Language Information Processing, vol. 20, no. 5, pp. 1-35, 2021.
[25] Newell A, Shaw J. C, and Simon H. A, “Report on a General Problem Solving Program,” In IFIP Congress, Pittsburgh, PA, vol. 256, pp. 64, 1959.
[26] Nicholson D. N, and Greene C. S, “Constructing Knowledge Graphs and their Biomedical Applications,” Computational and Structural Biotechnology Journal, vol. 18, pp. 1414-1428, 2020.
[27] O¨ ztu¨rk H, O¨ zgu¨r A, and Ozkirimli E, “Deepdta: Deep Drug-Target Binding Affinity Prediction,” Bio-Informatics, vol. 34, no. 17, pp. i821–i829, 2018.
[28] Patki A, Sarker A, Pimpalkhute P, Nikfarjam A, Ginn R, O’Connor K, Smith K, and Gonzalez G, “Mining Adverse Drug Reaction Signals from Social Media: Going Beyond Extraction,” Proceedings of BioLinkSig, vol. 2014, pp. 1-8, 2014.
[29] Rotmensch M, Halpern Y, Tlimat A, Horng S, and Sontag D, “Learning a Health Knowledge Graph from Electronic Medical Records,” Scientific Reports, vol. 7, no. 1, pp. 1-11, 2017.
[30] Sarker A, and Gonzalez G, “Portable Automatic Text Classification for Adverse Drug Reaction Detection via Multi-Corpus Training,” Journal of Biomedical Informatics, vol. 53, pp. 196-207, 2015.
[31] Shortliffe E, “Computer-Based Medical Consultations: MYCIN,” Elsevier, vol. 2, 2012.
[32] Mauik Panchal, Prof. Rutika Ghariya, "A Review On Detection of Fake News Using Various Techniques," SSRG International Journal of Computer Science and Engineering, vol. 8, no. 6, pp. 1-4, 2021. Crossref, https://doi.org/10.14445/23488387/IJCSE-V8I6P101.
[33] Whitebread S, Hamon J, Bojanic D, and Urban L, “Keynote Review: In Vitro Safety Pharmacology Profiling: An Essential Tool for Successful Drug Development,” Drug Discovery Today, vol. 10, no. 21, pp. 1421-1433, 2005.
[34] Xu J, Kim S, Song M, Jeong M, Kim D, Kang J, Rousseau J. F, Li X, Xu W, Torvik V. I, et al., “Building a Pubmed Knowledge Graph,” Scientific Data, vol. 7, no. 1, pp. 1-15, 2020.
[35] Yang Z, and Dong S, “Hagerec: Hierarchical Attention Graph Convolutional Network Incorporating Knowledge Graph for Explainable Recommendation,” Knowledge-Based Systems, vol. 204, pp. 106194, 2020.
[36] Zeng X, Zhu S, Hou Y, Zhang P, Li L, Li J, Huang L. F, Lewis S. J, Nussinov R, and Cheng F, “Network-Based Prediction of Drug-Target Interactions using an Arbitrary-Order Proximity Embedded Deep Forest,” Bioinformatics, vol. 36, no. 9, pp. 2805-2812, 2020.
[37] Zhang F, Sun B, Diao X, Zhao W, and Shu T, “Prediction of Adverse Drug Reactions Based on Knowledge Graph Embedding,” BMC Medical Informatics and Decision Making, vol. 21, no. 1, pp. 1-11, 2021.
[38] Zhang S, Yao L, Sun A, and Tay Y, “Deep Learning Based Recommender System: A Survey and New Perspectives,” ACM Computing Surveys (CSUR), vol. 52, no. 1, pp. 1-38, 2019.
[39] Zhang Y, Cui S, and Gao H, “Adverse Drug Reaction Detection on Social Media with Deep Linguistic Features,” Journal of Biomedical Informatics, vol. 106, pp. 103437, 2020.
[40] Lardon J, Abdellaoui R, Bellet F, Asfari H, Souvignet J, Texier N, & Bousquet C, “Adverse Drug Reaction Identification and Extraction in Social Media: A Scoping Review,” Journal of Medical Internet Research, vol. 17, no. 7, pp. e4304, 2015.
[41] Lee C. Y, and Chen Y.P. P, “Descriptive Prediction of Drug Side-Effects using a Hybrid Deep Learning Model,” International Journal of Intelligent Systems, vol. 36, no. 6, pp. 2491-2510, 2021.
[42] Wang C.S, Lin P.J, Cheng C.L, Tai S.H, Yang Y.H. K, Chiang J.H., et al., “Detecting Potential Adverse Drug Reactions using a Deep Neural Network Model,” Journal of Medical Internet Research, vol. 21, no. 2, pp. e11016, 2019.