A Novel RGB Channel Assimilation for Hyperspectral Image Classification using 3D-Convolutional Neural Network with Bi-Long Short-Term Memory

A Novel RGB Channel Assimilation for Hyperspectral Image Classification using 3D-Convolutional Neural Network with Bi-Long Short-Term Memory
  IJETT-book-cover           
  
© 2022 by IJETT Journal
Volume-70 Issue-3
Year of Publication : 2022
Authors : M. Preethi, C. Velayutham, S. Arumugaperumal
https://doi.org/10.14445/22315381/IJETT-V70I3P223

How to Cite?

M. Preethi, C. Velayutham, S. Arumugaperumal, "A Novel RGB Channel Assimilation for Hyperspectral Image Classification using 3D-Convolutional Neural Network with Bi-Long Short-Term Memory," International Journal of Engineering Trends and Technology, vol. 70, no. 3, pp. 201-211, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I3P223

Abstract
Hyperspectral image (HSI) contains high dimensionality of spectral information, which is not easy to classify every pixel. To confront the problem, we propose a novel RGB channel Assimilation for classification methods. This work discusses the classification of hyperspectral images based on Domain Transform Interpolated Convolution Filter (DTICF) and 3D-CNN with Bi-directional-Long Short Term Memory (Bi-LSTM). Before using the DTICF, the RGB images of HSI and the patch of the input image from raw HSI are integrated. Those obtained spatial and spectral features are finally given into the designed 3D-CNN with the Bi-LSTM framework. The excerpted colour features are classified by 2D-CNN. The probabilistic classification map of 3D-CNN-Bi-LSTM and 2D-CNN are fused. In the last step, additionally, Markov Random Field (MRF) is utilized for improving the fused probabilistic classification map. Based on the experimental results, two different hyperspectral images prove that novel RGB channel assimilation of DTICF-3D-CNN-Bi-LSTM provides good classification results.

Keywords
Bi directional-Long Short Term Memory, Deep Learning, Domain Transform Interpolated Convolution Filter.

Reference
[1] Bandos T, Bruzzone L, and Camps-Valls G, Classification of Hyperspectral Images with Regularized Linear Discriminant Analysis, IEEE Transactions on Geoscience and Remote Sensing. 47(3) (2009) 862–873.
[2] Chen Y, Lin Z, Zhao X, Wang G, and Gu Y, Deep Learning-Based Classification of Hyperspectral Data, IEEE Journal of Topics in Applied Earth Observations and Remote Sensing. 7(6) (2014) 2094–2107.
[3] Chen Y, Nasrabadi N.M, and Tran T.D, Hyperspectral Image Classification via Kernel Sparse Representation, IEEE Transactions on Geoscience and Remote Sensing. 51(1) (2013) 217–231.
[4] Dalla Mura M, Villa A, Benediktsson J.A, Chanussot J, and Bruzzone L, Classification of Hyperspectral Images by Using Extended Morphological Attribute Profiles and Independent Component Analysis, IEEE Geoscience and Remote Sensing Letters. 8(3) (2011) 542–546.
[5] Ellis D.M, Draper N.P, and Smith H.S, Applied Regression Analysis, Applied Statistics. 17(1) (2014) 83-90.
[6] Fauvel M, Benediktsson J, Chanussot J, and Sveinsson J, Spectral and Spatial Classi?cation of Hyperspectral Data Using SVMS and Morphological Profiles, IEEE Transactions on Geoscience and Remote Sensing. 46(11) (2008) 3804–3814.
[7] Guo Y, Cao H, Han S, Sun Y, and Bai Y, Spectral-Spatial Hyperspectral Image Classification with K-Nearest Neighbor and Guided Filter, IEEE Access. 6 (2018) 18582–18591.
[8] Haokui Zhang, Ying Li, Yenan Jiang, Peng Wang, Qiang Shen, and Chunhua Shen, Hyperspectral Classification Based on Lightweight 3-D-CNN with Transfer Learning, IEEE Transactions on Geoscience and Remote Sensing. 57(8) (2019) 5813-5830.
[9] He K.M, Sun J, and Tang X.O, Guided Image Filtering, IEEE Trans. Pattern Anal. Mach. Intell. 35(6) (2013) 1397–1409.
[10] Hochreiter S, and Schmidhuber J, Long Short-Term Memory, Neural Computation. 9(8) (1997) 1735-1780.
[11] Hughes G.F, On the Mean Accuracy of Statistical Pattern Recognizers, IEEE Transaction Information Theory. 14(1) (1968) 55-63.
[12] Hu W, Huang Y, Li W, Zhang F, and Li D.H, Deep Convolutional Neural Networks for Hyperspectral Image Classification, Journal of Sensors. 501 (2015) 258619.
[13] Jia S, Shen L, Zhu J, and Li Q, A 3-D Gabor Phase-Based Coding and Matching Framework for Hyperspectral Imagery Classification, IEEE Transactions on Cybernetics. 48(4) (2018) 1176–1188.
[14] Jun Li, and Jose M. Bioucas, Spectral- Spatial Hyperspectral Image Segmentation Using Subspace Multinomial Logistic Regression and Markov Random Fields, IEEE. 50(3) (2012).
[15] Kang X, Li S, and Benediktsson J.A, Feature Extraction of Hyperspectral Images with Image Fusion and Recursive Filtering, IEEE Transactions on Geoscience and Remote Sensing. 52(6) (2014) 3742–3752.
[16] Konstantinos Makantasis, Konstantinos Karantzalos, and Anastasios Doulamis, Deep Supervised Learning for Hyperspectral Data Classification through Convolutional Neural Networks, IEEE. (2015) 4959- 4962.
[17] Landgrebe D.A, Hyperspectral Image Data Analysis, IEEE Signal Process, Mag. 1053(5888) (2002) 17-28.
[18] Licciardi G, Marpu P.R, Chanussot J, and Benediktsson J.A, Linear Versus Nonlinear PCA for the Classification of Hyperspectral Data Based on the Extended Morphological Profiles, IEEE Geoscience and Remote Sensing Letters. 9(3) (2012) 447–451.
[19] Li J, Bioucas-Dias J.M, and Plaza A, Semisupervised Hyperspectral Image Segmentation Using Multinomial Logistic Regression with Active Learning, IEEE Transactions on Geoscience and Remote Sensing. 48(11) (2010) 4085–4098.
[20] Li J, Bioucas-Dias J.M, and Plaza A, Spectral–Spatial Hyperspectral Image Segmentation Using Subspace Multinomial Logistic Regression and Markov Random Fields, IEEE Transactions on Geoscience and Remote Sensing. 50(3) (2012) 809–823.
[21] Li J, Bioucas-Dias J.M, and Plaza A, Hyperspectral Image Segmentation Using New Bayesian Approach with Active Learning, IEEE Transactions on Geoscience and Remote Sensing. 49(10) (2011) 3947-3960.
[22] Li Y, Zhang H, and Shen Q, Spectral-Spatial Classification of Hyperspectral Imagery with 3d Convolutional Neural Network, Remote Sens. 9 (2017) 67-74.
[23] Lin Zhu, Yushi Chen and Pedram Ghamisi, Generative Adversarial Networks for Hyperspectral Image Classification, IEEE Transactions on Geoscience and Remote Sensing. 56(6) (2018) 5046 – 5063.
[24] Liu Q, Feng Z, Hang R, and Yuan X, Bidirectional-Convolutional LSTM Based Spectral-Spatial Feature Learning for Hyperspectral Image Classification, Remote Sens. 9 (2017) 1330-1340.
[25] Mandic D.P, and Chambers J, Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability, John Wiley & Sons, Inc. (2001) 1-14.
[26] Melani F, and Bruzzone L, Classification of Hyperspectral Remote Sensing Images with Support Vector Machines, IEEE Transactions on Geoscience and Remote Sensing. 42(8) (2004) 1778–1790.
[27] Oliveira M.M, and Gastal E.S, Domain Transform for Edge-Aware Image and Video Processing, ACM Transactions on Graphics (Tog), ACM. 30(4) (2011) 69.
[28] Qin Xu, Yong Xiao, Dongyue Wang and Bin Luo, CSA-MSO3DCNN: Multiscale Octave 3D-CNN with Channel and Spatial Attention for Hyperspectral Image Classification, Remote Sens. (2020).
[29] Radhesyam Vaddi, and Prabukumar Manoharan, Hyperspectral Image Classification Using CNN with Spectral and Spatial Features Integration, Infrared Physics and Technology, Elsevier. 107 (2020).
[30] Shaohui Mei, Jingyu Ji, Qianqian Bi, Junhui Hou, and Qian Du, Integrating Spectral and Spatial Information into Deep Convolutional Neural Networks for Hyperspectral Classification, IEEE. (2016) 5067- 5070.
[31] Shen L, and Jia S, Three-Dimensional Gabor Wavelets for Pixel-Based Hyperspectral Imagery Classification, IEEE Transactions on Geoscience and Remote Sensing. 49(12) (2011) 5039–5046.
[32] Shi X, Chen Z, Wang H, Yeung D.Y, Wong W.K, and Woo W.C, Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting, Advances in Neural Information Processing Systems. 28 (2015) 1049–5258. [Online]. Available: www.Papers.Nips.Cc/Paper/5955-Convolutional-Lstm-Networka-Machine-Learning-Approach-For-Precipitation-Nowcasting
[33] Sun X, Qu Q, Nasrabadi N. M, and Tran T. D, Structured Priors for Sparse-Representation-Based Hyperspectral Image Classification, IEEE Geoscience and Remote Sensing Letters. 11(7) (2014) 1235–1239.
[34] Szegedy C, Toshev A, and Erhan D, Deep Neural Networks for Object Detection, In Advances in Neural Information Processing Systems. (2013) 2553–2561.
[35] Yushi Chen, Chouhan Lin And Xing Zhao, Deep Learning-Based Classification of Hyperspectral Data, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, IEEE. 7(6) (2014) 2094-2107.