Deep Learning in Medical Image Super-Resolution: A Survey
Deep Learning in Medical Image Super-Resolution: A Survey |
||
 |
 |
|
| © 2023 by IJETT Journal | ||
| Volume-71 Issue-8 |
||
| Year of Publication : 2023 | ||
| Author : Vaishali Patel, Anand Mankodia |
||
| DOI : 10.14445/22315381/IJETT-V71I8P201 | ||
How to Cite?
Vaishali Patel, Anand Mankodia, "Deep Learning in Medical Image Super-Resolution: A Survey," International Journal of Engineering Trends and Technology, vol. 71, no. 8, pp. 1-12, 2023. Crossref, https://doi.org/10.14445/22315381/IJETT-V71I8P201
Abstract
Various deep learning (DL) algorithms, specific to convolutional neural networks (CNN), have grown exponentially and have become a methodology of choice for medical image processing. Many parameters degrade the medical images, like hardware limitations, the physical condition of the patient, the relative motion of objects, the limited scanning time of the image acquisition system, and many more. Obtaining medical images with the desired resolution to provide better assistance for the detection and proper diagnosis of the disease is still changeling task while limiting the level of radiation. In this aspect, a super-resolution is not a bad idea to obtain a high-resolution image using the information available in a degraded version of the image. This paper reviews the convolutional neural networks specific to medical imaging. They differ in terms of pre and post-upsampling, the density of CNN layers, and loss functions incorporated in the architecture. This paper provides a summary of DL-based super-resolution algorithms in terms of the quantitative evaluation, loss function and activation function. It also classifies available reference datasets used for medical image super-resolution. It was found that the DL-based medical image super-resolution achieves excellent quantitative and qualitative outcomes. However, this review also uncovers that deep learning techniques are complex in structure, computationally expensive and require large amounts of training datasets. A description of various medical image super-resolution deep convolutional neural networks, current challenges faced in this field, and directions for future work are provided.
Keywords
Convolution Neural Network, Deep learning, Medical imaging, Super-resolution.
References
[1] Sung Cheol Park, Min Kyu Park, and Moon Gi Kang, “Super-resolution Image Reconstruction: A Technical Overview,” IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21-36, 2003.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Watchara Ruangsang, and Supavadee Aramvith, “Efficient Super-resolution Algorithm Using Overlapping Bicubic Interpolation,” IEEE 6th Global Conference on Consumer Electronics, pp. 1-2, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Xin Li, and M.T. Orchard, “New Edge-Directed Interpolation,” IEEE Transactions on Image Processing, vol. 10, no. 10, pp. 1521–1527, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Hassan Aftab, Atif Bin Mansoor, and Muhammad Asim, “A New Single Image Interpolation Technique for Super Resolution,” IEEE International MultitopicConference, pp. 592-596, 2008.
[CrossRef] [Google Scholar] [Publisher Link]
[5] V. Banupriya, and S. Anusuya, “Improving Classification of Retinal Fundus Image using Flow Dynamics Optimized Deep Learning Methods,” SSRG International Journal of Electrical and Electronics Engineering, vol. 9, no. 12, pp. 39-48, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[6] R. Priyadharsini, and T. Sree Sharmila, “An Efficient Edge Preserving Interpolation Method for Underwater Acoustic Image Resolution Enhancement,” Archives of Acoustics, vol. 47, no. 2, pp. 267-274, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Zhi-Song Liu, Wan-Chi Siu, and Jun-Jie Huang, “Image Super-resolution via Hybrid NEDI and Wavelet-based Scheme,” Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, pp. 1131-1136, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Shalini Dubey, Shalini Sahu, and Pankaj Bazal, “Single Image Super Resolution using Interpolation and Discrete Wavelet Transform,” International Journal of Trend in Scientific Research and Development, vol. 2, no. 6, pp. 241-249, 2018.
[CrossRef] [Publisher Link]
[9] Na Sun, and Huina Li, “Super-resolution Reconstruction of Images Based on Interpolation and Full Convolutional Neural Network and Application in Medical Fields,” IEEE Access, vol. 7, pp. 186470-186479, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Yunfeng Zhang et al., “A Single-Image Super-Resolution Method Based on Progressive-Iterative Approximation,” IEEE Transactions on Multimedia, vol. 22, no. 6, pp. 1407-1422, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Cui Zhou, and Jinghong Zhou, “Single-Frame Remote Sensing Image Super-Resolution Reconstruction Algorithm Based on Two-Dimensional Wavelet,” IEEE 3rd International Conference on Image, Vision and Computing, pp. 360-363, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Priscilla Whitin, and V. Jayasankar, “A Novel Deep Learning-Based System for Real-Time Temperature Monitoring of Bone Hyperthermia,” SSRG International Journal of Electrical and Electronics Engineering, vol. 10, no. 1, pp. 187-196, 2023.
[CrossRef] [Publisher Link]
[13] Yi Tang, Jun-Hua Chen, and Zuo Jiang, “Piece-Wise Kernel Regression for Example-Based Super-Resolution,” International Conference on Machine Learning and Cybernetics, pp. 143-148, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Faezeh Yeganli, and Kuldeep Singh, “Finger-Print Image Super-Resolution via Gradient Operator based Clustered Coupled Sparse Dictionaries,” IEEE International Symposium on Innovations in Intelligent Systems and Applications, pp. 1-4, 2019.
[Google Scholar] [Publisher Link]
[15] Yi Tang, and Ling Shao, “Pairwise Operator Learning for Patch-Based Single-Image Super-Resolution,” IEEE Transactions on Image Processing, vol. 26, no. 2, pp. 994–1003, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Shuiping Gou et al., “Image Super-Resolution Based on the Pairwise Dictionary Selected Learning and Improved Bilateral Regularization,” IET Image Processing, vol. 10, no. 2, pp. 101–112, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Yapeng Tian et al., “Anchored Neighborhood Regression Based Single Image Superresolution from Self-Examples,” IEEE International Conference on Image Processing, pp. 2827-2831, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Kaibing Zhang et al., “Single Image Super-Resolution with Multiscale Similarity Learning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 24, no. 10, pp. 1648-1659, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[19] M. N. Rajesh, and B. S. Chandrasekar, “Deep Learning-Based Semantic Segmentation Models for Prostate Gland Segmentation,” SSRG International Journal of Electrical and Electronics Engineering, vol. 10, no. 2, pp. 157-171, 2023.
[CrossRef] [Publisher Link]
[20] Shengyang Dai et al., “Bilateral Back-Projection for Single Image Super Resolution,” IEEE Conference on Multimedia and Expo, pp. 1039-1042, 2007.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Vaishali Patel et al., “Hybrid Approach for Single Image Super Resolution using ISEF and IBP,” International Conference on Communication Systems and Network Technologies, pp. 495-499, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Rujul R Makwana, and Nita D Mehta, “Single Image Super-Resolution via Iterative Back Projection Based Canny Edge Detection and a Gabor Filter Prior,” International Journal of Soft Computing & Engineering, vol. 3, no.1, pp. 379-384, 2013.
[Google Scholar] [Publisher Link]
[23] Jun-Sang Yoo, and Jong-Ok Kim, “Noise-Robust Iterative Back-Projection,” IEEE Transactions on Image Processing, vol. 29, pp. 1219-1232, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Damber Thapa et al., “A Performance Comparison among Different Super-Resolution Techniques,” Computers & Electrical Engineering, vol. 54, pp. 313-329, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[25] NasimulNoman, “A Shallow Introduction to Deep Neural Networks,” Deep Neural Evolution, pp. 35 – 63, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Yoong Khang Ooi, and Haidi Ibrahim, “Deep Learning Algorithms for Single Image Super-Resolution: A Systematic Review,” Electronics, vol. 10, no. 7, p. 867, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[27] Ying Liu et al., “Single Image Super Resolution Techniques Based on Deep Learning: Status, Applications and Future Directions,” Journal of Image and Graphics, vol. 9, no. 3, 2021.
[Google Scholar] [Publisher Link]
[28] Hong Lin et al., “Generative Adversarial Image Super-Resolution Network for Multiple Degradations,” IET Image Processing, vol. 14, no. 17, pp. 4520-4527, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[29] Ming Zhao et al., “Super-Resolution of Cardiac Magnetic Resonance Images using Laplacian Pyramid Based on Generative Adversarial Networks,” Computerized Medical Imaging and Graphics, vol. 80, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[30] Heng Liu et al., “Perception Consistency Ultrasound Image Super-Resolution via Self-Supervised CycleGAN,” Neural Computing and Applications, vol. 35, pp. 12331-12341, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[31] Chao Dong et al., “Image Super-Resolution using Deep Convolutional Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[32] Christian Ledig et al., “Photo-Realistic Single Image Super-Resolution using a Generative Adversarial Network,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 105-114, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[33] Ahlad Kumar, Harsh Vardhan Singh, and Vijeta Khare, “Tchebichef Transform Domain-Based Deep Learning Architecture for Image Super-Resolution,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[34] Jin Zhu et al., “MIASSR: An Approach for Medical Image Arbitrary Scale Super-Resolution,” arXiv preprint Image and Video Processing, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[35] Kuan Zhang et al., “SOUP-GAN: Super-Resolution MRI Using Generative Adversarial Networks,” Tomography, vol. 8, no. 2, pp. 905-919, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[36] Faezehsadat Shahidi, “Breast Cancer Histopathology Image Super-Resolution Using Wide-Attention GAN with Improved Wasserstein Gradient Penalty and Perceptual Loss,” IEEE Access, vol. 9, pp. 32795-32809, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[37] Defu Qiu et al., “Multiple Improved Residual Networks for Medical Image Super-Resolution,” Future Generation Computer Systems, vol. 116, pp. 200-208, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[38] Hongbo Zhu et al., “Functional-Realistic CT Image Super-Resolution for Early-Stage Pulmonary Nodule Detection,” Future Generation Computer Systems, vol. 115, pp. 475-485, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[39] J. Andrew et al., “Super-Resolution Reconstruction of Brain Magnetic Resonance Images via Lightweight Autoencoder,” Informatics in Medicine Unlocked, vol. 26, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[40] Zhaoyang Songa et al., “Progressive Back-Projection Network for COVID-CT Super-Resolution,” Computer Methods and Programs in Biomedicine, vol. 208, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[41] Farah Deeba et al., “Wavelet-Based Enhanced Medical Image Super Resolution,” IEEE Access, vol. 8, pp. 37035-37044, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[42] Yan Lv et al., “Fusing Dense and ReZero Residual Networks for Super-Resolution of Retinal Images,” Pattern Recognition Letters, vol. 149, pp. 120-129, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[43] Chao Dong, Chen Change Loy, and Xiaoou Tang, “Accelerating the Superre Solution Convolutional Neural Network,” European Conference on Computer Vision, pp. 391–407, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[44] Shengxiang Zhang et al., “A Fast Medical Image Super Resolution Method Based on Deep Learning Network,” IEEE Access, vol. 7, pp. 12319-12327, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[45] Garima Pandey, and Umesh Ghanekar, “A Conspectus of Deep Learning Techniques for Single-Image Super-Resolution,” Pattern Recognition and Image Analysis, vol. 32, no. 1, pp. 11-32, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[46] Minakshi Gogoi, and Mala Ahmed, “Image Quality Parameter Detection: A Study,” International Journal of Computer Sciences and Engineering, vol. 4, no. 7, pp. 110-116, 2016.
[Publisher Link]
[47] Varsha Nemade, Sunil Pathak, and Ashutosh Kumar Dubey, “Hybrid Deep Convolutional Neural Network Approach for Detecting Breast Cancer in Mammography Images,” SSRG International Journal of Electrical and Electronics Engineering, vol. 10, no. 5, pp. 102-119, 2023.
[CrossRef] [Publisher Link]
[48] Kai Li et al., “Survey of Single Image Super-Resolution Reconstruction,” IET Image Processing, vol. 14, no. 11, pp. 2273-2290, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[49] Zhou Wang et al., “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transaction on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
[CrossRef] [Google Scholar] [Publisher Link]
[50] G. R. Meghana, Suresh Kumar Rudrahithlu, and K. C. Shilpa, “Detection of Brain Cancer using Machine Learning Techniques a Review,” SSRG International Journal of Computer Science and Engineering, vol. 9, no. 9, pp. 12-18, 2022.
[CrossRef] [Publisher Link]
[51] Zhihao Wang, Jian Chen, and Steven C. H. Hoi, “Deep Learning for Image Super-Resolution: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 10, pp. 3365-3387, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[52] Zhaowen Wang et al., “Deep Networks for Image Super-Resolution with Sparse Prior,” IEEE International Conference on Computer Vision, pp. 370-378, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[53] B. Suryakanth, and S A. Hari Prasad, “3D CNN-Residual Neural Network Based Multimodal Medical Image Classification,” International Journal of Engineering Trends and Technology, vol. 70, no. 10, pp. 371-380, 2022.
[CrossRef] [Publisher Link]
[54] Justin Johnson, Alexandre Alahi, and Li Fei-Fei, “Perceptual Losses for Real-Time Style Transfer and Super-Resolution,” European Conference on Computer Vision, pp. 694–711, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[55] Bee Lim et al., “Enhanced Deep Residual Networks for Single Image Super-Resolution,” IEEE Conference on Computer Vision and Pattern Recognition Workshops, vol. 1, pp. 1132-1140, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[56] Lina Zhang, Haidong Dai, and Yu Sang, “Med-SRNet: GAN-Based Medical Image Super-Resolution via High-Resolution Representation Learning,” Computational Intelligence and Neuroscience, vol. 2022, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[57] R. Tamilaruvi et al., “Brain Tumor Detection in MRI Images using Convolutional Neural Network Technique,” SSRG International Journal of Electrical and Electronics Engineering, vol. 9, no. 12, pp. 198-208, 2022.
[CrossRef] [Publisher Link]
[58] Ruchika Dhawan, and Umesh Ghanekar, “Single-Image Super-Resolution Using Rational Fractal Interpolation and Adaptive Wiener Filtering,” Proceedings of First International Conference on Computational Electronics for Wireless Communications, pp. 477-486, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[59] W.T. Freeman, T.R. Jones, and E.C. Pasztor, “Example-Based Super-Resolution,” IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 56-65, 2002.
[CrossRef] [Google Scholar] [Publisher Link]
[60] M. Sakthivadivu, and P. Suresh Babu, “Analytical and Empirical Survival Study on Natural Image Compression and Classification using Machine Learning Techniques,” International Journal of Computer Trends and Technology, vol. 70, no. 8, pp. 21-29, 2022.
[CrossRef] [Publisher Link]
[61] Michal Irani, and Shmuel Peleg, “Motion Analysis for Image Enhancement: Resolution, Occlusion and Transparency,” Journal of Visual Communication and Image Representation, vol. 4, no. 4, pp. 324-335, 1993.
[CrossRef] [Google Scholar] [Publisher Link]
[62] Shahrukh Athar, and Zhou Wang, “A Comprehensive Performance Evaluation of Image Quality Assessment Algorithms,” IEEE Access, vol. 7, pp. 140030-140070, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[63] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, “A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms,” IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440–345, 2006.
[CrossRef] [Google Scholar] [Publisher Link]