IJBTT

A Comparative Analysis of L1, L2, and L1L2 Regularization Techniques in Neural Networks for Image Classification

A Comparative Analysis of L1, L2, and L1L2 Regularization Techniques in Neural Networks for Image Classification
	© 2025 by IJETT Journal
	Volume-73 Issue-10
	Year of Publication : 2025
	Author : Deepa S, Rashmi Siddalingappa, Kalpana.P, Loveline Zeema.J, Vinay.M, Jayapriya, J,Suganthi.J, I. Priya Stella Mary
	DOI : 10.14445/22315381/IJETT-V73I10P108

How to Cite?
Deepa S, Rashmi Siddalingappa, Kalpana.P, Loveline Zeema.J, Vinay.M, Jayapriya, J,Suganthi.J, I. Priya Stella Mary,"A Comparative Analysis of L1, L2, and L1L2 Regularization Techniques in Neural Networks for Image Classification", International Journal of Engineering Trends and Technology, vol. 73, no. 10, pp.107-116, 2025. Crossref, https://doi.org/10.14445/22315381/IJETT-V73I10P108

Abstract
The research examines how L1, L2, and L1L2 weight regularization methods affect neural network performance, generalization, and sparsity using the CIFAR 10 dataset. A Convolutional Neural Network (CNN) trained with the same environment for each regularization method to evaluate test accuracy, weight sparsity, and computational speed. The study shows that L1 regularization produces sparse weights, which makes models more interpretable, and L2 regularization helps prevent overfitting while improving model generalization. The combination of L1L2 regularization enables individual image classification methods to reach test accuracy. The results indicate that the weight regularization plays a vital role in creating neural networks that are both stable and efficient. They are interpretable, and L2 regularization improves generalization and reduces overfitting. The combined L1L2 regularization achieves the balance between sparsity and performance, leading to higher test accuracy compared to individual techniques for image classification. The research results demonstrate that weight regularization stands as an essential factor for Creating Neural Networks that are robust, efficient, and interpretable, thus helping to enhance Deep Learning model performance.

Keywords
Weight Regularization, L1 Regularization, L2 Regularization, L1L2 Regularization, Convolutional Neural Networks (CNN), Generalization, Overfitting, Deep Learning.

References
[1] Alex Krizhevsky, Learning Multiple Layers of Features from Tiny Images, University of Toronto, pp. 1-40, 2009.
[Google Scholar] [Publisher Link]
[2] Andrew Y. Ng, “Feature Selection, L1 vs. L2 Regularization, and Rotational Invariance,” ICML '04: Proceedings of the Twenty-First International Conference on Machine learning, Banff, Alberta, Canada, pp. 1-20, 2004.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, MIT Press, 2016.
[Google Scholar] [Publisher Link]
[4] Hui Zou, and Trevor Hastie, “Regularization and Variable Selection via the Elastic Net,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 67, no. 2, pp. 301-320, 2005.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Kevin Murphy et al., “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929-1958, 2014.
[Google Scholar] [Publisher Link]
[6] Kaiming He et al., “Deep Residual Learning for Image Recognition,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
[Google Scholar] [Publisher Link]
[7] Sergey Ioffe, and Christian Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” Proceedings of the 32nd International Conference on Machine Learning (ICML), vol. 37, pp. 448-456, 2015.
[Google Scholar] [Publisher Link]
[8] Chiyuan Zhang et al., “Understanding Deep Learning Requires Rethinking Generalization,” arXiv Preprint, pp. 1-15, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Ilya Loshchilov, and Frank Hutter, “Decoupled Weight Decay Regularization,” arXiv Preprint, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Dongyoon Han, Jiwhan Kim, and Junmo Kim, “Deep Pyramidal Residual Networks,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5927-5935, 2017.
[Google Scholar] [Publisher Link]
[11] Karen Simonyan, and Andrew Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv Preprint, pp. 1-14, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Diederik P. Kingma, and Jimmy Ba, “Adam: A Method for Stochastic Optimization,” International Conference on Learning Representations (ICLR), San Diego, pp. 1-15, 2015.
[CrossRef] [Google Scholar] [Publisher Link]