Citation :

Nur Syabila Zabidi, Noraini Ibrahim, Asmau Osman, "A Systematic Literature Review on Fuzzy-based Metaheuristic Algorithms for Combinatorial Test Suite Generation," International Journal of Engineering Trends and Technology (IJETT), vol. 74, no. 6, pp. 187-200, 2026. Crossref, https://doi.org/10.14445/22315381/IJETT-V74I6P113

Abstract

Combinatorial Testing (CT), also known as t-way test suite generation, results in compact test suites that ensure that all t-way parameter interactions are tested. In this regard, metaheuristics such as Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), and the Sine Cosine Algorithm (SCA) have been used to generate compact test suites. However, these methods often rely on fixed parameter settings, which may lead to premature convergence and poor solution quality. To overcome this limitation in combinatorial testing methods, fuzzy logic has been proposed as a dynamic parameter-tuning method. Despite the potential of fuzzy logic in combinatorial testing, there is no systematic review on this topic. This research aims to guide a Systematic Literature Review using the PRISMA and Barbara Kitchenham frameworks. Out of 61 studies identified initially, only three primary journal articles were found to meet the selection criteria after a careful process following identification, screening, eligibility, quality evaluation, and data extraction. The results are synthesized into four major themes: algorithmic categories, fuzzy-logic implementation, parameter tuning, and performance results. The results show that fuzzy logic has been applied to teaching-learning-based, swarm-based, and ant colony optimization algorithms using high-level and parameter tuning approaches. All studies show improved results in test suite reduction, convergence speedup, and coverage efficiency.

Keywords

Combinatorial Testing, Fuzzy Logic, Metaheuristic Algorithms, Systematic Literature Review (SLR), T-Way Testing.

References

[1] Mark Harman, Yue Jia, and Yuanyuan Zhang, “Achievements, Open Problems and Challenges for Search based Software Testing,” 2015 IEEE 8^th International Conference on Software Testing, Verification and Validation (ICST), Graz, Austria, pp. 1-12, 2015.
[CrossRef] [Google Scholar] [Publisher Link]

[2] Abhishek Singh Verma, Ankur Choudhary, and Shailesh Tiwari, “Software Test Case Generation Tools and Techniques: A Review,” International Journal of Mathematical, Engineering and Management Sciences, vol. 8, no. 2, pp. 293-315, 2023.
[CrossRef] [Google Scholar]

[3] Muhammad Afiq Bin Ariffin et al., “Efficient Test Case Prioritization using the Rat Swarm Optimizer Algorithm,” 2024 IEEE 22^nd Student Conference on Research and Development (SCOReD), Selangor, Malaysia, pp. 513-520, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[4] Braulio J. Solano-Rojas, Ricardo Villalón-Fonseca, and Rafael Batres, “Micro Evolutionary Particle Swarm Optimization (MEPSO): A New Modified Metaheuristic,” Systems and Soft Computing, vol. 5, pp. 1-13, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[5] Pavi Saraswat, and Abhishek Singhal, “A Hybrid Approach for Test Case Prioritization and Optimization using Meta-Heuristics Techniques,” 2016 1^st India International Conference on Information Processing (IICIP), Delhi, India, pp. 1-6, 2016.
[CrossRef] [Google Scholar] [Publisher Link]

[6] Mohd Zamri Zahir Ahmad et al., “VS-TACO: A Tuned Version of Ant Colony Optimization for Generating Variable Strength Interaction in T-Way Testing Strategy,” Proceedings of the 2022 11^th International Conference on Software and Computer Applications, Association for Computing Machinery, New York, NY, United States, pp. 48-54, 2022. [CrossRef] [Google Scholar] [Publisher Link]

[7] N. Ramli et al., “T-Way Test Suite Generation Strategy based on Ant Colony Algorithm to Support T-Way Variable Strength,” Journal of Physics: Conference Series, vol. 1755, no. 1, pp. 1-13, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[8] Masrulehsan Mamat et al., “A Hybrid Sine Cosine Algorithm with Cluster Voting for Combinatorial Testing,” 2024 1^st International Conference on Cyber Security and Computing (CyberComp), Melaka, Malaysia, pp. 106-112, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[9] Amir Seyyedabbasi, “WOASCALF: A New Hybrid Whale Optimization Algorithm based on Sine Cosine Algorithm and Levy Flight to Solve Global Optimization Problems,” Advances in Engineering Software, vol. 173, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[10] Manju Khari, and Prabhat Kumar, “RETRACTED ARTICLE: An Extensive Evaluation of Search-based Software Testing: A Review,” Soft Computing, vol. 23, no. 6, pp. 1933-1946, 2019.
[CrossRef] [Google Scholar] [Publisher Link]

[11] Joshua E. Hernández-Suárez et al., “Search-based Software Testing Driven by Bioinspired Algorithms: A Systematic Literature Mapping,” 2024 12^th International Conference in Software Engineering Research and Innovation (CONISOFT), Puerto Escondido, Mexico, pp. 101-108, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[12] Changwu Huang, Yuanxiang Li, and Xin Yao, “A Survey of Automatic Parameter Tuning Methods for Metaheuristics,” IEEE Transactions on Evolutionary Computation, vol. 24, no. 2, pp. 201-216, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[13] Shubham Gupta, “Enhanced Sine Cosine Algorithm with Crossover: A Comparative Study and Empirical Analysis,” Expert Systems with Applications, vol. 198, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[14] Aasam Abdul Karim, Nor Ashidi Mat Isa, and Wei Hong Lim, “Hovering Swarm Particle Swarm Optimization,” IEEE Access, vol. 9, pp. 115719-115749, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[15] Aasam Abdul Karim, Nor Ashidi Mat Isa, and Wei Hong Lim, “Modified Particle Swarm Optimization with Effective Guides,” IEEE Access, vol. 8, pp. 188699-188725, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[16] Aminu Aminu Muazu, Ahmad Sobri Hashim, and Aliza Sarlan, “Application and Adjustment of ‘Don’t Care’ Values in T-Way Testing Techniques for Generating an Optimal Test Suite,” Journal of Advances in Information Technology, vol. 13, no. 4, pp. 347-357, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[17] Kamal Z. Zamli et al., “Fuzzy Adaptive Teaching Learning-based Optimization Strategy for the Problem of Generating Mixed Strength T-Way Test Suites,” Engineering Applications of Artificial Intelligence, vol. 59, pp. 35-50, 2017.
[CrossRef] [Google Scholar] [Publisher Link]

[18] Thair Mahmoud, and Bestoun S. Ahmed, “An Efficient Strategy for Covering Array Construction with Fuzzy Logic-based Adaptive Swarm Optimization for Software Testing use,” Expert Systems with Applications, vol. 42, no. 22, pp. 8753-8765, 2015.
[CrossRef] [Google Scholar] [Publisher Link]

[19] Caroline Heins, “Artificial Intelligence in Retail - A Systematic Literature Review,” Foresight, vol. 25, no. 2, pp. 264-286, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[20] Ariffin Abdullah, Rohayanti Hassan, and Zuraini Ali Shah, “A Systematic Literature Review of Combinatorial Testing,” International Journal of Advance in Soft Computing its Application, vol. 9, no. 2, pp. 128-139, 2017.
[Google Scholar]

[21] Alam Zeb et al., “A Systematic Literature Review on Robust Swarm Intelligence Algorithms in Search-based Software Engineering,” Complexity, vol. 2023, no. 1, pp. 1-22, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[22] Md. Abdul Kader, Kamal Z. Zamli, and Bestoun S. Ahmed, “A Systematic Review on Emperor Penguin Optimizer,” Neural Computing and Applications, vol. 33, no. 23, pp. 15933-15953, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[23] Hayrol Azril Mohamed Shaffril, Asnarulkhadi Abu Samah, and Raidah Mazuki, “A Systematic Literature Review on the Adaptation of Women in Fisheries-based Families on Climate Change Impacts,” Journal of Environmental Studies and Sciences, vol. 15, no. 3, pp. 648-665, 2024.
[CrossRef] [Google Scholar] [Publisher Link]

[24] Derek Cabrera, Laura Cabrera, and Derek Cabrera, “The Steps to Doing a Systems Literature Review (SLR),” Journal of Systems Thinking Preprints, pp. 1-28, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[25] Priti Bansal et al., “Construction of Variable Strength Covering Array for Combinatorial Testing using a Greedy Approach to Genetic Algorithm,” E-Informatica Software Engineering Journal, vol. 9, no. 1, pp. 87-105, 2015.
[CrossRef] [Google Scholar] [Publisher Link]

[26] Xu Guo et al., “An Effective Approach to High Strength Covering Array Generation in Combinatorial Testing,” IEEE Transactions on Software Engineering, vol. 49, no. 10, pp. 4566-4593, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[27] Md. Abdul Kader, Jamal A. Jamaluddin, and Kamal Z. Zamli, “An Educational Tool Aimed at Learning Metaheuristics,” Proceedings of the 2020 9^th International Conference on Software and Computer Applications, Association for Computing Machinery, New York, NY, United States, pp. 79-83, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[28] Asma Benmessaoud Gabis et al., “A Comprehensive Survey of Sine Cosine Algorithm: Variants and Applications,” Artificial Intelligence Review, vol. 54, no. 7, pp. 5469-5540, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[29] Jyotheesh Gaddam, “Development of Self-Adaptive Hybrid Algorithms : A Hyper-Heuristic Approach to Dynamic Optimisation,” Thesis, Deakin University, 2023.
[Google Scholar] [Publisher Link]

[30] Qusay Shihab Hamad et al., “Q-Learning Embedded Sine Cosine Algorithm (QLESCA),” Expert Systems with Applications, vol. 193, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[31] Abdelaziz A. Abdelhamid et al., “Innovative Feature Selection Method based on Hybrid Sine Cosine and Dipper Throated Optimization Algorithms,” IEEE Access, vol. 11, pp. 79750-79776, 2023.
[CrossRef] [Google Scholar] [Publisher Link]

[32] Hussein Almulla, and Gregory Gay, “Learning how to Search: Generating Effective Test Cases Through Adaptive Fitness Function Selection,” Empirical Software Engineering, vol. 27, no. 2, pp. 1-62, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[33] Hasitha R. Jayetileke, W.R. De Mel, and Subhas Chandra Mukhopadhyay, “Real-Time Metaheuristic Algorithm for Dynamic Fuzzification, De-Fuzzification and Fuzzy Reasoning Processes,” Applied Science, vol. 12, no. 16, pp. 1-38, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[34] David Moher et al., “Preferred Reporting items for Systematic Reviews and Meta-Analyses: The PRISMA Statement,” Annals of Internal Medicine, vol. 151, no. 4, pp. 264-269, 2009.
[CrossRef] [Google Scholar] [Publisher Link]

[35] Matthew J Page et al., “The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews,” BMJ, vol. 372, 2021.
[CrossRef] [Google Scholar] [Publisher Link]

[36] Barbara Kitchenham, and Stuart Charters, “Guidelines for Performing Systematic Literature Reviews in Software Engineering,” Keele University, UK, 2007.
[Google Scholar]

[37] Craig Lockwood et al., “Qualitative Research Synthesis: Methodological Guidance for Systematic Reviewers Utilizing Meta-Aggregation,” International Journal of Evidence-based Healthcare, vol. 13, no. 3, pp. 179-187, 2015.
[CrossRef] [Google Scholar] [Publisher Link]

[38] Anas Abouzahra, Ayoub Sabraoui, and Karim Afdel, “Model Composition in Model Driven Engineering: A Systematic Literature Review,” Information and Software Technology, vol. 125, 2020.
[CrossRef] [Google Scholar] [Publisher Link]

[39] Brett Stevens, and Eric Mendelsohn, “Efficient Software Testing Protocols,” Proceedings of the 1998 Conference of the Centre for Advanced Studies on Collaborative Research, IBM Press, IBM Canada, NRC, 1998.
[Google Scholar] [Publisher Link]

[40] C. Yilmaz, M.B. Cohen, and A.A. Porter, “Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces,” IEEE Transactions on Software Engineering, vol. 32, no. 1, pp. 20-34, 2006.
[CrossRef] [Google Scholar] [Publisher Link]

[41] Charles J. Colbourn, “Strength Two Covering Arrays: Existence Tables and Projection,” Discrete Mathematics, vol. 308, no. 5-6, pp. 772-786, 2008.
[CrossRef] [Google Scholar] [Publisher Link]

[42] Naseer Sabri et al., “Fuzzy Inference System: Short Review and Design,” International Review of Automatic Control, vol. 6, no. 4, pp. 441-449, 2013.
[Google Scholar]

[43] Erdal Kayacan, and Mojtaba Ahmadieh Khanesar, “Fundamentals of Type-2 Fuzzy Logic Theory,” Fuzzy Neural Networks Real Time Control Appl, pp. 13-24, 2016.
[Google Scholar]

[44] L.A. Zadeh, “Fuzzy Logic= Computing with Words,” IEEE Transactions on Fuzzy Systems, vol. 4, no. 2, pp. 103-111, 1996.
[CrossRef] [Google Scholar] [Publisher Link]

[45] Fakhrud Din et al., “Fuzzy Adaptive Teaching Learning-based Optimization for Solving Unconstrained Numerical Optimization Problems,” Mathematical Problems in Engineering, vol. 2022, no. 1, pp. 1-17, 2022.
[CrossRef] [Google Scholar] [Publisher Link]

[46] Ezgi Erturk, and Ebru Akcapinar Sezer, “Software Fault Prediction using Mamdani Type Fuzzy Inference System,” International Journal of Data Analysis Techniques and Strategies, vol. 8, no. 1, pp. 14-28, 2016.
[CrossRef] [Google Scholar] [Publisher Link]

[47] Stefan Lessmann, Marco Caserta, and Idel Montalvo Arango, “Tuning Metaheuristics: A Data Mining based Approach for Particle Swarm Optimization,” Expert Systems with Applications, vol. 38, no. 10, pp. 12826-12838, 2011.
        [CrossRef] [Google Scholar] [Publisher Link]