IJBTT

Comparative Study of Software Test Automation Frameworks

	International Journal of Engineering Trends and Technology (IJETT)
	© 2019 by IJETT Journal
	Volume-67 Issue-11
	Year of Publication : 2019
	Authors : Khalid Eldrandaly, Mahmoud Abd ElLatif, Nora Zaki
	DOI :  10.14445/22315381/IJETT-V67I11P216

Citation 

MLA Style: Khalid Eldrandaly, Mahmoud Abd ElLatif, Nora Zaki. "Comparative Study of Software Test Automation Frameworks" International Journal of Engineering Trends and Technology 67.11 (2019):94-105.

APA Style:Khalid Eldrandaly, Mahmoud Abd ElLatif, Nora Zaki. Comparative Study of Software Test Automation Frameworks  International Journal of Engineering Trends and Technology, 67(11),94-105.

Abstract
Software testing is one of the most critical phases of software development life cycle. The time and cost consumed by software testing are one of the most critical limitations of software testing. The testing process can be done manually or automatically. Recently, software automation testing is applied in many software organizations to guarantee the software quality and to reduce the cost and time consumed in manual testing. Software test automation framework is an independent application which maximizes the automation efforts by facilitating the execution of the automated test scripts. There are many software test automation frameworks (STAFs) are available in the marketplace. The automation testers face a problem in selecting the best STAF that meet their testing requirements. The main objective of this paper is to provide the automation testers with a good understanding of STAFs. This work aims to evaluate each STAF in terms of their scripting approach, features, advantages and disadvantages. Furthermore, it conducts a comparative analysis among STAFs by using the essential parameters of automation projects such Scripting capabilities, time, application size, scripting approach, modularity, scalability, reusability, maintainability, and complexity. This analysis aims to help the testers to select the excellent fit STAF.

Reference

[1] Sharma C, Sabharwal S, Sibal R. A survey on software testing techniques using genetic algorithm. IJCSI International Journal of Computer Science. Vol. 10, Issue 1, No 1, January 2013.
[2] Jamil MA, Arif M, Abubakar NS, Ahmad A. Software Testing Techniques: A Literature Review. In 2016 6th International Conference on Information and Communication Technology for the Muslim World (ICT4M). 2016 Nov 22. (pp. 177-182). IEEE.
[3] Hooda I, Chhillar RS. Software test process, testing types and techniques. International Journal of Computer Applications. 2015 Jan 1;111(13).
[4] Chauhan RK, Singh I. Latest research and development on software testing techniques and tools. International Journal of Current Engineering and Technology. 2014 Aug;4(4):2347- 5161.
[5] Nidhra S, Dondeti J. Black box and white box testing techniques-a literature review. International Journal of Embedded Systems and Applications (IJESA). 2(2),2012 Jun; 29-50.
[6] Amannejad Y, Garousi V, Irving R, Sahaf Z. A search-based approach for cost-effective software test automation decision support and an industrial case study. In2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops 2014 Mar 31 (pp. 302-311). IEEE.
[7] Thummalapenta S, Devaki P, Sinha S, Chandra S, Gnanasundaram S, Nagaraj DD, Kumar S, Kumar S. Efficient and change-resilient test automation: An industrial case study. In2013 35th International Conference on Software Engineering (ICSE) 2013 May 18 (pp. 1002-1011). IEEE.
[8] Kaur T, Goel SG. A Framework for Enhancing Performance and Minimizing Automation Testing Cost and Time Using Efficient Methods (Doctoral dissertation). 2015.
[9] Kumar D, Mishra KK. The Impacts of Test Automation on Software`s Cost, Quality and Time to Market. 7th International Conference on Communication, Computing and Virtualization. Procedia Computer Science. 2016 Jan 1;79:8-15.
[10] Gandhi GM, Pillai AS. Challenges in gui test automation. International Journal of Computer Theory and Engineering. 2014 Apr 1;6(2):192.
[11] Rafi M. Automated Software Testing. A Study of State of Practice/Dudekula Mohammad Rafi & Kiran Moses. School of Computing Blekinge Institute of Technology. Sweden. 2010:5.
[12] Taipale O, Kasurinen J, Karhu K, Smolander K. Trade-off between automated and manual software testing. International Journal of System Assurance Engineering and Management. 2011 Jun 1;2(2):114-25.
[13] Sharma S, Vishawjyoti M. Study and analysis of automation testing techniques. Journal of global research in computer science. 2013 Jan 10;3(12):36-43.
[14] Thummalapenta S, Sinha S, Singhania N, Chandra S. Automating test automation. InProceedings of the 34th International Conference on Software Engineering 2012 Jun 2 (pp. 881-891). IEEE Press.
[15] Böhme M, Paul S. A probabilistic analysis of the efficiency of automated software testing. IEEE Transactions on Software Engineering. 2016 Apr 1;42(4):345-60.
[16] Vishal S, Vaishali N. A review on Automating Test Automation. International Journal of Advance Research in Computer Science and Management Studies Volume 2, Issue 12, December 2014 pg.79-86.
[17] Clark JA, Dan H, Hierons RM. Semantic mutation testing. Science of Computer Programming. 2013 Apr 1;78(4):345-63.
[18] Puolitaival OP, Kanstrén T, Rytky VM, Saarela A. Utilizing domain-specific modelling for software testing. In3rd Int’l. Conf. on Advances in System Testing and Validation Lifecycle (VALID2011) 2011 Oct 23.
[19] Rodrigues A, Dias-Neto A. Relevance and impact of critical factors of success in software test automation lifecycle: A survey. InProceedings of the 1st Brazilian Symposium on Systematic and Automated Software Testing 2016 Sep 19 (p. 6). ACM.
[20] Vishal S, Vaishali N. Generic Test Automation. international Journal of Science and Research (IJSR) (2013). 2319-7064
[21] Cervantes A. Exploring the use of a test automation framework. In2009 IEEE Aerospace conference 2009 Mar 7 (pp. 1-9). IEEE.
[22] Artzi S, Dolby J, Jensen SH, Møller A, Tip F. A framework for automated testing of javascript web applications. InProceedings of the 33rd International Conference on Software Engineering 2011 May 21 (pp. 571-580). ACM.
[23] Lamba S, Rishiwal V, Rana A. An automated data driven continuos testing framework. International Journal Of Advanced Technology In Engineering And Science. 2015 Feb;3(1).
[24] Yalla M, Shanbhag M. Building automation framework around open source technologies. InProc. of Software Testing Conference 2009 (pp. 6-9).
[25] Rajesh P. Test Automation Framework. IJITKM Volume 7 • Number 2 Jan– June 2014 pp. 214-219
[26] Bajaj KS. Hybrid Test Automation Framework for managing Test Data. International Journal of Pure and Applied Mathematics. 2018;118(9):265-76.
[27] Raval RR, Rathod HM. Comparative study of various process model in software development. International Journal of Computer Applications. 2013 Jan 1;82(18).
[28] Balaji S, Murugaiyan MS. Waterfall vs. V-Model vs. Agile: A comparative study on SDLC. International Journal of Information Technology and Business Management. 2012 Jun 29;2(1):26-30.
[29] Aitken A, Ilango V. A comparative analysis of traditional software engineering and agile software development. In2013 46th Hawaii International Conference on System Sciences 2013 Jan 7 (pp. 4751-4760). IEEE.
[30] Singhto W, Denwattana N. An experience in blending the traditional and Agile methodologies to assist in a small software development project. In2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE) 2016 Jul 13 (pp. 1-5). IEEE.
[31] Umbreen M, Abbas J, Shaheed SM. A Comparative Approach for SCRUM and FDD in Agile. International Journal of Computer Science and Innovation. 2015;2:79-87.
[32] Kumar G, Bhatia PK. Comparative analysis of software engineering models from traditional to modern methodologies. In2014 Fourth International Conference on Advanced Computing & Communication Technologies 2014 Feb 8 (pp. 189-196). IEEE.
[33] Heba E, Mazen S. Agile and traditional requirements engineering: A survey. International Journal of Scientific & Engineering Research 4.9 2013. 473-482.
[34] Milad H, Amal EA, Mostafa-Sami M. Automated Software Testing Framework for Web Applications. International Journal of Applied Engineering Research. Volume 13, Number 11 (2018) pp. 9758-9767.
[35] Singla S, Kaur H. Selenium Keyword Driven Automation Testing Framework. IJARCSSE 2014 Jun;2277.
[36] Kumar A, Saxena S. Data driven testing framework using selenium WebDriver. International Journal of Computer Applications. 2015 Jan 1;118(18).
[37] Koneru S, Chandra P. A Comparative Study of Testing Framework with Special Emphasis on Selenium for Financial Applications. International Journal of Soft Computing12.3 (2017): 148-155.
[38] Tang J, Cao X, Ma A. Towards adaptive framework of keyword driven automation testing. In2008 IEEE International Conference on Automation and Logistics 2008 Sep 1 (pp. 1631- 1636). IEEE.
[39] Bajpai N. A keyword driven framework for testing web applications. International Journal of Advanced Computer Science and Applications. 2012;3(3):8-14.
[40] Sivanandan S. Agile development cycle: Approach to design an effective Model Based Testing with Behaviour driven automation framework. In20th Annual International Conference on Advanced Computing and Communications (ADCOM) 2014 Sep 19 (pp. 22-25). IEEE.
[41] Kaur H, Gupta G. Comparative study of automated testing tools: selenium, quick test professional and testcomplete. Int. Journal of Engineering Research and Applications. 2013 Sep;3(5):1739- 43.
[42] Laukkanen P. Data-driven and keyword-driven test automation frameworks. Master`s thesis. Helsinki University of Technology. 2006 Feb 24.
[43] Divya A, Mahalakshmi SD. An efficient framework for unified automation testing: a case study on software industry. International Journal of Advanced Research in Computer Science & Technology. 2014;2.
[44] Sadiq M, Firoze F. A method for the selection of software testing automation framework using analytic hierarchy process. International Journal of Computer Applications. 2014 Jan 1;102(1).
[45] Kumar P, Dr. Kavita. Automation Framework for Database Testing, IJARSE, Vol.No.5, issue No.6, June 2016.
[46] Fewster M, Graham D. Software test automation: effective use of test execution tools. ACM Press/Addison-Wesley Publishing Co.; 1999 Jul 1.
[47] Kaner C. Pitfalls and strategies in automated testing. Computer. 1997 Apr;30(4):114-6.
[48] Gustavsson R, Kostopoulos D. From Requirements Specification to Test Scripting: Towards Automated Support . Master Thesis in Computer Science with Specialization in Software Engineering. 2015.
[49] Samik R, Uday B. A Survey on Test Automation. International Journal of Innovative Research in Computer and Communication Engineering. Vol. 4, Issue 6, June 2016.
[50] Heet P , Danish C. Agile Testing Automation Framework. International Journal Of Engineering And Computer Science. Volume 5. Issue 09. September 2016. Page No.17996-18000.
[51] Rahman M, Gao J. A reusable automated acceptance testing architecture for microservices in behavior-driven development. In2015 IEEE Symposium on Service-Oriented System Engineering 2015 Mar 30 (pp. 321-325). IEEE.
[52] Hammond S, Umphress D. Test driven development: the state of the practice. InProceedings of the 50th Annual Southeast Regional Conference 2012 Mar 29 (pp. 158-163). ACM.
[53] Gómez LA. Analysis of the impact of test based development techniques (TDD, BDD, AND ATDD) to the software life cycle(Doctoral dissertation).
[54] Wanderley F, da Silveria DS. A framework to diminish the gap between the business specialist and the software designer. In2012 Eighth International Conference on the Quality of Information and Communications Technology 2012 Sep 3 (pp. 199-204). IEEE.
[55] North D. Introducing behaviour driven development. Better Software Magazine. 2006.
[56] Solis C, Wang X. A study of the characteristics of behaviour driven development. In 2011 37th EUROMICRO Conference on Software Engineering and Advanced Applications 2011 Aug 30 (pp. 383-387). IEEE.

Keywords
Software testing, automated software testing, STAF, agile methodology, BBD framework