{"id":2014,"date":"2019-10-18T01:40:37","date_gmt":"2019-10-18T01:40:37","guid":{"rendered":"http:\/\/kusuaks7\/?p=1619"},"modified":"2024-03-11T16:01:33","modified_gmt":"2024-03-11T16:01:33","slug":"definitive-guide-to-selecting-your-continuous-testing-solution","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/definitive-guide-to-selecting-your-continuous-testing-solution\/","title":{"rendered":"Definitive guide to Selecting your Continuous Testing Solution"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAs a software developer and testing expert with over 20 years in this industry, I\u2019ve seen\u00a0tools<\/a>\u00a0and solutions grow and disappear from the market. That\u2019s OK, and that\u2019s the life-cycle and \u201cprice\u201d of innovation. Unfortunately, I have also seen some wrong practices when it comes to choosing such tools, especially around the testing domain.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn this post, i would like to make some order in what needs to be top of mind for managers and practitioners when trying to either change and existing solution or pick a new one.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Focus on the Process<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tI cannot stress enough the importance of the existing processes that are being followed within the organization that is evaluating a new tool. Process means,\u00a0cadence<\/strong>\u00a0of\u00a0releases<\/strong>, how are testers, developers, and business work and collaborate?. Process also means,\u00a0 Agile vs. Waterfall vs. other methods like\u00a0ATDD\/BDD<\/a>. Process also refers to the quality and the definition of done.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\nIf the teams aim to release on a weekly basis, and their test automation suite is either unstable enough, the lab isn\u2019t able to cope with the size of the tests and platforms, or the time it takes to analyze a regression test report is too long, this needs to impact the decision process as well as the evaluation criteria process.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn most organizations that I\u2019ve met, the teams are distributed and share different\u00a0skills<\/a>\u00a0in both development and testing. This is also a driver to the way test development and execution is processed.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Consider Overall Organizational Objectives<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tTeams need to ask themselves what good looks like? Do they aim towards a \u201cright<\/strong>\u201d test automation coverage of over 80%? What is the maximum time that they have to test within each iteration or sprint? Before asking the questions around what can the tool do for me, teams need to feel solid about their objectives and where they plan to grow their teams.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tJust doing a checklist of tools functionalities isn\u2019t relevant, and cannot resist over time since the domain is changing constantly, the tools sometime becomes obsolete and your own objectives might change as well.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tBottom line<\/strong>: \u201cDon\u2019t let the tool functionality drive your selection criteria, but rather let your own objectives and process fit do it<\/strong>\u201c<\/em>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIn addition, teams ought to be focusing as part of the continuous improvement agenda on measurements, metrics and visibility into how they operate over time. For such, the organization needs to set objective metrics that matches where the teams are aiming to be in 6 months, 12 months, 2 years etc.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThere are various market reports like the recent\u00a0Google DORA DevOps report<\/a>\u00a0that was showing variance between Elite DevOps organizations and lower maturity organizations. These kind of market objective metrics can serve as baseline for setting the overall goals.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Get Down to Business \u2013 What can The Tool do For Me?<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tOnly after making sure across the board the the above questions are checked, and you have a good understanding on where you wanna be, what does your team needs to be successful and meet their goals, than go and create the right list of selection criteria for your testing tools.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tSuch list from my experience ought to include the following\u00a06 buckets<\/strong>\u00a0and be able to cover the functionalities of\u00a0test creation, test analysis, and test environment<\/strong>\u00a0(lab) for proper coverage and execution.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tSome tools come with AI and ML capabilities, some have record and playback, some require advance coding skills. You should consider all and in many cases, probably combine few tools to get to the 1 single goal you\u2019ve set.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tFor each of the above categories whether you\u2019re evaluating an open-source or a commercial too, you should focus on the test creation and analysis of each. Not all frameworks and tools were created equally.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIf to only focus on web test automation with Java Script, you can observe the plethora of test frameworks as listed by\u00a0Vitali Zaidman on Medium\u00a0<\/a>to see how may solutions are there per each category.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThere are various Java Script Tools that\u2026\n