Greg Paskal, evangelist in testing sciences and lead author for RealWorldTestAutomation.com, chats with TechWell community manager Owen Gotimer about testing as a craft, choosing the right test automation tools, and current testing trends around the world.
Most organizations understand that test automation is essential for modern application delivery processes. They’re just not sure how to make it a reality in an enterprise environment without exorbitant overhead and massive disruption. Enterprise organizations typically achieve small victories, but the process ultimately decays due to challenges in five main areas. Understanding these challenges will help us overcome them.
Scenario: I am working for a Product based company where schedule is very tight and we usually do not get much time between releases to create proper test cases, update and manage test suite, revisit the existing ones. Hence even though we have quite a large amount of TC repository, most of the testing goes adhoc and so are the defects, rendering those cases un-fruitful. Following traditional testing process consumes lots of test execution time which affect quality. Our process does not follow proper agile methodology or waterfall. Even though we have dates timelines, estimations from Dev, QA etc but 99% of times they tend to extend due to last minute change request (decision: upper management). Overall, since we are not following any proper methodology, following traditional testing method of creating ground level cases with steps/ updating on test management tool/ executing and then following defect management etc takes a toll.
When it comes to testing, there tends to be a differentiation between “production software” and everything else. But our ideas and principles about testing software are true for all software, not merely the code that will run in front of customers or the APIs that make things happen. Any software built for a purpose needs to be tested against that purpose, including the software running our test automation.
How have others been able to test a physical credit card on a POS system? Do you use your own card? Do you get a company credit card and expense everything? Is there a company you can purchase something like this from?
Testing continuous technological change can seem like chaos. There are many challenges that need to be managed, such as unavailability of power, excessive temperature, incorrect configuration, unexpected behavior of services, network downtime, and processing slowdown in production. By deliberately engineering chaos, we’ll be able to discover many of our systems’ weaknesses before our users do.
We have a very complex software product that is installed and runs on a Computer or Server. The Software is managed by a Management Console in the Cloud.
For years all of our End to End testing was manual. But this was very resource intensive and time comsuming. So a couple of years ago we moved to a more agule development approach and built a test automation system to run these End to End tests. However, we quickly found limitations for example we couldn't have automated tests check what is being displayed in the UI running on the Computer, or check what is being displayed in the Management Console in the Cloud. So we can never fully replicate in automation what the manual tests were doing.
Fixing a bug in one area of the software may break something in another area. To detect whether defects have been introduced, we need to perform regression testing—executing certain test cases again to see whether a change has affected other existing features. But how do you make time for another testing cycle prior to every production release? You need to get QA involved earlier in the software development lifecycle.
Talia Nassi, developer advocate at Split Software, chats with TechWell community manager Owen Gotimer about the fears, myths, and benefits of testing in production and how to get your stakeholders on board. Continue the conversation with Talia (@Talia Nassi) and Owen (@owen) on the TechWell Hub (hub.techwell.com)!
Traditional GUI automation is linear; it follows a set of steps. The first time you run it, it can't add any value, as the feature isn't done until the automation runs. Once test automation runs a second time, it effectively becomes change detection. This leads to a large number of "failures" that are not actually failures. Whether they are false positives or false negatives, we need a way to fix the automation tooling.