After finding and reporting a bug, a tester may get this response from a developer: "Please rerun the test on the latest version of the code and check if the bug still reproduces." This seems like a rational request; just as a change can cause a bug to appear, it can also fix a bug. But is following up the responsibility of the tester or the developer? And if the bug is no longer there, how do you classify and close it?
How can mature companies with complex systems achieve the level of test automation that modern delivery schedules and processes demand? There are four strategies that have helped many organizations finally break through the test automation barrier: Simplify automation across the technology stack, end the test maintenance nightmare, shift to API testing, and choose the right tools for your needs.
Companies that want to reduce testing costs usually try working with fewer people, or even cutting back on the amount of testing done. But with those approaches, quality usually suffers. Releasing a critical bug and suffering the subsequent pain usually costs multiple times what testing would. There are better ways to save money, and it can be done just by being smarter about our test cases and their structure.
As your QA team grows, manual testing can lose the ability to focus on likely problem areas and instead turn into an inefficient checkbox process. Using machine learning can bring back the insights of a small team of experienced testers. By defining certain scenarios, machine learning can determine the probability that a change has a serious defect, so you can evaluate risk and know where to focus your efforts.
In the era of agile and DevOps, release decisions need to be made rapidly—preferably, even automatically and instantaneously. Test results that focus solely on the number of test cases leave you with a huge blind spot. If you want fast, accurate assessments of the risks associated with promoting the latest release candidate to production, you need a new currency in testing: Risk coverage needs to replace test coverage.
Best practices for test automation emphasize reliability, portability, reusability, readability, maintainability, and more. But how can your existing automated test suite adopt these qualities? Should you address these issues with your current tests, or create an entirely new set of tests? Here are some questions that will help you determine if your test automation maintenance program is operating as it should be.
With 2020 upon us, software development firms seeking to increase their agility are focusing more and more on aligning their testing approach with agile principles. Let’s look at seven of the key agile testing trends that will impact organizations most this year.
Teams everywhere are looking to speed up testing without sacrificing quality, so once again, some of the top articles last year were about continuous integration, machine learning, and—of course—how to best implement and use test automation. But readers were also interested in what they shouldn't be doing, with two high-ranking articles about test practices we should stop and a tool you may be misusing.
The advantages of shifting left and testing as early as possible are obvious. But as you automate more testing, the test suite grows larger and larger, and it takes longer and longer to run. Instead, just automate the process of finding the right set of tests to run. The key to that is machine learning. This isn't AI bots finding bugs autonomously without creating tests; this is a different way to use machine learning, and it’s far simpler.
The test team uses the test automation system to execute thousands of test cases because … why not? The tests are running automatically, for free, so there is no incentive to improve test efficiency. Just run them all! But eventually, as more and more tests are added, the system becomes overloaded. Test runs are delayed and you get a bottleneck. Don't throw more money—or new systems—at the problem; do this instead.