Automation

In an agile world where having dedicated testers can be a controversial idea, manual testing is a tough sell. Not all contexts have their testing automated, so what is a manual tester to do when it comes time to release? Your team starts asking about regression testing, requests estimates, and expects justifications for the time being spent. Intuition isn't the answer, and retesting everything is not an option. In this session, seasoned tester Brendan Connolly will share his five core values for focusing your regression testing efforts: consistency over correctness, behaviors over bugs, intent over implementation, conformity over complexity, and common over complete. These values not only help with decision-making, but also serve as a heuristic foundation for understanding and communicating about regression testing, as well as define a clear intent and context for all your future regression tests.

Automation fails frequently in companies due to a variety of reasons, including poor team communication, lack of skills, flaky tests, and inadequate understanding of test coverage. Even when things are going well, the automated tests sometimes grow to a size where the test suites take too long to execute for the run to be viable. James Farrier is a test automation architect who will show you ways to leverage machine learning to address these challenges. You'll learn how to determine which tests are valuable to run after each commit or build in order to cut down the suite run time, how to automatically close and open defects based on test run results, and how to separate a test into different test runs to keep track of tests in different states. Finally, he will show you how to create a results dashboard that allows for team collaboration and a better understanding of test coverage so that testing can be further streamlined.

Some developers and testers think that APIs, whether internally developed or not, should just work. Unfortunately, without a significant amount of focus on testing API services, one little flaw can spell disaster for your application—especially security vulnerabilities where hackers will target your application. Elise Carmichael discusses the risks of not fully testing APIs, how to be sure that all services are tested fully, and the types of negative testing you need to do on APIs. She shows you a method to organize your tests using Postman, a tool for REST web services testing. During this session, Elise and the participants will write some Postman API tests with variables and then execute the tests with Newman, a command line application for running Postman tests that you can include in continuous integration scripts.

A difficult challenge organizations face when adopting continuous integration/continuous delivery (CI/CD) is ensuring that every build has been evaluated for performance. Although most organizations do performance testing, how they do it and when they do it define their readiness for deploying in fast-paced environments. Anjeneya Dubey’s team faced similar challenges and found ways to embed performance engineering as part of their CD pipeline. To achieve this objective, they made changes to processes, technologies, and culture. They added non-functional requirements to “normal” requirements and built a performance engineering platform for each API. This platform automates the performance tests, provides a centralized database for metrics, and automatically analyzes pass/fail on builds, based on dynamic thresholds.