Agile testing coach and practitioner Janet Gregory (@janetgregoryca) is the coauthor of Agile Testing: A Practical Guide for Testers and Agile Teams and a contributor to 97 Things Every Programmer Should Know . For the past ten years, she has been working with teams to transition to agile development. Find more information at janetgregory.ca or visit her blog.
1. Is there a universal definition of "agile testing" or can the definition vary from team to team and project to project?
I’m not sure if there is a universal definition, although Wikipedia has one. Agile testing to me is about involving testing activities throughout the software development lifecycle, no matter what methodology you use. However, I generally am talking with people using agile methods since those are the easiest methods to involve the whole team in testing activities. It requires a mindset change for the organization, the project teams, and individuals to make this happen.
How the practices are used may vary from team to team or project to project, but the mindset and the whole team approach to testing activities does not.
2. You recommend the automation pyramid as a "model for test coverage." But you also state "some circumstances may challenge this model." What are some of those circumstances?
The testing automation pyramid as we generally describe it shows 3 layers: the base – unit tests; the middle layer – API or service level tests; and the top layer – tests through the user interface. I’ll mention two circumstances that I’ve seen where the model is challenged.
1. There are a few teams – not many, but a few where the model doesn’t fit at all. Those teams need to understand the reasoning behind the pyramid and then adapt it to their needs with the lowest layer having the fastest feedback – what tests can be run quickly by the programmers to give fast feedback.
3. What's the best way to increase the ROI for a team looking to invest in automated tests for the first time?
I really hesitate to say there is one way, but in agile teams, we want to look for fast feedback. The most effective way I’ve seen this happen is using ATDD (Acceptance Test Driven Development) and the tools that support it. This practice enables team members to collaborate and get a shared common understanding of what is going to be tested and how they are going to test it. The side effect is that the team has automation at the API level.
I also recommend that teams look at their pain points – what is taking the most time, or repetitive that can be easily automated.
4. In the last 10 years that you've been coaching teams to transition to agile development - have you seen teams struggle with the same roadblocks, or have new challenges appeared that maybe didn't exist back in the day?
“Back in the day” – interesting phrase. My first agile team was 2001, 12 years ago. My main problem was there was nothing written or available on what testing (beyond developer and customer testing) meant. I had to discover much of what it meant by myself and with the help of friends like Lisa Crispin. As I moved from team to team, I find that many the issues I saw still remain today. The biggest difference is