There are many diverse ideas about what being a tester means in agile development environments. This leads to confusion between how agile testers and agile QA “fit” into agile teams and what the QA tester responsibilities are. John Stevenson explains why there appears to be some fear and a little distrust of agile environments among some testers, then offers suggestions for dealing with their confusion.
At my day job we’ve been driving toward a more agile way to develop software. Just like with any other change in work style, there is some confusion and, for a few, a little bit of fear. People are being asked to step out of their comfort zones and embrace a constantly changing, dynamic way of working. And—dare I say it—testing may be the most disrupted of the software roles on agile projects.
What do we think the role of a tester is? Can we even agree?
There are many diverse ideas about what being a tester means in agile development environments. This leads to confusion between how agile testers “fit” into development teams and what their role is.
In this article, I will start by looking at my experiences to explain why there appears to be some fear and a little distrust of agile among some testers, then offer suggestions that may be of use to testers for dealing with their confusion.
Confusion in the Literature
One of the few things people tend to agree on is that to deliver working software quickly, teams need automation in place which leads to the “testers must code” discussion. Experts like Elizabeth Hendrickson, Rob Lambert, and Michael Larsen have suggested that coding is the next logical step for testing, but there is no overall consensus on that. James Bach’s comment to Rob Lambert’s article suggests there is more for testers to do than only learning to code:
. . . [T]elling testers that they need to learn to code will discourage interesting thinkers from becoming testers. We can’t assume that people who like to program are just like other people in every important way.
Tester as an Agile Team Member
Historically, testing activities have happened at the end of the development (coding) effort because a tester’s responsibilities included proving the requirements were met, ensuring the software worked, and finding bugs in the mostly finished product.
The Agile Manifesto flips around titles and roles and says to focus on individuals and interactions over processes and tools.
This means that as testers, there is a need to work as members of the team and to engage with others within the team, regardless of titles. If you see a task that needs doing and you have the skills to do that task, then just do it.
Agile processes mean testers are part of the delivery team and should try to be involved from the beginning of the project. Testers should have the ability to “switch hats” and be involved to support and assist the team.
The following are only a few examples of the vast number of roles you could be taking on.
One huge benefit of small slices of work, such as a story taking a few days to complete, is that design meetings happen all the time and can be a lot easier to get invited to. Testers can contribute to these meetings, asking “What if . . .” types of questions, such as “What if a user did this?” or “What if the server we call with the web services does not respond?”
During the design discussions, try to create a model of what you think the system will look like. If you find yourself being confused about the information being presented or you feel something does not seem right, this is the moment to ask questions to clarify. The answers you get could be useful for noting areas to explore at a later date. Offer suggestions about how it would be easier to test if development were completed in a certain way, and look for ways to make the system design testable.
If there are uncertain areas about how a feature could or will be implemented, this would be a great area for you to explore after development. Make a note and maybe create a test charter in your test plan, perhaps using a mind map.
Consider from the information provided what could require checking and make a note of possible automation tasks. You may find yourself contributing to what checks are automated as acceptance or unit tests.
Automating checks does not have to be your job; it might be something you do part time, as needed. You don’t always need to write code or be a skilled programmer to do automation. Many tools exist that simplify the process. Knowing how to write code might be a useful skill if you feel it will benefit the team. If you do not have an interest in learning to code, it would be a good idea to at least try to learn to read code.
Some of the frameworks used extensively for test automation use English words and phrases, such as the Cucumber, Gherkin, and FitNesse frameworks.
They follow similar approaches for creating automated checks. First, create scenarios in a very high-level language that loops a bit like English; then run a parser over the scenario to automatically create “stub” functions; finally, fill in the functions with actual code.
For Cucumber, step definitions can be written in many different programming languages, and the level required to write these types of definitions is not too intense.
For more information on learning to code from a testing perspective, I recommend the excellent book Java For Testers by Alan Richardson.
Because of the tight timeboxes in agile, the skill of exploratory testing becomes incredibly powerful and valuable. This is where a traditional tester can shine by exploring the product to reduce risk and uncertainty.
Exploratory tests are one-time investigations into a specific risk, not repeatedly running the same checks over and over. Two places to pull test ideas from are those story kickoff design discussions and communicating with others within the team. Use this information to uncover where there is ambiguity in the system or areas where people have raised concerns or questions. These are strong indications that there could be issues that have yet to be uncovered.
Being an explorer is about mapping the system as you go and then reassessing your voyage as you discover exciting and interesting bits of information. As testers, we need to uncover assumptions and incorrect or missing information. Most importantly, we need to find out what the product actually does rather than what we believe it does or how it has been documented to behave.
This does not mean you should approach the process in a random, unplanned, or unstructured way. Your plan does not have to be too detailed, but it should be just enough to get you started. As you uncover more about the system and update your plan with the new evidence you have, you may want to make your work more systematic and document what you tested (as opposed to what you plan to test). Session-based test management can be a great way to manage the test effort and works well in agile environments, especially with kanban-style boards.
One of the tasks I feel is most overlooked within agile development is the tester’s responsibility to share information and knowledge, which is so much more than simplistic pass/fail details. The agile tester should be reporting useful and valuable information, such as:
- Issues or bugs they have uncovered
- Areas they didn’t have time to explore
- Recurring defects that could be prevented
- Blocks, wait states, and other problems obstructing additional testing
Agile testers need to be communicating as much as possible with the rest of the team, keeping them informed of what they have found. A great indication of a good test reporter is that when it comes time for the scrum meeting, there are no surprises for the rest of the team—the tester has already informed everyone about their findings.
Normally these conversations are about clarifying assumptions and ambiguity, or confirming a defect really is a defect before raising it on the bug-tracking system. The tester in these situations needs to be a salesperson, a negotiator, a user advocate, a bit of a journalist, an investigator, a scientist, and a psychologist; and that’s just to start!
Often, testers fall into a habit of simplistic reporting about how many tests ran or what percentage is done. It is better to tell the team what has and has not been tested and base completion upon the evidence of what was uncovered.
Tester Responsibilities on an Agile Team
The purpose of a tester in agile environments is about getting things done rather than roles. As a tester, you should try to act as a service to the project and ask yourself, “What can I do that is best for the team or for the project?”
There are many other roles a tester carries out during testing, such as a coach or mentor, a service provider, a sage, a confidant, and a person who is willing to listen as well as challenge others for the benefit of the team.
Take a look at your own team and think about the many roles testers can play. Then ask yourself what roles are missing and which ones you can fulfill for your team.
There is so much confusion in the industry about what testers should do in agile projects. Part of the problem is that the most widely used framework - Scrum - says almost nothing about it. This article helps to clear some things up.
Experienced agilists know that things like test plans are needed. Indeed there has lately been a series of discusison threads on this in various LinkedIn agile groups, and the overwhelming consensus is that (1) a test plan is needed, (2) it should be concise and allowed to evolve (not big-plan-up-front), and it should focus on test strategies - not detailed test procedures: e.g., it should say how functional testing will be done (unit, acceptance, integration), how performance and stress testing will be done, how security testing will be done, etc.
And then there is the issue of who is a tester. As John points out, anyone on a team can be a tester. In an continuous delivery setting, a tester is usually a test programmer. And it is highly technical. Non-techies can write cucumber scripts as John explains, but the implementation of those is pure code. BTW, the Java version of cucumber is not ready for prime time, so if you are a Java shop use JBehave - it works in a very similar way to cucumber.
But I personally feel that testing needs structure. Agile teams tend to encourage ad-hoc: try to include yourself in discussions, etc. But for testing, you need to dot the i's. You _need_ to be in certain discussions. Things _need_ to be done a certain way, or the process is compromised: e.g., the person who codes a story should never, ever be the person who writes the acceptance test code for a story.
Finally, the concept of test coverage is essential for continuous delivery and enterprise risk management. The old waterfall process relied on gates to make sure everything got checked: continuous delivery relies on testing. For that to work, there needs to be a way to measure the completeness of the tests. That is what test coverage is, and the concept applies not only to functional tests, but also to every category of risk: security, reliability, maintainability, etc. If there is an area that the organization cares about, there should be tests for that area, and their coverage should be assessed. And putting all that together usually requires someone who knows a-lot about those areas. It also requires some independent oversight.
None of that is mentioned by Scrum, but that does not mean it is not important. If you want to get to continuous delivery of reliable and secure systems, these things are necessary.
Good read. Really emphasizes how we're undervaluing testing and how we can begin to improve that outlook. I'm looking forward to expanding on these ideas and setting up concrete plans to incorporate them into my teams' methodologies.
Typo: " loops" -> looks