In order to understand a tester's value, we need to look at the role and understand the impact of the changing development process on this role.
Over the past seven years, I have been running the State of Testing™ Report with Lalit Bhamare from Tea-time with Testers. The annual report has provided a view of what is going on in the world of testing. We have been consistently asking some of the same questions (on purpose) over the years in order to reveal trends, while also adding new questions each year to fit the current times.
Among the questions asked are:
- What are testers’ main responsibilities?
- Where do you see yourself in five years?
- What are we doing and what should we be doing better?
But I think that the main question I’d like testers to answer is: What value do you provide your company? The ideal answer to this question is hard to come up with, so I’d like to offer some supporting questions that will assist in answering this:
- What decisions do you help make?
- How do you change the way your company delivers products or value?
- If you were not there, what would be different?
In order to understand our value, we need to look at the tester’s role and understand the impact of the changing development process on this role.
Testing’s Changing Role in Various Software Development Approaches
If we go a bit into the history of software development, when I started back in the day, the most commonly used model was the Waterfall model, which was actually the V model with a testing equivalent phase corresponding with every development phase. Later on, more iterative models came into practice, and today many organizations are shifting into DevOps after passing through Agile—although some are still working (and working well) with the Waterfall/V model.
From a testing perspective, a few things characterize the difference in models.
Previously, when working in Waterfall organizations, we started testing when the requirements and the product itself were completed, so we had close to no input on requirements. In addition to that, we received user feedback about the perceived quality of our systems only when our sales teams reported on sales results 3–6 months after the date of the release.
For testers this meant that the objective was to find all the bugs before the product was released. Quality was defined as “no bugs.”
Based on our current State of Testing numbers, 60–80 percent of organizations are using an iterative (Agile-like) approach where testers and developers work together. We get user stories, but we are still able to provide feedback on them before we start testing the functionality. And yet we still get feedback regarding the customer-perceived value of our product from external sources and only after some time has passed after the release.
Unlike the above-mentioned approaches, in DevOps we are merging the development team with the operations team. Together they plan, release, and monitor the product, and this allows them to know a lot more about how users are interacting with the system, which provides indicators on the perceived level of quality of the system in the eyes of our end-users. The feedback loop is something we can now do on our own.
DevOps can sometimes feel like Agile, but it is radically different. You don’t need to wait for anyone to get feedback. If for example I want to develop a new feature, my team can measure the impact of our efforts directly, and there is no need for anyone to provide us with the information we need. If there is a bug, we can fix it faster, so the cost of releasing bugs into production is significantly reduced.
The next phase of evolution is the transition to Modern Testing.
Modern testing was originally defined by Alan Page and Brent Jensen in the AB testing podcast. It is an evolution of how organizations should approach both testing as well as quality in a more holistic way. I would like to focus on the Modern Testing mission statement, which for me was an eye-opener: Accelerate the Achievement of Shippable Quality.
Modern testing teams are not in charge of testing or finding bugs, but rather they are in charge of releasing products faster after integrated development teams, which also include testers, have achieved the level of quality we would like. We are a part of the business and contribute to it.
The Value of Our Job Has Shifted from Testing to Quality Assurance
During my first years as a tester, I sometimes felt that I was busy metaphorically putting out fires and getting blamed for problems that I did not cause. This is something that can be changed when we focus more on quality and less on testing. I use the following Mind Map to understand what it is that we should do when we focus more on quality than on testing:
As you can see, there are quite a few things that can be done, and here are the main areas where we can focus.
User Story Creation and Validation
- A big issue that many organizations have is that they try to create a very wide functionality array, assuming they know what the customer wants. It’s better to use the MVP (Minimal Viable Product) approach where you release a lean version of your product, check the feedback, and only then develop the complete functionality based on what you learn.
- Define measurable success criteria to know if the user story was successful or not.
- Define the monitoring tools. When we talk about shifting right and testing in production, we will not be able to know if we are doing it right without measuring.
- Detect conflicts in user stories with other product areas. As the features continue to grow in products, we sometimes see user stories that have conflicts between different product areas. Testers hold a wide product understanding and can catch these better than some other team members.
- Accessibility is either something that is a part of the culture of the company or should be, and so is usability. This is something that the testing team can contribute to.
- Capture feedback from other departments in the organization. No matter how great and professional your product team is, I highly recommend also talking with other departments in your organization, such as support, sales, customer success, and others, to get their input. Have they seen this request? Are we covering everything related? This is something that we as the testing/quality team can actually bring into the user story validation.
Bring Customers’ Approach into the Process
Customers are the only ones who will actually be able to define quality, as they are the only ones who actually know what they need. And sometimes even customers have a hard time knowing what it is that they want or need. How can we do this?
In many cases, testers have better customer-facing skills (how to listen and how to speak) than other members of the DevOps team.
- Participate in customer-facing activities: calls, visits, and support interactions. Take notes. Before COVID-19, many companies had a habit of customer visits, which have been replaced by Zoom-like meetings. Try to take part in these meetings.
- Create customer personas for the development process. This is a common activity in marketing and business development that can also significantly contribute to the “customer in mind” development approach. When creating a feature, having a persona in mind will help your team make sure the feature is relevant and of good quality.
- Data is king. Whenever possible, bring data that can be used to improve your product. For example, when developing one of PractiTest’s features with the ability to import from Excel, I have used real customer examples to make sure our solution fits their needs in real life.
Test Training and Coaching
We are not going to “only test,” but testing will still be done. This means that we will need to train other members of the team to perform testing together with us.
- Test training for new developers: Developers will test differently than testers. They look at the world from a different perspective. Developers will often be better suited for the use of structured testing rather than exploratory testing approaches.
- Test process definition: Define the process and the players to make sure everyone is clear with what they should be doing
- Pair testing sessions with developers: Ideally, pair a tester with a developer or a developer who has experience in testing with a novice one.
- Test briefings and debriefings with developers: This will help developers understand the 360 degrees of product quality and get a more holistic view of the product, not focusing solely on their developed sections.
It’s not only about training but also about test enablement.
- Cookbooks: I really like the concept of a cookbook that guides in the simplest way.
- Generating environments: Generating realistic environments and data to use for testing turns hypothetical situations into something the person who is testing can understand and relate to, thus allowing them to test better.
- Create checklists on the system and heuristics: Developers typically know a part of the system, but they might be required to test the entire system. Checklists and heuristics will help them get familiarized with the system in a constructive way.
- Test management system: If we want our developers to really be a part of the quality loop, we need to give developers access to the test management system. This will help them follow the process and progress and will help them feel as responsible for the product’s quality.
When we focus more on quality and less on testing it doesn’t mean that testing will not be done or that our jobs are going to disappear. What it does mean is that the focus will be changed from finding bugs to enabling the achievement of quality products while supporting the company’s business goals.