The Independent Tester

[article]
Summary:

Has the agile world’s insistence on collaboration blown away the need for testers to be independent? What do we mean by “independence,” anyway? Consultant Fiona Charles argues that tester independence is essential, but that it is a state of mind that can thrive only when the whole organizational culture supports it.

“My testers currently report to the development manager. I’m thinking about moving them into a separate reporting stream with their own manager. That way, they’ll have more clout, and the project managers will have to listen to their concerns.”

Almost since the day designated testers appeared in software development, there has been debate about whether they need to be independent, and just what “tester independence” means. In the past ten years or so, there has been a trend away from isolating testers in their own organization, separated from the programmers. I believe that’s a healthy way forward for our software projects, but it does still leave the following questions open:

  • Does a tester need to be independent to be effective?
  • What do we mean by “independent”?

Where Do Testers Fit in Software Development?

Our understanding of where testers fit has evolved, most recently through the agile movement’s emphasis on cross-functional teams and collaboration. It used to be common for organizations to expect their testers to hold themselves aloof from what the programmers were doing. This was typically based on a belief that you would only get good testing from “independent” testers—that too-close collaboration would somehow compromise that independence. To foster test independence, those organizations removed their testers from the programmers’ sphere of influence, giving them a reporting stream of their own.

Organizational Independence

At the time, creating organizational independence seemed like a good idea. Managers feared that, without it, testers’ findings would be suppressed or overridden by the development leads they reported to. Testers wanted it, too, not least because they believed that having their own organization would bestow on them the credibility and legitimacy they often found themselves fighting for on projects led by programmer managers or project managers who had been programmers.

Some of those benefits were realized, but I’m not alone in believing that they were frequently outweighed by the negative unintended consequences. Too often, testers came to see themselves as the sole champions of quality in their organizations. Instead of working with the programmers and talking with them about what they were building, testers sat at their desks waiting for frozen requirements to come to them and then didn’t start testing until they got frozen systems. Many testers became gatekeepers—effectively, arbiters of quality—instead of development partners. In some organizations, the testers came to be viewed as obstacles, getting in the way of the development process while adding no real value. Rather than enhancing testers’ credibility, the combination of that view and those tester behaviors sabotaged it.

I’m describing all this as if it were safely in the past, when, in reality, it is still the situation in a lot of places. And, of course, there are many contributing factors. But organizational separation has often reinforced a dysfunctional relationship between testers and programmers, even where it wasn’t the root cause.

All of this goes through my mind when I hear a manager talk about wanting to give her testers “more clout.”

What’s the Real Problem?

First, what problem is that manager trying to solve? She believes that the project managers aren’t listening to the testers and giving their concerns due consideration. Why is that happening?

Is the real issue that the testers lack clout, or do they lack credibility? If it’s the latter, it’s doubtful that isolating the testers in their own organization would make project managers take them more seriously, even with their own manager advocating for them. It might well backfire and fuel antagonism.

If the issue is low credibility, why is that? Perhaps the testers are perceived as always “crying wolf” about minor problems. Or, perhaps they don’t have the skills needed to test adequately, and they repeatedly miss important bugs. Or maybe they’re highly skilled at testing but need help with their communication skills.

It’s always essential to ask if testers are afraid to defend the information they’re paid to uncover about the software. If managers don’t make it obvious that they value what testers have to say, testers won’t feel supported.

What signals does management send the project managers? It could be that the testers convey credible information, but the project managers feel management pressure to deliver on time regardless of product problems. Does management reward project managers for delivering quality products or only on-time projects?

If testers’ concerns are being ignored, it’s likely that several cultural factors are in play, and it will be futile to try and solve the problem without addressing the culture. Simply reorganizing the teams won’t do that—and it won’t give the testers any more clout.

The Truly Independent Tester

If we don’t believe that separate test organizations are the solution, does that mean the concept of tester independence is no longer important?

I think it is important. A tester needs to be both a full development partner and an independent observer—able to dive in and collaborate, and yet maintain his or her identity as a tester. That can be an extraordinarily difficult balancing act for anyone. If the whole culture doesn’t support it, the tester will fall off the wire.

Not everyone wants to hear what a tester has uncovered. Sometimes the tester’s information could delay a release and cause problems for management. And sometimes, not listening to that information results in bigger and more expensive problems.

It’s possible that a separate reporting stream for testers might still be useful in some circumstances. By itself, though, organizational independence will not guarantee that testers can uncover and communicate their information without fear to managers who will listen.

Instead, I suggest that tester independence is a state of mind—one that every healthy software development culture encourages and nurtures.

User Comments

4 comments
jagannath sriramulu's picture
jagannath sriramulu

Hi Fiona,

A Nice Article. First priority on a tester is to be a Partner/Collaborator and an independent tester. As you said, it is very important to have an organizational culture which values Collaboration instead of silos for Learning about the SUT.

Only aim of the Project Team should be to release a High quality product(with good enough testing) with in the best possible time & resource usage, So testing team should build their credibility of doing a good enough testing, not by building a political clout

Thanks,

jagan

September 6, 2012 - 11:33am
John Creson's picture
John Creson

Clout Vs Credibility

At first you may think you need more clout, but clout is possibly the problem, not a solution. In fact, you may think you have credibility, but what you really have is clout.

Clout can be wielded like a club, it can shut down conversations and questions. Clout can also be lost quite easily if you're wrong once.

Credibility can be nurtured.

Credibility is a 2 way street between Design, Dev, and Test that involves a shared empathy with customers' issues. This customer issues place is where we meet together. It is the focus of our work. Credibility demands that we understand each other. We have to teach one another. We have to learn from one another.

It is not good enough that designers meet customers and understand their needs. They can end up with a monopoly on customer interaction to the detriment of the whole team. The nuance in their design may be compromised or misinterpreted.

It is not good enough that testers understand the edge cases used daily in production. If no one believes it's important or has the same understanding of customer pain points, then it will wait until the customer reports it back.

It is not good enough for developers to code what is designed because "that is the design." It needs to make sense in context and for that we all need a shared context.

"I told you so" is not an acceptable response, if we had a chance, and we didn't take it, we need to understand why and work on how we can foster a better mutual understanding of our customer's dimensions.

Credibility is resilient. It is a shared understanding and a shared responsibility. It is empowering for the whole team. If it's lacking, it's a team problem.

Isolating testers will make it worse.

November 27, 2013 - 1:18pm
Mukesh Sharma's picture
Mukesh Sharma

Thanks Fiona. You bring an a very important topic to the table, where it is very important for a tester to maintain his independence and you right said, it is a difficult balancing act. One of my employees had also written on a similar topic in Sticky Minds earlier this year available at: http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&... and I thought you might be interested in reading it.

November 29, 2013 - 10:58am
Miranda Michalski's picture

I appreciate the points that you have made about the need for everyone to contribute to the quality of the project and to work together as a team, as well as asking the question of whether the need for independence is actually to resolve a culture issue or to compensate for the lack of tester credibility. 

Unfortunately, if testers reporting to a development manager on agile projects is the wave of the future, I'm concerned about the quality of testers.  If you can't get developers on a team to appreciate your contribution (i.e., defects against coding they wrote), how are they going to be able to understand how to develop you professionally and reward you for doing your job? 

Previously, my peers and I reported to someone who understood what our purpose was and would support us when we reported defects that would jeopardize the deadline or when we fought to test more than what development proscribed as necessary.  They also understood what types of training we needed to continue to expand our skills to be better testers.

Now, each of us report into the person who is most vested in having the project look successful.  In addition, that person now provides feedback on our work, determines our raises and decides how much of that pooled development/testing budget should go to training for testers.  The person telling you not to log that defect because you are jeporadizing the success of the project is the same person who will give you a negative review for not following directions.

There are laws that prevent a manager from sexually harrassing an employee, because the manager has power over a subordinate and can take punitive action against the employee if the employee refuses sexual advances.  The aren't any laws to protect testers put in a similar position.  There are cases testers where they can do what they are told and say the project their manager is running looks GREAT or find themselves getting bad reviews for "causing the project to fail." 

Personally, I have a great manager and am not experiencing these problems.  But I worry about my future.  For both the project and for the tester, I feel like the balance has gone too far in the other direction when testers report to development managers and I'm concerned.

October 22, 2014 - 2:47pm

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.