An Uncomfortable Truth about Agile Testing

One characteristic of agile development is continuous involvement from testers throughout the process. Testers have a hard and busy job. Jeff has finally starting to understand why testing in agile development is fundamentally different.

In organizations that have adopted agile development, I often see a bit of a culture clash with testers and the rest of the staff. Testers will ask for product specifications that they can test against to verify that what was built meets the specifications, which is a reasonable thing to ask for. The team often has a user story and some acceptance criteria in which any good tester can quickly poke holes. "You can't release the software like this," the tester might say. "You don't have enough validation on these fields here. And what should happen if we put large amounts of data in this?"

"This is just the first story," I'd say. ""We'll have more stories later that will add validation and set limits on those fields. And, to be honest, those fields may change after we demonstrate the software to the customerthat's why we're deferring adding those things now."

"Well, then there's no point in testing now," the testers would usually say. "If the code changes, I'll just need to retest all this stuff anyway. Call me when things stop changing."

I can understand their concern, but I also know we need to test what we've built so fareven if it's incomplete, even if it will change. That's when I realized what testing is about in agile development. Let me illustrate with a story:

Imagine you're working in a factory that makes cars. You're the test driver, testing the cars as they come off the assembly line. You drive them through an obstacle course and around a track at high speed, and then you certify them as ready to buy. You wear black leather gloves and sunglasses. (I'm sure it doesn't work that way, but humor me for a minute.)

For the last week, work has been a bit of a pain. When you start up your fifteenth car of the day, it runs rough and then dies after you drive it one hundred yards from the back door of the plant. You know it's the fuel pump again, because the last five defective cars you've found have all had bad fuel pumps. You reject the car and send it back to have the problem properly diagnosed and fixed. You may test this car again tomorrow.

Now, some of you might be thinking, "Why don't they test those fuel pumps before they put them into the cars?" And you're right, that would be a good idea. In the real world, they probably test every car part along the way before it gets assembled. In the end, they'll still test the finished car. Testing the parts of the car improves the quality downstream, all the way to when the car is finally finished.

Testing in agile development is done for much the same reason. A tester on an agile team may test a screen that's half finished, missing some validation, or missing some fields. It's incompleteonly one part of the softwarebut testing it in this incomplete stage helps reduce the risk of failures downstream. It's not about certifying that the software is done, complete, or fit to ship. To do that, we'd need to drive the "whole car," which we'll do when the whole car is complete.

User Comments

1 comment
Jeffrey Croft's picture
Jeffrey Croft

I think that before agile came along we had iterative based on developing Use Case flows which was very Tester 'friendly' as you could easily identify with the high-level requirements and each Use Case flow was a meaningful user action that delivered a purposeful outcome in terms of an 'actor'.

Developers did not like it so much as that's not the way they develop and deliver software i.e. to build up enough of each component to provide a complete Use Case flow.

Now with agile its much more Developer-centric with early sprints delivering small features that may or may not expose some functionality that's actually testable.One thing I do think that suffers is traceability with often a disjoint between the Requirements / Technical spec and the stories. What does the story deliver in terms of the overiding spec that the tester has been pouring over so much.

I know its the nature of the beast but testers should also be wise to what's 'firm' and whats likely to change so as not to invest a lot of time inventing complex test scenarios or criteria for stories that have features that are very much still in the discussion stage.

November 8, 2013 - 7:00am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.