Not Your Father's Test Automation

[article]
An Agile Approach to Test Automation
Summary:

If you think that test automation is mostly about executing tests, then you're missing out on a big opportunity. Or rather, you're missing a lot of small opportunities adding up to a big one. Consider this: stop thinking about test automation as merely executing automated tests, stop thinking about test automation as something you need expensive tools for, and start discovering automation you can implement in a couple of days and usually with extremely inexpensive tools or tools you already have available. In this week's column, Danny Faught and James Bach suggest taking a more Agile approach to test automation.

You don't need anything special to make test automation more Agile. Just adopt a very broad view of test automation and start exploring the Internet for tools that can help you. The rest takes care of itself. But in most cases it will work better if you establish a toolsmith role in your team. A good toolsmith should know how to program in a high level language, like Java, Perl, or Python, to whip up solutions quickly. A good toolsmith loves tools and learns about the free or inexpensive tools that are helpful. And a good testing toolsmith will know something about testing.

The method we use for Agile automation is straightforward: The toolsmith watches a tester work and determines how tools can help with what the tester is doing. The toolsmith's attitude is to help the tester on the tester's terms, rather than to push some private agenda.

If you have no toolsmith, then each individual tester should be on the lookout for automation opportunities. Keep in mind, the more the tester knows about tools and programming, the more effective he will be at finding useful tools.

Examples of Our Agile Approach to Test Automation
A tester had previously done a manual spot check on two comma-separated value (CSV) files and found no differences. With assistance from Danny, the tester used a tool to compare the two CSV files. The tool found an entire column of data that was wrong. After about an hour of further research, they found another free tool that did a better job of finding non-matching data.

James helped a test team query an auction system for a full report of auction status at any moment. The tool enabled them to automatically confirm that they had tested the scenarios and conditions they wanted to test, instead of simply hoping they had made no mistakes in executing their test cases. This team had tested for nearly two years without such a report, yet writing the tool took only three hours from start to delivery.

Danny has used the Perl WWW::Mechanize module to write a quick hack that loaded tens of thousands of records into a Web-based database front-end overnight. The next day, he quickly identified a performance problem with the application that was related to running with a large database.

We have both found situations where installation test tools could help the tester see exactly what an installation procedure did to the filesystem and registry.

In all of these examples, we produced real value with only a few hours of effort. We helped testers improve their testing with the help of tools. To us, that is the very definition of test automation: tool supported testing. This extends the scope of test automation far beyond the idea of happy robots that test while you sleep. Why not do automated test design? We've done that. Why not use automated test probes that alert the tester to certain kinds of problems? We've done that. It's all test automation, but it's not your father's test automation.

Deliver Something Useful in Less than Forty Hours, or Else...
Or else you picked the wrong task.

Incremental delivery is a key part of Agile automation. We find that there is a quantum difference between solutions delivered in less than a work week and those that take longer. Longer projects are riskier and absorb resources that might be used for other quicker tasks. Long tasks can be broken up into discrete deliverables of just a few days each-each one delivered and working for a tester before embarking on the next.

If your automation project

About the author

James Bach's picture James Bach

<strong>James Bach</strong> is the founder of Satisfice, Inc., a test training and consulting company. James is coauthor (with Cem Kaner and Bret Pettichord) of <a href="http://stickyminds.com/books.asp?ObjectId=488&amp;Function=DETAILBROWSE&... target="_blank"><em>Lessons Learned in Software Testing</em></a>. He has written many StickyMinds.com columns and spoken at Software Quality Engineering conferences. He can be reached at <a href="mailto:james@satisfice.com">james@satisfice.com</a>.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!