Avoiding the Script Cemetery


It's frightening how many companies are on their second, third, or even greater attempt to automate their testing—each time junking months or years of effort and work product. Here, test automation advisor Linda Hayes shows the way to avoid having to bury your automation project.

I know Halloween has passed, but I have noticed something spooky lately: lots of dead code. It's amazing how many companies are on their second, third, or even greater attempt to automate their testing—and each time at a cost of hundreds of thousands of dollars and many man-years of work. I know of one company that is on its third major rewrite, having just junked two years of effort by thirty people—for the second time. The horror.

It's not just the cost of a failed automation effort, it's the loss of the productivity you were after in the first place. The greatest benefit of test automation—cumulative coverage—is lost if you keep starting over.

Why does this happen? What can you do about it?

Slippery Slope
Because test automation tools are proprietary languages requiring special skills, many companies either recruit contractors to get them started or hire aspiring developers and train them. The team then develops, to the best of their ability and understanding at the time, an automated test library for the application under test. At the end of the contract, or the next time budgets require across-the-board cuts, or when the new hires want a career change, the original developers leave.

Left behind are tens, if not hundreds, of thousands or millions of lines of code, probably undocumented and certainly unique. An expensive work of art.

As soon as something changes and a different team has to work with this code, they will quickly determine that it is incomprehensible and will be easier to maintain if it is rewritten—maybe in a different language or tool. In other words, it would take longer to understand it than to just start over. This is the dirty secret of developers everywhere: maintaining someone else's code is a form of torture to be avoided at all costs.

And so, we start over and create yet another work of art. And on it goes. How do you break this cycle?

Accepting Reality
Start by accepting this reality: test automation is a development project. It is no different than the automation of any other task. Forget any nonsense about capture/replay and face the truth—we're talking about software here, and all that it implies. Either treat it like a development project or forget it.

Next, inventory what you need and make a build-or-buy decision for each component. Do you really need to develop a defect tracking system, or a report writer, or a test management repository, or can you buy one faster and cheaper? How much of what you need actually needs to be unique? Realize that there are both the cost of acquisition and the cost of ownership to be considered. If you build it, you have to support and maintain it.

Then, for the components you decide you must build, don't let anyone lay down a single line of code until you know what is going to be built and how long it will take. Design on the fly is costly and error prone. Whether you are interviewing contractors or employees, ask to see their proposed architecture and talk to their references. Insist on an early prototype, and short interval delivery schedules. Demand change control, code management, naming conventions—all the things we have learned you need to turn a bunch of code into an asset instead of an expense.

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.