Is Test Automation a “Project”?

[article]
Member Submitted
Summary:
Test automation can turn into a real pain in the neck if a designated team is in charge of it or if the automators work on it as a separate project. In this article, Lisa Crispin seconds Bob Jones’s recent call for whole-team test automation and elaborates on the dangers of relegating test automation to an isolated project rather than integrating it into the overall software development process.

Recently I read a post on a mailing list about “failed test automation projects.” This expression made me stop and think. Automating different types of tests is an integral part of software development, and that’s not only on agile teams. I worked in a waterfall shop back in the ‘90s where the programmers automated all their unit tests and ran them in a continuous build. My test team, which was involved in every project from start to finish, automated GUI-level functional tests as well as performance and load tests. Our product had no critical bugs in production.

I’m dismayed that many people still think test automation is something to be done by a team of test automation specialists and that QA should be a separate team, even in agile projects. I think every software team would benefit if they stopped thinking in terms of “test automation projects.”

Why Do We Automate Tests?
We automate tests to help our team work at a sustainable pace with a manageable level of technical debt. We automate regression tests in order to shorten the feedback loop between check-ins of new code and discovery of regression failures. We automate load and performance tests because, well, there is really no other way to do those types of tests. We automate repetitive steps to save time in setting up scenarios for exploratory testing. We automate the generation of test data for various purposes.

These are not optional activities. They’re all necessary in order to deliver high-quality software that provides business value in a timely manner without killing the team through overwork or sapping them of the will to live by having them do the same tedious keystrokes over and over.

The Problem with “Projects”
Because the term “test automation” contains the word “test,” managers think testers should do the test automation. They purchase a GUI test tool, usually one with a “record and playback” feature, and start a project for the QA department to automate all the manual regression tests. Perhaps they hope that once these tests are automated, they won’t need that pesky QA department anymore!

Some companies form a team dedicated to the test automation project but staff it with testers. After all, they’ve purchased a test tool that they believe is so easy to use that anyone can create the automated scripts.

In these scenarios, if the teams actually automate any tests, they’re likely to do so at the GUI level. The testers who create the scripts most likely have little or no programming experience. They aren’t familiar with code design patterns such as “Don’t Repeat Yourself,” so the test code is full of duplication. The test tool may not enable good code design for the test scripts. The scripts are brittle and hard to maintain.

I’ve heard of test teams that had several hundred thousand lines of test scripts that no longer worked. I’ve met testers on other teams who spend 100 percent of their time updating tests to keep them passing in the continuous integration process. This is a bad investment that adds to technical debt and slows the development process down even more.

Some companies are wise enough to staff their test automation department with programmers. These have a better track record. However, if they don’t leverage tests to help guide development, as in test-driven development and specification by example (also known as acceptance-test-driven development), then they are missing out on significant value that tests could provide. And, if they don’t collaborate closely with the programmers who write production code, then they may have to work with code that isn’t designed for ease of test automation.

Leveraging Different Skills
Let’s contrast the “test automation project done by test teams” approach with a whole-team approach where automation is an integral part of software development.

Learning to automate tests of any type is hard. At first, it’s a lot of extra work with no reward. You only realize benefits if you do it properly and stick with it.

Over the past several years, I have asked many people whose teams successfully automate tests to share the key factors that enable their achievement. In every case, they credit a whole-team approach to overcoming impediments to automation. Programmers who write production code are, hopefully, skilled at applying good design patterns. Writing automated test code is generally easy for them.

Testers are skilled at knowing which tests to write and to automate. When a programmer and tester pair to automate a test, not only is it much faster than a tester’s struggling to automate alone, but also it enhances collaboration and communication that benefits the whole product. I personally enjoy automating tests, but I’ve had to admit that the tests I design on my own are not nearly as clear and maintainable as the ones I do in collaboration with my programmer teammates.

Database experts on the team help provide test data and procedures such as database refreshes. System administrators help integrate automated tests into continuous build processes. The diversity of skills on a software development team helps overcome any impediment to automating any given test.

User Comments

1 comment
Gunasekaran Veerapillai's picture
Gunasekaran Veerapillai

Hi Lisa

The success of test automation lies in the reusability of the automation scripts for multiple releases and the savings the company realizes in terms of time and cost. In normal circumstances the end user turns into acceptance tester or the manual testers who tests the system for ages do not have the expertise on the automation tools. They may be happy to running the scripts and getting the benefits instead of creating the test automation scripts. But over a period, they will start maintaining the scripts by making the required changes also. In my experience, the dedicated experts’ team creating the automation scripts – training the whole team to run the same has yielded more benefits than involving the whole team to prepare the automation scripts.

February 9, 2012 - 3:06am

About the author

Lisa Crispin's picture Lisa Crispin

Lisa Crispin is the co-author, with Janet Gregory, of Agile Testing: A Practical Guide for Testers and Agile Teams (Addison-Wesley, 2009), co-author with Tip House of Extreme Testing (Addison-Wesley, 2002) and a contributor to Beautiful Testing (O’Reilly, 2009) and Experiences of Test Automation by Dorothy Graham and Mark Fewster (Addison-Wesley, 2011). She has worked as a tester on agile teamssince 2000, and enjoys sharing her experiences via writing, presenting, teaching and participating in agile testing communities around the world. Lisa was named one of the 13 Women of Influence in testing by Software Test & Performance magazine in 2009. For more about Lisa’s work, visit www.lisacrispin.com.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Apr 29
Apr 29
May 04
Jun 01