When Should a Test Be Automated?

[article]
Member Submitted
Summary:

The question is how to make a rational decision about which tests to automate. When I take a job as a contract tester, I typically design a series of tests for some product feature. For each of them, I need to decide whether that particular test should be automated. This paper describes how I think about the tradeoffs.

I want to automate as many tests as I can. I'm not comfortable running a test only once. What if a programmer then changes the code and introduces a bug? What if I don't catch that bug because I didn't rerun the test after the change? Wouldn't I feel horrible?

Well, yes, but I'm not paid to feel comfortable rather than horrible. I'm paid to be cost-effective. It took me a long time, but I finally realized that I was over-automating, that only some of the tests I created should be automated. Some of the tests I was automating not only did not find bugs when they were rerun, they had no significant prospect of doing so. Automating them was not a rational decision.

The question, then, is how to make a rational decision. When I take a job as a contract tester, I typically design a series of tests for some product feature. For each of them, I need to decide whether that particular test should be automated. This paper describes how I think about the tradeoffs.

About the author

Brian Marick's picture Brian Marick

Brian Marick has worked in testing since 1981. He concentrates on testing in Agile projects. Brian founded Testing Foundations: Consulting in Software Testing in 1992. He is the author of The Craft of Software Testing and Everyday Scripting with Ruby.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Apr 29
Apr 29
May 04
Jun 01