With more than thirty years of experience in the testing world, Dorothy Graham knows a thing or two about test automation. We asked Dorothy about how to get the most out of automation and how to avoid "intelligent mistakes."
Noel Wirst: You're giving a session at the 2012 STARCANADA conference that describes some of the "intelligent" mistakes in test automation. What makes these intelligent and not careless?
Dorothy Graham: A mistake is an action resulting from defective judgment, carelessness or a misunderstanding, so some mistakes are due to carelessness, but not the ones I will be talking about.
"Intelligent" means exercising good judgment, but if good judgment is based on a faulty premise or misconception, then a mistake is still made. If the misconception had been true, then the action would be sensible, and there is an element of truth in all of the intelligent mistakes I will be talking about - this is why they are easy to make and seem like a good idea at the time!
NW: When you say that “making testers become test automators may be damaging to both your testing and automation. How is this so?
DG: What I think is damaging is the assumption that it has to be the tester who also becomes a test automator, a person who works directly with the tool. Because the tools use scripting languages, which are programming languages, any tester who wants to use the tool directly must therefore become a developer.
There seems to be a bit of controversy developing around this topic, as some people are saying that all testers should learn to write code. I don't agree!
Of course there are some testers who will be very happy to become developers (script developers working with the tool's scripting language) - I have no objection to that, and it is very useful particularly in an agile team.
However, by implying that the only good tester is one who can code, we are denigrating our own discipline, and definitely "throwing the baby out with the bath water." Not all testers want to become programmers, and not all testers would be very good at it. If you are a tester who has come from a business background and is very happy dealing with tests and very effective at finding defects with your testing, why should you have to stop doing what you love to do something you don't like and won't enjoy?
Forcing all testers to become developers is damaging to those testers and therefore to testing, and it doesn't help the automation either, to have de-moralized people doing what they don't like not very well.
All testers should be able to write and run automated tests, whether or not they are developers - this is what a good automation framework will provide (and the framework needs developer skills for support).
And testing tools are not just for testers - developers can gain a lot of productivity by using them for their own testing too!
NW: You've mentioned that "many organizations never achieve the significant benefits that are promised from automated test execution?" Who promises these benefits, and why do so many testers lack the management needed to achieve them?
DG: I'm afraid that it is tool vendors who often get carried away and promise things that cannot be achieved, or omit to mention the effort needed to achieve good benefits. I was at a conference recently where one of the vendors (who will obviously remain nameless!) was promising that using their testing tool could achieve zero defects in the application being tested!
I was so annoyed I actually went and had a frank discussion with the representative who was there, and I believe they have now modified their web site, as I don't see this claim there any more.