Column info : The Six Million Dollar Automator

Comment:
by Jeff Dreyer 8/10/2007

I have been making this same argument for years (since 1997) when I first brought automation tools into my company. People latched on to automation as a resume builder and fell into the vendors' trap of "everyone can automate everything". I stood out like a sore thumb when I took the stance that automation is development skill and how it is implemented depends on a lot of factors and those factors can only be properly analyzed by a person with developer skills and a testing focus.

People tend to forget or just don't know (if they don't have a development background) that test automation yields an application in which its primary...Read On

Author's Response: 8/29/2007
Thanks for the feedback, Jeff!

Comment:
by Srinivasan Desikan 7/31/2006

Good article. I request the author to explain the concepts little more to make it clear.

One point I would like to add is that ABOK should be available with all engineers irrespective of whether that person is developer, test execution enginner or automation script writer. ABOK is a model/process around automation and automation is the job of everyone in the organization not just few people. Many organizations are going for a model where automation team just trains other teams and provide...Read On

Author's Response: 7/31/2006
Thanks for the comments, Srinivasan. I totally agree with you. The purpose of relaying the skills identified in this article is so that anyone will be able to effectively improve their test automation prowess. So whether you are a developer or software quality engineer, you will be able to better contribute to the overall goal of test automation. By the way, be on the lookout for additional white papers, presentations, training classes, and other items that delve deeper into each one of the skill categories presented in this article.

Thanks, again.

Comment:
by Nitianand Ramjattun 7/27/2006

This article speaks a lot about a subject which is poorly explored so far. It is possibly the only article that i have come across that answers most of the questions i used to have(both employee/employer perspective) some time back before I started my role in test automation. For completeness, I would probably add some general test automation concepts into this BOK. Well done!

Author's Response: 7/27/2006
Thanks for the feedback, Nitianand!

Comment:
by Richard Whitehead 7/26/2006

Following on from Howard's comment: the most successful automation effort I have seen had a distinct separation of roles between the people designing the test suite (definitely testers) and the people writing the actual scripts for the tools (definitely programmers). Not that that is a panacea, but it can help.

So how do we convince potential employers that lead testers don't need to be expert script-writers, and that they do need to be aware of all these other aspects you have identified (all of which I agree with, by the way)?

Author's Response: 7/27/2006
Excellent point, Richard. I think the most successful automation effort that I was involved in also had a strong separation of roles. It wasnt necessarily a distinction between "testers" and "developers", though. It was more along the lines of automation lead, manual testers, framework architects, and application automators. The automation lead had to have strong process knowledge, including concepts related to ROI and other items that set the direction of test automation. The manual testers were responsible for designing and implementing manual test procedures. The framework architects had to have strong coding skills, along with some other skills that Ive identified in the article, because they design, developed and maintained the framework that is used by the application automators. The application automators were somewhere in between. They were responsible for the actual application automation, and their development skills didnt have to be as strong, because the framework provided a buffer that simplified many of the automation activities. Strong test processes, documentation, management support and investment is required for such separation of activities, and such separation of "powers" should be scaled according to the scope of the testing effort.

Comment:
by Howard Clark 7/25/2006

I am truly amazed at all the verbiage that wraps around a seemingly simple concept. Take your pick of a software development methodolgy, and staff your automation needs with software developers, and you end up with the very skillsets the article speaks too. This line of demarcation between testers and developers needs to disappear. If for whatever reason it is impossible to acquire a development resource that is trained in the dark arts of testing, put a development resource under a tester's management. As a former developer by both experience and degree I find these issues I face on the testing side of the house to be trivial and almost...Read On

Author's Response: 7/25/2006
Howard,

I can understand where you're coming from to a degree. I do have some fundamental disagreements with some suggestions that you make. I believe that most will agree that there are clearly differences between the set of skills required for testers from those required for developers, so I’m not going to belabor that point. When talking about ‘test automators’, however, I could see how you suggest that there are some similarities between the skills required for automators and those required for developers. This is evident when you begin to look at the skills related to programming concepts, exception handling, and debugging. Brett Pettichord, in one of his papers entitled “Seven Steps to Test Automation Success” suggests that we need to implement our automation projects more like we implement our development projects. This is true, and makes sense, given the fact that test automators are in effect developing code. This does not, however, mean that test automators and developers are one in the same or that they have the same skill sets. I’ve worked on projects where developers have been responsible for automated test development. The result is very often an overly-extensive, scarcely-documented library of complex code with cryptic reporting, all of which makes the tests way to complicated for efficient maintenance. In addition, there is often something left to be desired regarding how the script addressed the test objective, and its test coverage tracking ability. This is NOT to suggest that there is never a need for complex code, NOR does it suggest that developers can’t make good test automators. What it does suggest is that if you try to approach test automation with only a developer mentality, you are going to miss the boat.

Comment:
by Jayatheerthan Sundararajan 7/25/2006

This is an excellent article on the various skills that an automation test team should have.

Return on Investment Analysis is another important skill, an automation lead should possess, which facilitates stakeholders on decision making process. In today's world the factors favouring test automation and benefits were not clearly articulated/demonstrated. Hence the test automation were never considered in the project budget. The factors vary differently from project to project and a careful...Read On

Author's Response: 7/26/2006
Jay,

Thanks for expanding on the importance of ROI computations. This is a key part to building a successful, sustainable automated testing effort. Glad you liked the article.

Comment:
by Jim Little 7/24/2006

As a recovering Automator I cannot agree more with Mr. Johnson. In my early years wowing colleagues with automated scripts that made the screen flash and move without human interaction seemed enough. As the years went on I found myself falling into the trap of "Auto gratia Automatis" or automation for automation sake. Trying to automate everything using only the tool of choice became a recipe for failure.

Automation, like QA as a whole, is a balancing act. It has factors above and beyond...Read On

Author's Response: 7/24/2006
Well put, Jim. Thanks for the comments.