How can you get your test automation project off on the right foot?
Commitment is Essential
The team needs to see test automation as necessary and critical. The whole team must be committed to test automation.
I saw the benefit of this kind of commitment on one project where the test plans and project plans assumed from day one that a means for automatically executing tests would be available. When the first person with responsibility for test automation admitted defeat, it was given to another. And then to another when the second person couldn't get it to work. This third team member was able to get additional assistance and finally got the automation to work. Because each participant knew the team was counting on the automation, the project stayed alive with increasing urgency and effort.
More common in my experience are teams that see test automation as something nice, as something to put some effort into, but not to bet on. They make sure they have fallback plans for manual testing in case the automation doesn't work out. Invariably these automation projects don't get the attention and assistance they deserve.
Nevertheless, I have seen some smaller successes arise from such circumstances. In each case, it was because of efforts by outstanding individuals who demonstrated personal commitment, technical accomplishment, and loyalty to the team. The loyalty of these individuals meant that they stayed on the team for years, always prepared to step in and help with the test automation. It was their baby and they weren't going to let it be abandoned.
Ultimately, commitment is key, whether from the team or from individuals. Without it, automation efforts--however laudable--invariably are abandoned.
Work Together as a Team
A second key I've observed that contributes to test automation success is effective partnership between testers and developers.
From my experience, testers who head test automation efforts without significant involvement from their developers are prone to make several errors. They fail to appreciate how their test automation project needs to be planned just like other software development projects. They underestimate their need to understand the test automation technology they're using. And they aren't in a position to simplify the test automation by adding testability hooks into the software under test.
On the other hand, developers heading up test automation efforts don't fall into these traps, but have been prone to providing automation that doesn't meet the real needs of testing. I've seen them generalize from experiences testing their own code, not appreciating the importance of finding errors that they didn't make. And I've seen them develop whiz-bang technology that looks cool, but doesn't really help get the testing done, because they don't understand the real testing problems.
The best efforts avoid these various traps by having testers and developers jointly charter the test automation effort. This is not just a matter of skill. You actually need to have members of the development team involved. Only they are aware of ways that they can make the software easier to test and are in a position to make it happen. And key tester involvement ensures that the results address bona fide testing needs. This kind of teamwork is very effective.
Less effective, but still capable of success, is having developers independent from the product development team assist a testing team with test automation. Some effort must be made to work with the product development team on testability issues, but in these cases full participation from the development team was not required for success.
Get an Early Start
Getting an early start with test automation has contributed to the success of several teams I've worked with. In other words, I've observed that test automation efforts have more challenges to overcome as a software product becomes more mature.
For instance, invariably some changes are required to the software under test in order to allow the automated tests to be effective. Early on, it is easier to champion these changes. I remember one mature software product that required some changes to allow the automated tests to avoid a nasty problem. But the original developer of the code wasn't around any more and no one wanted to risk changing the code. So the testers had to live with a test suite that occasionally raised false alarms.
Another challenge faced by test automation for mature software products is that automated test procedures are never quite the same as manual test procedures. They can be similar, but some things will need to be different because you no longer have an intelligent, perceptive human being watching every test and able to make reasonable assumptions about the source of errors. Every testing strategy has strengths and weaknesses. Changing strategy--which is what automation requires--may make testers and managers who are familiar with the old approach nervous. My experience has been that getting a testing staff and management comfortable with automated procedures, when they are more experienced with manual processes, is often the most difficult challenge to late-starting testing automation projects.
Often test automation efforts are started early in the life of a software product, only to be shelved in the rush to get out a release. In all the cases I've observed these efforts become only more difficult with time. It's like learning to ride a bike: if you don't get started early enough, you may never learn.
A lot of people considering how to automate testing focus on the technology—the languages or test tools. In my experience, this has never been the key to success. I've seen the very same technologies contributing to success on one team but leading others nowhere. The key factors are commitment, teamwork and a sense of urgency. If you have these, the technology takes care of itself.
Further Reading
Seven Steps To Test Automation Success, Bret Pettichord,
http://www.io.com/~wazmo/papers/seven_steps.html
