Fool Me Once: Follow-Up

[article]
Summary:

In her recent StickyMinds.com column, "Fool Me Once," Linda Hayes took a look at the difficulties encountered by those who dove into automated testing early on, only to be bitten by the record/replay shark. In this follow-up column, she offers some solutions to those who still struggle with making test automation successful.

In my previous column, I complained that most testing remains manual despite billions invested in test automation tools, largely because of the unrealistic expectations and unfortunate reality arising from the record/replay approach. Many readers subsequently took me to task for pointing out what went wrong without offering any solutions. Fair enough.

In my experience, there are four key elements to making test automation successful:

There are some possible benefits to allowing both to coexist, too. Consider these:

  1. Educating development to produce testable applications
  2. Understanding why yet more scripting won't work
  3. Learning to transition from a test tool to an automation application
  4. Setting realistic expectations

Each of these is summarized below with links to more detailed discussions.

Educating Development
Automation changes the relationship between development and test forever. Manual testers only worry about what they can see, but automation is all about what's under the covers. In order to build a reliable, maintainable, automated test, the tester must be able to identify application objects using identifiers or names that are understandable, persistent, and unique. Further, these objects must be accessible. Understandable names make the test readable and therefore maintainable, persistent names make the tests stable and repeatable, and unique names make it reliable. Accessibility means your tool can interact with the object.

Unfortunately modern applications often comprise code that is generated on the fly by servers, using gibberish identifiers that change from moment to moment. Also, flashy interfaces often are built with opaque objects that don't reveal their methods or properties. If this is happening to you, there is nothing you, as a tester, can do about it. Only development can fix this problem, but they first have to be educated. For more on this subject, see "Automated Testability Tips."

Understanding Scripting
As I pointed out in "Fool Me Once," record/replay was seductive because it looked easy, but it actually was deceptive because the recorded scripts weren't reliable or maintainable. The typical response was to add yet more code to handle timing issues, externalize data, and insert logic to handle unexpected responses. While these techniques may have addressed the immediate issues, they created even more. Programming skills were required, thus excluding the application experts who typically performed the manual tests. The resulting tests contained more and more code, requiring more time to develop and even more to maintain. In the end, the tests were too complex, costly, and time-consuming.

A detailed discussion of how this problem played itself out can be found in the Better Software magazine article "The Demise of Record/Script/Play."

About the author

Linda Hayes's picture Linda Hayes

Linda G. Hayes is a founder of Worksoft, Inc., developer of next-generation test automation solutions. Linda is a frequent industry speaker and award-winning author on software quality. She has been named as one of Fortune magazine's People to Watch and one of the Top 40 Under 40 by Dallas Business Journal. She is a regular columnist and contributor to StickyMinds.com and Better Software magazine, as well as a columnist for Computerworld and Datamation, author of the Automated Testing Handbook and co-editor Dare To Be Excellent with Alka Jarvis on best practices in the software industry. You can contact Linda at lhayes@worksoft.com.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Aug 25
Aug 26
Sep 22
Oct 12