Software process experts have long expounded the benefits of test automation in software development, yet few implement and those often fail to gain the expected benefit. With testing vital to the quality and success of a software project, TopCoder's Blake Tolbert shares tips and techniques on implementing a process that can drive large returns from relatively small upfront investment-creating an environment where developers learn from each other and strive to innovate their processes.
While working primarily with Fortune 1000 enterprises, a glaring majority of companies I have assisted do not have a testing strategy at all, and the rest achieve little of the benefit expected.
With the rise of outsourced, community built, and distributed developed software, testing is even more vital to the quality and overall success of a software project. It's clear that adoption of effective test automation at the enterprise is necessary, but implementation and adoption is difficult. In the following discussion, I'll explore and address the barriers to entry and how to overcome them, by sharing hands-on experience and advice from establishing and working with a wide variety of successful testing environments. By looking at the common strategies that led to success, we identify four core strategic areas to address for test automation.
By focusing in upon the Tools, Process, Culture, and Measurement, each of which must be aggressively implemented to overcome the test automation barrier to entry, you'll also see how adoption of each helps implementation of the others, in a complementary, intertwined fashion. Before going into further detail, lets define the areas I refer to in order to avoid any confusion:
- Tools–Writing unit tests has never been easier if given the correct tools. Test automation frameworks exist for virtually every development environment and continuous integration applications can analyze tests without any human intervention.
- Process–Testing must be ingrained in the overall development process. This might sound harsh but developers will not write tests if not required to do so as part of a process.
- Culture–Test writing is viewed negatively by most developers. It is easy to view tests as utilizing extra time that could be spent writing real code. Common ways to improve the test automation culture are discussed in this article.
- Measurement–The other strategic areas of test automation depend on or assist in the gathering of concrete measurements. The most common measurements are critical to overall success, as we'll discuss below.
If an organization is lacking in any of these four areas, test automation is likely to end up as an afterthought, as we continue to see time and time again. Now that we're on the same page, we can explore an overall test automation implementation strategy.
Culture is Critical
Organizations must create an environment that promotes testing. Developers must be willing to write tests and other stakeholders should understand the benefits of automated testing. Simple steps to enable this include:
- Publish test metrics –Valuable test metrics should be easy to gather given the correct tools and process. An automated weekly email with code coverage statistics, lines of test code, and other project specific metrics can be a source of pride for a development team and keep product sponsors up-to-date.
- Set code coverage goals –Setting clear code coverage goals provides guidance for developers. Saying “all committed code must have 80% code coverage unless otherwise approved” is much more effective than “write tests.” The testing tools and source code repositories available for most platforms can automate this workflow.
- Review test strategy –Monthly or quarterly test automation strategy meetings provide a forum for feedback and process improvement.
- Confidence–The above steps create confidence throughout the organization which further promotes the culture. Once this cycle is started the results can be fantastic.
Testing Automation Starts with Process
Development organizations often provide testing tools to developers without a mandate to write tests, and then wonder why those developers are not creating extra work for themselves. Tools seldom make a difference without establishing the right process and approach in which to use them. Before implementing your organization’s testing process,