Many companies that elect to buy a packaged application rather than build a custom system do not maintain a traditional QA department. Acceptance testing of their own configuration is performed by users conscripted for the duration of the project who return to their day jobs as soon as possible.
The downside to this approach is that there is little reusability between projects. Borrowed resources don’t have the incentive, time, or skills to implement automation or invest in processes and documentation that will survive the project. The result is a laborious, expensive, manual testing process of uncertain coverage that becomes the long pole in any project.
This is a problem for both the customer and vendor. The resource and schedule drain discourages the customer from implementing upgrades or change packs as often as they become available. As a result, the customers often demand targeted patches for issues that may have already been resolved in a bundled package or they delay the purchase of new products.
This is a headache for the vendor, who can’t sell new products and ends up supporting multiple versions and patches in the field and can’t possibly test every combination. The chaos that inevitably ensues degrades quality and stability for the customer and diverts the vendor's development and support resources to constant fire drills. The irony is that less stability both increases the need for changes and decreases the appetite for them.
Converting Costs into Cash
Some software vendors have seen the customer’s testing challenges as an opportunity. After investing in test automation for their internal needs, they realize it can be packaged and implemented for their customers as a new product and service offering. This is a win-win-win for the customer, the vendor, and the vendor QA team.
For the customer, test automation reduces the demand for resources, compresses the implementation schedule, and increases the stability of the deployment. endor revenue increases from new products and services while support costs decrease due to greater stability. QA team members can monetize their efforts and also develop a closer relationship with their customers—sharing test cases and results in a common format and platform. This extends their understanding of customer needs and therefore their test coverage, which improves quality overall.
There is a lot to like in this model, and as long as automation has been around you have to wonder why this offering isn't already standard for most vendors. The answer can best be found by reviewing the following case study, which exposes the issues but also suggests solutions to making this a reality.
Case Study: Testing as a Product
A financial services vendor’s QA manager perceived the potential for marketing his automated test library to customers and sought to make it a reality. He presented the concept to the implementation services team, who loved it but noted that there was no documentation and training for the library. Because the automation assets were developed for internal consumption, they were not documented for external use, and neither was training formalized.
Producing documentation and training materials required additional investment the QA manager could not afford without a bigger budget, so he suggested that the services team should pick an "alpha" customer who would be forgiving of the rough edges in exchange for favorable terms. If the project was a success, then the QA manager would have a business case for the extra investment. In exchange, he also agreed to have one of his QA resources work with the services team to provide on-the-job training.