Offshore Testing: The PITS?


Offshore resources have proved useful, and companies continue to try and cash in on the cost savings. But those savings might not be what you or your company expected. In this column, Linda Hayes warns that outsourcing has some pitfalls we should always look out for.

The use of offshore resources for development, while not a panacea, has been accomplished with varying degrees of success by corporations in many industries. The promise of the same value proposition—a reduction in resource costs—has encouraged some to pursue the transfer of testing offshore as well. Interestingly, the value proposition is not as clear for testing as it is for development. In fact, some companies have reversed course and returned functional testing back onshore. The past five years or so of experience has illuminated four key factors affecting the success or failure of these efforts: process maturity, independent verification, time to value, and security.

P: Process Maturity
Just as companies found that their own development processes and project oversight had to be significantly improved before offshore development could be effective, they also discovered that their test processes lacked the maturity, necessary detail, and structure needed to be successful offshore.

High-level test scenarios, rife with phrases such as "enter valid data" and "verify correct results," presuppose the tester has domain and application expertise. When working with offshore resources that have neither expertise, the test documentation must be reworked to provide explicit step detail and data values in order to be usable. And while this investment might be worthwhile if the tests are reused often, a high rate of change in the application under test may negate the value.

I: Independent Verification
For mission-critical software with a high component of specialized domain knowledge, it is essential to independently verify the functionality. And offshore development already poses a risk of misinterpretation of requirements due to the lack of domain knowledge and experience.

Some have sought to mitigate a portion of this risk by engaging separate organizations for development and testing, but this approach drives up time and costs due to extra overhead for contract and project management. Internal experts—the very resources whose time is supposed to be saved—may end up with even more responsibility. Thus, acceptance testing often ends up being performed internally as a check against external development.

T: Time to Value
Offshore resources are never a quick fix, especially for testing. A 2005 report from AMR Research found that it took between fourteen months and three years before the offshore testers had sufficient familiarity to be effective in finding the root cause of problems. Others have found that the lack of domain knowledge resulted in spurious defects being reported that actually increased development overhead due to review and response times. In fact, one organization identified that as many as 33 percent of reported issues were traceable to tester error.

So companies that choose offshore resources as a way to remedy a resource deficiency, or to handle an influx of projects while keeping costs flat, might find themselves with higher costs and resource demands for one or more years. Without careful planning and realistic expectations, this confluence may end up eroding quality and extending schedules.

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.