Who's Testing Your Software?

[article]
Summary:

Due to shrinking budgets, organizations have scaled back testing beyond the point of acceptable risk. Some companies have eliminated QA resources altogether, pushing testing responsibilities back onto programming staff, which is itself spread too thinly to get it done. Unfortunately, this means that software (and even some hardware) is being released in an untested state. It's important to ask who is doing the testing on your project to ensure the testing is being done at all.

Software customers and investors beware. You need to look harder at the process of software development. Organizations, in a move to save cash, have scaled back testing beyond the point of acceptable risk. Some companies have eliminated QA resources altogether, pushing testing responsibilities back onto programming staff, which is itself spread too thinly to get it done. As a result, software (and even some hardware) is being released in an untested state.

Times like these may make such cost-cutting look prudent. But what looks frugal on paper now, can in a competitive real world, run to high hidden costs and lost revenues. What's advantageous in the short-term for management may be disadvantageous for customers and investors. Unsuspecting customers are stuck trying to stabilize fragile product, while the high costs for fixing problems in the field are being shifted to investors as a hidden drain on future profits. If you're depending on a software development company either as a customer or an investor, the question of who's doing the testing is vital. Your revenue depends on the answer.

Software is an invisible machine. You can't peel back the screen and look under the hood to see it run. You can't hear the engine labor over creaky algorithms, or see it falter when the modules don't mesh, or smell it when your data crashes and burns. Testing is the only way to find out what you're buying-or selling.

Software is still hand-crafted to solve unique problems and fit different business needs. You can't budget for development without testing, because without testing you can't know if what you've developed even works. And you can't assume that testing is free. It's going to fall into someone's budget-and both parties to a development contract need to know who's responsible for testing what.

A customer these days ought to ask who's testing their software. If the developers are testing it themselves, this is a higher risk. In the first place, testing isn't their job: a career in development doesn't make them experts at this very distinct (and in some ways opposite) skill-set. In the second place, it's hard to be critical of your own work. It's hard to see defects in your neatly woven code: and when your customers tell you the emperor doesn't look dressed, your first impulse is to try to convince them they need Vitamin A.

So, a company that claims to be delivering tested software should show that it has dedicated resources to do the testing. Customers and investors should expect to see an in-house testing organization that's not pruned too small in proportion to the developers. If a company has, say, one test engineer and a dozen developers, you are looking at high-risk, superficial coverage.

If you find your software project testing is outsourced, that's not necessarily bad in itself: any self-contained project can be outsourced successfully, with effective planning and good management. But a software development organization that outsources testing should have a written system test plan, and be able to produce it, so you can make your own judgment about the management. The outsourcing firm should have more detailed technical test plans, and if the work is custom, you might want to have your own technical staff look them over. You should be confident that the development organization is exercising appropriate control over its test contractor, as you are exercising appropriate control over them. Customers need to look at the whole situation, and decide whether a distant level of control lies within your comfort zone of risk. And remember that communication between development and test can become thorny when

About the author

Sheryl Smith's picture Sheryl Smith

Sheryl Smith is Chief Testing Geek at TestDesigns, Inc. She specializes in software test architecture and setup/improvement of QA groups. She shows management how to recognize when QA is functioning effectively, and how it needs to relate to other groups in the organization. Email: sheryllsmith@earthlink.net.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Aug 25
Aug 26
Sep 22
Oct 12