The Drawbacks of Developing Your Own Test Tools


This article describes the disadvantages of creating and designing your own testing tools as opposed to purchasing the test tools from a commercial vendor. Creating in-house test tools present a barrage of issues and problems that are often overlooked and consequently stifle the activities of the testing team. The article analyzes some of the most salient drawbacks of creating test tools.

Some companies need to create their own test tools due to certain project requirements situations. These situations occur when existing commercial vendors of test tools do not offer products that allow record and playback for a given IT environment. However, under most situations it's not practical or pragmatic to create, design, and develop test tools given the vast number of choices for commercial test tools. Companies that embark on a mission to create their own test tools for either test management, or record/playback will need to answer some key questions:

How long will it take to create the new test tool?

Who will manage the effort to create a test tool?

Where will the requirements come from to create the new test tool?

How will the creation and development of a new test tool detract attention from existing QA activities?

How will the new test tool be tested to ensure its dependability?

Who will create the training materials for this new test tool?

Is it more expensive to create from scratch a new test tool rather than purchasing a commercial off the shelf test tool?

In addition to reviewing the presented questions above, a company that is preparing to create its own test tools should also be aware of the other issues associated with developing test tools:

Maintenance Costs and Service Support
Vendors of commercial test tools, for a fee, will offer companies various types of support levels, new software upgrades, patches, installation support, consulting services, etc. However, with the creation of in-house test tools a company would now have to create a plan and framework for supporting the test tools for bugs, defects, upgrades, or patches. This option might be more expensive and less reliable than simply paying maintenance fee to a vendor. Furthermore, the quality of service for maintenance support from a commercial vendor is more extensive and comprehensive than a company's in-house test tool support. This is because commercial vendors have more service support engineers with access to the vendor's R&D department. Additionally, vendors of commercial test tools also have a repository of known issues or limitations with their test tools whereas companies creating their own test tools may not have such repository.

Missing Online Help
In-house created test tools often ignore features such as context sensitive help, online help search, and online help index. The creation of online help features for in-house test tools offer testers a smoother transition and learning curve to the test tools. The downside of developing online features for in-house test tools is that they are time consuming and resource intensive to create.

Test Tool Knowledge Limited to In-house Testers
A company should beware when creating their own test tools. When a test tool has been created from scratch, only the developers and testers of that tool will have the necessary knowledge to repair, patch, or upgrade it. This situation can deteriorate into a problem if the company has a high tester turnover and has to hire new testers.

Not Proven
In-house created test tools are not proven and have been widely used anywhere. While on the other hand, vendors of commercial test tools have already deployed and implemented their tools. Companies that are ready to begin or have initiated critical testing activities might not be well served by test tools that are unproven and have never been deployed to other projects. Another issue to consider is that many commercial vendors of test tools offer tools that interact with one another. In-house created test tools might not offer integration of different test tools or offer test tools whose integration have been effectively

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.