The Homegrown Tools Syndrome

[article]
Summary:

Test management is a generic process, yet much effort goes into developing tools in house to do this work. Learn the reasons for this phenomenon and suggestions for avoiding it.

Test management is a rather generic process. The basic building blocks are the same: test cases, test suites, test cycles. How much different could it be from one team to another, or even among different companies? And yet, I have seen much effort spent developing tools in house to do this work.

Why does this happen? Why spend time and effort reinventing the wheel, when this wheel can be brought home from open source or commercial options?

And, it’s not just test management. I’ve witnessed this happening with other generic tools, such as test execution engines, bug tracking, and requirements management tools.

To me, the fact that the phenomenon happens across a wide range of tools is an indication that the reasons are predominantly non-technical. It’s not that test management is an art that is not yet fully understood by commercial tool developers and therefore has to be designed in house. There are other reasons at play that cause this seemingly redundant effort.

In this article, I propose a number of reasons that I think lead to internal tool development. While I refer to “test management tools” (TMTs), the logic applies to a number of other tool types.

"We Are Special"

The common wisdom when discussing tool selection is "Find a tool that supports your process." This is one of the main reasons why teams end up developing their own tools.

How's that?

Pages

User Comments

2 comments
Kobi (Jacob) Halperin's picture
Kobi (Jacob) Halperin

Good points Michael,

Another issue is politics - choosing a full ALM requires consent of other group leaders (System Eng. and Dev), often not easy to acquire - so testers switch into temporary fallback, which result in large amount of duplicate work (rewrite) in separate tools + the loss of traceabilty and E2E management visibility.

Many are unaware of the vast offerings of free ALM tools these days (such as XStudio), and the ability to to adapt these tools by sending feedback and change requests.

Situation is even worst in automation frameworks - where many turn into bad coding habits, unique programming languages, and limited logging and mainly drill-down abilities.

Here the situation is worst, since most free tools are aimed at specific arena, and when one seeks to automate both legacy desktop GUI, web based GUI, Embedded CLI and test equipment - he must select one area and find bypasses for handling the rest.

Not to mention integrating between automation frameworks, ALMs, Bug Trackers, CMs and the dev IDE.

Thanks,

@halperinko - Kobi Halperin

August 28, 2012 - 1:31am
Oren Reshef's picture
Oren Reshef

I think you jumped to conclusions too soon in saying that "the fact that the phenomenon happens across a wide range of tools is an indication that the reasons are predominantly non-technical", I evaluated more than a dozen tools and couldn't find that one that will justify the effort in buying it, modify it, or migrate hundreds of test cases from out in house system.

In my industry of highly embedded devices most (all is a strong term to use here) TMTs requires a lot of customization, so you end up with a thin layer of test management that can be easily developed in house.

This also leads to another reason you missed- company size. In a company of more than 15,000 employees letting a small team (10-20 engineers including managers) develop such a tool is cheaper than buying it, and guarantees long term support.

August 28, 2012 - 4:41am

About the author

Michael Stahl's picture Michael Stahl

Michael Stahl is a software validation architect at Intel on a team that validates the drivers of Intel's graphics hardware. In this role, he defines testing strategies and work methodologies of the test team and sometimes even gets to test something himself (which he enjoys most). Before starting his career in testing in 2000, Michael worked at Intel’s manufacturing facility in Jerusalem, Israel, as a chip-level test engineer. Michael is an executive board member of the Israeli Test Certification Board (ITCB), holds a full Advanced ISTQB Certification, and chairs ITCB’s advisory board. He has presented papers at SIGiST Israel, STARWest, EuroStar, and other international conferences and is teaching a course on testing at the Hebrew University in Jerusalem. Read more about Michael at www.testprincipia.com.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Sep 22
Oct 12
Oct 15
Nov 09