Further Thoughts on Boutique Testing


checks. The developers use test-driven development, the testers write scripts on demand, and we often have a high-level tool like Fitnesse to communicate requirements or have developers write Selenium code to demonstrate that the feature is “done.” The thing I am talking about is having someone external—even contract—develop automation that drives a GUI after the code is “passed” to QA.

This can work and has for some organizations with certain types of problems. The work we did at Socialtext is often held up as a case study for a good framework. A few years later, I'm less excited. So few companies have the perfect mix of problems that lends itself to this sort of automation approach. Our case study may have been an exception, not the rule, but it's open for debate.

Adam Yuret, a boutique provider for management and test services recently put it this way: "I think test automation fetish is a pretty good heuristic for organizational dysfunction—not to say automation is always bad, but I think many dysfunctional organizations misidentify their problem as being a lack of automation."

Boutique Testing Services Can Be Sold by a Larger Company

When I wrote the original article on the Boutique Tester, I thought high-end testers would contract directly with companies. But, like I wrote above, it turns out that many companies want testing as a true service—like tap water they can turn on and off, with little concern over the "who" or if that person is available right now. The easiest way to do that is to hire a service-providing company, one with hundreds or thousands of testers who can be accessed at will.

Since that article was published, I've seen two companies move into that space: uTest, which offers crowd-sourced testing (as well as specialized test) services, and PQA in Canada. It turns out that PQA has been around for a long time, but they only recently got aggressive about competing in the US. Two of my friends, John Kotzian and Elena Houser, have done independent test work for uTest, accepting projects as they come and working on them when they want to from home. John tells me he even exceeded the cash salary of his old day job in his first year with uTest, working about forty hours a week.

On a related note, another challenge is that boutique testing only offers part of the solution, not the whole thing, so it creates a little more management work for the client. One fix for this is to partner with a developer and test on the projects the developer leads. My colleague Catherine Powell did this for several years. If you want to live the boutique life but can't take the risk, it might make sense to find a partner.

Experience in the Subject Area Matters

The first article downplayed the importance of subject-matter experience. It implied that any properly trained tester could drop into any project and be extremely valuable immediately. Three years later, I'd like to clarify things a bit. Yes, there are certain sorts of "common attacks" or "quick attacks" that can be used on most software applications. They are skills many testers don't have, and with those skills you can be valuable on most things. Still, I don't tolerate someone with an MBA claiming that a good manager can manage absolutely anything, and I'm not sure we can claim something similar as testers. Those "quick attack" skills are easy to learn, easy to apply, and mechanical. There other aspects to testing, like understanding the problem domain, the combination of inputs that have the greatest risk

User Comments

1 comment
Ajay Balamurugadas's picture
Ajay Balamurugadas

Hi Matt,

Thanks for this article. I liked this article for two reasons:

Your experience speaks in most of the sentences. It was like listening to the open lecture by James.

It is a good continuation of your first article on 'The Boutique Tester'. Not only you pointed the benefits, you also highlighted where one could go wrong.

I am sure that this article will be very helpful to many testers out there.


Ajay Balamurugadas

February 14, 2012 - 2:42pm

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.