Exploratory tests, unlike scripted tests, are not defined in advance or carried out precisely according to a plan. So where and how do they fit with the other tasks testers must perform? James Bach, a chief proponent of exploratory testing, provides some insight on how best to exercise exploration in your testing effort.
If you, like me, find the exploratory approach to testing valuable (see my recent column "What Is Exploratory Testing?"), then the questions arise: When do you do it? How does it relate to the software lifecycle? StickyMinds Review Team member Yogita Sahoo sent me an insightful list of questions and comments about exploratory testing (ET). I'll be responding to some of them in future columns. For instance, she writes, "In my personal experience, if I start exploring things when I'm supposed to carry on with my allotted tests, the whole thing gets jumbled up. I can neither concentrate fully on ET nor on my regular test cases. That results in less productivity." Let me address that.
A simple way to think of ET is concurrent test design and test execution. To help Yogita better, I want to use a more specific definition: Any testing is exploratory to the extent that the tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests.
This definition includes pure exploratory testing, where you explore the product and design a test strategy and specific tests based on your understanding of your mission as a tester, but without any specific guidance. It includes chartered exploratory testing, where you have a specific assignment for what to test and what techniques to use, but no designated procedures. It also includes improvisational testing, where you elaborate on a test procedure or use it to inspire a set of related tests. It includes test procedure creation, too, inasmuch as you perform tests while documenting them.
It's more about how to do ET well, than when to do it.
All testing done by humans is exploratory to some degree, because humans are not robots. That means the question is not when do you do exploratory testing, but rather how exploratory is the testing that you do, and how well do you do it. When I consult about test process, I don't suggest that my clients perform exploratory testing. Instead, I help them become aware of all the ET that they already do, which is probably mixed up with some form of scripted testing (or pseudo-scripted testing, where testers say they follow the procedures, but don't). Once I can get their exploratory testing to "come out of the closet," I can help them improve their skills and overall test strategy so that they do ET, and any other testing, better.
Exploratory testing is all you have at the beginning…
ET fits at the beginning of the test project because test procedures don't yet exist for the new technology being developed. Even if they do exist, you have to learn the product (that requires exploring and questioning it), and the procedures would have to be reviewed and upgraded. The process of writing test procedures is exploratory. Watch anyone, or yourself, writing a test script, and you'll see those thought processes at work.
…and it's how you create diversity in tests later on.
ET fits into the middle of a test project, even when you have lots of scripted tests. Be exploratory in the sense that a tourist on a tour bus is exploratory. Let your allotted tests take you to visit different parts of the product, then improvise on the theme of those tests, briefly. Spend a few minutes working through variations of the tests, then get back on the tour bus and do the next scripted test.
It's also how you assure that there is enough variation and creativity in the test cycle.
Also, throughout the project,