5 Ways to Overcome Your Hatred of Test Documentation

[article]
Summary:
Writing test documents is a good practice to have: It enforces an orderly thought process, explains what you’re planning, and improves the test strategy. But knowing it's useful doesn't make it any more fun. Michael Stahl knows this, so he has five tips to help make the idea of test documentation a little easier—or at least a little more difficult to hate.

As far as I know, there isn’t yet a support group for testers. But if there were, I believe many of us would confess to having a severe aversion to writing test documents.

Part of my job definition is to be the annoying person who pushes people to write documents. I defined the document templates, and I review most of the test documents my team produces. I can tell you that it’s just about as you imagine: unexciting and much less fun that running tests and finding an interesting bug.

Worse, because I am the one preaching about test documents, I have to be a good example and do what I ask others to do. I can’t skip writing test documents for any of the projects I am responsible for, even though I really don’t enjoy it. In fact, when writing test documents, I have to get up every now and them and take a walk in the hallway just to wake up.

Even though it’s not the most interesting task, writing test documents is a good practice to have. It enforces an orderly thought process about the test challenge ahead, explains what we’re planning and why we do things this way and not another way, and improves the action plan, the test strategy, effort estimation, and task partition.

Documented plans also let stakeholders know what they can expect to get from the validation team. It allows feedback if we made mistakes in our plan and reduces the number of surprises caused by misunderstanding or mismatched expectations.

None of this should be news to you, but it still doesn’t make the process any more fun. However, I may be able to make the idea of test documentation a little easier, or at least a little more palatable. You’re going to have to do it anyway, so why not try to overcome your hatred of test documentation and try to get in a better mental state about it? Here are five tips.

1. Write the test document for yourself

Get used to the fact that very few, if any, people will read the test document. This is OK. The main beneficiary of a written test document is the author. Writing it helps you order your thoughts and organize the test plan into clear steps you (and anyone else) can understand.

Besides, if you don’t expect anyone to read the document, you won’t be disappointed when that’s just what happens. In their book Exploring Requirements: Quality Before Design, Donald C. Gauss and Gerald Weinberg relate to this disappointment. They write, “The document is nothing; the documenting is everything.”

Writing the document is the goal. If it is read by others, that’s a bonus.

2. Use your human thought process

In today’s Twitter world, we feel that any idea can be conveyed in one or two sentences—and if we need more, someone must have already found a trick or developed an app that will do the job for us. We believe that if we only think hard enough, we will find a technological solution that will write documentation for us, like Doxygen did for code documentation.

It’s frustrating to spend a lot of time doing something manually when all the time you feel it could be automated and be done more efficiently. But writing a document is a thought activity, and luckily no one yet has found out how to automate that. So stop torturing yourself about wasting time and revel in the fact that this is a human job that isn’t replaceable.

User Comments

7 comments
Dave Maschek's picture

I write detailed test cases for two reasons. One, because they are easier to automate if they are detailed. Two, because the automation team probably won't get to my test case anytime soon. So, two months from now when I have to re-run the test case, I will have all the detailed steps in front of me. If the author is referring to a test plan, I agree that it is helpful to write something out in order to clarify the problem. But I would keep the test plan as brief as possible and include it in the introduction to the test case.

March 21, 2017 - 2:01pm
Michael Stahl's picture

One of the reasons I give people why they need to invest in good documentation is that they will be the first to enjoy it when they come back to their code or test cases after a few weeks. I see we agree on this.

We probably have somewhat different opinions on how detailed a test plan needs to be and if it should be a stand alone document or part of the test cases; but as long as you do documentation that fulfills your needs, I salute you.

 

Michael

March 22, 2017 - 12:14am
Matt Griscom's picture

There IS a way to automate documentation for the lowest, most detail-oriented and product-facing aspects of a check: See MetaAutomation, the pattern language and code samples available on the site metaautomation.net.

Unlike BDD/Gherkin, this coding technique generates hierarchical steps of a check at runtime, and even uses those steps as a reference for the next run. The artifact of a check is pure data so very powerful and flexible for self-documentation, analysis, notifications etc.

 

March 21, 2017 - 2:09pm
Richard Nixon's picture

Isn't it better to write the testing document as actual code.

Tools like Cucumber are great for this...
* write tests in business language with business product owners.
* write the technical tests with software developers or do it yourself

If you do then it will become a living document that gets used regularly at every release.

March 24, 2017 - 7:10am
Michael Stahl's picture

Richard -

What you propose is a good way to document the test cases. But does this give you an overall picture of your plans?

There is stuff that is better considered ahead of writing the test cases:

- Strategy (e.g. what to test, what not to test; prioritization)

- Partition of the work among different groups

- Thinking what tools that will be used; what tools will need to be aquired or developed

- Defining processes (how you get builds from dev; how you report bugs; etc.)

... and many more details.

 

This is the stuff I recommend to think about ahead of time, before starting to write (well documented) test cases.

 

Thanks for the comment!

 

Michael

March 24, 2017 - 10:43am
Tim Thompson's picture

We write detailed tests for the initial develpment. For any UI related tests we created a test case library that allows for cloning test cases and requiring only minimal edits afterward. That has the benefit of speeding up test case creation, keeping tests consistent, and not forgetting something. We keep a reference to the test case library entry and if anything changes across the entire app we can quickly find all those tests that each need the same change. I second the point made in the article, writing test documentation and crafting test data is probably the best way to accomplish systematic testing with large coverage. It does not exclude exploratory testing, both approaches have value.

We also write a separate set of tests for regression that focus mainly on function. Those tests are not only targeted for automation, we also had great success with crowdsourcing testing by bringing in support reps and trainers to help out when we have a lot to test and little time to do it.

Detailed and regression tests in one place and in a common format allow for sharing work. At times we don't get the work load assignment quite right. Having clear and easy to follow tests allows for passing on some test work to others, either within QA or within the department. A development group berefts itself from a lot of agility without proper, well-maintained, and complete tests.

March 27, 2017 - 8:04am
Michael Stahl's picture

Tim -

sounds like a good and flexible system!

 

A general note; The responses so far are focused on the test cases: how well they are written and documented.

What if someone (developers; project manager) want to know what is your test strategy? or what is it they can expect to get from the test team? If all you can give them is a set of well documented tests, they need to "build the forest from the trees" - that is, understand your strategy by looing at (sometimes hundreds of) tests.

A test PLAN - NOT a test case list - is the document that will provide this information.

 

If in your world "test plan" is synonym to "test case list" - look up the definitions (for example in the ISTQB glossary: http://www.istqb.org/downloads/category/20-istqb-glossary.html) to see the difference.

 

Michael

March 28, 2017 - 10:02am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.