Don’t Guess Your Tests—Strive for Complete Requirements

[article]
Summary:

Many teams struggle with test creation due to miscommunication or a lack of requirements, testers not being present during design phases or discussions, a shortage of time, or incomplete information. But that doesn’t mean you should turn to guesswork. Your tests will suffer in quality and completeness. We must always strive to get the desired requirements.

Test creation is perhaps the most important aspect of software testing. Though most of our focus is on test execution and reports generation because that is the most visible output testing has, the backbone of the entire process lies in the creation of accurate and meaningful tests.

Many teams struggle with test creation during their agile journey, and they often seem to be doing a lot of guessing during test design or creation. This can be due to miscommunication or a lack of requirements, testers not being present during design phases or discussions, a shortage of time, incomplete information about which test environment or tools to use, or unknowns such as relevant positive and negative scenarios to test or overall purpose and scope of the application.

Hurried guesswork during test creation leads to bigger problems later in the project and poses a risk to the overall quality of the product.

Say your team has to test a webpage. Apart from some functional specifications about controls on the page, they do not have much information. But they have so many things to decide on before starting to test, such as what browsers to test on, which tool to use for automation based on the types of controls to be identified, what workflows to execute as part of integration testing of this page with the entire website, and so on.

It is also very important that those functional specifications about the webpage, its controls, and validations on each control are complete and thorough. When the team is designing and creating the test plans and test scenarios, ambiguous or missing requirements would lead them to assume a lot of parameters and guess some cases according to personal perceptions. For instance, a tester may assume Chrome should be the web browser under test and start using his installed version as the default without asking for customer preferences and all the versions that need support.

Suppose a username field on the webpage does not specify its upper limit and allowed characters validation in the specification. The tester may assume it to be a standard ten-character limit with only letters allowed and start testing accordingly, without asking relevant questions about regional and domain specific requirements. If the website has French users whose names contain special characters, or Indian users whose names frequently exceed ten characters, these inputs would fail from the tester’s assumptions and guesswork applied during testing.

Another problem teams may face is guessing test data. Are you using proper techniques to derive test data values, or just guessing random values to input? Suppose you are testing the function of addition. Are you randomly sending inputs of numbers and verifying the output? We should be using proper design techniques such as equivalence class partitions or boundary value analysis to derive a proper set of test values.

This is how poor communication or a lack of direction results in bad test design. We need to minimize this guesswork and make informed decisions before we begin the actual testing.

Getting Testers Involved in the Requirements Phases

My team was quick to realize this when we faced missing and vague requirements in one of our agile projects. We noted that due to a lot of factors mentioned above, we were subconsciously assuming things when designing tests and later on realizing our assumptions were incorrect, meaning many of our defects were marked invalid and we had to redo a lot of work. We decided to fight this problem.

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.