Dion Johnson use the martial arts metaphor four common issues with automated tests and how test automation specialiasts can "train" their scripts to identify, capture, and handle these problems. In this week's column, Dion talks about how to make develop test automation scripts with the balance they need so that you need fewer scripts to cover more test cases.
In his second installment of the four-part Taekwondo-mation series, Dion Johnson teaches the art of balance in three straightforward steps. Balance requires equilibrium, and in test automation that translates to increasing the coverage of a single script without increasing the amount of code in the automated test. In this week's column, Dion explains how to increase the coverage by randomizing the data being tested.
Application behavior is dynamic, so it's time that we train our automated scripts to be more dynamic via the techniques provided in the automation martial arts known simply as "Taekwondo-mation." It comprises relatively simple techniques that can be employed without the implementation of an overly complex framework.
Taekwondo-mation, like many of the martial arts, places a focus on four key principles:
1. Flexibility-Dynamically inputting data
2. Balance-Inputting dynamic data
3. Self-defense-Exception handling
4. Strikes-Dynamic-path handling
I will discuss each of these principles in a separate article. The first article in this four-part series covers flexibility. In this second installment, I discuss Balance.
Achieving balanced requires being in a state of equilibrium. In life, balance is important for increasing the quality of life. In the martial arts, balance helps reduce the risk of injuries. In test automation, balance is important for both of these reasons as applicable to software testing.
In general, balance in testing typically refers to the adequacy of the test coverage in the test bed as a whole, which is more of a test-planning concern rather than something specific to automation. Balance in taekwondo-mation focuses on the test-automation script and on ways to subtly increase the coverage of a single script without increasing the amount of scripting inside of a single automated test. One way of achieving automated-test-script balance involves randomizing the data that is entered or selected in the application by the script. Other than the one-time creation of reusable functions that are responsible for randomizing the data, no additional scripting is required for each individual test script.
Before going any further, we must ask how randomizing data helps to achieve balance. The answer is in the fact that randomized data increases the application data coverage over time (cumulative coverage) within a script. This means that while each individual test execution may only traverse a single data scenario, the data scenario may change slightly during each test run, so multiple data scenarios are covered over time by the same script. To better understand, let's examine an old game known as block breaker. The object of the game is to break all the blocks at the top of the screen by using the paddle at the bottom of the screen to direct the ball into the blocks. Each time the ball hits the paddle it bounces up and breaks a different block.
|Figure 1: Block Breaker Scenario 1|
In figure 1, we see that the ball is about to break a block to the left of the screen.
|Figure 2: Block Breaker Scenario 2|
In figure 2, we see that by using the same paddle in a different scenario, the ball is about to break a block on the right side of the screen. What changed? The trajectory of the ball is what changed. So while just a single block is broken on each bounce of the ball, over time all of the blocks will be broken. This is what cumulative coverage is all about.
Balance in an automated script helps to increase quality of the application by increasing the test data coverage of that script over time.