A Strategy for Risk-Based Testing

[article]
Summary:

The fact that you test an application extensively does not itself render the application more stable. However, NOT testing an application increases the risk that the software may not comply with the requirements and won't necessarily provide the expected business value. Here is a method for approaching risk-based testing.

In the ten years that I have worked in the software industry, I have come to realize that there is a widespread, but essentially wrong, belief amongst most software companies that software quality is a function of testing. The fact that you test an application extensively does not render it more stable, nor does it provide added value to the customer. How many times have you heard, "This software is not stable enough, we should have had more QA."? Unfortunately, testing has nothing to do with stability. You test to ensure that the software functions as it was initially requested by those who dictate the requirements. Therefore, not testing software increases the risk that the software may not comply with the requirements and won't necessarily provide the expected added business value.

This leads us to risk-based testing, which I would describe as Steve Wakeland defines IT risk, as 'the likelihood that a program fault will result in an impact on the business'. Further, I would specify a NEGATIVE impact on the business. Let me explain.

I classify bugs with severity ranking, as do most people, and define severity by measuring the negative impact a bug has on the business. The severity ranking is usually high, medium or low. High meaning the customer cannot work with the software and wastes time and money, medium meaning the user has to use some alternative workaround to achieve the aimed goal and low meaning the customer can STILL work with the software (feel free to specify additional levels for a more granular ranking). Thus, if I classify bugs found by QA in terms of severity by measuring (or guessing) the negative impact it has on the business, then I should focus my testing using the same mindset, i.e. negative impact.

Thus, testing and risk (as defined by Steve Wakeland) are related. How? Is it a straight line as shown in Figure 1? In that case, the only strategy is: Test anything; it's sure to lower the risk.

Strategy for Risk-based Testing1

Figure 1

Ideal Relationship
Pareto was probably right when guessing that 20% of the total feature set allows a user to do 80% of his/her work. What if the relationship between the testing effort and risk was more like Figure 2?

Strategy for Risk-based Testing2

Figure 2

We can define a better testing strategy that optimizes the QA effort to minimize risk. It is important to note that no matter how much testing you invest, there is always a risk. Therefore, releasing the software is directly related to the level of risk you are willing to accept.

However, if you see testing as an investment, the strategy becomes: What is the minimum testing effort that I should invest in order to maximize risk reduction? In other words, what should I test first to be in Zone A of Figure 2?

I control my testing effort with a simple methodology based on usage. Although users have their own way of using software, you can usually predict the top-ten features list that allows them to do most of their work.

Methodology
Step 1: Identify the 'vital' functionalities that could prevent the user from using the software if a bug was found. (This bug would be a high severity.) A good example would be a login page for a Web application that does not work. An efficient way to gather this list of functionalities is to survey the end-user community, ask a domain expert, or assemble statistics from the log of a previous version of the application. Since the risk increases with the frequency of use, you should look at the most used feature

About the author

TechWell Contributor's picture TechWell Contributor

The opinions and positions expressed within these guest posts are those of the author alone and do not represent those of the TechWell Community Sites. Guest authors represent that they have the right to distribute this content and that such content is not violating the legal rights of others. If you would like to contribute content to a TechWell Community Site, email editors@techwell.com.

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!

Upcoming Events

Sep 22
Oct 12
Nov 09
Nov 09