The Relationship between Risk and Continuous Testing: An Interview with Wayne Ariola


In this interview, Wayne Ariola talks about the relationship between risks and continuous testing, misperceptions of continuous testing, who should be interested in it, how we are all in a new era of testing, and why it's the best time in the history of software to be a tester.


Cameron Philipp-Edmonds: Today we are joined by Wayne Ariola of Parasoft. He is going to be speaking to us today about continuous testing. To start things off, let me thank you for joining us today.

Wayne Ariola: My pleasure, as always.

Cameron: All right, can you also tell us a little bit about yourself and your role at Parasoft.

Wayne: Sure, I've been with Parasoft for, I think it's twelve or eleven years now. My current role is chief strategy officer, so I'm responsible for the direction of the company from our technical perspective and the ways we can help clients achieve better results reducing the risks associated with application failure. What I do is work very closely with clients to help them achieve their business goals with software.

Cameron: All right, fantastic. Now today you're going to talk to us about continuous testing, so in your words, can you tell us what continuous testing is?

Wayne: Sure, today we're in a position in which software is truly the interface to the business. There's just no doubt about it. Software [Skype] is the interface to our discussion. It didn't always use to be that way, but, today it certainly is. Whether you're a bank or driving a tractor in the field, software is optimizing how you're interacting with your customer or how that customer is interacting with what they are doing. With that, the risk of software failure has an extended cost. Or the cost of quality, which is really the cost of failure, is much greater today than it ever has been. As we progress down this technical chain of events it will become even more expensive and there will be more risk to the business when software fails.

When I'm talking about continuous testing, I mean we need a method in which we can more confidently evaluate our products (software) continuously to understand the risks associated with it at any point in time. And when I say products, I mean software projects or any kind of application that is evolved through a software development group.

Cameron: Now you mentioned risk there a couple of times. What exactly is the relation of the word risk to continuous testing? Is it kind of a one-to-one ratio?

Wayne: Absolutely, there's no doubt about it. Just like any manufactured product, what you're trying to do is diminish the risk of failure, right? Let's just face it: Software is complex—it's not easy.

It's just one of those things that as you begin to evolve software, the application gets pretty complex, pretty quick. With change the complexity associated with, or the risk associated with failure, increases. There's a couple of great studies by the FDA by the way, along this theme. In most of the FDA recalls (and I'm paraphrasing this dramatically), they found that it wasn't the initial application that had risk, but it was the subsequent change that introduced risk into the application. So the second release, the third release; those had a greater risk for application failure. And as you can imagine, as we look at an application and we start tearing into one area, rebuilding around something or adding code to one element, things get a little more complex. As they get a little more complex, the risk of failure increases.

With that, we really need to understand where the risk associated with the application is at any given point in time so we can make trade-off decisions associated with release candidates. That's really the translation between what the development team has developed and what the business is willing to adopt, or accept, as a risk in terms of going to market.

Cameron: It sounds like continuous testing is really great for diminishing risks and having a great handle on things. Who really should be interested in continuous testing and why should they be?

Wayne: That's a great question. As every industry matures, you will see this trend replicated associated with process maturity. The maturity goes along really three distinct elements which are speed (or time), scope, and quality. When you look at these trade-off elements, they are in constant conflict. If you could imagine, the business is always saying, "I need more. I need it faster." As I mentioned before, software is distinctly the interface of the business. If software is that differential component, then you need to truly be able to deliver software faster. So the business is saying “I need more and I need to be able to differentiate faster.”

Now we have this scope and quality issue that development teams need to be able to accommodate. If you look at any single industry, and one of the best documented is the auto industry, you need to actually put these things into balance in order to come out with a quality product that enhances the brand and doesn't necessarily promote any distinct type of risk. I did a little research project last year. I took all the public companies that had software failure announcements between 2012 and early 2014. On the day of the announcement, the companies that were public lost an average of 2.3 billion dollars of shareholder value upon that announcement.

Cameron: Wow. Just wow.

Wayne: When we sit there and we talk about risk and about controlling the quality of software... I doubt that the developers who were working really, really hard to turn that product out were thinking that they were going to lose 2.3 billion dollars of market share because there was a bug in their software.

Cameron: Gosh, that's unbelievable.

User Comments

Gallop Sol's picture

Continuous testing can be thought of in two ways. One is when you are involved in the software development process and follow agile testing methods which involves continuous testing in all iterations. Second is when you have a live software which is going through regular patches and you continuously test it for various Integration tests and UAT's. I think he is talking abt the 2nd one.

December 11, 2014 - 3:40am
Jon Hagar's picture

So maybe we need to think about continuous Verification, Validation, and Testing includling test automation over the complete product lifecycle, which starts during product/business analysis (should we build anything) and continues during deployment and operations until retirement.   Many managers and testers, just focus on a part of the problem of creating software. 

December 11, 2014 - 1:11pm

About the author

Upcoming Events

Oct 01
Oct 15
Nov 05
Nov 14