Hans Buwalda, CTO of LogiGear Corporation, is an expert in test development and testing technology management and a pioneer of keyword-driven test automation. In this interview with Heather Shanholtzer, Hans discusses what "big testing" means and some of the challenges it presents.
Hans Buwalda is an expert in test development and testing technology management and a pioneer of keyword-driven test automation. He works in California as CTO of LogiGear Corporation and is co-author of Integrated Test Design and Automation. In this interview with Heather Shanholtzer, Hans discusses what "big testing" means and some of the challenges it presents.
Heather Shanholtzer: What is "big testing"?
Hans Buwalda: It is a relative term that can have different meanings for different organizations. A good definition could be "big for you"—the size of the testing efforts needs attention in itself, apart from the test content. Here are some things that would make a test big for me:
- Big volume of tests—for example, targeting a large system under test
- Concurrent variation of code branches
- Complex functionalities, needing many test situations or values
- Big variation in target platforms and configurations
- Big maintenance—rapid and frequent changes to the system under test
- Big or complex hardware infrastructure needed to support testing
Heather Shanholtzer: Why is it necessary in today's computing environment?
Hans Buwalda: Big testing might be necessary simply because the application under test is large or complex. Or, you may have many versions or configurations of your systems under test and their environment, making many tests necessary.
However, in my classes, I also like to first talk about keeping the size of tests manageable. in testing, "big" is not always "beautiful." Careful test planning and design can help keep testing under control.
Heather Shanholtzer: What are some of the challenges unique to big testing projects?
Hans Buwalda: You have to plan and manage them like large-scale operations. Apart from the design angle, there will be an industrial-scale "production" focus. Many people may be involved in developing and assessing tests and their automation and large-scale equipment use may be necessary. Being smart and efficient and keeping timelines are critical for success. Also, if you do big testing, you need a method that makes the test cases more maintainable and scalable. I started an approach called Action Based Testing, which helps me achieve this. It focuses on test design, rather than technology or tooling, as the main driver for automation success.
Heather Shanholtzer: Does big testing require a big testing team?
Hans Buwalda: There is no general answer to that. "Big" could also just mean needing many computers to execute the tests. Big team size can help timelines, but it can also be counterproductive, since it can lead to too many and too shallow test cases. An outsourcing strategy could be considered as well, as it can help to more easily scale up and scale down based on project needs. But, it needs to be done with even more care to make it effective.
Heather Shanholtzer: Is there a place for agile methodologies in big testing?
Hans Buwalda: Absolutely! Testing being big does not preclude it from being integrated in sprint team efforts. In fact, even if the system under test is more traditional, I still recommend agility for testing. This is also a key part of Action Based Testing.
Heather Shanholtzer: You taught a tutorial on big testing at the STAREAST conference in April 2012. For those who weren't able to attend, what was the main take away message you wanted to impart?
Hans Buwalda: My main message was: think before you do. The bigger the testing, the more important that becomes. Plan and organize your tests before you develop them in detail. See if there are techniques that can help reduce size, like exploratory testing, code inspection, and agile approaches. Also, pay attention to test environments. In particular, take a good look at virtualization and clouds to achieve needed scalability.