Understanding Software Performance Testing, Part 2

[article]
System Architecture and Infrastructure, Selecting the Types of Tests to Run
Summary:

Most people don't fully understand the complexities and scope of a software performance test. Too often performance testing is assessed in the same manner as functional testing and, as a result, fails miserably. In this four-part series we will examine what it takes to properly plan, analyze, design, and implement a basic performance test. This is not a discussion of advanced performance techniques or analytical methods; these are the basic problems that must be addressed in software performance testing.

This is the second part of a four-part series addressing software performance testing. The first part originally appeared in the April 2009 issue of the Better Software magazine. You can also find part one on StickyMinds.com.

The general outline of topics in this series is as follows:

- The role of the tester in the process
- Understanding performance and business goals
- How software performance testing fits into the development process
- Selecting the types of tests to run
- Quantifying and defining measurements
- Executing the tests and reporting results
- Understanding tools
- Quantifying and defining measurements
- Executing the tests and reporting results
- Understanding load (operational profiles OP)
- Selecting the types of tests to run
- Quantifying and defining measurements
- Executing the tests and reporting results
- Understanding tools
- Quantifying and defining measurements
- Executing the tests and reporting results
- The system architecture and infrastructure
- How software performance testing fits into the development process
- Selecting the types of tests to run
- Quantifying and defining measurements
- Executing the tests and reporting results
- Understanding tools
- Quantifying and defining measurements
- Executing the tests and reporting results
- Understanding load (operational profiles OP)
- Selecting the types of tests to run
- Quantifying and defining measurements
- Executing the tests and reporting results
- Understanding tools
- Quantifying and defining measurements
- Executing the tests and reporting results

In part two of the series I will focus on dealing with architecture and infrastructure issues and selecting the types of tests available to testers. The goal here is not to address every possible aspect of architecture and infrastructure or all possible test types, but to provide a general set of guidelines that can be used in developing a reasonable software performance test plan.

System Architecture and Infrastructure
The test architecture and infrastructure is probably the most common reason software performance tests results are invalid. To properly test a system or application, there are several key elements that must be looked at to ensure the test has at least a reasonable chance of providing useful information.

The first areas we will look at are issues relating to the physical architecture. There are two primary elements here: 1) the applications and systems, 2) the platforms on which the software will operate and 3) the network (if one is involved).

If the physical layout of the test system is substantially different from the production or target system, the test may not provide accurate performance data. When we look at the physical layout of a system, there are several key
elements that must be addressed.

The physical footprint of the system is critical. If the production system is a multiple-server system working in clusters but the test system is a single server or non-clustered individual servers, depending on the scalability factor , the test may have no value at all. The scalability factor is the ratio of equipment used in the test as compared to the actual target environment. A scalability factor of 5:1 or greater may yield no useful results. Some believe that a ratio of 10:1 is acceptable, but I find that once you get beyond 5:1 the differences can be dramatic in certain aspects of the architecture and infrastructure.

Figure 1 shows a 3:1 scale model.

Figure 1

This may give us reasonable numbers, but they will not be a guarantee of what will happen in the production world. There is also a missing element to this picture. To have a "cluster" of servers you need some form of

Pages

About the author

Dale Perry's picture Dale Perry

<span class="Text">With more than thirty years of experience in information technology, <strong>Dale Perry</strong> has been a programmer/analyst, database administrator, project manager, development manager, tester, and test manager. Dale's project experience includes large-systems development and conversions, distributed systems, and online applications, both client/server and Web based. He has been a professional instructor for more than fifteen years and has presented at numerous industry conferences on development and testing. With Software Quality Engineering for eleven years, Dale has specialized in training and consulting on testing, inspections and reviews, and other testing and quality-related topics.</span>

StickyMinds is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, TechWell.com is the place to go for what is happening in software development and delivery.  Join the conversation now!