1. We’re dealing with potentially very high numbers to convey peak usage of systems, and many times the numbers will be inaccurate. For example, "Technically, max users on our Web site figures out to 103,894 but we’ll just round down to 100,000."
2. Frequently, load testing is at best a scaled-down simulation of real-world testing, not simulating the actual load of a real-world environment. For example, "Production has ten Web servers behind a load balancer. At 100,000 hits an hour, that’s 10,000 hits per Web server. Let’s simulate 10,000 hits on a single Web server, and we’ll have validated the system by calculation."
3. Load testing is frequently performed at the system level, thus touching potentially hundreds of data validation points, but not actually testing any one of them.

The result is that often load testing becomes synonymous with "loading up the system and seeing what breaks." This illogical, and frankly inconclusive, method to load testing does nothing but mask real potential problems. Moreover, it’s very difficult to quantify such tests and it often leads to inaccurate or wrong conclusions.

I want to emphasize a few basic points to keep in mind when designing load tests. These points don’t cover how to execute or run load tests, but they should be considered before the tests are actually run. They may give you a way of designing the plan by which you will conduct load testing.

The first step in constructing a logical stress test plan is to pick a goal that you want to validate. You must have a specific hypothesis in mind that you are testing with your load tests. Some examples of bad test goals are

• to make sure our system handles heavy load
• to see what the breaking point of our system is
• to find the bottleneck in our system

Examples of good, accurate test goals are

• to verify that the database is not the bottleneck of system capacity
• to explore what happens when 10,000 people try to navigate our Web site’s most popular path simultaneously
• to verify our Web server load balancer is appropriately routing connections

Second, it is vital to pick distinct test parameters and validation points. We all learned in science classes that when conducting an experiment, one must pick control parameters and a test parameter. Load testing is very much like conducting an experiment, in that you may not really know what the system is going to do when you apply your test coverage. Thus, you need to be able to identify distinct parameters that you will hold constant, and preferably a single test parameter to which your load will apply.

Once you have identified what you are going to test (i.e., your test parameters), it’s time to decide how to test—not what tool to use, or what hardware will be required—but by what methodology you are going to conduct the exact load test. Before you can assign numbers and values to the parameters you identified previously, however, you need to establish the assumptions from which you are operating, and from which you will make your calculations of the test and control parameters. For example, if our test goal is "to verify our Web server load balancer is appropriately routing connections," our assumption is "a connection simply represents a literal TCP/IP connection request to the load balancer that is sustained over time." Obviously, connection routing cannot be tested with connections that are being established and terminated quickly, so we need to identify the persistence assumption.

Perhaps a more relevant example will be to outline the assumptions to validate the test goal, "To explore what happens when 10,000 people try to navigate our Web site’s most popular path simultaneously." What are the assumptions?

1. The most popular path of the Web site is: log on, navigate to Content…
2. One "person" represents a single connection to a Web server, followed by actions that will navigate the user according to assumption 1.

Once you have your assumptions in place, you now have logically made the transition from a load goal to being able to actually test. You have identified the control and test parameters, and you have assumptions by which you will make calculations of actual values. This is where you fill in numbers and values for all parameters, both control and test, that you identified previously. You can then execute!

Make sure that as you prepare and run the load test, you jot down every step you took. You will want to be able to repeat your findings if necessary, especially if things go wrong. It may be that as the load test starts, the environment or test setup will require a tweak to get it going—make note of that. It may be that you realized halfway through that someone else was doing bandwidth-intensive work on the network—make note of that as well. Keep track of all your control parameters and alter your test parameter according to plan.

Document everything—starting with the initial steps of identifying your load test goal and defining assumptions, straight on through to your calculations and execution notes. All of it should be written down and documented somehow, whether in Notepad or a template in Microsoft Word. Not only will it make reproducing the test much easier, but when it comes to sharing results of the test, you will have at your fingertips all the information required to put together a good report.

Finally, make sure that you have defined a quantitatively clear stopping point for your load test, and that you run it to completion. A load test goal like "establish 10,000 connections to the load balancer and make sure they route properly to Web servers" is a goal with a clear definition of the stopping point: establishing 10,000 connections. On the other hand, a load test goal like "generate 50 percent load on the system" is unclear and has no definite stopping point.

Load and stress tests that are constructed in a logical manner, with up-front planning and strategy before execution, will yield meaningful results that you will be able to quantitatively convey.