When it comes to load testing in the cloud, going bigger is not always better. This article describes how organizations can utilize load testing to help right-size a production system being built in the cloud, allowing their project teams to build an ideal system that is not too small and under-performing for their needs and not too large and costly.
When application development teams build an application on physical hardware, initial hardware costs are generally a one-time consideration. They size their systems to support periods of peak load and some predetermined amount of buffer.
Physical systems are usually built oversized in relation to expected peak loads to mitigate the risk that a bad sizing calculation or false assumption has been made. It is more favorable to have a system that is somewhat oversized than one that is somewhat undersized because expected growth should be taken into account. We don’t build infrastructure just to support today’s loads; we make an educated guess as to how much our user base will grow in one, two, three years and beyond, and we build a system to support that growth.
As we know in technology, processing capacity correlates positively with cost, and the greater the processing capacity of the hardware in question, the larger the cost. We make an initial outlay for our servers, network equipment, and other hardware; we build out our environment; install, test, and configure our software; and deploy the finished product to production. With physical hardware, once we have made the initial purchase of hardware and deployed our system to the target environment, the ongoing cost for the infrastructure is comparatively low.
External cloud implementations offer an inverse cost structure compared to physical implementations. In the cloud, initial hardware investment is minimal, while ongoing (generally monthly) costs accrue based on the size and capability of the machine types utilized. The more capacity you purchase from the cloud provider, the higher the monthly cost. Regardless of whether you utilize that capacity, you are still paying for the availability of that capacity. This availability cost adds additional complicating factors to sizing and performance decisions. Performing properly targeted load testing allows a company to right-size its cloud installations, balancing ongoing costs with performance requirements and guiding decisions, which will have a direct impact on the cost of a system’s cloud footprint.
Let’s look at a basic example to illustrate two different approaches to building an application in the cloud.
The first approach is to build the system on the largest cloud instances available without regard to ongoing cost. This is a tempting solution, particularly when an organization is under pressure to deliver a project in a compressed timeframe, which is usually the case with application development projects. Teams under extreme pressure from their IT leaders to deliver something of value to production look for ways to reduce delivery times without adding risk to their projects. Machine instance sizes are selected not because of a thoughtful consideration of sizing needs, but because the developers and architects are certain performance issues will not be a problem on larger instances. Because the initial costs associated with spinning up large machine instance in the cloud are either the same or only marginally more expensive than spinning up a comparatively smaller instance, the project team implements an oversized infrastructure in the cloud for very little initial cost. Whereas on physical hardware the cost difference between a small server and large server can amount to tens of thousands of dollars up front, the cost to spin up a large cloud instance can be almost nothing. So, larger instances are utilized, load testing is either ignored or is reduced to save delivery time, and the application performs acceptably on the chosen infrastructure. This approach, however, causes a number of issues soon after the application is deployed to production.
First, this “build it big” method of developing for the cloud encourages teams to build flabby solutions without consideration for performance or load handling. Why bother tuning an application for performance when a developer can simply choose the largest available machine type and have it run acceptably? This code, however, will invariably be inefficient, creating a “lock-in” effect—the code now must reside on larger, more expensive instances because it can’t run efficiently on more moderately sized instances. The only way now to reduce the cost of the underlying cloud infrastructure is to go back and refactor the code for performance, another project (and an additional set of costs) in and of itself.