|
|
|  |
 |
Better Software Home > In This Issue > Featured Article
|
 | |
January/February 2010 | |
 |
|
Virtual Labs in the Cloud
by Ravi Gururaj and Srihari Palangala
There are two key management challenges that software engineering labs face: The bottleneck from IT operations and unconsolidated administration and lab infrastructure.
The IT administrator must respond to users’ requests quickly. Since manual intervention is required when the administrator readies the infrastructure, there often
is a significant wait time before software engineers gain access to the requested resources.
On the other hand, IT administrators have a difficult course, because they need to prioritize the requests received as well as provision, manage, tear down, and free up resources.
It is not uncommon for developers and testers--the users of the labs--to have their server infrastructure closely guarded and self-managed. These users often operate in silos and find it difficult to share lab information and artifacts. IT administrators are faced with the uphill battle of trying to coordinate, sync, and ensure organizational compliance in the many servers used across the organization. Since there is no centralization of the lab infrastructure resources, it is difficult to drive efficiency of resource utilization.
Virtual Labs on the Private Cloud
To address these challenges, IT administrators have moved toward lab server consolidation
powered by virtualization. This lab consolidation is usually accompanied by a management layer called virtual lab automation (VLA). Together, they enable the lab to operate as a private, on-premises cloud.
VLA is a management approach in which the recurrent, labor-intensive manual tasks, processes, and infrastructure necessary to provision development and test labs are automated, streamlined, and centrally managed. VLA ensures rapid, highly repeatable, resource-optimized deployment of complex, multi-machine development, test, and pre-production environments across virtual machines. VLA can integrate with existing software and hardware infrastructures in order to maximize your existing investments.
On-premises VLA provides a solution to the two key management problems described above, through end-user, policy-governed self-service, and consolidated and centralized lab management.
VLA enables automation-supported, self-service lab provisioning for end-users. These virtual labs can be constrained by resource quotas and access policies for each class of end-user. End-users gain on-demand lab access to IT resources via a Web browser interface. The VLA management system ensures that no single user can monopolize or misuse the shared pool of resources.
Infrastructure administrators can use a single tool to manage across a multi-site, multi-hypervisor virtual farm infrastructure deployment. A hypervisor is a computer software platform that abstracts the underlying hardware and allows multiple operating systems to run on a host computer concurrently while sharing the hardware resources. This unified management view provides the infrastructure administrator with a coherent view of resource utilization across all lab sites. To a great extent, multi-hypervisor VLA management framework ensures the administrator has eliminated the risk of vendor lock in.
While VLA solves some critical management problems for virtual labs on the private
cloud, there are other challenges to consider: required upfront capital expenditure,
capacity planning for the peak, and in-house infrastructure administration.
On-premises virtual labs require organizations to fund the required hardware and software. In today's economic environment, it may be difficult to convince management to invest in on-premises equipment.
The on-premises virtual lab consists of a shared infrastructure that is accessible to many groups and users throughout the organization. The user demand for the infrastructure resources will vary
as a function of time, making capacity planning a challenge for IT managers. The easiest solution for administrators is to plan for the peak lab requirements, which means that infrastructure would be substantially underutilized during the average and low utilization periods as shown in figure 1. This leads to inefficient use of capital resources for equipment that is not utilized optimally.
Figure 1: Hybrid lab management offers the best of both private and public cloud infrastructure
On-premises private clouds require dedicated administrators to maintain, tune, and manage the infrastructure. For example, a hardware administrator manages the hosts in the virtual lab and a software administrator manages the virtual lab instance, grants user permissions, sets up the LDAP integration and authentication, and performs other tasks.
So, while there are tremendous benefits with an on-premises solution, it may not always work due to the challenges cited above.
Virtual Labs on the Public Cloud
Along with an on-premises infrastructure running hypervisors, VLA technologies and products also can leverage the public cloud infrastructure, which is available as infrastructure as a service (IaaS). This can address the challenges of running virtual labs on your internal, private clouds.
Two benefits come from leveraging the public cloud as the infrastructure platform for your virtual labs: transactional access to resources and abstraction of the resource view from users.
The public cloud provides users with the ability to access and manage resources (compute, storage, and networking) in a utility-style, pay-for-whatyou-use mode.
The public cloud infrastructure is powered by virtualization, which abstracts underlying hardware from user interactions. This provides users with dynamic, scalable, elastic, and graded access IT resources. The abstraction also provides the platform for increased collaboration and artifact reuse among the many users.
The ability to have your VLA management software work on public clouds provides several benefits:
Convert capital expenditure to operational expenditure: Your team can avoid making upfront investment and capital expenditure on hardware and software to run the virtual lab. Instead, users can leverage the pay-as-you-go operational model of the public cloud.
Let the IaaS vendor handle the infrastructure administration: Your team does not need to rely on an administrator to manage and maintain the hardware and software in the lab. Instead, the infrastructure is provided as a service and is associated with specific servicelevel agreements to meet your virtual lab requirements.
Pay as you go: Shifting to the public cloud provides tremendous economic flexibility and the ability to pay only for resources that you use. This flexibility helps the engineering team manager control and cap operational costs.
When using the public cloud, concerns of security and compliance may arise. In such cases, users can look at hybrid VLA solutions that allow you to manage virtual labs over both private, on-premises infrastructure, which is typically more secure and under tighter compliance norms, and public clouds, which give you a more flexible operational model. As shown in figure 1, administrators can design the on-premises lab to handle average steady-state lab demand and, for all requirements over the average, users can “burst” to use the resources in the public cloud. Vendors are now building virtual lab management solutions to handle seamless management over both public and private clouds. This use of both public and private clouds has economic benefits. Customers no longer need to build and plan for peak capacities and risk ending up with a lot of underutilized capacity on premises.
Engineering Implications and Benefits
As discussed above, virtual labs can be enabled on private as well as public clouds. Once deployed and adopted, how do these virtual labs help the development, build, and test processes? Let’s examine each of these in some detail.
Software Development
Provision, manage, and tear down multi-machine configurations quickly: In a virtual lab, multi-machine configurations (think n-tier configuration test beds for your application) that model production-like environments can be created. This provides networking capabilities to connect the various virtual machines in the configuration.
Network isolation (IPZone): Engineers can deploy and redeploy a configuration without worrying about the IP or MAC address conflicts between the various deployed instances. The capability to IPZone a configuration is tremendously valuable. For example, a tester can hand off to development a configuration snapshot capturing a bug and then continue testing. Development engineers can use the saved configuration without causing any interruptions or conflicts with the testing efforts.
Virtual desktop infrastructure environments and consistent development environments: Virtual labs can be configured to provide developers access to standard machine configurations with the appropriate software and stack configured on the machine. This ensures that all your developers have access to consistent development environments. Furthermore, developers have the ability to capture, save, and back up the virtual machine with ease.
Commercial software licensed in virtual machines: Several large independent software vendors like IBM, Microsoft, and Oracle have started to offer their software programs and packages as part of standard virtual machines in the public cloud. For example, Amazon Web Services offers special Amazon Machine Images (AMIs) preconfigured with IBM, Microsoft, Oracle, and other software. These AMIs are priced on a per-hour usage basis, and developers and testers can use them in their development and test environments without needing to purchase the expensive full software license.
Outsourced, partner-controlled access: Virtual labs are accessible across the globe via a Web browser. Users and teams can be set up with appropriate policies, quotas, and specific persona permissions. Now your development team, test team, and external partners, working from different parts of the globe, can share artifacts such as scripts, licenses, and image templates, as well as collaborate on the software engineering project.
Software Build
Virtual labs can spin up pristine build environments: The build process in software engineering requires access to a pristine environment. A virtual lab in the cloud lets users use automation to spin up a pristine machine. A typical scenario involves the powering on of an instance of a build virtual machine from the library, connecting to the source code repository, downloading the latest stamped source code, executing the build commands in the virtual machine, and outputting the latest build files. If required, these build files can be transferred out of the virtual machine (to a common file server, for instance) and then powered off or discarded.
Virtual labs can drive increased levels of automation: Agile methodologies and continuous integration (CI) are powerful process methodologies. Virtual labs can complement the CI workflow process. For example, if a CI build needs to run on a certain platform and no virtual machine image exists that matches the requirements, users can automate the spawning of a new virtual machine, even saving it for future use if required. This new, spawned virtual infrastructure can execute the CI build, trigger the set of automated tests to be run on the latest build, and transfer the results of the test run to the results repository. Once the CI run and test execution are complete, the virtual infrastructure in the cloud can be torn down to free up the resources.
Software Testing
Compatibility testing: Today, software needs to be tested on many different platforms. A virtual lab in the cloud provides the ability to spawn pristine virtual infrastructure with minimal wait times. Software can be compatibility tested in these environments and certified against myriad software system requirements. The alternative to this approach is to wait for your in-house IT administrator to provision and set up the software environments for each of the configurations in the compatibility test matrix, something that is time and cost prohibitive.
Test repeatability and avoiding "norepro" bugs: Software engineers are constantly faced with the no-repro bug, where development is unable to reproduce the bug encountered by software testing or found through the use of the software in the field. These bugs are unattended to or, in the worst case, closed because they cannot be reproduced. Virtual labs in the cloud provide two key capabilities to help in this scenario: Testers can suspend, snapshot, or capture the test environment configuration as the defect occurs, which will help the developers gain easy access to an environment where the bug can be reproduced. And, testers and developers can repeat the sequence and introduce predictability for test runs--starting with a clean base OS image, installing or customizing the necessary software stack environment, installing the version of the developed software for testing, and repeating the series of steps for the tests.
Scalability tests: Virtual labs in the cloud help testers create test beds for large and massive scalability tests. Creating a test environment that flexes the software’s scalability aspects can be done best with virtual lab infrastructure that allows testers to spawn up, grow, and then throttle down on the test bed environment.
Centralized virtual labs enable user sharing and collaboration: Virtual labs in the cloud are a great asset for testing staff and teams when there is a need to reuse, automate, collaborate, and share various lab artifacts like virtual machine templates, scripts, licenses, and ISOs. The centralized management and operations enable user and team management with easy sharing and collaboration. For example, testers can share entire environment configurations with developers.
Test environments for the manual tester: Manual testers often require access to pristine, multi-machine test environments. With virtual labs in the cloud, testers can not only deploy but also design custom virtual machine environments in minutes via a Web interface that is used to manage virtual labs in the cloud.
Business Value and Implications
As seen above, virtual labs in the cloud offer tremendous benefits from an engineering standpoint. Let’s examine the benefits of the management capabilities and infrastructure from a business value standpoint.
Eliminate provisioning bottlenecks and accelerate software time to market: Environment provisioning, re-provisioning, and waiting for the environment availability are critical blocking factors for smooth and quick development and testing cycles. In traditional physical environments, the provisioning takes eons and blocks users on key tasks in the project plan. With virtual labs in the cloud, your testers, developers, and external partners have self-service access to infrastructure and software with complete policy, quota, and lease governance controls. This access to virtual labs ensures that engineering staff members have access to the required resources when they need them and how they need them. The gain in engineering cycle time is a key business benefit. Anecdotal evidence suggests that customers can reduce the provisioning time by as much as 80 percent.
Increase test matrix coverage to improve software quality: Catching bugs early is a key goal for every software engineering effort. Bugs discovered late in the testing cycle or, worse, bugs found when software is deployed in the field, are very expensive for the business. Virtual labs in the cloud help advance the test matrix coverage. For example, more tests can be run during every unit, release, system, or regression set of tests. The increased test matrix coverage guarantees improved software quality since it helps bring the field or production environments into the software engineering cycle and reduces the risk of discovering latent software deficiencies after release. Anecdotally, virtual labs in the cloud reduce by as much as 95 percent the time required to reproduce bugs and provision environments.
Manage infrastructure and improve cost control: A virtual lab in the cloud is a resource that is used by multiple software development engineers and teams. The costs of this resource can be shared between the many business units or engineering managers within and outside the organization, leveraging it during their software development and test efforts. Virtual labs provide management capabilities to enable lab access and audit trails, and they control and allocate costs internally to the various units or engineers accessing and leveraging the virtual labs.
Improved worker productivity: Virtual labs in the cloud enable powerful automation in terms of both policygoverned self-service of virtual machine environments and automation of operations within the guest virtual machines. This powerful automation capability helps increase worker productivity--workers now have the ability to refocus their efforts on strategic, higher-valueadd activities.
Offer tiered access to shared resources for your workers: The shared infrastructure resource can be divvied up with different service-level agreements for different teams and users accessing the resources. For example, testers involved in the release of a product require higherpriority access to the resource than do developers and testers conducting unit tests or system tests while working on the next major version of the software. This management capability when overlaid with the policy-governed, selfservice, shared virtual lab in the cloud makes it compelling for the business--it can now unleash its users to the virtual lab in the cloud, knowing that they have access to all the benefits with little to no downside.
Cloud computing is gaining mindshare and importance for IT in small, medium, and large businesses. Organizations are looking at both private and public cloud infrastructures to tackle their acute lab infrastructure operational issues. Virtual labs on private or public cloud infrastructures provide powerful infrastructure solutions to the operational issues, including the ability to run a policy-governed, self-service virtual lab and to drive consolidation and centralization in the lab. Virtual labs in the cloud provide both engineering and business benefits. The key engineering benefits include improved development, build, and test. Virtual labs in the cloud provide business justification by accelerating software time to market and improving software quality, cost control, and worker productivity. {end}
Ravi Gururaj is founder and CTO of VMLogix. Ravi has more than nineteen years of product development, management and marketing experience, having held positions at companies including Trilogy, UberWorks, and Dialogic, and having founded two software start ups that designed and licensed reusable software components for application
developers worldwide.
Srihari Palangala is the director of product marketing at VMLogix and has spent more than eight years in software product management, marketing, and business development having held positions at Microsoft and OPNET Technologies. |
| | |
