Jesse Dowdle sat down with us recently to discuss the benefits that embracing "massive" continuous integration and deployment brings to testing. Jesse is presenting the session "Massive Continuous Integration and Light Speed Iterations" at the 2012 Better Software Conference East.
Noel: I read where you said, "AtTask decided to rapidly rethink their development model." What brought on that decision, and how can other companies benefit from doing the same thing?
Jesse: AtTask is software as a service, a model that has exploded over the last few years. In the early days it was typical for a company to deploy monthly, and our infrastructure and application stack were built to that expectation. As the industry was progressing, our company was growing as well, so we were experiencing the pain of multiple code branches, less than stellar automation, etc. Ultimately, we were having trouble hitting our monthly releases.
AtTask never does anything small: it's kind of a cultural thing for us. Instead of just fixing the monthly deployments, we decided to shoot for a big hairy goal… what if we deployed every day? That goal is what led us down this path.
Noel: Continuous integration appears to rely heavily on directed, constant communication. What advantages does this create, and what are companies lacking without it?
Jesse: Continuously integrated builds, particularly at scale, work best when the results of the system have high visibility in your organization. This visibility creates a terrific rallying point. We think there's a lot of value in knowing, at any given moment, what the quality of your build looks like.
When you couple that with automated deployment scripts (which we also have), you create a level of flexibility within your development group to respond to changes in production within minutes. That level of agility has lots of power. We use it to split-test features, hot-fix high visibility defects, measure performance impact, and a host of other things. Ultimately, we can move more code around faster than our competitors.
Noel: For those wonder if they can afford "massive" continuous integration, what are the costs involved, and how can companies keep those costs down?
Jesse: Building out an automated system is all about trading fixed cost for elastic cost. We used to take our team off-line for a week to do the QA required for a release. Doing this monthly meant we were spending 8% of our payroll annually just certifying builds. Once we went to a fully automated cloud-based system, we were able to reduce the time to certify a build to about an hour, and required the attention of just a couple engineers during that time.
The primary cost (after investment in automation) is the cloud infrastructure to run your tests. We use reserved instances and small machine sizes to keep it affordable. Additionally, all of our automation is designed to tear itself down when its job is done, that way the meter is never running on unused compute. The whole system is fully elastic.
Noel: With available tools and technologies constantly being introduced to the software developing world, how do you see continuous integration becoming even more effective in the future?
Jesse: It's becoming easier to stand up a complex application stack with all of its dependencies in a virtualized way. Being able to replicate your stack (or parts of it) is critical making something like this work. With companies like Amazon investing in tools to simplify the process, I think it's going to become a standard practice, just like