So, you want to improve the quality of the testing done by your organization? The test quality improvement journey has several aspects to consider: the identification of improvement actions, which improvement action to start with and how to continue to improve. This document focuses on those improvement actions and ways to implement and improve on an existing set of good practices.
The Strategic Test Improvement Roadmap (STIR) is a tool which our organization has used to identify good testing practices and to provide a roadmap to implementing and improving on those practices. Our “definition” of practice is: a repeatable performance of an activity, capturing the skill gained from experience. In other words, practices are those testing activities that are done on a daily basis. And, as experience is gained, the quality and performance of those activities improves. At first glance, this appears fairly simple. However, there are many good testing practices out there. How do you know where to start? How do you know which practices you are capable of implementing? How do you know which practices will help you the most?
Background of our Testing Improvement Journey
At this point, let’s step back for a moment. In 1998, Bill Woodworth was appointed the IBM Director of Test, a position covering the entire corporation. His mission was to improve the way that testing was being done at IBM. He created the IBM TEST Community Leaders (ITCL), a group of 30 of the top technical people in test (Top Guns), with a corresponding 30 managers, who held key middle management positions in test. The idea behind the ITCL was to bring testers together to share experiences with each other and to strengthen test across IBM by allowing the individual units to grow and improve. The purpose of this group was for the same reason many testers attend conferences like STAR; to find new ideas and to share what is currently being done so that their organization can improve on the way test is being done. This concept sounds simple enough, but in reality we ran into many challenges. We were working with a very diverse set of testers and products, and the culture within each of the groups was different. As stated earlier, the practices are really what are done on a day-to-day basis. Each of the areas represented did testing differently, had different processes, and used a wide variety of practices. They developed products that spanned many functions; microcode, base operating systems, middleware, and applications. On top of that, each area’s organization represented a different culture. Some placed a high value on test, while others did not. This made it very difficult to identify "best practices" that were applicable to such a diverse set of testers and gave us many challenges when we actually tried to deploy a set of practices. We noticed that some teams were very successful at implementing and deploying new practices. While other teams floundered and failed miserably, unable to determine where and how to start. In order to make each of our teams successful we needed a structure to communicate what the good practices are and to show them how to deploy those practices in such a way as to maximize their success.
Development of the Strategic Test Improvement Roadmap
As stated earlier, the ITCL was made up of top technical leaders from various test areas. Many of them had more than 10 years of experience in test and found it fairly easy to apply that experience to deploying new practices. However, in many cases, they were not the norm and that worked against us. Finally, Dave Jewell, one of the Top Guns, presented some fairly basic but extremely important insight into deploying practices. He noticed that practices vary by the level of skill and experience that is required to implement them. That is why we saw tremendous success with some of our practices and why only a few groups were able to deploy other practices.