A recent question in the Quality Engineering-Metrics message board (on StickyMinds.com) asked about measurement at the different levels in the Software Capability Maturity Model. This article begins a series that will highlight the measurement requirements at each of the CMM Levels, starting with Level 2. Measurement in the CMM is often misunderstood when people focus only on the "Measurements and Analysis" sections of the model. This article offers an in-depth explanation of the CMM.
To appreciate the scope of the measurement activities needed to satisfy the CMM, you have to look at the entire model, from Commitments and Abilities through all the activities to Measurements and Verifying Implementation. If these terms (Commitments, Abilities, etc.) sound like arcane bits of CMM-ese, read chapters 3 and 4 of The Capability Maturity Model: Guidelines for Improving the Software Process1 to understand these terms and their relationship. The terms are not as important as recognizing that measurement is spread throughout the CMM.
Another important issue is the goal of the measurement. If you are looking for a checklist of what you need in order to achieve "Level 2," you will miss the value of measurement. Instead, you should approach measurement from a business-value perspective: "What is important to the management of software development in my business?" From this perspective, identify the data you need to run your business, and translate that into measures. Starting with Level 2, the most significant measures are size, effort, and schedule. Also part of Level 2 measurement are the Critical-Computer-Resources (think CPU, disk, memory, other performance measures), rate of requirements change, configuration management measures, and quality assurance measures.
In this series of articles, the word "measure" will mean "a unit of measurement (such as source lines of code or document pages of design)."2 Level 3 adds quality measures and changes the way measures are used. Level 4 uses the same basic measures, but again changes the way they are used. Level 5 again changes the way measures are used as a foundation for continuous improvement. This is a straightforward list of what needs to be estimated, collected, analyzed, and saved for future use. I've avoided justifications and explanations of their usage, as these answers are well documented and published in many software engineering books.
Level 2 Measurement Basics
Level 2 is about commitment to deliver products in terms of content, cost, and schedule. Thus, the measures should support the project manager and project team in meeting this goal. Any element that is estimated is also tracked for actual expenditures and is then available to improve estimating accuracy in the next cycle. The following measures support the Project Planning and Project Tracking and Oversight Key Process Areas (KPAs).
Size is a fundamental measure, since it can be used to estimate effort and defects. Whatever dimension is used to measure size, it should provide a normalizable unit of measure (a volume or count) that can be used when estimating effort and defects. If you have 100 units of size, it should allow you to determine there are nn units of effort, or mm defects. If you have 500 or 1,000 or 10,000 units of size, the relationship to effort and defects should follow some sort of relationship to the smaller size number. This is a basis for most estimating models (SLIM, COCOMO, etc). There may be lower or upper sizes where the relationship breaks down (in the original COCOMO model, 2,000 lines of source code was the lower size that was recommended). In many cases, the size measure can be a count, for instance, of the number of problem reports per week, the number of enhancement requests, or the number of help desk calls per day, all of which can be translated into effort.
Size estimating at level 2 puts you into the mode of estimating a target, measuring against the target, and taking mid-course corrections. Think of the fire control on World War I battleships—they estimated range, wind, etc., then fired, corrected, and fired again. This is Level 2. Level 3 adds radar control. Level 4 adds computer-controlled fire control. Note that the size-to-defect relationship is a Level 4 Software Quality Measurement capability; however, when defining and collecting measures at Level 2 and Level 3, you should look ahead to see what you would need in the future.