metrics

Conference Presentations

Measurement Problems that Plague Us

Measurement problems in software organizations are many: challenges with effort tracking; difficulties motivating the workforce to comply; resource management in multitasking and matrix organizations; attempts to "standardize" project status reporting or dashboards that run amuck; the misconceptions that fester and hinder defect collection and analysis throughout the life cycle; and management’s failure to truly understand and actually use measurement data and information to make decisions. Beth Layman reviews these thorny and recurring measurement problems-problems that plague even those organizations with established measurement cultures. She provides case studies of the problems and discusses their typical root causes. Beth concludes with practical advice on what it takes to institutionalize valuable, workable measurement practices within your organization.

Beth Layman, Layman & Layman
Five Core Metrics to Guide Software Project Endgames

By its very nature, the endgame of software projects is a hostile environment. Typical dynamics include tremendous release pressure, continual bug and requirement discovery, exhausted development teams, frenzied project managers, and “crunch mode” for testers. However, hope and relief are available. By using metrics derived from defect discovery and repair patterns, you can learn how to guide projects toward a successful release with less stress and more certainty. Bob Galen shares patterns that you can mine from your defect metrics to focus your entire team on a few key performance indicators. Examine defect patterns, maturation rates, and other organizationally unique patterns that you can leverage as project milestones. Learn about the Pareto Principle and its implications on defect actions and workflow. Next time, you’ll be better prepared for your project endgame and increase the likelihood of an on-time delivery.

Bob Galen, iContact
The Joy of Legacy Code

Even though the code may have been written only five years ago, there it is-a sprawling unintelligible mess that nobody wants to touch. For most people and teams, this reality is a cause for fear and loathing-something we want to sweep under the rug. We can, however, choose to see “bad” code as a challenge to restructure and refactor into a maintainable design that serves the business for years to come. Although legacy code presents many constraints on design choices, it offers the opportunity for incremental improvement. Michael Feathers shows you how to practice design within the boundaries of what some see as unintelligible code and explains ways to make the improvement process manageable. See how you can escape the fear that holds you back from productive action. Find out how to start with what you have now and progress toward a structure that supports the work at hand and immediately adds value.

Michael Feathers, Object Mentor
Harmful Project and Management Patterns Revealed

Every organization has its own project patterns. Some management teams take a long time to start a project. Others interrupt and divert the project teams once they've started. Some project teams never finish because the product must be “perfect.” Still others believe there is a single solution to all their problems such as “Two weeks of overtime and we'll be caught up!”, or “If the testers just tested more”, or “We just need some more time designing.” Project managers also fall into patterns: overly optimistic, pessimistic, risk-encouraging, risk averse, etc. Johanna Rothman explores different project and management patterns to help you understand which patterns are working-and which are harmful-for you.

Johanna Rothman, Rothman Consulting Group, Inc.
The Elusive Tester-Developer Ratio

Perhaps the most sought after and least understood metric in software testing is the ratio of testers to developers. Many people are interested in learning the standard industry ratio so that they can determine the proper size of their test organization. Randy Rice presents the results of his recent research on this metric and explores the wide range of tester-developer ratios in organizations worldwide. Learn why this metric is almost always not the best way to determine your test organization’s staffing levels and how to understand and apply this metric in more helpful ways. Find out how different tester-developer ratios relate to test effectiveness. Take away a new appreciation of your own tester-developer ratio and ways to meaningfully convey this metric to management to help rightsize your test team and improve the ROI of testing. Determine the "right ratio" of testers to developers in your team and company.

Randy Rice, Rice Consulting Services
Deception and Estimation: How We Fool Ourselves

Cognitive scientists tell us that we are hardwired for deception. It seems we are overly optimistic, and, in fact, we wouldn't have survived without this trait. With this built-in bias as a starting point, it's almost impossible for us to estimate accurately. That doesn't mean all is lost. We must simply accept that our estimates are best guesses and continually re-evaluate as we go, which is, of course, the agile approach to managing change. Linda Rising has been part of many plan-driven development projects where sincere, honest people with integrity wanted to make the best estimates possible and used many "scientific" approaches to make it happen-all for naught.

Linda Rising, Independent Consultant
The Uncertainty Surrounding the Cone of Uncertainty

Barry Boehm first defined the "Cone of Uncertainty" of software estimation more than twenty-five years ago. The fundamental aspect of the cone is quite intuitive-that project uncertainty decreases as you discover more during the project. Todd Little takes an in-depth look into some of the dynamics of software estimation and questions some of the more common interpretations of the meaning of the "cone." Todd presents surprising data from more than one hundred "for market" software projects developed by a market-leading software company. He compares their data with other published industry data. Discover the patterns of software estimation accuracy Todd found, some of which go against common industry beliefs. Understanding the bounds of uncertainty and patterns from past projects help us plan for and manage the uncertainties we are sure to encounter.

Todd Little, Landmark Graphics Corporation
Using Source Code Metrics to Guide Testing

Source code metrics are frequently used to evaluate software quality and identify risky code that requires focused testing. Paul Anderson surveys common source code metrics including Cyclomatic Complexity, Halstead Complexity, and additional metrics aimed at improving security. Using a NASA project as well as data from several recent studies, Paul explores the question of how effective these metrics are at identifying the portions of the software that are the most error prone. He presents new metrics targeted at identifying integration problems. While most metrics to date have focused on calculating properties of individual procedures, newer metrics look at relationships between procedures or components to provide added guidance. Learn about newer metrics that employ data mining techniques implanted with open source machine-learning packages.

  • Common code metrics and what they mean
Paul Anderson, GrammaTech, Inc.
Measure Quality on the Way In - Not on the Way Out

If you have been a test manager for longer than a week, you have probably experienced pressure from management to offshore some test activities to save money. However, most test professionals are unaware of the financial details surrounding offshoring and are only anecdotally aware of factors that should be considered before outsourcing. Jim Olsen shares his experiences and details about the total cost structures of offshoring test activities. He describes how to evaluate the maturity of your own test process and compute the true costs and potential savings of offshore testing. Learn what is needed to coordinate test practices at home with common offshore practices, how to measure and report progress, and when to escalate problems. Jim shares the practices Dell uses for staffing and retention, including assessing cultural nuances and understanding foreign educational systems.

Jan Fish, Lifeline Systems
Measuring the End Game of Software Project - Part Deux

The schedule shows only a few weeks before product delivery. How do you know whether you are ready to ship? Test managers have dealt with this question for years, often without supporting data. Mike Ennis has identified six key metrics that will significantly reduce the guesswork. These metrics are percentage of tests complete, percentage of tests passed, number of open defects, defect arrival rate, code churn, and code coverage. These six metrics, taken together, provide a clear picture of your product's status. Working with the project team, the test manager determines acceptable ranges for these metrics. Displaying them on a spider chart and observing how they change from build to build enables a more accurate assessment of the product's readiness. Learn how you can use this process to quantify your project's "end game".

  • Decide what and how to measure
  • Build commitment from others on your project
Mike Ennis, Savant Tecnology

Pages

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.