In this interview, TechWell's Mike Sowers goes in depth on the measurements, metrics, and management side of testing. He discusses both the good and the bad on his test automation journey in large and small enterprises and communicates the real challenges.
For thousands of years, human language has provided us with beautiful and complex ways of sharing important ideas. At the same time, language can derail attempts to communicate even the most basic pieces of critical information. We testers are the heralds of vast amounts of data, and...
Imagine you’re a test manager starting a new assignment. On the first day of work, you’re presented with a list of metrics you are to report. Soon, you realize that most of the metrics are not really connected to what should be measured. Or, consider the situation where you’re told that...
Scott Barber is the chief performance evangelist for SmartBear and an author of several books, including Performance Testing Guidance for Web Applications. In this interview, Scott chats about useful test metrics, communication, and the problem with performance testing programs.
On a fairly regular basis, test and QA management have to explain their value and role to their clients. Sanjay Zalavadia writes that in these situations you must choose metrics that provide insight into what you are doing instead of obscuring it. This will help tell your story in a compelling way.
Counting is easy. However, what makes measurement really valuable-and really hard to get right-is knowing what to count and what to do with the results. If your organization is mostly tracking resource usage, costs, and schedule data, it is making a big mistake. What about the users? The customers? The overall business strategy? Sharing the lessons he has learned from fighting-and surviving-many software measurement battles, Ed Weller offers a step-by-step approach for implementing a practical and valuable metrics program. After understanding what measures are most important to the business strategy and all stakeholders, the next step is to decide what data supports those measures and how to capture it. With data in hand, you can create simple and informative ways to make the resulting metrics visible and easy to digest. The biggest challenges-avoidance, disbelief, and rationalization-come next.
Edward Weller, Integrated Productivity Solutions, LLC
Measurements and metrics are a hot topic again. Theorists often rail against them as meaningless and potentially harmful. Practitioners fear them because they don’t want to be metric-ed out of a job. Managers want them so they can better understand what is happening in the software development lifecycle and try to make their processes more efficient. David Gilbert shares the simple set of measurements and metrics Raymond James has implemented and describes the practical benefits they have gained. By first asking stakeholders what they really wanted to understand and then developing metrics to support their goals, David and his team have made measurement work at their company. They educated the stakeholders on key metrics concepts-first order measures and subjective relativity-and established a common model everyone agreed to follow.
Are your testing and quality assurance activities adding significant value in the eyes of your stakeholders? Do you have difficulty convincing decision-makers that they need to invest more in improving quality? Selecting metrics that stakeholders understand will get your improvement project or program past the pilot phase and reduce the risk of having it stopped in its tracks. Todd Brasel and Kent McDonald show you how to avoid getting bogged down in the minute details of test results reporting and approach quality measurement from a systems perspective. They offer practical lessons about using metrics to promote quality improvement initiatives to upper managers and executives-the people who make the real decisions about quality. Learn about common traps and problems of measurement programs and how to avoid them.
Do you have a process in place to analyze defects, identify the defect categories and common pitfalls, and correlate the results to recommended corrective actions? Forced to get more done with less, organizations are increasingly finding themselves in need of an effective defect analysis process. David Oddis describes a systematic defect analysis process to optimize your efforts and enable higher quality software development. David’s approach promotes collaboration in the post-deployment retrospectives performed by the development/test teams. Join David as he facilitates an open conversation and provides guidance and tips via a real world walkthrough of the strategy and process he employs to analyze defects. Learn how these findings can lead to opportunities for process improvements in your requirements, design, development, test, and environment domains.
During the past ten years, static analysis tools have become a vital part of software development for many organizations. However, the question arises, "Can we quantify the benefits of static analysis?" William Oliver presents the results of a study performed at Lawrence Livermore National Laboratory to do just that. They measured the cost of finding software defects using formal testing on a system without static analysis; then, they integrated a static analysis tool into the process and, over a period of time, recalculated the cost of finding software defects. Join William as he reveals the results of their study, and discusses the value and benefits of static testing with tools. Learn how commercial and open source analysis tools can perform sophisticated, interprocedural source code analysis over large code bases.
William Oliver, Lawrence Livermore National Laboratory