analysis

Conference Presentations

Test Metrics in a CMMI Level 5 Organization

As a CMMI® Level 5 company, Motorola Global Software Group is heavily involved in software verification and validation activities. Shalini Aiyaroo, senior software engineer at Motorola, shows how tracking specific testing metrics can serve as key indicators of the health of testing and how these metrics can be used to improve your testing practices. Find out how to track and measure phase screening effectiveness, fault density, and test execution productivity. Shalini describes the use of Software Reliability Engineering (SRE) and fault prediction models to measure test effectiveness and take corrective actions. By performing orthogonal defect classification (ODC) and escaped defect analysis, the group has found ways to improve test coverage.

CMMI® is a registered trademark of Carnegie Mellon University.

  • Structured approach to outsource testing
Shalini Aiyaroo, Motorola Malaysia Sdn. Bhd
Inside The Masters' Mind: Describing the Tester's Art

Exploratory testing is both a craft and a science. It requires intuition and critical thinking. Traditional scripted test cases usually require much less practice and thinking, which is perhaps why, in comparison, exploratory testing is often seen as "sloppy," "random," and "unstructured." How, then, do so many software projects routinely rely on it as an approach for finding some of its most severe bugs? If one reason is because it lets testers use their intuition and skill, then we should not only study how that intuition and skill is executed, but also how it can be cultivated and taught to others as a martial art. Indeed, that's what has been happening for many years, but only recently have there been major discoveries about how an exploratory tester works and a new effort by exploratory testing practitioners and enthusiasts to create a vocabulary.

Jon Bach, Quardev Laboratories
Your Development and Testing Processes Are Defective

Verification at the end of a software development cycle is a very good thing. However, if verification routinely finds important defects, then something is wrong with your process. A process that allows defects to build up-only to be found and corrected later-is a process filled with waste. Processes which create long list of defects are . . . defective processes. A quality process builds quality into the software at every step of development, so that defect tracking systems become obsolete and verification becomes a formality. Impossible? Not at all. Lean companies have learned how wasteful defects and queues can be and attack them with a zero tolerance policy that creates outstanding levels of quality, speed, and low cost-all at the same time. Join Mary Poppendieck to learn how your organization can become leaner.

Mary Poppendieck, Poppendieck LLC
Let's End the Defect Report-Fix-Check-Rework-Cycle

Find out how teams transitioning to Agile practices must re-think their workflows and project metrics originally designed to handle many hundreds of defect reports that occur in typical testlast development cycles. Richard Leavitt discusses how a real-world implementation of key practices like early testing and continual integration-though not without bumps and bruises-lowered the number of open defect reports by an order of magnitude. These practices also can improve how the team communicates, reduce delays, and provide more direct measures of project status, feature progress, and release readiness.

Richard Leavitt, Rally Software Development
The Next Stop in Test Automation: Test Environment Setup

To achieve the most effective test automation, you need to go beyond automated test case creation and implement automated environment setup. In tracking the testing time spent over several projects, the Clustering group Amit Mathur worked with found that more staff time was being spent on setup than on actual testing. He discusses and demonstrates the Environment Configuration System (ECS) his group developed to automatically set up test environments and trigger automated test execution. Find out how ECS has dramatically improved the ability to automate many more manual tests. ECS employs a scripting language (Perl) and libraries for environment setup and test cases. Based on interface specifications and dependency rules, ECS bundles environment setups and test cases for complex test scenarios. Learn how you can adapt the ECS approach for your applications and environment.

Amit Mathur, VERITAS Software Corporation
Systematic Techniques for Fault Detection and Isolation

Selecting the appropriate testing techniques and test cases improves test efficiency, reduces time to market, and gives you confidence that the system is ready to ship. Using real-world case studies as examples, Madhav Phadke explains the fundamentals of robust test case selection and how code coverage can improve your test results. He discusses ways for testers to support debugging and faster repairs by isolating defects to a specific part of the software. Learn to select test outputs based on "total function evaluation" rather than end customer outputs and ways to use orthogonal arrays for testing combinations of parameters. Take away a list of free or inexpensive tools that can speed up your testing process.

Madhav Phadke, Phadke Associates
Risk: The Testers Favorite Four Letter Word

Identifying risk is important-but managing risk is vital. Good project managers speak the language of risk, and their understanding of risk guides important decisions. Testers can contribute to an organization's decision making ability by speaking that same language. Learn from Julie Gardiner how to evaluate risk in both quantitative and qualitative ways. Julie will discuss how to deal with some of the misconceptions managers have about risk-based testing including: Testing is always risk-based. Risk-based testing is nothing more than prioritizing tests. Risk-based testing is a one-time-only activity. Risk-based testing is a waste of time. And risk-based testing will delay the project.

Julie Gardiner, QST Consultants Ltd.
Lipstick on a Pig - How Illusion Leads to Crisis in Real World Projects

Change, ambiguity, and risk are key issues whether you are running a software project, managing a development team, or leading an entire organization. We learn it over and over again. It's not a matter of "if" change will happen-it's a matter of "when." When a crisis inevitably arrives, how do you respond? As Jerry Weinberg observed in The Secrets of Consulting, "It may look like a crisis, but it's only the end of an illusion." Andy Kaufman looks at key project illusions that threaten success as we lead projects and people in the realm of software development. Whether you're a project team member or a senior executive, Andy provides practical tips you can immediately apply in your organization.

Andy Kaufman, Institute for Leadership Excellence and Development
The QA/Testing Perspective on Software Security

Most everyone now realizes that we cannot solve security vulnerabilities with firewalls, virus scanners, and other tactics that build an electronic “moat” around systems. According to Julian Harty, security is not an operational issue, not a developer issue, and not a testing issue. It is a systems issue that you must focus on throughout the software’s life. From a QA/testing perspective, we need to look early in the development process for adequate security requirements. Then, we should assess the designs for vulnerabilities and participate in security code reviews. When specialized, security tests find bugs that get past our early prevention efforts, causal analysis helps prevent the recurring security defects. Dig into system security issues with Julian and learn about manual techniques, commercial software, and home-brew automation tools to help you find security vulnerabilities-before the bad guys do.

Julian Harty, Commercetest Limited
Service-Oriented Architecture - Exposed

Service-Oriented Architecture (SOA), incorporating methods for Web services to communicate dynamically, promises to significantly improve organizational operating efficiency, change the way companies conduct business, and even alter the competitive landscape. However, Service-Oriented Architecture is a strategy rather than an objective, and, like any strategy, it is of no value unless it is implemented. With illustrations from companies who today are using SOA to transform their organizations, Sharon Fay shares current practices for exposing Web services and XML to internal development teams, outsourced development, external trading partners, and customers. Learn why reuse is a key method for supporting integration of SOA implementations and how it is being accomplished. Take away a set of metrics that you can use to measure the level of SOA adoption, development productivity gains, and organizational agility.

Sharon Fay, Flashline, Inc.

Pages

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.