|
 |
Home > Detail: Strike Three - Time for a New Batter
| A StickyMinds.com Original |
|  | |
Strike Three - Time for a New Batter
By James A. Whittaker
   
Summary: In the past few decades, three “big hitters” have stepped up to the plate of building better software—formal methods, tools, and process improvement initiatives. Unfortunately, they have all struck out, according to James Whittaker in this week’s column. Read on for his critique of these supposed cure-alls and for his call for a new kind of player on the field. | | |
|
|
 |
In the late 1970s the software quality problem was all too apparent. Software was hard to write, hard to maintain, and often failed to meet its requirements. Even at this early stage in the history of software engineering, a move was afoot to correct this problem. Beginning with structured programming, researchers studied better ways to write code. The push for formal methods began. If you’d only do things more formally, the plea went, your code would be better! A few even went so far as to make “zero defects” their Holy Grail. The search was on. Unfortunately, the search still continues.
Following close on the heels of the formal methods advocates were the tool vendors. If you’d only use the right tools, they cried, your code would be better! Many tools have greatly simplified the task of developing software. Development organizations spend hundreds of thousands of dollars on tools. Software is still buggy.
The latest entry into the fray has been the process improvement people. If you’d only work more carefully, they beg, your code would be better! Now our managers are as busy as our developers. Developers not only have software to develop but also make-work reports to write. Software is still buggy.
This article discusses these three “silver bullets” and exposes their Achilles’ heels. I suggest that the answer to the problem must, by definition, be technical—not managerial—in nature. I end by launching the search for a fourth proposal.
Formal Methods
Formal methods are a great idea. Essentially, they equate programming a computer to solving a math problem. You can go about it using a mixture of creativity, intelligence, and lots of practice solving similar problems. However, there are a couple of problems with formal methods that can’t be overlooked.
First and foremost, software developers won’t use them. This puzzles formal methods advocates to no end. However, it’s fairly plain to the rest of us: no one can show developers just how to apply formal methods. The examples in books and papers are far too simple to scale the ideas to real development problems. Plus, once you get outside your comfort zone, the formal methods fall apart. You remember how you felt when you learned calculus after mastering algebra? Algebra problems were easy because you’d solved hundreds of them. But the methods you used didn’t seem to apply to calculus problems. All the rules had changed. It turns out this same situation plagues software problems too. They can be as different as algebra and calculus. Why should we be surprised that what works on one problem is useless on another? Formal methods don’t scale in software because they don’t scale in mathematics either.
Second, one can use formal methods and still write buggy code. Formal methods don’t address anything but the algorithm. But we all know that an algorithm can be correct on paper but fail on a real computer. The problem is that real computers have space and time limitations, other applications, and very complex operating systems that must be dealt with by code that has nothing to do with the main algorithms of an application. Indeed, code to process inputs and handle errors is often much larger and more complex than the algorithm that actually gets the main work done. There are no formal methods for handling such code.
Formal methods are important, but they will only take you so far toward reliable software. Strike one.
Tools
Tools can make the software development task much less painful but they cannot guarantee zero defects; in fact, they cannot even guarantee fewer bugs. Since the tools themselves can be buggy, they create one more unknown in a project. When a defect is found, is the product or the tool at fault?
Tools range from simple and indispensable text editors and compilers to more elaborate environments for analysis and design. Very few outrageous claims are made from the developers of editors and compilers—it’s the design tool vendors that corner that market. What’s more valuable to a project anyhow, a nicely drawn E-R diagram or a programmer who’s expert in the target implementation language? Would you buy $100K worth of tools or hire a person who intimately understands the problem domain in which you are involved? Tools are great when used properly, but they can only offer you steep learning curves and limited functionality. Plus, they bring along a whole new set of bugs to worry about: their own.
You see, if tools really were a silver bullet, they wouldn’t be buggy, would they? Strike two.
Process Improvement
The latest attempt at getting the software quality problem under control has been made by the process improvement people. Obviously, controlling and improving the software development process is in everyone’s best interest. However, since software development is a technical problem and process improvement is a management problem, it simply cannot have a profound effect on quality. Good organizations can produce bad software. Bad organizations can produce good software.
Furthermore, process improvement initiatives are rarely met with enthusiasm from rank-and-file technical people. ISO certification is a pain. SEI assessment is demeaning. Both take away creativity and add management overhead. Heck, part of the joy of working in this field is not being micro-managed. Why would any developer in his or her right mind actually think this is a good idea?
Well, it is a good idea, but it won’t help appreciably with the quality problem. I was once a part of a partnership in which a consulting company that specialized in formal methods was training an SEI level three—on their way to level five—organization. An example code fragment was used extensively throughout the training. Neither the formal methods advocates who wrote the buggy code, nor the mature process organization that studied the code, noticed that it possessed a fatal flaw. Not even during the course of proving the code fragment correct was the bug ever discovered. Why? Because the formal methods people were concerned about math and the process people were concerned about documentation. No one was looking at the code! Fortunately, the test organization was paying attention and the bug was caught.
Management solutions cannot solve technical problems. Strike three.
The Fourth Proposal
What we need is for someone to come up with silver bullet number four. Except this one shouldn’t be silver. In fact, I think it’s important that proposal number four should be bland-colored, say brown, so that no one really notices it. It shouldn’t be something so revolutionary (as a gold or platinum bullet might be) that it makes no sense and people avoid it. It should be so ordinary that developers can integrate it seamlessly into their normal pattern of design. It should be so straightforward that developers remark “this is simple, what’s the big deal?” Not only should it be these things, it must be in order for it to be used and have some positive industry impact. Otherwise, it’s just more ivory-tower nonsense that real practitioners don’t appreciate.
It turns out that parts of such technology exist, are readily understandable by capable developers, and will not fundamentally change the manner in which software development occurs in an organization. If you are a great developer now, then you’ll still be a great developer. (This is a fear of great developers that keeps them from being strong process advocates. They know they are not great at filling out forms.) If you are a mediocre developer, then perhaps you’ll become better. Either way, it is likely that the code you write will be more understandable and have fewer bugs than it did before. In future installments of this bimonthly column series, I will survey many of the techniques that will one day contribute to the fourth, this time brown, bullet and show how developers and testers can adjust—but not change—their development process to improve quality and maintainability.
It’s time for a new batter.
About the Author
Dr. James A. Whittaker is a professor of computer science and
the director of the Center for Information Assurance at Florida Tech. He is author of How To Break Software (Addison-Wesley) and co-author of an upcoming book, How to Break Software Security. His group focuses on highly efficient test techniques and tools for penetration testing. Contact him at jw@cs.fit.edu.
Back to Top
StickyMinds.com Weekly Column From 1/13/03
|
|
|
 | RESOLVE SUPPORT ISSUES from your Desktop!
Minimize downtime with a remote support solution that lets you resolve issues right from the desktop
| | See how EASY REMOTE SUPPORT can be. Try WebEx FREE!
DELIVER SUPPORT MORE EFFICIENTLY. Remotely Control Applications. Leap Securely through Firewalls!
| | SOLVE SUPPORT ISSUES on the First Call!
REMOTELY CONTROL AND CONFIGURE SYSTEMS. Easily install applications, updates. All from your Desktop!
| | Census: Web-based Bug Tracking and Defect Tracking
Track software bugs, defects, enhancements, support calls, and more. Issue tracking software that is scaleable, fully customizable and integrated with VSS. Includes e-mail notifications, role-based workflow, change history, and Crystal reporting.
| | TAKE CONTROL OF REMOTE COMPUTERS
Support, configure and install applications and updates remotely for greater efficiency.
|
| | Get your product or service listed here.
| |
|
|
|
|
|
