The Missing Ingredient
Deming's core message was that we should stop inspecting defects out of products and start building quality in. Obviously preventing defects or finding them when they are cheapest to fix is preferable to finding them all at the end when they are many, many times more expensive to fix. Yet few software development projects write tests up front, do inspections, or frequent integration despite the benefits.
Why not? Because it is hard work in most environments. It's hard work trying to inspect hundreds of pages of documents. It's hard work trying to write tests for many pages of requirements at the beginning of a project. It is even harder to keep the tests up to date as the requirements change. It's harder still when you realize that you have to inspect the tests.
So what was different in manufacturing environment that allowed manufacturers to adopt an equivalent processes?
The primary difference is that when the Japanese manufacturers adopted Deming's quality practices and philosophy they also adopted what are now known as the lean manufacturing techniques. Lean factories have much less work-in-process compared to traditional factories and this allows them to spot defects much sooner. For instance, compare a factory where it takes three months from when material is released until the product is shipped with a factory where it takes only two days. If a systemic defect occurs in the first process, then it will take three months for the defect to be found in the first factory, but only two days in the second. At the point that the defect is found every product within the factory will have the defect and will need to be scrapped or reworked. The cost of rework and scrap is significantly lower in the factory with lower work-in-process (WIP).
Lean factories achieve much lower levels of WIP by working in smaller batches of work. The equivalent in software development is working in small increments as used by the Agile approaches. It is much easier write tests up front and to automate them when you are working on a much smaller set of functionality at any point in time. When working this way, defects are discovered much sooner, when they are easier to fix, the quality of the application is more transparent, and the odds of delivering on time much higher.
"Balancing Agility and Discipline: A Guide for the Perplexed" by Barry Boehm and Richard Turner.