In my article, Agile Removes Limitations—You Must Now Change the Rules, I proposed that
- Process improvements, such as agile, remove limitations that existed in the past, like the inability to deal with change and uncertainty
- Based on Goldratt's teaching, in order to get holistic effects from such improvements, we must:
- Determine which limitations no longer exist
- Examine the rules we put in place to deal with these old limitations
- Remove or change these rules to deal only with new or remaining limitations
I believe that many organizations attempting to introduce process improvements of any kind fail to execute the second point. The process improvement is introduced but existing structures still assume old limitations. The result is either less dramatic improvement than we'd like or reversion to the old ways of doing things. This kind of "force fitting" agile into existing organizational processes will certainly hamper your results.
Let's now apply this approach to two common challenges in software product development: compliance and audit processes. Many organizations, especially large companies and those operating in heavily regulated industries, have compliance processes in place that must be met in order to approve software for release. These processes often require activities that appear to conflict with all we have learned about agile, for example, requiring waterfall-like sequential flow of heavy documentation artifacts through the life of a feature request.
Generally speaking, compliance is about managing risk. In businesses that face more risk than others—e.g., those developing medical device software upon which peoples' lives depend or software that carries millions of dollars of financial transactions form one place to another—a process, similar to the one below, is used to help manage risk.
- Identify and assess risks
- Establish controls that attempt to mitigate these risks
- Monitor and assess the implementation and effectiveness of the controls
- Communicate results
- Remediate gaps or observations
Audits generally are the implementation of steps three and four. Auditors share an opinion on the controls we establish and whether we are actually implementing them effectively. They generally do not specify what the controls are or which activities need be taken to implement them. The business establishes controls, and auditors basically test whether controls are being implemented.
Is There Already a Problem?
When I started to learn about compliance processes, I immediately recognized a common pattern. Despite the compliance steps above being cyclic, most of the compliance processes I have seen in action seem only to repeat steps three through five. That is, once risk was identified and controls were established, they were pretty much set in stone and rarely changed. At this point, I began to realize that there was nothing in principle conflicting between agile and compliance processes. In implementation, however, compliance and audit can appear as an obstacle to agility, sometimes even requiring that people create artifacts for no other reason than to complete a compliance controls checklist. Though this might satisfy the needs of the audit, it does not do the business a service—money is being spent on an effort that probably isn't actually reducing risk.