Agile Software Development – Past, Present, Future

Almost ten years after the Agile Manifesto was published, Russell Pannone reflects on the past, present, and future of agile software development.

I’ve come to realize system-software development methodologies have in common with nature the life span of infancy, childhood, adolescence, adulthood, and aging. I have also come to realize methodologies, sometimes as part of their life span, enter into a relationship or marriage with other methodologies; like has happened with Agile Software Development and Lean Manufacturing.

Additionally, methodologies of the past and present have an associated taxonomy, new jargon and technical terminology or idiomatic expressions of the practitioner. They also tend to reuse old jargon but with different connotations.

Just like the seventeen like minded individuals did in 2001, when they came up with the Manifesto for Agile Software Development, it is time to reflect on today’s reality, specific to the fundamental practices underlining our industry’s and your agile/lean product (system-software) development approach and look to the future.

“You can clutch the past so tightly to your chest that it leaves your arms too full to embrace the present.” - Jan Glidewell

We use to have the pony express delivering letters helping to keep folks in touch with one another. Then came the telegraph and telephone. Now we've gone back to letter writing, in the form of email, text messaging, or Twitter to keep in touch.

Let’s not dwell on the past, but reflecting on our past and present creates a better future.

To help you reflect, ask yourself a few questions and give honest objective answers to them. You will then be able to see whether or not you have progressed the way you wanted. The 5 Whys is a simple problem-solving technique that you can use. Made popular in the 1970s by the Toyota Production System, the 5 Whys strategy involves looking at any problem and asking: "Why?" and "What caused this problem?"

Very often, the answer to the first "why" will prompt another "why" and the answer to the second "why" will prompt another and so on; hence the name the 5 Whys strategy.

Following is an example of the 5 Whys analysis applied as an effective reflective technique in the world of being agile when applying the Scrum framework:

    1. Why is our Product Owner, unhappy? Because we did not deliver the product release when we said we would.
    2. Why were we unable to meet the agreed-upon timeline or schedule for delivery? The stories took much longer to develop than we thought they would.
    3. Why did it take so much longer? Because we underestimated the complexity and uncertainty of the stories.
    4. Why did we underestimate the complexity and uncertainty? Because we did not have acceptance criteria associate with each story, our stories were not at the right level of detail and we did not realistically size each story prior to committing to a schedule for delivery.
    5. Why didn't we do this? Because the higher powers were still working under the traditional waterfall planning mental-model, as a result we felt pressure to work faster and skipped steps. We clearly need to review our approach to writing stories, sizing stories, estimating duration and committing to a schedule for delivery and then get buy-in and visible support from the higher powers.

“The present is never our goal: the past and present are our means: the future alone is our goal.” – Blaise Pascal

In the span of the last 35 years or so the following system-software development methodologies have been widely practiced and evangelized:

    • Structured Programming
    • Structured System Analysis and Design
    • James Martin’s Information Engineering, popularizing visual modeling in the form of data modeling and process modeling
    • Object Oriented Analysis, Design and Programming
    • Extreme Programming
    • The Unified Software Development Process
    • Rational Unified Process
    • Agile Software Development
    • Agile Software Development with Scrum
    • Agile Software Development with Kanban
    • Lean Software Development

As depicted in Figure 1.0, system-software development has also come a long way, but a long way to where?


Figure 1.0 – SDLC 3.0[1]

Today, we have a multitude of variant agile software development approaches:

    • Extreme Programming
    • Agile with Scrum
    • Crystal
    • OpenUP
    • Agile with Kanban
    • Agile/Lean with or without Scrum or Kanban

Additionally, we have thousands of books and articles, yes like this one, written on the subject.

“When it comes to the future, there are three kinds of people: those who let it happen, those who make it happen, and those who wonder what happened.” - John M. Richardson, Jr.

Looking to the future, ask yourself what is working best for you and take control of your adoption of agile/lean product (system-software) development through frequent introspection and continuously adapting to your reality, both iteratively and incrementally.

The what, why, and how of agile/lean product (system-software) development and delivery is not one persons vision alone; to become reality it needs to be a "shared" vision through negotiation and compromise between individuals, the team and the organization.

Next, adopt an agile/lean approach that collaboratively and adaptively promotes developing value-added system-software product increments in a continuous flow from requirements to deployment.

Then, establish a set of norms around your adoption. Here is an example sub-set of norms, when “being” agile using Scrum.

Attributes of the Product Backlog:

    • Stories
    • Priority
    • Story Size (The unit of measure used for story size, at the Product Backlog level, needs to be consistent across all development teams in your organization)

Attributes of the Sprint Backlog

    • Stories
    • Tasks
    • Priority and dependency
    • Level-of-effort (The unit of measure used for level-of-effort, at the Sprint Backlog level, is consistent across all development team in your organization)
    • Definition of "done"
    • Unit tested
    • System (Functional and Regression) tested
    • Integration (End-to-End) tested
    • User Acceptance tested

A team’s velocity from Sprint to Sprint is used to highlight the trend of a team’s ability over time to deliver stories and the point-in-time commercial or operational value was delivered

    • Velocity is not to be used to compare one teams rate of getting stories done over another team’s rate

It is worth noting that a set of norms, to the agile/lean product (system-software) development team, is like sheet music to a group of musicians. Recognizing, the more familiar the musicians are with the musical score and the more experience they have playing together, the less dependent the musicians are on the sheet music; except when introducing new musicians to the musical ensemble. This metaphor is applicable to an agile/lean product (system-software) development team and it’s set of norms.

Word to the wise - we are climbing a slippery slope when setting norms. We need to be keenly aware your norms should not be rigid or prescriptive and should evolve through collaboration between self-organizing and self-directing cross-functional teams based on reality.

Norm setting can only work if the team is truly able to arrive at consensus. Norms won’t stick if members have reservations about them. However, once consensus is reached, the team is equipped with a guide that can serve to strengthen positive practices. A set of norms can serve as a common reference if contrary behaviors arise. Finally, written norms are handy for potential members and newcomers who want to quickly get a sense of the team’s adoption of being agile.

Norms in hand, a team can move forward inspired and motivated to uphold the team’s approach and confident in the security such guidelines provide.

Moving forward in today’s reality, I recommend the following as a manifesto for the marriage of agile and lean product (system-software) development:

    • Avoiding the high cost of discovering defects late in the development cycle by discovering defects early in the development cycle which is accomplished by eliminating waste, increasing feedback loops and developing code from the point of view of provability and outside-in design
    • Cross-functional, collaborative and adaptive teams developing and delivering value-added product (system-software) increments in a continuous flow from requirements to deployment
    • Emphasis is placed on the need for teams to nurture group cohesion, and paying attention to norms that serve as a guide that strengthens positive practices
    • Minimizing frustration levels and making the art and science of system-software development enjoyable and not a burden or death march
    • Delivery of commercial or operational value early and often, giving ourselves the best opportunity to beat the competition to market, realize revenue and discover insights that we can use to help us improve
    • The what, why, and how of agile/lean product (system-software) development and delivery is not one persons vision alone; to become reality it needs to be a "shared" vision through negotiation and compromise between individuals, the team and the organization


[1] Used by permission from Fourth Medium Consulting -

About the Author
Russell Pannone is the Founder of We Be Agile and the Agile Lean Phoenix User Group, as well as the Agile-Lean Adoption Lead. With almost 30 years of system-software development and delivery experience, my focus is on working side-by-side with folks on real projects helping them deliver valuable system-software to production early and often, giving those I collaborate with the best opportunity to beat the competition to market, realize revenue and discover insights that we can use to help us improve.

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.