Behavior-Driven Development and Automation: Establishing Order

[article]
Summary:

Though automation is often mentioned in the same breath as behavior-driven development, they are not equally important. If you want to use behavior-driven development, do just that: Work on getting the approach right, and forget about the automation at first. Here's why you should think of automation as more of a bonus to the BDD process.

My colleague Greg Paskal recently sent a message with the subject line “Behavior-Driven Development - Legit or Fantasy.” In the email, Greg posed several great questions and concerns regarding BDD. Before I had time to get my thoughts together on the matter, my inbox was full of replies. I noticed that much of the conversation was automation-centric. This struck me because I’d recently had other conversations regarding BDD and automation.

Those points helped form my contribution to the conversation, and I’d like to share it with you here.

Putting the Cart before the Horse

From a Scrum point of view, I think of a story as a placeholder for a conversation. If we can agree with that as a good enough definition, then we can see behavior-driven development or acceptance test–driven development as a framework to help teams have that conversation.

The general idea is that during the conversation, the cross-functional Scrum team, including the ScrumMaster, product owner, and the technical staff, gains some sort of shared understanding of what the story is supposed to produce. The BDD specifications (notice I didn't say test cases) become examples: testable artifacts that are the result of the shared understanding. The fact that these specifications are testable lends them to being treated as test cases, for better or for worse. It doesn't help the case much that often these development frameworks are lumped into a category called "test-first approaches."

Now that we have these testable things, if they are written down, can't we parse them and automate everything? Absolutely. Thus, you have BDD tools such as Cucumber, SpecFlow, and Fit/FitNesse.

The biggest flaw I see in all of this is people saying things like, "Oh, yeah, we're doing BDD, so we're going to use Cucumber." The fact that we can specify the conversation results (i.e., the BDD specifications) in a way that can be parsed, and that we have additional software to check the product against the specifications, is thought of as merely interesting—it’s a bonus, a convenience, that we can do that checking via a machine. The result of the conversations is not automation. The result is a shared understanding of the stories; the specifications are the artifacts of that shared understanding.

Here's the dirty little secret: The automation is not the hard part; the changing of your whole cross-functional team to be "test-first" is. Most teams aren't set up for this. Here are some common problems:

  • Developers don't want to wait until there are test cases written before writing any code
  • Testers are not used to being at the front of the pack, so it takes time for them to acclimate to that role
  • Product owners, product managers, and other assorted business people are not as available as they should be, so they can't or don't participate in the conversation as much as is required

This change in mindset is the hard stuff.

If you want to take a BDD approach, do just that: Work on getting the approach right, and forget about the automation. If you can't get the approach and corresponding culture right, BDD will fail, either partially or completely.

For BDD, the conversations are the foundation; we need them in order to even come close to building the correct thing. Once they are in place, then we can talk about the bonus that is automated checking of the specifications.

Frameworks Shaping Script Creation

Yes, you can use BDD-esque tools outside a BDD approach. I used to have a different mindset on this: I thought these tools added an unnecessary layer of complexity if you weren’t doing BDD, unless you had nonprogrammers participating in creating automation script. I've become more moderate on that stance.

Before that email conversation about BDD and automation, I had talked with another colleague, Chelsea Lovas, about her experience creating scripts in different types of frameworks. She used to use the automation framework we created at a previous company that was based on pure script writing. She now uses the framework we have at our current company, which uses SpecFlow as a layer we can leverage to automate BDD specifications. She says she can create scripts far faster using the current framework as opposed to the previous one.

Part of the speed increase is due to VisualStudio’s IntelliSense, as well as a more refined approach to page objects. Nevertheless, Chelsea says SpecFlow gives her some reusability that was missing in our other framework. She also finds that the shared "steps" in SpecFlow have helped reduce maintenance effort. Her team does some BDD-like work, but many of the scripts are created after the code is written. All in all, it's hard to argue with results.

If you take one thing away from my thoughts, I hope it's this: Behavior-driven development has nothing to do with automation; the fact that automation is derivable from BDD specifications is a convenience, not the primary focus.

User Comments

7 comments
Ken Pugh's picture

Paul,

Your approach matches the one I advocate.   I talk about ATDD/BDD starting with communication, not automation. The issues you describe are some of the ones I've found at many companies. 

Thanks for your article.

Ken Pugh

Author, Lean-Agile Acceptance Test-Driven Development: Better Software through Collaboration

 

May 3, 2016 - 11:36am
Darren Davie's picture

Great article it speaks for itself. No more to add.

May 3, 2016 - 6:16pm
Paul Grizzaffi's picture

Thanks Ken and Darren. I appreciate the feedback and I'm glad you found the article valuable. All the best to you.

May 20, 2016 - 9:16am
venkatesh Malepati's picture

 

I went through the article regarding BDD and  your thoughts regarding Test first approach. Every sentence in your article so true. BDD involves the whole team to define proper specifications and come up with an agreement on specifications. From my experience, developers  will not understand test first strategy. In many organizations they don't impose developers to write  unit tests. Most developers think that ,  developing unit tests are also part of QA team. Thank you Paul for sharing lot of good information.

August 8, 2016 - 9:35pm
venkatesh Malepati's picture

 

I went through the article regarding BDD and  your thoughts regarding Test first approach. Every sentence in your article so true. BDD involves the whole team to define proper specifications and come up with an agreement on specifications. From my experience, developers  will not understand test first strategy. In many organizations they don't impose developers to write  unit tests. Most developers think that ,  developing unit tests are also part of QA team. Thank you Paul for sharing lot of good information.

August 8, 2016 - 9:35pm
Tim Walker's picture

It was more than 15 years ago that a colleague suggested that "Executable Specifications" were the "holy grail" for iterative development. I believe that more today than I ever have. That thought led to so many trainings and discussions and "Executable Requirements with FIT and FitNesse". The current state-of-the-art is Cucumber/Gherkin (and a few similar). In delivering and coaching teams to many successful deliveries with Test First/CI/CD/Devops, including the federal government, utility companies, geospatial and more, I believe this to be the hook from which the rest of the Agile Quality approaches hang. Great article! Thank you. 

June 21, 2017 - 3:13pm
Paul Grizzaffi's picture

Thanks Tim! I appreicate you taking time to respond and Im glad you liked the article! It's not been on my radar quite 15 years, but I did do some time on FIT and FitNesse as well.

June 21, 2017 - 11:09pm

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.