The Impact of Quality-Driven Development

[article]
Summary:
When the development and QA teams work independently of each other, there can be some duplication of test efforts—which results in wasted time. The solution: quality-driven development, with QA-implemented automation run in the development environment. This is the story of one team's venture into this new process.

My team had everything in place for continuous quality improvement: behavior-driven development, unit tests, and integration tests. These tests are typically run by developers while they write the code to make sure the code performs as expected. Yet when the code gets to me in QA, I still find myself running the same tests and finding issues.

The behavior-driven development (BDD) examples are written during or after testing, so the bugs haven’t been caught yet when the code reaches me. The unit tests tend to focus on low-level components, allowing defects from the interactions of components to slip through. And the integration tests use the REST APIs and can’t catch visual problems or problems with JavaScript calling the REST APIs.

Because the automated checks don’t verify these things, a new build could introduce new problems—which creates a need to retest functionality. We realized we had to raise the quality of the product as it is being developed.

Introducing Quality-Driven Development

My organization implemented the idea of quality-driven development (QDD) with the promise of eliminating the test duplication effort. This process includes some new measures:

  • Automated tests created by QA run in the development environment.
  • We fix issues on the fly. If a tester finds a problem, she can explain it to a programmer and get it fixed. That means if a tester finds a bug in a story, the story is not done. There is no need to file a ticket and get it assigned to someone later; the story is not done. Just fix it.
  • We run manual test charters only once.
  • No story is marked done until the automation itself can demo the feature, driving the browser and showing the product owner that the intended use case can actually happen.

It sounds wonderful—at least on paper. Now let’s get real.

The Results of Our Experiment

Initially, I had lots of questions about this approach. How feasible is it to run QA-implemented automation in the development environment? Would it be more efficient to just have automation set up after the development effort? This approach seemed time-consuming, especially when the developer was unable to call a task done until the automated tests ran successfully. It was clear to me from the outset that the development and QA teams would need to completely buy into the new process.

Our organization also introduced a new automation tool to automate browser interactions and our own quality management software. This imposed a learning curve for me, as I not only had to automate new features, but also was required to learn the tool and its capabilities along the way. This caused my effort estimation to be way higher during sprint planning. Initial automation scripts and testing took awhile, but in some instances I finished earlier than planned. This got me questioning whether I was doing things right in some cases versus others.

Still, I was confident that our efforts would all eventually pay off. Any new process will have its struggles. The challenge lies in how we handle these hurdles and progress toward the goal.

First, I needed to learn the automation tool. I reached out to a colleague with several years of experience who guided me through the process and ironed out a solid way of using the tool effectively. With his help, I was able to streamline the automation tasks.

At first, it was a struggle to keep up with the automation and complete when coding is complete. It also became clear very quickly that if unplanned changes came about, the existing automation framework would not function optimally. I really had to

User Comments

4 comments
Andrew Sieffert's picture

Great article Praveena.  I am curious to hear how difficult it was to implement BDD, especially in its early stages.  My guess is QDD wouldnt have been possible without first becoming quite adept at BDD.  Our development organization is currently new to Scrum and we are working through some of the growing pains as part of that transition.  BDD isnt even on the radar for us but I was curious if you have any suggestions, should we adopt it early, should we grow into as we familiarize ourselves with Scrum?  Your input would be greatly appreciated.

December 1, 2015 - 6:03pm
Praveena Ramakrishnan's picture

Hi Andrew,

It is a good idea to start thinking about BDD early on as we grow with Scrum. It gives time for the team to Adopt to BDD as it quickly becomes a part of weekly sprints. In our early stages, the development team implemented some of our tests using cucumber and Selenium webdriver and then it was demo-ed to a broader team. Like any other development needs, we had to think about laying down the framework for the automation and then building on top of it. 

 

 We did hit roadblocks- 

 Every sprint, we focused more on functional development and kept BDD at the end. BDD never got picked up for that sprint and eventually ended up being pushed out to the next sprint. Before we knew it, we had accumulated a BDD tests backlog that needed to be complete. Eventually, we got a grip on the automation, and QA also started to contribute to implementing BDD thereby reducing the backlog. 

 

 BDD tests are now integrated as part of the Jenkins build process. We are now at a point where we don't deploy if the Jenkins build fails for BDD tests! 

 

 Hope that helped.

-Praveena

December 2, 2015 - 8:29pm
Praveena Ramakrishnan's picture

It is a good idea to start thinking about BDD early on as we grow with Scrum. It gives time for the team to Adopt to BDD as it quickly becomes a part of weekly sprints. In our early stages, the development team implemented some of our tests using cucumber and Selenium webdriver and then it was demo-ed to a broader team. Like any other development needs, we had to think about laying down the framework for the automation and then building on top of it.

We did hit roadblocks-
Every sprint, we focused more on functional development and kept BDD at the end. BDD never got picked up for that sprint and eventually ended up being pushed out to the next sprint. Before we knew it, we had accumulated a BDD tests backlog that needed to be complete. Eventually, we got a grip on the automation, and QA also started to contribute to implementing BDD thereby reducing the backlog.
 
BDD tests are now integrated as part of the Jenkins build process. We are now at a point where we don't deploy if the Jenkins build fails for BDD tests!
 
Hope that helped!

December 3, 2015 - 10:50am
Steve Martino's picture

What tool did you choose to use to build you automation framework?

December 23, 2015 - 9:18am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.