Agile, DevOps, and What They Mean for Testers: An Interview with Jeffery Payne


In this interview, Jeff Payne, the CEO and founder of Coveros, explains the fundamentals of agile and DevOps and their roles in the world of testing. He gives his thoughts and lessons for how to leverage DevOps and agile to further your career while creating and testing better software.

Jennifer Bonine: Hello, we're back with our virtual conference interviews, and we have Jeff Payne with us. Jeff, thanks for joining us.

Jeff Payne: Very happy to be here.

Jennifer Bonine: Pleasure to talk with you. Let's talk about ... You've been here for a while, so that you're on day four.

Jeff Payne: Sunday, Monday, Tuesday, Wednesday, yes.

Jennifer Bonine: Yes. Sunday, Monday, Tuesday, Wednesday, four days.

Jeff Payne: I'm still speaking. I can still speak, that's the amazing part.

Jennifer Bonine: You're going on day four. Why don't you tell us a little bit about what you're hearing this year at the conference because you've been here the whole time? What are the delegates talking about for those that aren't here? What's the big hot topic or confusion pinpoints slash for everyone this year?

Jeff Payne: One of them is definitely DevOps. Everybody's still talking about DevOps. What does it mean? What is it? How do testers play into it? How do we get started in it? How do we get everybody together? Is it the same thing as Agile? Is it different than Agile? Everybody is talking about that, trying to figure it out. Of course, every vendor is selling DevOps, whatever that means.

Jennifer Bonine: Whatever that means, but they've got something to sell.

Jeff Payne: They're all selling it. Everyone says continuous integration, delivery, DevOps, it's in front of every product name now. Yeah.

Jennifer Bonine: I maybe need to get aware, right? Everyone's selling it, everyone's talking about it. What would you say kind of a simple answer for the folks out there that may be just as confused as some of the people here, where are some good resources? Where is a good place to go to get introduced? I know people throw that term around a lot and it can be amazing. I think I was looking at a number of jobs that are out there that say—

Jeff Payne: For DevOps.

Jennifer Bonine: DevOps. How do people even know what that means or what it is? Your take on that just for them.

Jeff Payne: I have a couple of books I like. Most people who are in DevOps know of the book The Phoenix Project, Gene Kim. That is a great place to start for anybody, because it's not only an informative book, it's written as a story, and that makes it a lot ...

Jennifer Bonine: It's easy to read.

Jeff Payne: Yes. It's quick and easy to read. It describes the problem and how you can solve the problem. Fundamentally, DevOps is a cultural shift; it's a philosophy about getting everybody collaborating, communicating together, and figuring out how do we make our process better. End-to-end in software. I like in it too also just really applying the agile principles of collaboration and communication and getting everybody to work together as a team, just extending that out to operations and getting everybody in the life cycle from requirements all the way through sunset of your apps together working on making it efficient.

Jennifer Bonine: Right. Just broadening that scope, right?

Jeff Payne: That's the way I look at it, yes. Others don't, though. There are those who are going to sit in on what is called agile versus DevOps because we're running into companies now who are saying, "We're done with that. We're done with agile. We're now doing DevOps because that's totally different."

Jennifer Bonine: Right.

Jeff Payne: I'm like, "No, not really."

Jennifer Bonine: Yeah.

Jeff Payne: "No, not quite."

Jennifer Bonine: Right.

Jeff Payne: Now, you can do some things that are considered DevOps without doing agile, but it's really hard to do agile well without DevOps and it's really hard to do DevOps well without agile. They're very related.

Jennifer Bonine: Yeah. Go hand in hand.

Jeff Payne: Exactly.

Jennifer Bonine: At least, like you said, in a lot of new concepts as that come out, like agile anything else, DevOps probably has competing opinions on how to interpret what it is.

Jeff Payne: Absolutely.

Jennifer Bonine: You'll hear different philosophies, buy, yeah, a good point on broadening that scope and the two going hand in hand for you in particular. Some people may have different opinions, but the book The Phoenix Project definitely-

Jeff Payne: It's a great book.

Jennifer Bonine: Great book, easy read.

Jeff Payne: Yes.

Jennifer Bonine: For those of you who hate picking up things that are more text bookish like, definitely not like that at all.

Jeff Payne: Not like that at all. Yes.

Jennifer Bonine: Very relatable, like you said, it's a story, so people tend to remember stories.

Jeff Payne: Yeah, it's an entertaining story too. It's a good story.

Jennifer Bonine: Yeah, it's a good story.

Jeff Payne: It's entertaining.

Jennifer Bonine: Absolutely. Definitely a book to check out for those of you out there watching in virtual land.

Something else interesting that happens here at the conference that people may not be aware of is on Friday, there is all day specialized conferences within a conference that go on.

Jeff Payne: Right.

Jennifer Bonine: There's what's called the leadership summit that takes place on Friday here and you are the chair of that.

Jeff Payne: I am.

Jennifer Bonine: You are responsible.

Jeff Payne: Yes, I am in charge, if my voice is still here.

Jennifer Bonine: You'll be going on day six at that point.

Jeff Payne: That's right, yes.

Jennifer Bonine: Can you tell the folks out there a little bit about what happens at that and how that evolves? I know it's really driven by the participants. Even if you come to a past leadership summit, no two are the same from what I understand.

Jeff Payne: That's right. The summit is real easy. We start out with ... I kick off reception Thursday night, good food, good drink, but as part of that, introduce the speakers. We have a couple of speakers this year. Martin Ringlein and Cindy Gilmore are our speakers.

Jennifer Bonine: Okay.

Jeff Payne: The way we do it is at the reception on Thursday we ask people as a leader what's keeping you up at night? The whole idea behind the leadership summit is to get out of the trenches. We spend all week talking about testing and animation, agile and DevOps and all this stuff.

Jennifer Bonine: Yeah, all the details.

Jeff Payne: What we want to do is emphasize. Let's elevate it up now. Let's talk about the things that leaders are worried about, your people and your process and your boss and your career and growing people and educating and training, all these things above the techy stuff. We ask the participants what keeps you up at night? We take that information from them on cards and then the next day, after Martin and Cindy give some introductory featured talks on leadership from their perspective, we break the organization or break all the participants into tables. There are different topics on those tables and each of the tables works to solve a particular problem that was one of the most important ones that was identified the night before.

Jennifer Bonine: The night before.

Jeff Payne: We go around the speakers and moderators and facilitate and help them come up with the answers. Then by the end, every table presents what they came up with during the summit and then we transcribe it and send it out to everybody so they have a copy too.

Jennifer Bonine: Wow.

Jeff Payne: That's how it works.

Jennifer Bonine: That's awesome.

Jeff Payne: It's great.

Jennifer Bonine: Yeah. I've noticed, like with the conference, the attendance is up, so it's seeming like the same thing with our leadership, the leadership summit and the leadership events that it's drawing more and more participants into those groups and wanting to be a part of that and making partner that.

The other thing, I think, that's interesting that we're seeing as a trend with leaders and with just people at the conference is the continual networking. Do you ever see folks who've come back from a past leadership summit coming back or networking with other folks or keeping that dialogue open beyond even just this week or that day?

Jeff Payne: Yes. In fact, in some past leadership summits, it really depends on if the participants want to do this. We've put out a basket or a bowl of people who want to have an ongoing conversation and engagement to carry the conversations forward. If you're interested, you drop your card in and we created a mailing list and people could continue to collaborate afterwards and talk about how the things they learned they're applying. That's definitely the case. The other nice thing about the leadership summit is if you're listening and you're nearby, you can just sign up for just the summit.

Jennifer Bonine: Oh, that is nice.

Jeff Payne: People can just come for Friday, if they want to check it out ...

Jennifer Bonine: Just one day, yeah.

Jeff Payne: ... but couldn't make the conference. We get some people that just come for the leadership summit as well who couldn't make the conference.

Jennifer Bonine: That's a nice way to do it too. If you say, if you're sitting out there going, I'm too busy, I can't go for a whole week, but I'm really interested in the leadership topics and what people are thinking about around leadership in this space, that would be a good way to get engaged and have an opportunity to participate.

From the topics. The last time you did this would have been at STAR—

Jeff Payne: STARWEST.

Jennifer Bonine: Just to give people some insight into maybe what was keeping people up at night that was months ago, but what were some of the things that were themes? I guess, two part question then, do you anticipate those themes have changed?

Jeff Payne: I actually have all the data. I've been doing this on and off since around 2010. I keep all of the topics. I actually was thinking about writing a paper about what the interest it.

Jennifer Bonine: That would be cool, yeah.

Jeff Payne: There are definitely things that are always ... Half of them I might as well just write right now because ...

Jennifer Bonine: It's going to be this.

Jeff Payne: ... they're going to ask. How do I find good people? Is always on the list. Everybody's trying to always figure out how do I do a better job of recruiting and hiring the right people? Retaining people is always on the list. How do I retain people and keep them around? Some of them are very people-centric. The other one that's always on the list is how do I manage my boss? I'm in leadership, but my boss is a ... I've got to lead and I've got to manage up.

Jennifer Bonine: Right. Very few of us don't have bosses.

Jeff Payne: Exactly.

Jennifer Bonine: Right.

Jeff Payne: Yes. Then, usually there's a little bit of a flavor of a technical concept as in we're moving toward Agile, what does that mean and how do I facilitate change? You see that one a lot. DevOps is starting to pop up. For a while, we heard a lot about how do I manage outsource? How do I manage offshore?

Jennifer Bonine: Not so much.

Jeff Payne: That topic not so much anymore. Either it's been solved with the way it's working or people are doing less of it. I don't really keep track.

Jennifer Bonine: Right, or doing something different, but it's not on there.

Jeff Payne: Some of those things popped up and you saw them for a few years and then they weren't really top of mind any longer.

Jennifer Bonine: That's interesting.

Jeff Payne: It is interesting, yeah.

Jennifer Bonine: Then, what do you think about ... Now we're starting to see more on the millennials and the new folks in the workforce, some of you out there may be considered millennials. Do you anticipate seeing some of the leaders saying, "How do I manage this new generation in the workforce?" We're seeing a lot of questions around how do I manage technical millennials?

Jeff Payne: Sure. Absolutely.

Jennifer Bonine: How do I make this work in my organization? Not only do I have to retain the people I have, but I may be in an industry where people are twenty-five, thirty years in the company, they're getting moved or retire, there's going to be a large gap in the workforce. You've got to bring in a bunch of new people. We may not be in one of those cool industries, we're not Google or Facebook or anything. We're an insurance company in Ohio. How do we get people and then manage those new folks that we've got that are going to be completely different than our current set of workforce?

Jeff Payne: I definitely hear that out there. I guess the way I look at it is every generation goes through this. As you become a manager and a leader, then you deal with the next generation, maybe the one after that. What we try to do at Coveros is find out what people of that age ... what motivates them? What do they care about? The other thing I've learned through my career is different things motivate different people.

Jennifer Bonine: Absolutely.

Jeff Payne: If you just focus on one motivation that might work for some of your people. Other people could care less about it. You almost have to get personal and figure out what motivates individual people? There are also themes that motivate classes or groups of people or generations of people. You've got to understand that and make sure your company is providing the kinds of things that attract those kinds of people.

Jennifer Bonine: Promoting that new ways, right?

Jeff Payne: Exactly.

Jennifer Bonine: The way you maybe used to attract talent is changing.

Jeff Payne: Yes. We do so much now from our recruiting perspective ...

Jennifer Bonine: They're like, where did he get that cell phone from?

Jeff Payne: We're starting to do a lot of social media-based marketing and sales and recruiting because it's the way you attract and get to today's kids coming out of school.

Jennifer Bonine: Right. I wonder, silly for those of us who've been in our careers a while, but remember when people used to put ads in the newspaper?

Jeff Payne: No, I don't remember that, no. I'm sorry, I don't.

Jennifer Bonine: Maybe I'm just old and you're not, Jeff.

Jeff Payne: No, I've never heard of such a thing. What's a newspaper, by the way? What is that?

Jennifer Bonine: Right, exactly. Are those going away? Do we still have those? It's funny to think of those things that used to be traditional mechanisms for attracting workforce.

Jeff Payne: Yeah, gone.

Jennifer Bonine: Gone. Completely gone.

Jeff Payne: Jobs ads.

Jennifer Bonine: What's a job ad? Why would I do that?

Jeff Payne: Clipping it out of paper, right? Pasting it into a little book.

Jennifer Bonine: Mailing in your resume.

Jeff Payne: Yeah, exactly. Yeah. Mailing in your resume.

Jennifer Bonine: The thing of the past.

Jeff Payne: Exactly.

Jennifer Bonine: Yeah, so maybe getting creative. Before we end it ... I was going so fast. Any final thoughts or things you want to share with the folks out there and then also a way to get in touch with you if they have more questions on maybe the DevOps topics or the leadership summit?

Jeff Payne: Yeah. The conference is great. Numbers are up here. There's a lot of buzz here. I've talked to a lot of people about how rewarding it is to come and obviously, talked to other people who are dealing with the same problems you are dealing with, but also solving some of those problems. You can take some of that and go home. I think the conference has been great.

Jennifer Bonine: Yes, the energy here ... Every conference I think has a vibrant energy to it and this one just has some great energy, I think, that's here.

Jeff Payne: Yeah, exactly.

Jennifer Bonine: You can tell, it's just good.

Jeff Payne: It's good. I'll be speaking all week. I'm about to run off and give a talk on agile test management. It should be fun, doing that later today. If you need to get a hold of me, Twitter handle, right?

Jennifer Bonine: Yes, perfect.

Jeff Payne: Social media.

Jennifer Bonine: Social media.

Jeff Payne: My Twitter handle is @JefferyePayne. I tweet a lot, so people can follow. I talk about agile, DevOps, security, all that fun stuff. You can also reach me at

Jennifer Bonine: Right. They'll be able to find you.

Jeff Payne: They'll be able to find me.

Jennifer Bonine: I heard "The king of Payne" is the new nickname.

Jeff Payne: That is really old. Whoever came up with that, that's ...

Jennifer Bonine: I know. That's what they said.

Jeff Payne: I'd like to know who.

Jennifer Bonine: I'll let you know later.

Jeff Payne: Okay.

Jennifer Bonine: Thanks for joining us.

Jeff Payne: They either know me for a long time or they think they invented something new. It's been around for a long time.

Jennifer Bonine: Exactly. Thanks for joining us, Jeff. Thanks for being here.

Jeff Payne: Thank you, it was fun.

Jennifer Bonine: I appreciate it.

Jeff Payne: Bye bye.

Jeff PayneJeffery Payne is CEO and founder of Coveros, Inc., a software company that builds secure software applications using agile methods. Since its inception in 2008, Coveros has become a market leader in secure agile principles and recognized by Inc. magazine as one of the fastest growing private companies in the country. Prior to founding Coveros, Jeffery was chairman of the board, CEO, and co-founder of Cigital, Inc., a market leader in software security consulting. He has published more than thirty papers on software development and testing, and testified before Congress on issues of national importance, including intellectual property rights, cyber terrorism, and software quality.

About the author

Upcoming Events

Sep 22
Oct 13
Apr 27