Steve Berczuk is a regular contributor to TechWell and StickyMinds and a principal engineer and ScrumMaster at Fitbit in Boston. In this interview, Steve discusses configuration management and agile, helpful tools, and how testing has evolved over the years with the rise of agile.

Jonathan Vanian: I'm here today with Steve Berczuk. Steve, thank you for taking time to chat with us today. You're very familiar around these parts. You frequently write great stories for us on TechWell and on StickyMinds. Let's start out by having you talk about your career thus far for some of our readers and listeners.

Steve Berczuk: Okay, so I started off and went to school. I got an electrical engineering degree and segued into a job doing quality assurance for a mainframe software company. I decided that that wasn't really what I enjoyed doing. Really lucked out and got involved with a group at Eastman Kodak of all places, which was doing some really cool stuff when I started working.

I got a job at Kodak where people really understood what they were doing. They understood software development really well and they understood configuration management. They understood working with co-distributed teams.   

After that, you know, was one of those times when companies were laying off people and then I went through a couple of other companies. A couple startups where I basically, the stuff I took for granted that I learned in my first software development job, people just didn't know how to do.

JV: Interesting.

SB: Over time, that's what led me to write, I wrote some papers. I met Brad Appleton. Eventually I wrote this book on configuration management patterns, which to me were things that I just assumed everyone knew. Some of the feedback I get when I look at the book reviews, "We all know this, what's the big deal?"  Not everyone does.

JV: Yeah and when you say that you found out that people didn't actually know what they were doing, what do you mean by that?  What did they not know?  

SB: It's simple things like how to reproduce environments or how to not have integration builds fail. I went to do some work and I went home and things stopped working because they didn't understand the concept of private workspaces—that was a big one.

Then I segued into a bunch of things. In between, I'm very interested in configuration management. Then, I segued into learning about patterns. Then, I learned about agile. The three of them really have a lot in common, I think. Configuration management and agile, they give you an infrastructure so that you can focus on the interesting stuff, which is building.

JV: Right, sure.

SB: Patterns are also a way of instilling key knowledge, so you're not spending a lot of time reinventing stuff.  What you're doing is you're building interesting things.

JV: Right, what's actually very interesting is that you say that CM and agile are related. For a lot of people, maybe they don’t believe that to be the case. Can you explain that relation between the two? 

SB: To me they both form a framework. They help you just focus on the work. CM helps you, if you have your CM story in place you don’t have to say “What version of source code am I using?  What version of the jargon am I using? How do I build this?”

JV: It's all there for you. 

SB: There's tools in place. The ideal is the one-button build or the one-button setting-up-a-workspace thing, and CM allows you to do that. It also enables you to do agile because then you can actually maintain a velocity because you're not spending time wrestling with trivial issues.

JV: Right, CM just keeps things running. It keeps all those gears moving and you can go back to different versions.

SB: It helps you maintain flow—I guess, another term people like to throw at you. You just focus on writing code and building value.

JV: How do you feel about the response towards CM in today’s market, given the rise of Agile? Do they still know what a configuration management is?

SB: I mean the agile thing. A lot of the early stuff on agile was about keep the main line working; focus on tests. For the most part that's not a bad approach. You have another group of people, especially a lot of tool vendors, push the idea of tools that enable you to see it streaming different stream. Sometimes it also just defers problems.

JV: How does it do that in that it just defers problems?

SB: If your argument is this is the common thing with a lot of approaches using tools. One conversation that happens a lot is the build keeps breaking, so what we’re going to do is we’re going to have people develop our branches. Then, we’ll do pull requests and we’ll stage the integration.

That in itself is not a bad thing. What the problem is is if the branch goes off for a number of days then you have to spend a lot of energy merging. Then, if you're merging a bunch of different changes you're just going to get a different kind of breakage. The problems still there, you're just finding it later.

JV: Right.

SB: I'm not against branches and pull requests and things like that, but they're better as long as you're still integrating in the end and doing all the right practices; they’re not bad. Some people just use them as I'm going to build up my own universe and we’ll just defer the problem until later.

JV: Can you talk about the growth of agile?  How it’s evolved over the years?

SB: Yeah, it's been an interesting thing. When people were first talking about agile it was people who were really into it. They read all the books. They tried to understand the principals of what was going on. There was actually a big cultural paradigm shift. It wasn’t about just a bunch of practices. 

Agile is about focusing on value, getting rid of waste. Then, it becomes a little more buzz worthy. People are like “We unit test.” The joke I've heard people say, like Bob Martin does in talks, is “We’re agile. We don’t do documentation,” and they basically—all the things that are important—they just throw them away. They're not understanding why they're not doing them.

Now, it's even to the point that people out of school will say at a career fair, they’ll ask if you're doing agile without really having done a lot of it. They’ve done some unit testing, but they’ve done school projects or they’ve done small projects. 

Which actually leads me to another interesting thing, I've been thinking about. I guess the good news about that is that people now are doing agile are maybe starting to think a little more that agile’s more than just about the practices or not just about even the values. There's got to be a culture change that underlies what goes on.

JV: Right.

SB: So, my wife’s cousin’s got an interesting assignment. He's actually started a company a long time ago, called Culture Change, that's involved with safety, developing safety cultures. When I was married into the family I met him and we talked. I said there's got to be something about this that ties into agile software development. There has to be. He wasn’t a software guy; he was a manufacturing guy, so we had some nice conversations.

Last year I was noticing that Joshua Kerievsky had been talking about this thing called, I don’t know if I'm going to pronounce it right, anzen engineering.

The theory there is thinking about safety as you're doing your work; how does this increase or hurt safety. Safety in a larger sense, not just is someone going to hurt themselves.  Is it going to add project risk? 

JV: Right, definitely something that changes.

SB: I've known Josh from my patterns days or from the early patterns conferences. I said did you know about this?  It might be interesting and he went to one of Steve Simon’s workshops and just thought it was amazing. Steve called me and said Josh is a great guy. It was great to see that happen.

Anyway, I think that might be an omen or portent of people who are doing agile are maybe re-realizing that it's not just about the tools or techniques or whether you're doing Scrum or XP.  It's about an underlying set of culture and value changes.

JV: We need to talk about safety and some of those things. I mean we could even talk about security. How does agile development fit in with the old school way of thinking? With CM, everything has to be secure and safe, but now it's a rapid pace. How are those two being resolved?

SB: I was saying about the pieces right? Agile’s not just about moving fast. It's about moving fast and testing and validating as you go steps.

JV: Sure, yes.

SB: I think the big difference is that you can do agile just as well. I don’t know if this is entirely true, but when I was first talking about CM in the context of agile I'd say something along the lines of the old way is that you ensured quality by making things move very slowly. You would reduce risk by making things move very slowly.

JV: Right.

SB: Agile’s more about reducing risk by finding problems sooner because you can't ever really eliminate the problems and by going slower you actually introduce business risk. 

JV: Sure, which in itself is a problem.

SB: Yes.

JV: Got you.

SB: Right.

JV: You're talking about seeing kids out of college and this is actually I think very fascinating. Are these developers being trained right to enter today’s work?  Are they getting the fundamentals down?  .

SB: You see I don’t even know if that's even a good question because … I mean it was a good question, but I don’t know if it was the right question.

JV: (Laughs)

SB: The main thing I see as a problem sometimes is—the difference between, say a good beginning programmer and a less, good’s not the right word to use, is knowing what they don’t know. 

JV:Right.

SB: I have come across people who would insist that this is the way they do it because that's what they learned in class. This was involving even say C++ programing where it was a very idiomatic language.

A lot of times what you're learning in academic exercises there's certain things that have evolved in terms of practices; idioms that everyone does it that way. The books all say to do it this way, but on one does it that way. Again, that's more true of C++ than something like Java.  

JV: The harsh reality of the work.

SB: Not knowing it is fine; it’s knowing that you don’t know it or being aware of having a bit of understanding and some humbleness about what, understanding your ignorance.

JV: Right.

SB: That to me is the big difference. I think there's always going to be a mix of people who have come at it from both sides.

JV: Got you, so let's shift gears a little bit about testing. You're talking about agile; you're obviously trying to test often, test early. In what other ways have you seen testing involved over the years in regards to agile?   

SB: Like I say, one of the interesting things is I think there's been a new thing. There's also DevOps.

JV: Yeah. We see several webinars on that each week.

SB: Yeah, it sort of comes along with agile because that's the next step, right?  You can test things then you get into the deployment environment which is of necessity often a little bit different than your development environment, particularly for certain types of products. 

There's a bunch of tools around testing in DevOps. There's a company called ScriptRock that does a bunch of things, a bunch of tools that allow you to test that your environment is configured correctly. I mean other tools do this too. I just happen to know a little bit about ScriptRock.

There are tools … you can use CM to deploy the same thing. I have this version of Tomcat, this version of that, but validating that I can connect from this server to that server things like that. Tools that provide a framework for that I think are very promising and that's probably the next thing to get adopted.

I mean a number of people probably have their own home-grown tools to do this and it's intrinsically not a hard thing. Like anything else having products and tools that do that make it easier to adopt.

JV:Right, so you see a lot more tools that are more useful in today’s world than they were in the past?  I was talking to Joe Townsend the other day and he was saying how in the ‘90’s there was a real failing on the tools from what the marketing was explaining that they could do. He's finding that tools are a little bit more advanced today.

SB:Yeah, I think part of it is also a lot of open source tools are because they're tools people use who use such as development tools. People used them so they are actually very useful if you're using them for what they're for.

The other thing I was realizing, and I'm just thinking about what we talk about, is there are a lot of good commercial configuration management tools and for a while the open source tools … the difference was people used things like Subversion. Often because they just thought they were, because that's what they wanted to pay for.

There were other tools that were really better, but there's people that didn’t want to spend money. I think now a lot of the commercial tools, and a lot of the open source tools, like, for example, Git solves a lot of problems.  There's problems around using Git, around work flow. I think the successful tools are going to be the ones that support the use of open source tools. 

JV: Right.

SB: For Git there's a couple. I mean Git has something. Atlassian has Stash for managing the repositories, the permissions; providing some default work flows;  integration with your IDE and things like that.    

JV: It's like the tools just need to be able to communicate with each other, essentially.

SB:Right, so the commercial value-add is if you know what you're doing you can use your own bare bones. If you know that you're going to have time, GIT out of the box is fine. A lot of people, that's not where their valued added is. They don’t want to spend time figuring out all this stuff.

JV: Got you. Where do you see CM in the future?  What do you see in store? 

SB: It's interesting. The other thing, and I was talking to a colleague about this, is that we were talking about Git. The distributed version control systems seemed to be the thing to do and they have a lot of advantages. What we also realized is that often people who, not just because they're distributed, they're good because to be distributed they have interesting features and they do things like merging and so forth very well.

People, for example, don’t always need to use all the distributed features. Everybody is not working in Linux. Most use cases are you have some sort of centralized repository or two most corporate applications. You're actually interacting with that. Sometimes people get carried away by trying to use all the distributed features. Then that gets to the problem of, again, delaying integration so that at the end of the day you find a problem, but it's a week later.

JV: Yeah, because you're just someone who’s busy trying to play around with every little thing, every little tiny feature. When all you need it to do is one specific thing.

SB: Right, this integration thing; the sooner you get something all integrated and built and tested the sooner you’ll know there's a problem and you can fix it. Again, that's sort of the premise of why agile works, is that you're finding problems sooner because you can't really avoid all the problems. 

JV: Is there any reason why you think companies feel much more open discussing transitioning agile than in the past beyond they just see everyone that's doing it? 

SB:I think they see that everyone else is doing it. When I go to groups and things and talk to consultants who are—I work for a company so I do this I see some things, but a lot of—I talk to consultants who see it a lot more.

JV: Right.

SB: People get strangled with a lot of the values. They say they want to be agile because it's a good recruiting thing because people want to work for agile companies. It's a good buzzword thing why wouldn’t you be agile? 

I think a lot of people still struggle with a lot of the culture change aspects. How do you do requirements? How do you do deliverables   How do you do deployments and all the downstream stuff?

JV: I was going to ask you about some of the common things you see companies get wrong when they decide they want to go agile? I mean, obviously, culture being much easier said than actually done.

SB: I think the product management stuff is often the hardest part because you're coming up with small requirements and it’s working things into Sprint.

JV: How does that shift in where they already have a product management team?  They already have their ways of doing that. How do you merge those two groups together? 

SB: Right. It's possible. It just requires changing the way you're doing it, changes in some job roles.

JV:All right, Steve I think we’re coming to a close. Anything you want to leave the reader with?  Any final thoughts?   

SB: No, I enjoyed talking with you. Again, the thing that I'm probably struck by is just the whole culture change what Josh Kerievsky’s doing. I feel like that could go somewhere. I'm not quite exactly sure where it's going to go.

It's just interesting that a lot of the things that we’re talking about are things people were talking about five years ago or ten years ago about agile. More people are aware of it. I guess that's the challenge is for it not to fade into just obscure buzz wordiness.

Actually, there is one more thing I do want to mention though is …

JV: Go ahead, feel free.

SB: There's this structured, like Scrum that have fixed iterations. Then, there's Kanban and they all have their place. What I have noticed people trying to do is they fail with Scrum because they just can't get their backlogs done at the end of two weeks. They say we’re going to do Kanban because that means we’re just one thing at a time. They work for different constituencies.

I guess that's where there's more about agile out there so people can say well if this isn't working I’ll try this without really realizing that they're not getting the four things. They still may not do as well as they could. 

JV: Right. It's important to really know the method you are trying to apply.

SB: And why you're applying it.

JV: And why you're applying it exactly.

SB: The same for tools as well.

JV: Totally, definitely. All right, this is a good place for us to stop. Thank you so much for taking time Steve.

SB:Thank you. 

 

Steve Berczuk is a Principal Engineer and ScrumMaster at Fitbit in Boston, MA. The author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, he is a recognized expert in software configuration management and agile software development. Steve is passionate about helping teams work effectively to produce quality software. He has an M.S. in operations research from Stanford University and an S.B. in Electrical Engineering from MIT, and is a certified, practicing ScrumMaster. Contact Steve at [email protected] or visit berczuk.com and follow his blog atblog.berczuk.com.