ROI Robbers in Test Automation: STARWEST 2015 Interview with Greg Paskal


In this interview, TechWell speaks with Greg Paskal, a technology innovator in quality assurance. At STARWEST, he had a presentation titled "ROI Robbers in Test Automation."

Jennifer Bonine: All right, we are back with more virtual interviews today, and I have with me Greg Paskal. Greg, nice to have you here.

Greg Paskal: Hello! Good to be here.

Jennifer Bonine: For those folks watching out there that maybe haven't gotten a chance to meet you yet, or didn't see your session that you're doing … did you already do it, or is it coming up?

Greg Paskal: I did. I just finished it about an hour and a half ago.

Jennifer Bonine: Yeah, that's what I thought. For those who didn't get a chance to see it, maybe give them a little background on you, and how you got into this, and how you ended up here at this conference.

Greg Paskal: Sure. I've been an automation engineer probably officially for about fifteen years, but actually much longer. I started in some form of automation almost thirty years ago in the aerospace. It was when we were writing code on Apple IIs and the early days of that, but we were testing space flight hardware at the time. Then I was working for Kinko's corporate offices, and I was officially handed a copy of WinRunner, an HP tool, and that's how I got started in the current role that I'm at.

Jennifer Bonine: Okay. Your first part, that aerospace stuff, was that public sector, or was it government?

Greg Paskal: Government, yeah; aerospace, for sure.

Jennifer Bonine: Very nice. And then transitioned to more of the commercial, the ...

Greg Paskal: Yes.

Jennifer Bonine: ... the Kinko's-type environment and stuff.

Greg Paskal: That's right.

Jennifer Bonine: You kind of have, I gather, a passion around test automation.

Greg Paskal: I do. I even love the idea of removing the word test from test automation. I talk about that a lot. An automation engineer is wired in a certain way: to see opportunity and things that can be done through a tool that would make it efficient. So I talk about that idea often.

Jennifer Bonine: And just calling them automation engineers.

Greg Paskal: Remove the word test, and then all of a sudden you realize there's a lot of potential that you could do with these very expensive tools that you never thought of before.

Jennifer Bonine: Yeah. I think it's great, too, because a lot of folks out there that maybe are interested in it may not come from your traditional testing background or world, right?

Greg Paskal: Sure.

Jennifer Bonine: There's a lot of automation engineers out there who I've seen lately that maybe come from development backgrounds, or coding backgrounds, or programming backgrounds, too, that are saying, "I want to get involved in this. I'm interested in it. Is this for me?" That kind of thing, as well.

Greg Paskal: Absolutely.

Jennifer Bonine: You have a new book, I hear.

Greg Paskal: I do.

Jennifer Bonine: Yeah.

Greg Paskal: It's called Test Automation in the Real World.

Jennifer Bonine: I know. I don't know if I have it, but do you have one of your cards on you, too?

Greg Paskal: I do, but it's actually very easy to find online.

Jennifer Bonine: Oh, good! Okay, tell us how to find it.

Greg Paskal: You go to, and it will lead you to Amazon and Apple, where you can download a copy from it. It's an e-book.

Jennifer Bonine: Oh, nice!

Greg Paskal: It's just my first one. It's exciting because it's taking all those years of experience and sharing the things I've learned. This book isn't necessarily a "how to automate," it's a "why to automate." It really gives you the things to watch out for, the things that will be very helpful—like, how do you hire an automation engineer? How do you perform an automation evaluation, so you're automating the right things? This book talks a lot about those.

Jennifer Bonine: For some of those folks out there that may be watching in our virtual audience that say, "I'm struggling with how to find the right person for our automation roles ..."

Greg Paskal: Perfect.

Jennifer Bonine: This would help them with ...

Greg Paskal: This would help them enormously. A lot of folks have automation, or they'll have specific tools listed on their resume, but they don't necessarily have the experience, and that particular chapter in the book talks about the things that you would want to find and identify in an automation engineer: that they have the experience, that they're able to show you that what they've built is scaled over a few years, something that is more than just, "Look, I've built it. It's working." You need beyond that to really know the person you're bringing in has the right skills to do it well.

Jennifer Bonine: Yep. For folks out there that maybe say, "I've got people in my organization," or "I'm interested in moving into this," any suggestions on how to do some evaluation to say what types of things are you going to need? Are you truly interested, or do you just like the word?

Greg Paskal: Right. Many people are enamored by the idea of automation. A lot of manual QA testers see it as a way to kind of bump up their skill set and progress in their career. I really look for folks, when I'm looking for an automation engineer, that have done some development in their backgrounds, even if it's dabbling. Maybe they've created a webpage—something that lets me know they understand the basic concepts of scripting. It takes a lot more than that, but that's a good set. Of course, you really want to find someone that's got a QA mindset as well. I believe that particular chapter talks about how important both those things are, and what happens if you have just someone who's a great coder but doesn't have the mindset of QA, and vice versa.

Jennifer Bonine: Yes, so kind of blending those two things together.

Greg Paskal: Absolutely.

Jennifer Bonine: Right mindset, plus the interest or the aptitude and wanting to code is important.

Greg Paskal: Yes, definitely.

Jennifer Bonine: For those of you out there looking for that, that may be a good place to go to get some information to not only hire people from outside your organization, but evaluate even from inside: Do you have the right people, or someone, besides liking the thought of it ...

Greg Paskal: Yes.

Jennifer Bonine: ... that'll actually be able to execute and deliver on that.

Greg Paskal: That's very true. This book also talks about the expectations. When you begin to automate, it's one of those fields that's sold often with all sorts of conflicting things of what you're going to get out of it right away.

Jennifer Bonine: Right.

Greg Paskal: So it sets a reality check on that. There's a particular chapter that talks about the automation evaluation and how you walk through to determine if what you are considering is a right candidate for automation. Just because you two can see it doesn't necessarily mean it's worth doing in the long run.

Jennifer Bonine: Right. Exactly. Besides automation, you have an Open Test Technology Forum, is that right?

Greg Paskal: Yes, I do. I'm so excited about this. In the beginning of this year, I really wanted to bring together, or have a forum where folks from our company could come together who are not only QA folks, but people that were in development and others. We've got people from marketing and others that come together, and we basically share some of the challenges we're dealing with. I initially started it from a testing perspective, but I'm now considering removing the "test" piece of this and calling it the Open Technology Forum.

It seems simple, but you need to give people kind of the permission to come in and share openly and realize, "Look, we all can potentially learn; there's no kind of dumb questions here." The things that bubble out of that—if you are someone with the mindset of, maybe, quality in the software development lifecycle, we just identified one of those that came out of it. We were able to say, "There's something here that's worth us approaching and pursuing further," and sure enough, we were able to get greater quality in our software lifecycle by something that came out of that. It came from one of our marketing people, believe it or not.

Jennifer Bonine: Very nice. And so this Open Technology Forum that you have now, is it just inside of your organization, or can anyone get access to it right now?

Greg Paskal: Right now it's only within our organization. I've actually shared it with Lee [Copeland], because he had some interest, and possibly even a keynote down the road that it could turn into, because I think it's such a powerful idea that, yes, it can definitely span outside the four walls of a company where other people can contribute, and you can think of new opportunities to bring quality or other improvements within a company through it. I love learning about going through this process. You really have to have someone that can moderate a conversation well, and things like that. I'm learning all about that still; I would be lying to say, "You'd probably do a great job at it; you're naturally wired for that." Those are some key things I'm learning, and really trying to understand as I go through the process.

Jennifer Bonine: Maybe for someone out there who's saying, "We could really use something like this in our organization," and it's not available yet, any tips or techniques on how to go about setting something like that up, that they could maybe use?

Greg Paskal: Believe it or not, the deeming of something like this is important. It needs to be something that people feel is ... That's why I like the word "forum." In their mind, they know, this is people coming together and talking. I think that's important. And of course, I'm a big believer in those relationships in the company, so I like to know a lot of folks in our company. I know folks from our HR, marketing and sales, and dev. If you can get some of those people to come to begin with, it kind of seeds the process in starting. I believe in this case, our HR department is even interested in what we're doing for the open forum, because they see it as a way of having collaboration across what are typical silos within the company.

Jennifer Bonine: Broken down, yeah.

Greg Paskal: Broken down. So partner with those folks that can be good at getting the word out about it, and who can get behind it and will empower it to become better and better over time.

Jennifer Bonine: Yeah. I think it's so important, to your point, to name it correctly so everyone feels it's inclusive—that they can participate, that it's not just for the test group or the development group; it's for everyone to come together.

Greg Paskal: Absolutely.

Jennifer Bonine: The business and marketing, and getting those various perspectives, like you said, I think is really valuable. Good idea for folks out there, if you're looking for something like that to get started in within your own organization, to look at doing something similar.

Greg Paskal: Yeah, absolutely.

Jennifer Bonine: You also, besides your new book that's out, you have an iPhone app.

Greg Paskal: Yeah, I dabble in ...

Jennifer Bonine: You're busy!

Greg Paskal: I dabble in lots of things. Just ask my wife, she knows—I'm always tinkering with something. She calls it the wiggles. I've got to get there on Friday to do something new. The particular app we're talking about—I have seven apps in the iTunes Store ...

Jennifer Bonine: Oh, wow!

Greg Paskal: This particular app that we're referring to is called a Minimal Essential Testing Strategy, and it actually was the first STAR Conference I ever spoke at, was on this. I spoke at STAREAST and STARWEST; I believe it was in 2003, believe it or not.

Jennifer Bonine: Oh, wow!

Greg Paskal: And it's a methodology for when something's thrown over the wall and you're told you have an hour to test. It's a strategy for the reality that we all face like that.

Jennifer Bonine: Oh, wow; very cool.

Greg Paskal: It's easy to find online. I've got people that really believe in it: Randy Rice and other big name testers out there really believe in this methodology. It's a good thing to have as a tool in your tool belt. So when I wrote the iPhone app, the thought was, "This is literally in my phone and I can use it," but for folks that really would want to use it, I would suggest you go online and search for "Minimal Essential Testing Strategy," and there are documents you can download, PDFs that have the methodology right there.

Jennifer Bonine: Oh, cool.

Greg Paskal: Yeah. You can apply it to a lot of different genres, for sure.

Jennifer Bonine: Oh, that is so neat. Again, Minimal Essential Testing Strategy app, and for those of you out there that have had this ... I'm sure we can all relate, when someone comes and says, "You've got an hour! That's all we've got. Do what you can."

Greg Paskal: That's exactly right.

Jennifer Bonine: This will help all of you.

Greg Paskal: This gives you a strategy. That's exactly what it was designed for.

Jennifer Bonine: It shows you what to do, in that hour, to spend it well. I know that seems to be a common theme. A lot of folks who are in testing get nervous about, "Well, I don't have enough time. What do I do?"

Greg Paskal: That's right.

Jennifer Bonine: Then we kind of get that analysis paralysis, so it can help with how to tackle that—what seems like an overwhelming challenge.

Greg Paskal: That's exactly how the methodology got started. It was my first official QA role, and I owned the dot-com site, as the tester, and you know how with that kind of thing, they're like, "This change has got to go live in an hour," and so I thought the next time that happens, I want to make sure I do at least this. And I just literally wrote it down on a little piece of paper next to my desk, and it grew and it grew and it grew; and that became this entire methodology, now.

Jennifer Bonine: Wow!

Greg Paskal: Yeah.

Jennifer Bonine: That you can all take advantage of.

Greg Paskal: Yeah.

Jennifer Bonine: Some great things that we learned. Our time goes so fast.

Greg Paskal: Sure.

Jennifer Bonine: We're already out of time. Thanks.

Greg PaskalA technology innovator in quality assurance, Greg Paskal pioneers best practices across tools, techniques, and talents. Exploring existing and new approaches in test automation, Greg enjoys mentoring automation engineers in highly sustainable, maintainable approaches for greater ROI in automation efforts. He authored Test Automation in the Real World, contributed to industry publications such as StickyMinds, and recorded podcasts for TestTalks. Greg has spoken at conferences on his Minimal Essential Testing Strategy, now developed into an iPhone app. He founded the Open Test Technology Forum to encourage collaboration and focus on greater quality across the enterprise. Learn more about Greg at and

About the author

Upcoming Events

Sep 22
Oct 13
Apr 27