TrainingConferencesAbout UsContact UsAdvertiseSQE.comRSS Feed

StickyMinds.com: brain food for building better software

Log In
 Clarify Your Search Criteria

Tips on Using Our Search Feature(s)
 
StickyMinds.com Home
ResourcesTopicsCommunityPowerPass
Home  >  Detail: Hey Vendors, Give Us Real Scripting Languages



A StickyMinds.com Original
Article Picture
Hey Vendors, Give Us Real Scripting Languages

By Bret Pettichord

Send This Content to a FriendGet a Short Link to This ContentPrint This ContentSee User Comments About This Content

Summary: Most test tools come bundled with vendor-specific scripting languages that I call vendorscripts. They are hard to learn, weakly implemented, and most important, they discourage collaboration between testers and developers. Testers deserve full-featured, standardized languages for their test development. Here’s why.


Rally Software Development
For most of the test tools available today, you have to write test scripts in the tool’s scripting language. What drives me crazy is that most vendors have their own special language that you must learn in order to use their tool. They often have recorders to help you generate code, but invariably you need to go in and make changes and write code yourself. And you end up having to learn another language—another vendorscript—another dialect that is good for nothing but the one tool. 
 
Undermining Collaboration 
Because a vendorscript is a specialized language, developers are less likely to know it and little inclined to learn it. I frequently counsel testers and developers to work together on test automation projects. There are many reasons why this is a productive collaboration, but vendorscripts get in the way. They split testers from developers and from each other into tool-specific language isolation. This reduces the opportunities to share, collaborate, and improve the craft. 
 
We have enough trouble bringing developers and testers together in the testing process. The last thing we need is an inherent obstacle from the get-go in our testing tools. Ask a developer to learn Visual Basic? No problem. They can always use that knowledge in the future. Ask a developer to learn a specialized language unique to one tool? You're dreaming. You may already be having trouble getting the developer to pay attention to testing, and now you’re asking for more. 
 
Why spend money on a tool that’s going to undermine the collaboration efforts between testers and developers? 
 
Dialect Stew 
There are various dialects of vendorscript. Some use C-like languages. What does this mean? It means that the code looks like C, but it doesn’t have pointers, so you can’t use any of the complex data structures that you learn about in any C course or book. C is really a poor basis for a scripting language. It is compiled, while test scripting languages are interpreted. This is why the C-like languages can't support pointers. 
 
Others use a Visual Basic-like dialect. If you know Visual Basic, the code will look familiar, but you won’t be able to make use of standard Visual Basic libraries. Moreover, the things you learn about Visual Basic may or may not apply to the vendorscript. 
 
Still another tool uses an object-oriented mongrel vendorscript. Being object-oriented is nice until you learn that modifications to a class won’t be inherited to its subclasses. What? This vendorscript thinks that this is a better arrangement for testing. Whether this actually helps testers is debatable, but what testers really need is not a special language designed for testing. Rather they deserve and work better with a full-fledged language. 
 
Some of the newer test tools are using real scripting languages, such as JavaScript or Visual Basic for Applications. I can only hope that this tend continues. Moreover, it would be nice if some of the older tools would be reworked to support standard languages. Standardized languages have formal specifications, which tend to favor ease of programming instead of ease of implementing the interpreters. This makes them easier for you and your test team to use. 
 
Reinventing the Wheel 
The drawbacks of vendorscripts were illustrated in a recent article that described how a couple of test automators had collaborated to add stacks and queues to a vendorscript. The point of the article was to demonstrate what test automators could do when they worked together, but the lesson I drew was that vendorscripts can lead test automators to waste a lot of time and effort reinventing wheels. Stacks and queues are elementary data structures that require little effort to implement in standardized languages.  
 
I can sympathize, because I’ve been in the same situation. I have had to build libraries to support string manipulations, date math, and calculating permutations for various vendorscripts. Some vendorscripts have libraries for such things, but the libraries for standardized languages are much larger and more robust. 
 
Back in the Old Days 
Lots of nontesting tools have embedded vendorscripts as well. And it used to be even worse. About a decade ago, John Ousterhout decided to do something about it. He designed Tool Control Language (TCL), made it easy to integrate with C, and released it into the public domain. TCL is now used in some public-domain test tools. 
 
Ousterhout makes a distinction between system programming languages (such as Pascal, C, C++, and Java) vs. scripting languages (such as Perl, Python, Rexx, TCL, Visual Basic, and Unix shells). System programming languages are better for building from scratch and optimizing for performance. Scripting languages are better for rapid development and reusing code. They are great for test automation. 
 
Some of the test tool vendorscripts pre-date Ousterhout’s advances—created before TCL and when Perl was still in its infancy. But that was then. There are plenty of options now. 
 
Retiring Vendorscripts 
So with plenty of options, vendorscript should be retired. With one exception, there are no third-party books that you can use. You’ll have to rely on books from the vendors, which may or may not be frank about the vendorscript’s shortcomings. Courses are also hard to come by, expensive, and may require travel. If you are hiring, it will be hard to find people who are already trained. 
 
Test automation is hard enough without us having to flesh out the vendorscripts that come with many tools. Developers wouldn’t stand for programming languages with such limitations. We shouldn’t either.  
 
Further Reading 
 
Scripting: Higher Level Programming for the 21st Century, John K. Ousterhout, IEEE Computer, March 1998. 

Breaking the Language Barrier, Christopher Meisenzahl and Ferry Firmansjah, STQE magazine, Nov 2000.


About the Author
Bret Pettichord helps teams improve their software testing and test automation. As a principal of Pettichord Consulting, he offers consulting and training. He is the editor of the Software Testing Hotlist.

Back to Top
 

StickyMinds.com Weekly Column From 02/26/01 

Member Comments
Add Your CommentExpand Comments
 
Comment:    
by Ghassan Dawood 6/18/2008

I agree 100% with the author. I am aware of one tool that gives you a choice of scripting languages (VBScript, JScript, C#Script and DelphiScript) and that is why it became my favourite tool. Also licensing is far cheaper than the famous brands. It does a brilliant job for me.

Gazz
ghassan.dawood@findel-direct.co.uk


 
 
Comment:    
by Edward Teune 10/3/2007

How about regression testing WITHOUT scripting languages?
Developers can develop product and business analysts and area owners can test what the software was supposed to do in the first place.
Functional testing without the need for developers.
Self-healing scripts created without any code at all.
Now THERE's an original idea.
Original Software http://www.origsoft.com


 
 
Comment:    
by Julius Alba 3/18/2006

What if you could marry a real programming language and encapsulate it with simple to use APIs to automate Web Testing? Check out Newbie at http://www.mynewbie.net

Julius Alba
Founder, Newbie Labs
www.newbielabs.com


 
 
Comment:    
by Violet Weed 10/25/2005

I'm not sure who is commenting about automation test tools, most sound like non-programmers. I, however, have been programming since I graduated from high school at 13, got legally emancipated, and went to work for IBM as a jr. programmer... some 44 years ago. I have been programming (not record-playback) with WinRunner (and XRunner, etc.) since 1995, the first year those tools came out (xrunner). I wouldn't use a scripting tool like silk, etest, or that rinkydink samie tool if you paid me to... oh wait!, I'm being paid to use samie right now, but only to do a comparison with WR. Think I will give a slanted review? No. I won't have to,...Read On

 
 
Comment:    
by Chris Bilcz 5/4/2004

Can anyone comment on Rational RobotJ? RobotJ is a java based automation tool for testing web applications.

Author's Response:
10/1/2004    
Definitely NOT a vendorscript.

 
 
Comment:    
by Rafael Ruiz 4/11/2002

Hey, as I was reading the article and the comment from Marek Jastrzebski, I recognized my own experience. We were two departments writing tests in the 'C-like' language used by Winrunner, and we got involved in those old non-profitable methodology wars: while one department was applying programming techniques, the other was just using the 'record and play' feature. Results: by the end of the project the first department had structured and easy-to-modify libraries, but unfortunately they covered just a few functionalities, because we testers were too busy 'reinventing the wheel', as Bret says. The second department had many efficient...Read On

 
 
Comment:    
by Frederic Torres 2/18/2002

If you have to do functional web testing and you like VBScript try this new tool http://www.w3runner.com.

 
 
Comment:    
by Michael Jacobson 11/21/2001

It's nice to see someone else that speaks out against all these proprietary (non-portable) script languages (a.k.a. "vendorscript"). About 3 years ago we switch out internal (home grown) test automation tool from our script interpreter to TCL (and have never looked back). TCL has many nice features with the main ones being readability of the script and its support libraries. Its user community offers great support and it is one of the easiest languages to learn (especially if you skip the GUI "Tk" stuff). I just can say enough about TCL verses the other script languages (including VBA, ECMA, PERL). With its BSD style software licenses...Read On

Author's Response:
11/28/2001    
I've used TCL and am rather fond of it. You're right: any vendor can bundle it with their tool. I don't know why more don't.

 
 
Comment:    
by Mark Felling 10/30/2001

Excellent comments everyone. Check out TestQuest, Inc. (www.testquest.com) This tool (TestQuest Pro) is non-intrusive so it can test any OS from Handhelds to PC's to Interactive TV, uses standard "C" as a languange and runs on an Interpreter. Additionally they recommend "programming expirence" and implementing an "Architecture" --- NOT using Capture / Playback, though it is available, to efficently use any automation tool.

Author's Response:
11/28/2001    
Actually TestQuest Pro uses one of the C-like vendorscripts that I criticized in the column. It does not support pointers like real C.

 
 
Comment:    
by Mary Sweeney 6/11/2001

Hey, Bret In many cases you can dispense with the less-than-perfect scripting language provided by the vendor and simply utilize a full-featured programming language, or scripting language, like say VB or PERL. Why not do that? --Mary

Author's Response:
11/28/2001    
Great idea. Readers should know that Mary's written a book: Visual Basic for Testers that describes how VB can be used directly.

 
 
Comment:    
by Brian Hurst 5/23/2001

I am glad to see that this article has gotten positive responses, and even included a reference to TestPartner from Compuware. I am an SE for the TestPartner product and have been making my living the last 6 months (since its release) traveling around the country proving object level support where other vendors are falling short. This is due to various reasons. 1) In an application using Microsoft languages, TP will obviously have superior support to any other tool because our scripting language is TRUE VBA, licensed from Microsoft, which really shows its advantage against ActiveX and COM envmts. 2) Also, for non-standard controls...Read On

 
 
Comment:    
by Marek Jastrzebski 5/10/2001

Thanks for a great article. I have worked with AutoTester(back in 94) WinRunner and RationalRobot. Each time I get to a critical limitation. After 4 months I write 80% of the code by hand. In Robot I have to build my own libraries for reusability. It feels like fighting with the tool. Recently a manager told me that I do too much programming with Robot. Someone reviewed my code and said that It's too complicated. I asked them to point where. The answer was "What are those headers and libraries for? You should use the record and playback built in the tool" - I almost wanted to quit right there on the spot. I think managers of testing...Read On

 
 
Comment:    
by Tracy Benton 3/1/2001

We are starting to list our requirements for an automated testing tool--thanks for giving me another big one for the list!

 
 
Comment:    
by Danny Faught 2/27/2001

Bret, you're right on. I worked with a home grown test automation tool that used a proprietary language. The developer of the tool swore he'd never invent a new language again. Consider the build vs. buy economics from the vendor's point of view - you can create a new programming language, or use one that already exists. Using an existing language is likely to be much cheaper, especially if you use a language with a freeware implementation. Even if you intend to create a very simple language, if the users perceive it to be a programming language at all, they will demand all the usual programming constructs, and they will ask you to add them,...Read On

 
 
Comment:    
by Chad LeRoy 2/27/2001

Of course if you are unlucky enough to work with a Visual FoxPro application you have no automated testing tool to use. Just another example of the limitations of “modern” automated testing tools.

 
 
Comment:    
by Chris DeNardis 2/26/2001

I agree with what has been said here. I believe the only selling point between the automated testing tools is not only what language does one program in, but how easy it is to use. When I select a tool I make sure it has several "languages" available that are more common, and can be trained just about anywhere versus just at the vendor. Otherwise it is like a two edge sword - I spend a lot of money on a tool, but no one can program it, unless I spend a lot of money on training.

 
Back to Top


Marketplace

Web based bug tracking - AdminiTrack.com
AdminiTrack offers an effective web-based bug tracking system designed for professional software development teams.

BugSplat - Automatic Crash Analysis
Fast online exception analysis. Capture customer crash data online.

Six Sigma Certification
100% Online-Six Sigma Certificate from Villanova - Find Out More Now.

Census: Web-based Bug Tracking and Defect Tracking
Track software bugs, defects, enhancements, support calls, and more. Issue tracking software that is scaleable, fully customizable and integrated with VSS. Includes e-mail notifications, role-based workflow, change history, and Crystal reporting.

Need Agile Test Cases?
Create statistically complete test cases simply and quickly.

Get your product or service listed here.
Subscribe to Better Software Magazine
Subscribe to Better Software Magazine

First Name:

Last Name:

Email Address:


Home   |   Resources   |   Topics   |   Community   |   PowerPass



© 2008 StickyMinds.com. All rights reserved.
StickyMinds.com is a division of Software Quality Engineering.
Privacy Policy    Terms & Conditions    Link to StickyMinds.com    Feedback


Borland

MKS, Inc.



STARWEST 2008

 
Agile Development Conference 2008