Thoughts on the future of WebTest
I asked Marc and Dierk what they foresee for the future of Canoo WebTest. Since WebTest is an open source tool, many WebTest users simply add their own features. If a contributor writes a selftest for the new feature and checks it in, it can become part of the official build. Because I’m not a Java programmer, and my teammates haven’t shown quite enough interest to dig into WebTest and HtmlUnit code, I’m dependent on the developer community for new functionality. Dierk and Marc shared some ideas for features or “steps” they may implement when they or others have time:
- Cleanly support Javascript alerts, confirm, and prompts with something like nested tags in all “action” steps
- Ability to call Javascript functions from WebTest in order to test them
- A “recorder” for WebTest (a “capture” type tool to make it easier to specify tests; currently there is a limited-use tool called “Click-o-mat”)
- Accept all types of Ant tasks within a WebTest “testSpec” (test specification)
- Report Javascript errors in the same way that WebTest currently reports HTML parser messages
- Integrate email steps and extend it to verify different parts of email (i.e., custom headers that may be used to hide information)
Who Uses Canoo WebTest?
Dierk Koenig and Marc Guillemot both refer to themselves as “JustAProgrammer.” Dierk, who lives in the Black Forest of Germany and works in Basel, Switzerland, for Canoo Engineering AG (www.canoo.com), got involved in XP early on and contributed much to the early XP publications and training. He wrote his first test tool in 1990. Marc, who has worked in France and Switzerland, currently works in Germany. As a student, he wrote an MS Word template that facilitates typing math formulas in Word (www.amath.net). He found WebTest a couple of years ago when he was looking for a way to save time on repetitive tests. He contributes to both WebTest and HtmlUnit. I asked Dierk and Marc some questions about Canoo WebTest.
LC: Many WebTest users are programmers. Perhaps this is natural for an open source tool. The tool is fairly easy to use, even if you aren’t a programmer. Do you see the audience for WebTest as both programmers and non-programmer testers? Do you assume that users have a technical background, or do you want to accommodate beginners as well?
MG: I can imagine that WebTest is currently only well-known by programmers, but I can imagine as well that in some companies it is installed by programmers and then used by non-programmers, the programmers acting as the "interface" to the mailing list for instance. In my opinion, a good technical background is not needed to write most WebTests, but it can become more complicated, for instance, when using XPath or regular expressions. As far as it doesn't reduce the possibilities of WebTest, I think that simplifications are welcome.
DK: In the beginning, on our mailing list, we had reports from projects on which the
Web application users, who happened to be the secretarial staff, were writing the WebTests.
When WebTest is first introduced, tests are generally written by programmers.
Over time when testers, application users, and business representatives get involved with testing, they partially take over this task.
In most cases that I have seen so far, young and inexperienced programmers were assigned to the task of "writing the tests." This is usually a bad thing to do because it makes writing the tests a second-level activity, which it isn't.
Writing the tests was most successful when technical and business people paired on the task.
LC: Is Canoo WebTest targeted at testing a certain size application?
MG: I can only speak for myself, as an independent developer, without experience on really large applications. I don't see any reason to limit the size of the application to test. It's only a question of test organization. The "divide and conquer" strategy may apply here too—the tests don't need to all be in a single test suite. Nevertheless, I've also seen a memory problem because the resulting XML file is parsed again after each test sequence. This is probably not complicated to fix . . .
DK: I use it for three sizes: minimal, typical, and large.
What Do the Users Think?
To get another Canoo WebTest user’s perspective, I talked to Benjamin Feiermann, a consultant with Interactive Objects Software in Germany. He’s currently working on a project for a large bank, where his main focus is WebTest.
LC: What features or characteristics of WebTest do you find particularly advantageous? What “sold” you on using it?
BF: I think one of the most important qualities of the framework is its open-source characteristic. Due to this quality, every WebTest user has permanently up-to-date documentation, a stable status of the framework itself, and a possibility to extend it with its own specific features. The active mailing list supports all this.
The simplicity of the framework allows a newbie not to spend a lot of time learning it. And the fact that you write your scripts in XML (based on Ant) and don't have to learn a new or similar script language makes Web testing easier.
LC: Would you agree that anyone using WebTest needs some programming and other technical skills, or support from a programmer? Lots of non-programming testers seem willing to try a tool like Mercury QuickTest Pro but seem intimidated by a tool like WebTest.
BF: I don't think that we can call WebTest a tool. It's just a framework that gives you a basis for your tests. And perhaps the reason that people without programming skills are often put off is that they think, "Oh, no, I have to write SCRIPTS to test my homepage? Oh, no!” But I think anyone who has already worked with Web applications and who has already manually tested a Web site can write WebTest scripts without much effort. WebTest elements , , , and are self-explanatory, and WebTest scripts should display the flow of manual testing. Commercial test tools test for you; WebTest supports your tests and allows you to remain flexible.
LC: Have you experienced any downsides to WebTest?
BF: There are some situations in which you need some structures like "IF-ELSE" loops, etc., logic you have in programming and script languages but don't have in WebTest. It was the aim to design WebTest so simply that there are no complex structures and statements in your scripts, and you don't have to test them in turn. But testing of complex Web sites often requests the usage of such logical statements. With new features being added to WebTest such as Groovy support, we will have the possibility of using a real scripting language which has all of the logical structures and statements we miss in WebTest. But a tester should still keep his scripts simple and well readable.
LC: Have you used any extensions or customized steps of your own, or have you used Groovy or some other scripting language with WebTest—anything that doesn’t just come zipped up with it? If so, can you give me an example? What benefits have you seen, and have there been any difficulties?
BF: Yes, I wrote some steps, which reasonably solve the problems and needs I described in the previous answer. For example, there is a composite step named , which tests the equality of two values or variables and executes the inner steps if the value of the comparison is true. It's not a problem for me as a programmer to write an additional functionality, but if a "normal" tester without programmer know-how needs some new features, he, of course, needs support from a programmer. However, I don’t believe this situation happens very often. On the WebTest Community Web site you find many customized steps which can be used in recurrent situations. I didn't work with Groovy in connection with WebTest, but I will surely do it next time when I write some tests for any new applications.
