Tridion Integrated Test and Development Team Adds Efficiency with Automated Testing Tools

Background
Tridion is Europe's leading enterprise content management vendor and winner of the prestigious Tornado100 award in 2003. Tridion's R5 Enterprise Content Management solution enables organizations to create, manage, distribute and deliver content for multi-lingual Web sites, channels and online applications. The applications built with Tridion's R5, create competitive advantage for some of the most successful companies in the world including ADAC, AKZO Nobel, Alitalia, Blackwell Publishers, Carrefour, Heineken, Honda Europe, ING Real Estate, KLM, Renault, Scania, Smurfit Communications, Orient Express, Unilever and VisitBritain.

Tridion's R5 is continually enhanced to provide as much power and control to its end-users as possible. The concept of 'one version of the truth' remains central to all Tridion R5 developments, providing the user with the ability to create once, and then deliver content everywhere.

Tridion's R5 team of developers and test engineers work together to continually test new functions and new versions of the software. The product has an ambitious development cycle and a rigorous testing plan that includes operability, browser compatibility and usability testing, plus customer feedback sessions and focus groups. The Tridion development team was looking for additional ways to save time and money by increasing the team's efficiency.

"Our users demand the best in performance and reliability from our applications and since we innovate continually this presented the challenge of being able to reduce our design, build and test cycles," said Johnny Hopper, Senior Technical Tester, Performance Testing, Tridion. "We realized we needed to add another testing tool to our arsenal, so we began to review what we needed and what was available."

Testing Tool Requirements
Since Tridion was automating its entire load and functional testing cycle, the team wanted a tool that they could work with immediately. Thus, a solution based on an open language that the entire team understood became the first requirement for a tool. The team could not afford to spend time learning a proprietary language that would slow its ramp up time or that of new team members.

"We all knew that identifying problems quickly speeds the test and development cycle, but without being able to re-produce the actual error, our test and development efforts would be slowed," added Hopper. For this reason, testing tools that included features such as the Document Object Model (DOM), a programming interface specification that provides a mechanism for accessing document content verification, also had high appeal to the Tridion test and development team.

As the team began its evaluation of new testing tools, they reviewed shareware as well as traditional testing tools. The team had the following requirements:

• provide an efficient functional and content fault diagnosis process including the ability to reproduce complex errors for debugging
• streamline the testing and development cycles to speed time to market
• leverage existing corporate development skills and competencies
• enhance the testing team’s reporting capabilities
• ensure correct purchasing of additional hardware and software components

Evaluation Process
The evaluation process began with a detailed review of the tools available in the market. The team's initial findings were that many shareware tools (the existing tool of choice for the integrated test and development team) could measure raw response times well, but didn't provide enough data about functional errors in the content being returned without intensive programming by the team.

In fact, Hopper explained "we discovered that the amount of development effort required to check for content errors with many of the shareware tools was prohibitive and with some of the shareware tools it was necessary to write dozens of lines of code just to locate a returned value on a page."

"The traditional testing tools seemed to steer us towards the idea of creating a separate testing group which we would need to staff with highly specialized contractors who understood the proprietary nature of these tools," said Hopper. "Since our test and development engineers are integrated, we needed to ensure that whatever tool we chose would enhance our team’s work. We were looking for a tool based on a standard software language that everyone already understood."

Traditionally, the more complex the application, the more time it takes to test. Software developers and test engineers turn to automated testing whenever possible to reduce testing time by building scripts to replicate users' behavior. Then the tester would exercise the application in a real-world scenario by scaling the test to potential user volumes. With the information from these tests, time could be spent detecting when an application 'failed' or where it can be enhanced instead of manually running lines of scripts.

Many of the traditional testing tools, which were developed in the late 1980’s, solved a major test challenge–they reduced the time it was taking to test more complex applications by replacing the slow, error-prone manual process. These tools automated the process of manual testing by allowing a user to record probable user keystrokes and play them back quicker than a test engineer could manually. Also, due to Year 2000 conversions, many traditional tools added test management modules for the coordination and documentation of the tests they had completed to ensure compliance.

Tridion's mission critical high-end Web applications required a testing tool that would provide an automated testing process to address where they were losing the most time - with developers and testers attempting to reproduce complex test scenarios to exercise their code. Tridion’s objective for testing its Web Application was completely different from what the traditional testing tools were built to solve.

"Time to market with new Tridion R5 product features became one of the biggest factors in our success with customers," said Patrick van Gent, VP of Product Marketing. "Since many of our client's entire Web strategy is based on Tridion products, we have become an increasingly larger part of their ability to be successful. New functionality from Tridion products creates more opportunities for us to work more closely with our customers to create extra value for them."

Tridion chose RadView’s WebLOAD for a variety of reasons. The tool utilizes an industry standard language, JavaScript, facilitating and encouraging collaboration between developers and testing professionals. This brings immediate productivity benefits, because it provides a common language that can be understood and maintained by both teams, which encourages earlier and more frequent testing within the development lifecycle.

"JavaScript is an efficient, accessible language that is a perfect fit for Web testing," said Omri Reuter, Vice President, Research and Development at RadView, Inc. "RadView’s choice of JavaScript as the scripting language for our tools was an easy one. Every Web developer is familiar with JavaScript and if anyone on the Quality Assurance staff is unfamiliar with a scripting language, JavaScript is the quickest and most accessible language to learn."

Testing Results
Using JavaScript, the Tridion engineers built a test engine comprised of a complex library of functions. This engine automatically runs and scales a multitude of test scripts made up of dynamic user configurations that show how enhancements and new features will affect the customer’s current products on each platform.

"With WebLOAD, we easily built test functionality into this suite of scripts instead of the more labor intensive process of manual testing. In fact, it would have been nearly impossible to run tests of this nature without a product like WebLOAD" added Hopper. "But by investing the effort in building the automated tests, we gained efficiency over time without sacrificing quality." Another key solution for the Tridion team was the ability to validate every byte of returned data during the load testing.

"Our customers demand that our applications are extremely reliable in performance and functionality at release time. We needed a tool that allowed us to check everything within our application throughout the load and performance testing, and WebLOAD's utilization of DOM made this possible."

The DOM technical standard, defined by the World Wide Web Consortium (W3C), provides a mechanism for accessing document content, typically HTML pages. WebLOAD automatically presents each page returned from a Web application during a test in the DOM format, which makes it much easier for a test author to check for content on a page without laborious programming.

"We know that many other tools do not check for returned page content, preferring to force a prospective client to separate the disciplines of functional and load testing because it is just too difficult to manage," added Reuter. "We designed WebLOAD to make this an easy, natural part of the testing process."

"We have been very happy with our choice of RadView's WebLOAD for functional, load and monitoring of pre and post production builds of Tridion R5,” said Hopper. "Use of WebLOAD offers us the power of JavaScript as a scripting language for load testing in addition to integrated functional testing and monitoring to ensure the quality of the Tridion products and the efficiency of our development team," said Hopper.

As experts in content management, Tridion believes that information is the driving force of success. By facilitating easy access to the right content in the right language at the right time, Tridion’s products enable faster, reliable and better-informed decisions to be made by its customers. RadView also believes that information is the driving force for testing success and that using a common scripting language and providing access to testing details enables its customers to produce higher quality, more efficient software.