I recently had a wonderful experience at an industry conference and learned
some interesting things I thought I’d share with you. Usually I hate going to
conferences on the topic of testing and quality assurance (they can be so boring
when under-qualified, over-hyped, or overly-academic people get up to speak). However, I recently returned from the Practical Software Quality Techniques (PSQT)
conference in Bloomington, Minnesota, held in conjunction with the PSTT
conference (where T=Testing replaces Q=Quality). And, I am even more happy to
say, the P=Practical word in the title was particularly well chosen.
Presentations were by "been there, done that" kinds of people.
I went out of my way to choose sessions on Web application development (there
were eight parallel tracks), mainly because I don't know much about Web
development. My chief interest was to what degree Web development differs from
traditional software development. The answer I learned, partly to my surprise,
is "plenty."
What’s different about Web application development? Web application
development tends to be:
- requirements-free: There are seldom any stated requirements at the outset
of a project. (One speaker described that phenomenon, only partly in jest,
as "Specs? We don't need no stinkin' specs!")
- rapid-development: There is seldom time to do anything more than what one
speaker called "slam it and jam it," using a process he called
"FAD--Frantic Application Development."
- short fuse and relatively small: Most projects are expected to last about
three months, using a maximum of four people.
- testing-primitive: There are few generally accepted processes, and the
tools that are available often don't fit the problems well.
- user-intimate: Although you never meet your users, they are only a mouse
click away from letting you know what they don't like--and from going over
to the competition.
- a "living project" (two speakers used this term): Maintenance,
usually by the developer, is a "must," and in fact "all
products are in permanent beta test" (modifications are ongoing).
- performed with limited management support: Most managers think Web
development is easy.
- highly aware of performance: Performance is vital--and extremely
unpredictable.
- concerned with content: Content is king; it doesn't matter whether the
application is "pretty" or not, if the content isn't interesting
or useful.
- a whole new field: Developers are more "Wild Web developers"
than software professionals.
In fact, that last item is key to understanding the difference between the
Web and traditional software. One speaker expressed this difference very well,
saying, "We've moved back two decades in testing approaches" (that is,
no one is building the notion of Web applications testing on the lessons learned
from the history of software testing), but went on to say "I'm not sure
that traditional approaches will ever be applied." In another session, the
issue became even more clear. Is it pure backwardness to build Web applications
without requirements, or is it simply true that in any rapidly evolving
discipline, it is impossible to know what the requirements are until the first
(several?) prototypes are built? And in the absence of firm requirements, isn't
it impossible to perform the rudiments of traditional testing and quality, such
as requirements-driven testing? As the Web applications field matures, it will
be fascinating to see what comes of these issues.
What characterizes a Web application? Andrea MacIntosh of QA Labs, Inc.,
said that the typical architecture is a browser tied to a Web server tied to an
application server tied to a database. And what characterizes Web applications
testing? The same speaker said that it's about multiple environments (you can't
test a new version of a site on the live site), multiple platforms (you can't
necessarily control the browser/operating system/languages/tools choices of your
users), a new approach (you test "back to front," meaning starting
with the server side and working out to the client side), and the focus is
different (you must "think user," structure by whatever canned
components you can use, and prioritize your testing according to where the risk
of failure is largest). Two other speakers, addressing the issue of performance
(and not specifically testing), said the same thing about "back to
front" approaches. Brian Hambling and Angelina Samaroo, of ImagoQA, advised
starting performance work at the back end, focusing especially on any legacy
systems and their (baseline) performance. (Incidentally, regarding performance
requirements, most speakers agreed that a maximum response time is about eight
seconds--anything more than that requires telling the user what is going on).
What about the state of the Web applications practice? Arthur Hicken of
ParaSoft noted that "the average Web site has one bad link for every three
and a half pages, and twelve errors for every page of HTML." "Ninety
percent of browser incompatibilities," he went on to say, "are caused
by using nonstandard HTML, and the other 10 percent are because the browser does
nonstandard stuff." (He noted that many people blame Microsoft for
deliberately doing nonstandard things so that their competitors could not get
their code to work, but Hicken blamed the competitors themselves for their own
nonstandard practices). And Mark Bonewell of Medtronic described his own lessons
learned from a first-time experience testing a Web application. He noted that
there were almost no requirements (by the time he said it, this had become a
familiar theme at the conference) and that it was the first time that most
programmers on the project had done Web development.
As I said, I usually hate going to conferences on the topic of testing and
quality assurance. But Web testing conferences should offer new and changing
information for some time to come. Each conference, like each Web development
project, will have to address fluid and unstable "rules" as the field
develops.
