Would you like to do a test exercise right now? Don't worry, you won't be graded or compared to anyone else. This is just a learning exercise.
Go to the site tinyurl.com/testwords. This program was built as a test exercise during WHaTDa, or the Workshop on Teaching Test Design, earlier this year. I'll explain the rules, and then you can test.
The Rules
At the top part of the page, you’ll see a simple function that tells you whether the text you enter is a palindrome. (A palindrome is text that is the same forward and backward—so, "bob" is a palindrome, but "matt" is not.) Take a few minutes to make a list of text you plan to enter, along with, perhaps, indicating on your list whether each item should be identified as a palindrome by the function.
We could stop there, but that would be too easy, wouldn't it?
If you want to have some fun, find a partner. Have your collaborator make the list of ideas for text to enter also, then swap lists. Look at what the other person came up with and see if that gives you any ideas. Switch papers back, draw a line under your last entry, and add anything.
Finally, go through and run your test ideas. During the course of the testing, you might come up with a new idea to test—document those, too.
That's a straightforward exercise ... but it barely scratches the surface.
There's More to Testing Than This
The exercise above is all about the algorithm: input, transformation, expected result. It doesn't tell us anything about who the user is, how he or she intends to use the software, or anything about the platform.
There are many user-oriented questions we could ask to improve our testing:
- Does the user care about capitalization? Does "Bob" = "boB"?
- Does the user care about two or three spaces? HTML combines more than two spaces into only one, so "a bc cb a" would look like a palindrome.
- Does the user care about special characters? Should they be ignored? Is "{[]}" a palindrome, as its reverse is actually "}][{"?
- What about security and HTML injection? Typing HTML into the textbox can actually inject that HTML in the page, so typing "nbsp;" injects a space; “<HR/>” injects a line.
- What about extremely long strings? Is our user likely to cut and paste something that is hundreds of kilobytes in length?
- Are we limited to the extended ASCII character set? Japanese "palindromes" don't pass, but does that concept even make sense?
- Does the software only look for words, or would it recognize a string of sentences that would be a palindrome? Should periods be stripped out?
Because we have never met the user or had the user described, we don't know if these things matter.
We could even do another exercise here where we play "customer" for each other and answer these questions, to see how that changes the testing approach.
But forget about the customer for a moment. Let's get back to the original idea of testing specific palindromes at the input level. Our test ideas are a simple list of what we type in, what we expect to see, and maybe a checkbox if it’s correct.
But the software could fail in a number of different ways that are outside those parameters. Here are just a few:
- The software is extremely challenging to read on an iPhone.
- The JavaScript doesn't run if the user turns JavaScript off.
- Memory leaks could occur if the software is running too long.
- The software could encounter performance issues.
- Old or unsupported browsers could render the text wrong or run the JavaScript improperly.
- The software might not comply with accessibility guidelines for vision-impaired readers.
There are also other usability concerns:
- User interactions are painful. There is no help, and there are no margins around the text.
- Should the software accept only text characters and spaces? Perhaps the textbox should have an input mask.
- Should the text resize itself based on the size of the screen being used to view?
- How many simultaneous users can use the application at the same time?
- When you enter a word to determine whether it’s a palindrome, the generated text tells you yes or no and then gives you the text you entered, reversed. However, the word reversed is spelled reserved; that's a bug, but not an algorithm bug.
It's easy to call these problems out of scope, or say they are "-ities", like usability, scalability, or security. Some test teams explicitly carve off all these concerns and say they are only dealing with functional testing. Yet the questions raised by this sort of thinking can lead directly to high value for customers—or not.
In test training, I find it useful to explore three aspects of the software: I start with the user or the business (what matters and why it matters), then test inputs and transformations ("functional testing"), then platform and other considerations.
Broadening Testing Perspectives
We've explored a tiny piece of functionality with just one input, one output, and, conceptually, only three or four lines of code. And look how much we found to test.
So, let's make this interesting. Keep exploring the palindrome algorithm, and present the bugs you find in the comments section below. I’m looking for the best functional bug and the most interesting bug. Feel free to use the comments to ask me questions as if I were the customer, too.
Let’s have some fun and discover some new useful test strategies.
User Comments
1) Doesn't (seem to) handle arabic palindromes either: تُوْت is one such valid value, but fails here.
2) Doesn't handle sentance palidromes: 'rise to vote sir' fails.
3) Palidromes doesn't translate unicode values to characters prior to checking, '♠♠♠'
4) Anagrams always returns 5 possible anagrams regardless of input value length
Nice Article, Matt. Short, sweet, and beefy.