It often seems a fact of life, that processes are made to be broken.
In nearly all the organizations where I’ve directed testing efforts–large or small--there have been points where we've knowingly dodged best practices with feature design and product requirements. Sometimes money talks over reason. Sometimes high client demands require a rapid introduction of an enhancement or functional change. Sometimes you need to fix something—quick.
Recently, I encountered an example of this, where a new report was required in our software to address an immediate need of a new customer. In this case, it was the addition of a report to be filled out by Parks & Recreation Officers, during Boating Accidents, which was a somewhat new concept for our company. The deadline for completion was firm, and the information needed came late.
As I heard the stirrings of the new development being introduced, and could foresee needing to soon be testing the report, I decided I'd dig and find what I could to obtain at least basic elements for test planning.
Not expecting much, I went to key people, asking for product requirements. ("Asking, at least shows I'm trying to follow and enforce process—and any information is better then none"—I thought).
Here are some responses from my questioning:
Project Lead... "No, we’ve really just been developing this on the fly, with regular interaction with the client—and I don’t hold QA too responsible, since we expect to have the client continuously sign off on the screens and functionality."
"But wait, I do have this sample report that we obtained from the client, that I’m providing to the technical writer to review–that the client found mimics some of what they need. At least 50% of what you see on here should be on the newly-developed report in our application. I can probably clarify some of the other differences you may encounter."
Developer 1:
"No Requirements or Use Cases; I just mocked up the report print layout based on conversations."
"Oh–But, I did create these 12 sample reports that make good test cases to try. Look at these printed documents, and see if you can produce them the same way, in the product."
Developer 2:
"Are you kidding? We've been developing at light speed to finish this on time. No Requirements or Specs."
"Oh, but look at this .xml file I generated. I actually coded a comment by each report rule with testing in mind. Search and pull out each of these rule comments in the file, and that should provide a basic set of rule requirements for the report, that you can spin into an informal test plan."
Developer 3:
"I did the report submission interface, and just coded to a database schema the vendor provided. As this report submits, you can't really see what is validated and communicates across the interface to the national Coast Guard database."
"But, let's try some scenarios against a test database, and we have a way to tell, behind the scenes, if each condition is met, and that the data is actually transmitted for submission correctly. Here's the schema they gave me, so you can see all the elements that pass across, and the valid expected values for each required field. Draft some data validation cases, and we can work together to confirm all data transmission looks good."
Developer 4:
"Well, it may not help much, but I have this Coast Guard regulation guide, that I used quite a bit in determining what data elements are necessary, to be reported. Search the electronic format of it, for all regulatory code sections, and you should be able to validate the options and guidelines involved in each section, against this source."
We followed up these discussions with a QA Brainstorm Session and answered these questions:
What are basic tests we often conduct for reporting in general?
Examples:
Data entry in all fields, functional behavior
Add/Change/Delete
Report review, approval, and submission processes
Print preview. Print functions. Export/PDF
Post submission–historical record searching
Basic field limits/format tests
Tab Ordering; Shortcut Keys
What defects/regression tests from other reports can we conduct?
Examples:
Data retention defects
Memory handing issues
Interface display issues
State transition issues–screen workflow
Local and server record storage conflicts
Are there any other logical sources we can research/learn/apply?:
We had some background in the QA group with the Marine industry that proved helpful
We looked up industry standards for things like boat HULL# assignments, state vehicle registration procedures, boating safety regulations
We reviewed surrounding sources for things like common boat makes, models, and types
We researched and discussed general terminology–things like: vessel, beam, propulsion, docking, mooring, towing
As you can see, with a little digging, and asking the right questions to the right people, we produced many items to formulate test cases, and optimize the test team's efforts.
Even when we don't see them in the formal, documented ways that are best–there are always requirements–in some fashion, as a foundation to software development. Even in the most difficult settings, see how you can gather and structure facts that will drive the best testing possible.
Lets Hang!