Comment:     by David Oliver 1/28/2008 Hi Michael, Thanks for the article. Guess I'm not the only person to see those things. David Oliver
	Comment:     by Atulesh Kumar 1/3/2008 I feel it is result of "Job Done Vs Job Done SMARTLY". Work should be done keeping End-user in mind and it should give the comfortness. It happens and happend, Software Testers has reported the defect and rejected by the Developer because it has programmed that way. In this case the developer has done his duty to develope the website and as per his/her code, the pop-up works fine. But if it is done smartly, you would not have faced such problem. Thanks.
	Comment:     by Robert Vanderwall 1/1/2008 This article was well written and enjoyable. However, it seems to encourage testers to use emotion over logic. Emotional responses to violations in internal models can certainly be useful, however that response is predicated on the tester having such an internal model. Mr. Bolton was fortunate in that the application under test, even though it had no requirements, was familiar and he had a solid well grounded internal model already constructed. Many testers are not so fortunate. Let me give an example from my own experience. I've been a SW tester for about 20 years, now, and many years ago, we were building a product based on...Read On Author's Response: 1/2/2008     >Thanks Michael, for a well written thought-provoking article. And thank you, Robert, for some thought-provoking comments, as well as the kind words. >This article was well written and enjoyable. However, it seems to encourage testers to use emotion over logic. Can you identify where it says that? >Emotional responses to violations in internal models can certainly be useful, however that response is predicated on the tester having such an internal model. Mr. Bolton was fortunate in that the application under test, even though it had no requirements, was familiar and he had a solid well grounded internal model already constructed. Many testers are not so fortunate. In an actual test project, this isn't merely a matter of good luck. It also has a great deal to do with testers' management--and self-management. In my teaching and consulting, I've observed many occasions in which management focuses testers entirely on the functional correctness of the software, and not at all on other aspects of the system. Testers often make this imposition on themselves, too. As a group, toolsmiths in particular seem to vulnerable to this kind of self-imposed blindness. Moreover, the online application that I was testing--an airline reservation system--did not have "no requirements". It had no written requirements document, available to me. It certainly had requirements, the foremost of which was to allow me to purchase a ticket quickly, easily, and conveniently. Some statement to that effect probably occurred as a preamble to the actual written requirements document--but was that statement tested? If it was tested, did anyone act on the testers' observations? >In this application was a calculated field and when the testers looked at the field, it seemed reasonable. When the customer looked at the field, they thought it was correct as well. After several months in the field, we discovered that the field was, in fact, incorrect and had cost the company thousands of dollars. The problem was that we had no authoritative reference for the calculation. The customer had only a weak understanding of the calculation. Yes. That's why I said that test oracles "include comparable programs, documents, mathematical principles, personal experience, and people--'live oracles.'" (That's my emphasis, there.) I also said "Although emotional reactions don't come with a guarantee, they can often aid our decisions about the existence or the significance of a problem." That's "aid", not "replace". >While I certainly don't suggest that every (or even many) projects need heavy weight requirements documents, I do suggest that the formality cannot simply be discarded without some adequate substitute. If you can point out the place where I suggest that other sources of oracles should be rejected, I'd appreciate it. I don't believe I suggested such a thing, and I'm puzzled that you would make the inference. It's not an uncommon kind of inference, but I still don't understand the thought process that leads to it. For example, one correspondent inferred from my last column that I was attacking documentation, when I was in fact attacking quantitative measures of things that can't be counted usefully. >In summary, I agree with Boltons assertion that emotional responses can be helpful guides, but that the article inadequately relied the importance of strong domain knowledge. Here's the puzzle that I'm trying to figure out: we can read volume after volume after volume of testing literature (and believe me, I've done it), and refers incessantly to functional correctness, domain expertise, falsifiability, and the like . The amount of attention paid to the human side of the equation--heuristics, emotions, usefulness, value--asymptotically approaches zero. I've never heard a complaint from anyone--except from a handful of students of Jerry Weinberg--about that imbalance. Why not? Again, I do appreciate your comments. ---Michael B.

Back to Top