How We Test Software at Microsoft
Published:
2008
Pages:
405
Upcoming Events
Oct 13 |
Agile + DevOps USA The Conference for Agile and DevOps Professionals |
Apr 27 |
STAREAST Software Testing Conference in Orlando & Online |
User Comments
BEFORE RELEASING its products, does MS detect and report every failure that its entire Customer base will encounter over the first, say at least, 5 years of its release?
If so, then please disregard the rest of my statements.
Otherwise this is by far the most subjective review I have ever read on this website that ironically includes both Fact & Fiction....
an Objective with possibly deniable Fact: "how Microsoft managed to become the biggest software engineering company".
a Subjective with UN-deniable Fiction: "how they have embedded quality within all of their processes, especially with utmost concern for customer needs."
Or just might be that I am not one of MS's targeted Customers.
If we had released as many defects with Customers' resulting failures that we have experienced with MS's products, we would not be working for any company where I have worked. And we use just the most basic and most common of MS's products.
This is a World-wide company with an International dictionary that does not even include the term "Testware".
Did anyone else notice that -- much less find it significant?
After reading the posted review, a MS television commercial sprung to mind -- the one where an MS QA/Test Manager appears to be attempting to balance himself on something and while doing so says (& I paraphrase): "I guess it is my job to break things."
If my job was to test cars and I pushed one of a 1000 foot cliff, then gee, wonders among wonders, I could report that I broke it and get a pat on the back -- certainly not from my Developers.
Obviously this is a company that has little, if any, understanding of the Problem Domain Curve over a product's Lifecycle -- the graph that illustrates the differentiation between a probable 'realized' Customer failure and those that while possible are virtually improbable.
I would not be surprised that many if not most/all of the MS Testware Developers still use terms like: "exhaustive testing", "positive & negative testing", "corner tests", ....
This thinking went out in the early 1980's -- along with the term "bug" that was replaced with "defect" with much Thanks and due Gratitude to Edsger Dijkstra, and Bill Hetzel for recognizing that ALL Test Cases/Scenarios are 'positive'.
BEFORE RELEASING our products, we did detect and report every failure -- and most defects -- that our Customer base encountered over the first 5 years of our the products' release. It quite probably could have been much longer if not for the limits of our measurements due to personal changes.
Can you spell Risk Management -- the Heart of the STEP methodology.
How can this be successfully achieved without these metrics that INCLUDE REAL WORLD Customer experience?
ALL Test Cases through Real World Operational Profiles were auto-generated.
These realized a minimum of 80% Statement Coverage of the intended functionality.
Moreover ALL TEST RESULTS ANALYSIS was automated with complete Traceability to each & every Requirement using an Expert System Help facility, transcending archaic and woefully limited binary Pass/Fail technology.
Our employer was a Software/Hardware vendor with hundreds of thousands of Customers. The tested software was a minimum ~1/4 to a maximum 1/2 of all its released software products' code.
This was my (might well be considered less than) Humble attempt to say that I have learned a thing or two on the Road to Zero Defects.
I have yet to experience the released results of MS embedding "quality within all of their processes, especially with utmost concern for customer needs."
If my experience with MS's products is what you call 'quality', then we are most definitely not on the same page, much less in the same book, much less in the same library -- is saying much less than on the same planet going too far?
An honest tale speeds best, being plainly told.
Source: King Richard III", Act 4 scene 3
Every man has business and desire,
Such as it is.
Source: Hamlet", Act 1 scene 5