As Testers, How Do We Know What We Know?

Software testing is a process of acquiring knowledge about software. But, how do we know what we know or how to acquire knowledge? Rick Scott shows what the study of epistemology means for testers.

One of the central questions—if not the central question—of software testing is "What do we know about this piece of software?" When we know enough about the software under test, we can stop testing it. But, how do we acquire knowledge? Given that our perceptions are sometimes unreliable, can we actually be certain of learning anything through the senses? How about through intuition or feelings? Are there some things we can't know? Epistemology is the branch of philosophy that studies knowledge and attempts to answer questions such as these.

Building Blocks: Knowledge and Truth

For a concept so fundamental, philosophers surprisingly don't agree on a single definition of “knowledge.” That said, the most widely known (though, again, not universally accepted) formulation is that knowledge is justified true belief .

Let’s take these one at a time. For some proposition to count as knowledge, it requires:

    • Belief: You have to be aware of the proposition and, further, think it to be true.
    • Truth: It has to actually be true.
    • Justification: There has to be some reason for believing it to be true.

Theories of Perception

Because the ways in which we can gain knowledge about the universe are fundamentally dependent on how we believe the universe is constructed, epistemology builds on and is closely related to metaphysics.

We perceive the world through our senses and our mind, but these perceptions can be mistaken. In dreams we perceive things that do not exist. We may hear a noise but find out other people in the room have not heard it. Two people may witness an event yet not agree on what actually happened. It seems that we cannot rely on our senses for knowledge. How, then, are we to know anything about the world?

As one might expect, the answer to this question depends heavily on how the philosopher answering it constructs the world. Plato, an early Greek philosopher, asserted that true reality does not consist of the physical world around us, but of forms—perfect, abstract, immutable concepts such as beauty, harmony, and blue. We can't perceive the forms directly; instead, we see them manifested (albeit imperfectly) in things like beautiful landscapes, harmonious music, and blue skies.

Plato believed that we are all born with intrinsic knowledge of the forms but that this knowledge is latent. In his view, true knowledge comes from becoming acquainted with the forms, not by carefully studying their imperfect manifestations in the physical world. Since the forms are perfect and our knowledge of them is innate, perfect knowledge of some area of expertise is something we can hope to attain.

Idealism and Realism

Broadly speaking, later philosophers tended to take one of two different approaches to the problem of perceiving reality: that of Idealism or Realism. Idealism answers that our perceptions are reality. Objects exist only as bundles of sensory data in the minds of those who perceive them. Our perceptions are all we can be truly certain of, and they are the measure of all things.

Realism, by contrast, declares that reality exists independent of perceivers or perceptions. Direct realism says that we can directly perceive the world through our senses and thereby form an accurate impression of it. However, we already know this position to be problematic, because our senses are unreliable. Indirect realism says that, despite the fact that an external world exists, we cannot perceive it directly. We only know what our senses tell us about the world, and those perceptions can be valid, incorrect, biased, and so forth.

Both Realism and Idealism seem at least somewhat plausible. The

User Comments

Sanat Sharma's picture
Sanat Sharma

A great article bridging the gap between philosophy and technology. I read this article thrice to have a deep understanding of it. But sometimes, I got confused about what the author is trying to say.

-- Sanat Sharma

August 16, 2011 - 1:24am
Gerard Miller's picture
Gerard Miller

Hi Rick,

Excellent article. Sometimes what we know is true may not be true all times in all conditions. The context is real-time systems.


From Jack Ganssle article from 7/29/2011

For instance, what is the likelihood the sun will rise tomorrow? Dumb question; for four billion years the probability has been 1.0. Surely it's safe for an engineer to think that the sun will indeed appear tomorrow as it always has. Five or six eons from now it will be a burned-out cinder, but our systems will be long landfilled by then.

Recently a developer told me about a product he worked on that changed the display's color scheme depending on whether it's night or day. It does a very accurate calculation of sunrise or sunset using location data. Turns out, a customer took one of the units above the Arctic Circle where it crashed, the algorithm unable to deal with a sun that wouldn't rise for months.

The sun may not rise tomorrow. Don't count on anything.

August 16, 2011 - 8:53am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.