As Testers, How Do We Know What We Know?


problem is that neither leaves us with a satisfactory explanation for how we can know anything! Our conventional conception of knowledge is that anything that we can know should be the same to everyone, and we should be able to know it with certainty. Idealism grants knowledge that is certain for you but not everyone. Realism yields knowledge that can be shared amongst everyone, but since we cannot be certain that our perceptions accord with reality, our knowledge can never be certain.

Rationalism and Empiricism

Given this problem of perception, then, by what means can we acquire knowledge? Rationalism answers that we can gain knowledge solely through the use of reason. For instance, without making any observations of the world, we can formulate the idea of a triangle and describe its characteristics without having to resort to the use of sensory input. We can make logical concepts defined in our own minds agree with each other without reference to any external phenomena.

Empiricism, on the other hand, argues that we must rely on our perceptions (imperfect though they may be) to acquire knowledge. We come into the world as a blank slate, without any ideas on which to build our knowledge, and must experience things such as triangles before we can begin to reason about their qualities.

Logical positivism—also known as logical empiricism—holds that these two viewpoints are not necessarily mutually exclusive. Reason alone can demonstrate the truth of abstract constructs such as logic and mathematics, while controlled observation of the external world leads to knowledge through being able to verify one's theories about it.

The scientific method is based upon logical positivism, amongst other philosophical perspectives. To learn about the world, we must assume that it behaves by consistent rules and that we can observe the results of testing these rules. We think of knowledge and the truth of theories in terms of reproducibility and demonstrability. A theory is "true" to the extent that it lets us predict the observed result of some test or set of actions. As testers, we form theories about software and perform experiments to validate or refute these theories.

Takeaway Points for Software Testing

How does this inform our testing strategy? Historically, the more we've studied epistemology, the less sure we've become of our knowledge and our ability to acquire it! We started off with at least the hope of attaining perfect knowledge and ended up with the realization that we can never know anything about the external world with complete certainty. What benefit, then, have we drawn from several millennia of contemplating epistemology?

First, we can draw a distinction between two categories of knowledge: the realm of logical constructs, about which we can reason without resorting to the senses world, and all other knowledge, which requires observation of the world to obtain.

Second, our perceptions can be mistaken, so any knowledge that depends on our perceptions can be flawed. Imperfect perception is a reality of being human, so we shouldn't dwell on an occasional perceptual error.

Third, the truth of a theory about how the world works is measured via repeatability. A theory is true to the extent that it can predict reproducible results. A theory that we cannot test is one that we cannot obtain knowledge about.

Fourth, we can refine our knowledge by using the results of our experiments to refine our theories. We then test our revised theories, leading to a virtuous circle in which our knowledge becomes more accurate.

Finally, we can never be completely certain in our knowledge. We don't know with complete certainty that the sun will

User Comments

Sanat Sharma's picture
Sanat Sharma

A great article bridging the gap between philosophy and technology. I read this article thrice to have a deep understanding of it. But sometimes, I got confused about what the author is trying to say.

-- Sanat Sharma

August 16, 2011 - 1:24am
Gerard Miller's picture
Gerard Miller

Hi Rick,

Excellent article. Sometimes what we know is true may not be true all times in all conditions. The context is real-time systems.


From Jack Ganssle article from 7/29/2011

For instance, what is the likelihood the sun will rise tomorrow? Dumb question; for four billion years the probability has been 1.0. Surely it's safe for an engineer to think that the sun will indeed appear tomorrow as it always has. Five or six eons from now it will be a burned-out cinder, but our systems will be long landfilled by then.

Recently a developer told me about a product he worked on that changed the display's color scheme depending on whether it's night or day. It does a very accurate calculation of sunrise or sunset using location data. Turns out, a customer took one of the units above the Arctic Circle where it crashed, the algorithm unable to deal with a sun that wouldn't rise for months.

The sun may not rise tomorrow. Don't count on anything.

August 16, 2011 - 8:53am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.