Logic and Software Testing

Formal logic is what runs computers, but it is only a part of the logic used by a software tester. In this installment of his ongoing series on philosophy and software testing, Rick Scott explains.

Finally, we come around to a branch of philosophy in this series that most people will immediately associate with software. Logic is what runs computers, right? After all, they are logical machines. However, the software we are testing often seems to behave in a way that is anything but logical. So, what exactly is logic, and how is it relevant to software testers?

What Is Logic?

We bandy the word “logic” around a great deal, but our mental definition of it tends to be a bit amorphous. While we have a good general idea of what logic is, we tend to conflate it with “thinking" and “reason.”

Thinking is, broadly, any mental process. Reason is a kind of thinking—specifically, a mental process of arriving at an answer to a question or of deciding amongst choices. Logic tries to enumerate rules for reasoning—rules that allow us to reason in an orderly manner and help to ensure our conclusions are sound. As such, logic is invaluable for ensuring that we make robust decisions, that we are systematic in our consideration of difficult issues, and that we can perceive the flaws in erroneous arguments before they mislead us.

Formal Logic

Formal logic (also known as “mathematical logic”) is the flavor of logic that comes to mind when we speak about computers. It’s based on “propositions” or “Boolean variables” that can be either true or false. These are combined with logical connectives, such as AND (true when all constituent propositions are true) and OR (true when any constituent proposition is true). This, along with conditionals (IF ... THEN), is the foundation of how computers “make choices” or “reason” at both the hardware and software levels. For example:

IF the user is logged in,
AND the user has the correct permissions,
THEN show the user the configuration page.

IF this exception is uncatchable,
OR we haven't provided a way to handle it,
THEN crash.

Testers make use of formal logic in ways beyond understanding the machines we work with, particularly when it comes to testing strategy. Whether we explicitly say so or not, we often plan our test steps and allocate our time using formal logic: IF performance is slow AND it’s slow in browsers other than Internet Explorer 6, THEN I'll spend more time on performance testing; otherwise, I'll devote more time to inspecting the new UI. In those blessed (albeit infrequent) scenarios where we can enumerate all the possible inputs and outputs of a test scenario, we can use truth tables to make sure we don't miss anything and sometimes tools such as Boolean algebra or Karnaugh maps to separate the inputs that should affect the outcome from the ones that shouldn't.

Informal Logic

As software professionals, half of our job is interfacing with technology and half of it is interfacing with people. While it is laudable to have found an important bug buried deep in a application’s dark recesses, it does no good for the users of the software if you can’t convince management or the development team that it is worthwhile to fix it.

Informal logic (sometimes called "persuasive logic") is how we form arguments and attempt to reason with each other in our everyday lives. As opposed to the mathematical structure of formal logic, it deals with argument and reasoning in natural language. An argument consists of one or more premises, a line of reasoning, and a conclusion reached thereby. Informal logic outlines what constitutes a sound premise and what constitutes valid reasoning so that the conclusions we reach are both justified and defensible.

As one of the core skills

User Comments

shrinivas Kulkani's picture
shrinivas Kulkani

You missed "symbolic Logic" - also how does all of it ties to software testing

Shrinivas Kulkarni


October 10, 2011 - 4:05am
Harry Theus's picture
Harry Theus

This article was both informative and entertaining. I enjoyed reading it.

September 22, 2011 - 11:36am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.