On Beauty, Quality, and Relativity

The saying “beauty is in the eye of the beholder” rings true whether you’re staring at a centuries-old painting, listening to a busker’s music reflect off the tiles in a subway station, or testing software. It’s one thing to evaluate quality, but how do we evaluate how we evaluate quality?

Most people tend to rate art on a “beauty scale.” But beauty is, and always will be, in the eye, ears, nose, mouth, hands, and brains of the beholder. A decision about beauty or lack of beauty in a particular work of art is instinctive and natural. In fact, you probably won’t even have to make that decision; you’ll either be captivated by a piece of art or you won’t.

Art critics, however, operate on a different level. They are not interested in mere beauty but also in characteristics like shapes, patterns, symmetry, colors, textures, composition, proportions, presentation, framing, inherent meaning, uniqueness, fulfilled intent, and skill of the artist. These are attributes that contribute to the overall “beauty” of a work of art, but there is more to the evaluation of art than meets the eye. The appreciation of a work (this applies to non-visual art forms, as well) is also influenced by the setting in which it is presented to an audience.

This became clear through an experiment conducted by the Washington Post in 2007. On a cold January morning, in a Washington, DC metro station, a man with a violin played six pieces by Bach (Johann Sebastian, not James Marcus). This lasted for about forty-five minutes. During that time, approximately 2,000 people went by, most of them on their way to work. The man played continuously. Six people stopped and listened for a short while. Twenty people gave money but continued to walk at their normal pace. Only one person recognized the musician, and she couldn’t believe her eyes and ears.

Here’s the catch: That violin player wasn’t just an anonymous street busker. He was Joshua Bell, one of the greatest violin players in the world. That morning, he was playing the “Chaconne,” one of the most intricate pieces of music ever written, on a 300-year-old Stradivarius violin with an estimated worth of $3.5 million. He collected a total of $32 dollars. The night prior, he earned considerably more playing the same repertoire at the Boston Symphony Hall, where regular seats went for $100.

This experiment addressed some interesting questions:

    • If we’re confronted with beauty in a commonplace environment and at an inappropriate hour, do we perceive it? Do we stop to appreciate it? Do we recognize talent in an unexpected context?
    • What is beauty, really? Is it a measurable fact or merely an opinion—or is it a little of each, colored by the immediate state of mind of the observer?
    • If a great musician plays great music that no one is hearing, is he any good? This is an old epistemological debate that continues to engage philosophers.

Each passerby had a quick choice to make: Do I stop and listen? Do I hurry past with a blend of guilt and irritation, heading for that first office coffee? Do I cough up some cash, just to be polite? Does it make a difference if the player is really bad? What if he's really good?

Lots of people didn’t bother to listen that January morning. Does that mean we should label all those commuters as unsophisticated yahoos? No, because when it comes to beauty, context matters.

The 18th-century German philosopher Immanuel Kant, who took beauty seriously, said the same thing in his Critique of Aesthetic Judgment (1790). He argued that one's ability to appreciate beauty is related to one's ability to make moral judgments. Furthermore, to properly appreciate beauty, the viewing conditions must be optimal—and "optimal" doesn't mean heading to work, focusing on that report to the boss, or fetching the kids from school in time.

Much has been said about the impact of biases on our noticing—how it is easy to miss things. If we fail to recognize genius in a metro station, how many other things are we missing through incongruent contexts?

As I was first reading about the Joshua Bell experiment, I couldn’t help but seeing the similarities with software testing. After all, testers are often asked to evaluate the quality of a piece of software. But, just like beauty, quality is highly subjective. Quality characteristics are all factors that contribute to the overall quality, which in the end is the sum of all the quality scores of all persons involved. Such a thing is hard to gauge.

If we take a look at Jerry Weinberg’s definition of quality as “value to someone,” shouldn’t the context in which we are assessing that quality be taken into account as well? After all, there are a lot of factors that are capable of influencing the quality of our judgments:

    • Feelings: Isn’t quality perceived differently by testers when experienced in different circumstances—for instance, when they are in a hurry, when they don't want to be bothered, or when they're not in the mood?
    • Expectations: What about a tester’s view on quality when he already "expects it to be mediocre" (like one might expect from most street musicians)?
    • Words: Seemingly trivial conversations can have quite an impact. Testers are often primed by simple phrases from everyday tester life: “Oh, you don’t need to test that; I didn’t change anything there”; “Can you perform this little test for me? Don’t sweat it, it will probably just take an hour or so”; “Any news on those test results? Boss wants them by noon—just saying.”
    • Distractions: Your daily life and work environment is a tasty smorgasbord of distractions: kids, desk, mails, colleagues, and your computer screen with those social media that need scouting 24/7. All of these interactions may provide much-needed defocusing, but they are not creating an optimal setting for our quality-related efforts.

Quality has a lot in common with beauty. To properly assess the characteristics of a particular product (or work), the testing (or viewing) conditions need to be optimal. If quality is “value to someone,” maybe we should extend that with “in optimal conditions” or “in an ideal context."

This leaves us with another question: Is there such a thing as “optimal testing conditions”? What are those, really? People are people; we will always be biased and fooled by our own minds. Neither can we deny that we are living in the age of distractions. So, how can we set up for a totally unbiased, unspoiled look at quality?

We probably can’t, but what we can do is try to make those conditions as good as possible. How? By creating an environment for testing in which testers get enough freedom in their work, or by being more aware of possibly disturbing factors, taking them into account, and acknowledging them when testing. We can try to eliminate as many distractions (interpersonal, technical, and environmental) as possible. That way, our judgments on quality will be of high quality.

User Comments

Linda Rising's picture
Linda Rising

I have heard the Joshua Bell story many times and I love it! I think the message is a good one. We see and hear what makes it through our unique set of filters. I'm sure you've read the recent study that showed professional violinists couldn't tell the difference between superb modern violins and priceless Strads :-)! Thanks for this great comment on beauty!!

March 6, 2012 - 9:39am
Ajay Balamurugadas's picture
Ajay Balamurugadas

Excellent article Zeger. I liked the way you highlighted the similarities between beauty and software testing. The points about "optimal testing conditions" are interesting. If we wait for the optimal testing conditions, we might never test. At the same time, good testing might require good conditions. :)

Thanks for the article.

March 6, 2012 - 11:37am
Tim Thompson's picture

I think that establishing a process and sticking to it might help keeping quality assessments consistent. The process needs improvements as needed, but those improvements can be made when there is sufficient time for this.
What I mean by process is defining the testing tasks upfront, especially the creation of detailed test plans. And with detailed I mean detailed to the nth degree. All these plans have to be reviewed, ideally by other testers and business analysts (or whoever created the requirements). Once there is broad consensus that the test plans are on point and sufficient then testing can begin. The benefit is that the time testers need to 'think' is removed for the most part from the time when tests are executed. That allows for focusing on many other things that the tests failed to cover, that is typically things that are not related to the items to be tested, but that are still incorrect.
That translates to the Joshua Bell story. The people in the subway might have stopped and listened more willingly if their minds weren't occupied by other tasks such as catching the correct subway to be at work on time and which things need to be done today.
The Joshua Bell example is like exploratory testing. Testers might find the most obvious flaws in software (or violin playing) by simple consumption, but it is impossible to cover all aspects and document what was done and which results were achieved by the test. I agree that it is faster in finding the big ticket items, but for the nitty gritty stuff you need to sit in the concert hall and focus on the one thing.

November 22, 2014 - 10:09am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.