In the 2011 Boston Marathon, everyone running had a wearable attached to their clothing. In the race bib with their name and registration number, there was also an RFID, or radio-frequency identification, chip, which recorded the runner’s exact race time by detecting when the runner crossed the start and finish lines.

The first time this system was tried, there was only one glitch: not all the RFID chips registered with the readers. As a tester I found this fascinating, but the Boston situation was personal—I was in the race.

No, the failure did not create a life-threatening situation. But it did create a great deal of consternation and disappointment for runners whose race times were not recorded. For runners with an Olympic qualifying time or personal record, their elation and joy at the finish line turned to grief and anguish when they found out their times did not register and were not recorded with the Boston Athletic Association. I was one of those runners.

As a tester, I began to question not only what had and had not been tested, but also the impact of the failure on the user. I realized that what all wearables have in common is that they have a purpose or function that is coupled with human interaction, providing value by enabling the user to achieve a goal. Unless the runner ran the course and stepped on the mats, the chip in the runner’s bib would have no way of providing any value.

This analysis led me to realize that the human user must be an integral part of the testing of wearables. Furthermore, the closer a device becomes to a human, the more important the human’s role in testing becomes. When a networked device is physically attached to us and works with us and through us, the more personal, even emotional, the interaction is. With wearables, the user becomes a part of the Internet of Things. From this experience, I devised a method to test this collaboration, which I call human experience testing.

Human Experience Testing

Human experience testing focuses on testing the interaction with the wearable and on how the value is provided. It differs from usability testing in scope, depth, and approach. It involves testing in the “real world” of the user: when and where the device will be used; the interaction between the device and the users’ actions, senses, and emotions; and how that interaction will provide value.

The most effective way to design human experience tests is using human-computer interaction design principles. A persona is a model of a potential user. Personas are developed by listing all the characteristics of the user, including age, family background, level of education, occupation, socioeconomic status, physical size and condition, gender, hopes and desires, point of view, values, and life expectations.

By developing personas, we delve into the users’ personalities and characteristics and we begin to understand their expectations of the device. Then we create user value stories to test the ways in which the human—or, in the case of some wearables, animal—user will achieve value from the device.

For any particular mobile device or wearable, there may be several different personas, and there will be several user value stories for each persona. If the device will be worn by an animal—for example, the Whistle, a fitness tracker for dogs—there will be multiple personas for both the dogs and the dog owners, and there will be several user value stories attached to each set.

Test scenarios based on the personas and user value stories include testing the physical, sensory, orientation, geographical, and contextual expectations the users have for the device or wearable. The test scenarios also involve the emotional, physical, and sensory reactions, biases, and mindsets, as well as the social expectations and interactions, of the user.

As an example, let’s develop a persona to test my sports watch, based on my marathon story above. Let’s begin with my demographic characteristics. Because I am a marathoner, I am a distance runner. I run on the road; a trail runner would have a different persona. I am obviously conscious about health and physical fitness, and running is likely a lifetime sport. Most marathoners are over thirty years of age, however; personas should be built for various age groups. I am a woman and small in stature. A persona should also be written about a male runner. Marathon runners will train and race in all types of weather conditions.

Now let’s complete my persona with the psychosocial characteristics. I am a member of the running community, and I train both alone and in groups. Running is a source of fulfillment; I thrive on both individual achievements through personal records as well as competition with my fellow runners.

From this persona we can derive many of my expectations, which will become test scenarios for my sports watch. For example, from my demographic characteristics, because I am small in stature, the watch must fit my wrist, and as I’m older, I need the screen to be legible without my glasses while I’m moving. From my psychosocial characteristics, because I’m competitive (or at least like to think I am), I require the watch to be accurate and reliable.

Personas as a test activity work well; they can also dovetail nicely with a requirements activity, or deciding what we will build. The personas provide the basis of the test scenarios for testing the interaction between the user and the device, but for complete human experience testing, we must also test how the device provides value. This testing is designed from user value stories.

Like any story, a user value story has a beginning, middle, and end. It has a main character: one of the personas that had been built previously. It discusses common practices when using the device, and how the device might help in these situations. It includes realistic situations and field conditions such as weather. Sensory perceptions, including sights, sounds, tastes, smells, and what the user may be touching, are also extremely important to include, as are the user’s feeling and emotions. It is written in paragraphs, the same way you would write a story.

There are usually at least three user value stories written for each persona. There is a “happy ending” story where interaction provides the desired value, a “sad ending” where the wearable fails to provide the expected value, and a “medoicre ending” where some value is provided but it is not what the user wanted.

So, let’s return to my 2011 Boston Marathon run. Yes, the user value story of my interaction with the RFID chip in my bib is an example of a sad ending. However, there is another user value story from that day: I was also wearing a sports watch, and that story has a happy ending. My sports watch captured and saved my race time, which the Boston Athletic Association accepted as a qualifying time, and I was able to run again in Boston in 2012.

Although my stories of my RFID chip, my sports watch, and the 2011 Boston Marathon are neither life-threatening nor life-changing, they illustrate the importance of testing the human experience when testing wearables. And the closer a wearable becomes to the user, the more important this framework of testing becomes.