When it comes to what testers should focus on, people always say you have to think like a user. I used to think I was good at doing that—until I started actually sitting next to users.
Now I know there are all kinds of questions that I should think about more often as a tester. In what kind of physical environment do users use the application? What kind of pressure is affecting them when they try to solve their problems with the application? What frustrates them? What do they value?
All these questions are easier to answer if you're able to sit next to the people who use your application in their everyday environments. I’d like to share what my experiences sitting next to users have taught me about how our users think, how our applications meet their needs, and what you should consider when it comes to your users’ interactions with your product.
Sit Down with the User
I work at an insurance company in Finland as a test specialist in the IT claims solutions unit. We develop applications (some for the computer, others for the web) that deal with claims our customers report. This includes applications that our internal claim advisors use to serve our customers.
These internal applications are the ones I'm able to observe when I sit next to our claim advisers. The advisers use the apps, for example, when they try to find information about the customer or when they are registering a claim on behalf of a customer. While some of the applications they use on a daily basis are developed by our unit, many are developed by other units or by external vendors.
In addition to testers, our developers, managers, and team leaders are all encouraged to sit next to our users at least once a year. I’ve gotten to do that three times in the last six months. It's easy to schedule a session to sit next to a claim adviser. If they are on the same city as I am, the session can be scheduled for the next day. I usually observe an adviser’s work for approximately two hours. I’ve found it to be a good length of time, as it can be exhausting to focus longer and maintain the ability to make observations, at least without breaks.
Because a major portion of the claim adviser’s time goes to answering calls from customers, I also wear a headset so I can hear the conversation (although not comment myself, as I'm muted) and understand better what's happening. (I am bound by professional confidentiality, so any customer-related information is not written down or talked about outside the session.) In between the calls I'm usually asking the adviser clarifying questions and listening to how they describe the challenges they face with our applications.
When I sit next to the claim advisers, I begin making observations immediately.
I first pay attention to their work environment. In one of the sessions, I noticed that the user had two external monitors that were quite small—perhaps seventeen inches. But I noticed that her colleague had only one external monitor, which was twenty-seven inches. This was useful information for an application we later developed because I knew to pay extra attention to testing it with different screen sizes.
I also pay attention to what users do when they jump from one application to another. There was a session where a user copied and pasted information between applications. Pasted information was in an undesired format, but I was more interested in the fact that the user needed the information in the other application. This knowledge will help us in a project coming up where we will build an application to replace the one the user copied information from. We can at least try to make copying the information easier than it is now.
Even if you can improve your products based on what you see your users doing, chances are good that all their problems cannot be solved by your applications. It’s important to observe what other applications or tools they are using for solving their problems. For example, I noticed that one of our users was using a website for calculating a distance between cities. It wasn’t a need we could do something about immediately, but it will help when we further develop our applications or replace them with new ones. These kinds of problems can be easily forgotten if requirements reflect only existing features of an application.
Interact with the User
Besides observing the claim advisers while they handle the calls from our customers, I’ve also asked clarifying questions after each call. It’s useful to know immediately after the call if there was anything the claim adviser was frustrated or confused about. What was making it harder for them to serve our customer? What could make it easier?
And it’s not just about learning how claim advisers interact with our systems; growing empathy matters also. When you get to know your users as human beings, they turn more into “us” instead of “them.” For these reasons, I also pay attention to personal items on their desks and sometimes engage in conversation about informal subjects.
Listening to the phone calls customers make to our claim advisers has also helped with empathy because I better understand the pressure they face while using our applications. The person the adviser is talking with could have just been in an accident or other stressful situation, which requires mental engagement from the claim adviser as well as product knowledge. This is something that is usually missing when I’m testing, as I have the luxury of focusing only on testing the application.
Unless you have an exceptional memory, you will most likely benefit from taking notes while observing and interacting with a user.
I personally prefer writing down notes in my Moleskine notebook. A physical notebook gives me the freedom to either write or draw quickly, depending on what I want to document. Based on my own experience, I also feel that a physical notebook doesn’t create the impression that I’m focusing on something else than our session. If I would write notes on a laptop, there’s a danger the claim adviser will think I’m doing something else. However, I know many people prefer taking notes on a computer, so if that’s what you choose to do, I think it might be useful to mention explicitly to the user that you’re writing notes.
What you write down is something you need to figure out yourself, but here are some of the things I typically take notes on:
- What has frustrated our users
- Something I’ve learned about our business domain
- What the calls were about, so I can remember the session as a whole more easily
- What the adviser had on their desk (monitor, notebooks, calculators, etc.)
- Improvement suggestions
If you have been inspired and would like to sit next to a user of your product, where do you start?
First, of course, you need to find a user. It’s easy for me because many of our applications’ users work in the same company as I do and we have a culture of sitting next to our users, but if you don’t have that kind of culture, you can still just ask an internal user if they will allow you to observe them when they use your product. People are often delighted that someone wants to listen to them and give them a chance to share their experiences.
In the case of an application that is not intended for internal users, it requires a bit more creativity. A couple of years ago I was testing a global e-commerce site. We wanted to get feedback from people who were not that familiar with the site, so we decided to go with crowd-sourced testing. I sent an email to about fifteen people, all of whom had been involved with the product but hadn’t used it themselves, asking for their help in getting feedback—and I mentioned that there would be snacks available.
About ten people arrived for an exploratory testing session, where they pair-tested the site based on high-level objectives I gave them. Choosing people unfamiliar with the site proved to be a good tactic, as they made a lot of observations and suggestions for improvement.
Whether you’re getting feedback internally or externally, it’s important to remember in both cases that information is more valuable if the situation where you observe users using your product is as close as possible to the real everyday situations the users typically experience.
When you’ve managed to observe a user and gained new information about your product, share that information with your team. This opens a window of opportunity for getting a better collective understanding about the actual impact our applications have.