In part 1 of this interview with Jonathan Kohl on mobile testing, he talked about some of the basics of testing applications for mobile devices and some of the challenges that traditional software developers face when going mobile. In part 2 of the interview, we discuss the testing of location-based services, the value of a good social network for crowdsourcing, and how poor optimization practices may literally burn a hole in your pocket.

Joey McAllister: What are some of the approaches to testing location-based applications? Is this a good excuse for testers to get out of their offices and go on long walks with the devices? Are there virtual tests if you can’t get out of your office or if you need it to work in another country?

Jonathan Kohl: Some of those things you can test with scenarios. I worked with a corporate client, and we tested some of these with their application. Someone realized that a microwave is a good Faraday cage, which means that there’s enough metal there that the radio waves can’t get through. So, wireless broadband, WiFi, and telephone calls—all that stuff that the device might depend on—aren’t going to work if the phone is in the microwave. While they were doing an exercise testing network and location, they were able to mimic some of these effects by putting a phone in the microwave. (They didn’t turn it on, of course.)

When we use the devices, most of us know the dead spots. We know where WiFi is weak, especially if you’re working in a big building where there’s a lot of metal. You can use those to your advantage. If an airport application needs constant Internet connection, you can test it with poor network connections and say, “Maybe we should store key information on the device itself, so no matter where people are, it gives them the core information.”

When it comes to testing things in different places, that’s where things get a little bit tricky. Yes, you do need to get out and walk and move around, not only for the actual location being different than at your desk, but also because there are sensors in the phone that help you determine things.

There are several ways of determining location. Common ones include:

• Using your web IP address the way we do with web browsers. Those correspond to databases that these services have, so there can be a lot of errors if my postal code is assigned to the wrong city.
• They can take your location off of the cell tower you’re closest to. That’s OK, but it might be a very powerful tower and you might be in another locale or municipality. It says you’re here, and actually you’re there.
• More accurate is triangulation, where they use multiple towers to hone in to a location where you might be.
• GPS is probably the most accurate, but they all have tradeoffs—levels of accuracy, expense, and battery life.

Without knowing what kind of location-based service is being tested, I want to try to take different things into account. What happens if I’m near tall buildings or if I’m coming in and out of dead spots? I can often infer what the application is doing based on the types of errors I get, but it’s incredibly important to have it be correct.

I was teaching mobile application testing in Australia, and one of the applications people were testing was a transit application. It used location-based services, but it was about three blocks off. Three blocks off is probably fine for a social networking app, but it’s not fine when I need to take a bus, train, or subway in a city I’m completely unfamiliar with.

You can use different ways of testing those scenarios locally. Some remote-device-access services will give you some of these benefits. Probably the easiest thing is some form of crowdsourcing, whether it’s with a crowdsourcing service or people in your own network. Social networks are amazing things. There are some things you just can’t do on your own, and some of these services help you extend your reach. If you’ve got enough followers who are interested, say, “Hey, people with different devices in different places, try this and tweet back” or “Next time you’re in the airport in Japan, see if this application works or not.” There are creative ways that we’re going to have to use to try to solve some of these platform- and location-based issues.

Joey McAllister: What are some particular concerns you’ve encountered with popular mobile products like games, social applications, and mapping software?

Jonathan Kohl: A couple of the big things that are difficult are security and privacy. The devices themselves aren’t that secure with their network connections, so it’s easy for people to snoop in, crack your accounts, and find out personal information. But, privacy is a huge issue. Some of the big social applications get in trouble in Canada because they violate our privacy laws and they have to clean things up.

What I find a lot is people complaining about using social games where there’s accidental leakage of their personal information. “Oh no! My boss can see my party pics from when we went to Mexico!” The applications tend to update quite a bit, to roll out new features, and to change things as they search for new revenue models. Bugs can occur where your personal information goes out there and you don’t want certain people to see it.

With social games, people don’t understand that you can be giving private information away. People will just hand their kid an app on an iPad, and there’s some creepy person nearby who’s sending things that aren’t appropriate because they don’t realize that it’s a kid using the app. We take these devices everywhere, and it’s really easy to end up in strange situations where you’re giving away private information or allowing intimate access to strangers.

There’s a huge growth in the data services space. You have all these powerful social applications with all this data, and there are APIs so you can tie in your application or your service. People can make connections with this data. One that got a lot of press lately was the Girls Near Me app. You could find single women within a certain radius of where you’re using the application, and it could be really creepy if people didn’t realize that they’d somehow made their data available to be accessed this way without their permission.

Joey McAllister: A lot of applications can now connect with third-party applications on a device. When you’re testing your applications, what are some ways to factor that in?

Jonathan Kohl: One of the things I’ve found when testing mobile apps is “interruptions and interactions”—push notifications, for example. You may be working with something that’s business critical, and a push notification telling you the latest score or a weather change pops up and causes problems.

What’s more fascinating is when applications are written to optimize. One of my friends told me about a problem he ran into. He got a brand new phone, and it would heat up in his pocket and just about burn his leg when his battery was dying. The phone heating up means there are more processes being used and probably contention going on. What he found was that two applications had different optimization algorithms. When the battery was dying, one application would say, “I’m going to turn WiFi off to optimize.” The other application running at the same time said, “I’m going to turn WiFi on to optimize, because wireless broadband takes more battery.” They were in conflict, and they would just go crazy. They’d burn up the cycles on the phone, and these devices aren’t very good with dissipating heat.

It’s vital to have an understanding of how these applications might work. You can’t predict those kinds of things, and that’s why it’s important to try things under different lighting effects, under different battery power, different types of network connections—weak connections, strong connections, changing between connections. One of the biggest issues with web applications is that they’re huge. They may eat up my data plan if I’m traveling, but they also take a lot of processing power to display.

It’s a fascinating thing to find these weird errors when you’re using the application in conjunction with other things. You can use some heuristics, like trying to starve resources. How many apps can I fire up before this thing starts to slow down? Do I know that this particular activity—say, lots of animation and gesturing with a game—tends to wear down battery life? One thing that’s really important is just having awareness of how your device works and being able to create conditions like low battery or poor network connection.

Joey McAllister: Do you have anything specific that you would like to share about what you're working on or what you’ll be presenting at STARWEST 2012?

Jonathan Kohl: I’m going to self-publish a book on testing mobile applications, and I’m working hard on my mobile applications testing course with SQE Training.

As for STAR, I just want to give people a bit of an insight into some of the characteristics that you might see when you’re testing these things—some of the opportunities and challenges. There’s a lot of fun, and it’s kind of given me a shot to the arm with my testing work, because it’s just neat to have fun with these applications and get paid to do it. When people are doing exercises in my course, there’s a lot of laughter. People are having fun. “I can do this? This is allowed? I can get up and move around, and I can wave things around and test the sensors?”

I just want to give people a bit of an insight, because it is really growing and it’s going to get bigger. I see there being a lot more opportunities for people next year where they’ll have an app thrown in front of them. What do I do? Where do I start? I’m trying to blaze a trail for people so they can learn from my mistakes and experience and have some structure.