Our company, like many other small software development companies, faced the common "time and resources" issue when it came to manually testing our own mobile applications. As the number of projects increased, the number of bug reports and unstructured documents and files grew as well. We also lacked a standard way to build the applications or to submit bugs. All these issues began to slow down projects.
And then there were the changes. We were using TestFlight, a beta testing tool, for build distribution and crash reporting, but then it was acquired by Apple and stopped support for the Android platform. In search of alternatives, we switched to Ubertesters, a cloud-based platform that gives developers access to a crowdsourced team of mobile app testers with a range of tools and devices.
This is the story of that experiment, along with step-by-step instructions on how to execute a project in this space.
Here’s how Ubertesters works at a high level:
- Create your organization and project within the platform.
- Integrate Ubertesters’s software development kit with your build.
- Upload the build.
- Add the devices to test on
- Add members to your project. These can be developers, testers, or any other people who can contribute to the project.
- Share the build with your testers (internal or external).
- The tester gets a push notification that a new build is available for testing and installs it to his or her device.
- The tester sends reports with the attached screenshots of bugs found.
- The bugs are fixed.
Now, here’s how to actually use this platform.
First, you need to create an organization in the Ubertesters workspace. The organization is a set of projects written by developers for different devices.
Create a project within your organization for each new application. In order to do this, you need to choose what operating system your app was designed under. You also can simply drag and drop .apk or .ipa files to the window.
When working with iOS, the developer must add the unique device identifier, or UDID, of the device to the Provisioning profile in iTunes. If the device is not added to the profile or the UDID is wrong, the tester will not be able to install the build. Forgetting this step was our main issue when working with Ubertesters, but the tech support team was extremely fast with their help, so we had no significant pauses at this point.
In order to start using Ubertesters, download and integrate the necessary SDK. If you are writing a cross-platform application, you will also have Xamarin, Unity, PhoneGap, and Appcelerator among the supported frameworks.
Add devices with the appropriate operating system to your project. After connecting and configuring the devices, invite members into the project: developers, testers, and project managers.
If it is necessary to collect statistics on the specific category of users (such as for our outsourcing testing team), we create a separate "distribution group" for them and form it as we wish.
Once a new build is uploaded, testers who are added to the project get push notifications, download the build through their devices, and start the testing process.
One pleasant surprise compared to similar platforms is the set of features for faster feedback from the testers. Testers now submit bugs directly from the device with the provided screenshots and description.
With previous tools, we faced the common stupid issue of testers spending more time making reports than testing. If we were lucky, a bug report came as a text description, and then we had to take the time to specify the exact place a bug appeared.
The screenshot editor that is built in to the Ubertesters SDK allows the tester to take a screenshot, draw a circle or arrow to point out the problem, crop, add a description, and submit the report to the platform directly from the mobile device. In addition to the images, all the necessary information about the device, such as the model and screen resolution, is collected automatically.
You can also export bug reports to other bug trackers, which is very convenient for those who work through Redmine or JIRA, for example.
It is also possible to track sessions of every particular device or participant of the project, in order to check how long the session was active and how long it just hung in the background. Though we do not use this feature, I think it would be an ideal solution for project managers who are trying to control the entire process of development and testing.
In the Wild
After two weeks of successfully using the Ubertesters trial version, the speed and quality of our application testing have increased significantly.
In general, the Ubertesters platform is a decent service that can be compared to TestFlight, with a range of additional features such as support of test cases, build management, and automatically collecting the information about testing devices. It also speeds up the testing process because you do not need to send emails with each new build; testers get all notifications immediately.
I hope my experience will help you make your mobile app testing process more efficient and productive.
What tools and tips do you recommend to save time when testing mobile apps?