Reducing Redundancy in Responsive Testing

[article]
Summary:

When testing responsive design, it's important to make sure that it renders well on a variety of devices. But how can you do that efficiently, without testing on dozens of smartphones, tablets, and desktops? Adam Rosenberg tells you how to make informed testing decisions based on OS, viewport widths, and your team so that you can reduce redundancy.

Three years ago, before responsive design was considered fundamental, my company won a project to design and build a responsive site. It was up to me to recommend which devices and browsers we should test on. Responsive design is now more widely used, and what I learned from this initial responsive project is still very relevant today.

My testing recommendations would have broad consequences: The test matrix usually becomes the “support” list as the build moves forward. That’s why it was crucial for me to understand what responsive really means for testing and to use that knowledge to make recommendations that would be valuable over the entire course of this and future projects.

The first thing I had to understand was how responsive sites actually work. Of course, the aim of responsive design is to build one site that displays properly and changes its layout depending on the width of the browser on the device that is being used. A common misconception about responsive design is that the layout format depends on device detection. Actually, only the viewport width of the browser determines the layout. Here’s what this means for designers, developers, and testers: It’s only necessary to consider screen width, not specific devices themselves.

After detecting viewport width, the site uses CSS media queries to serve up different CSS layouts, depending on what breakpoint the browser falls between. Breakpoints are the specific viewport widths at which the layout of the page changes. For example, on the site we were building, the designers had already identified our two breakpoints: 600px and 1200px. That means we actually had three different layouts: a small layout that displayed on any device with a viewport width less than 600px, a medium layout on any device between 600px and 1200px, and a large layout on any device 1200px and up.

Note that I referred to the different sizes as small, medium, and large instead of mobile, tablet, desktop. Layouts are not device-type specific. The small or large layout might display on some tablets, while some phones might show a medium layout.

In order to make an informed recommendation about what devices to test on, I needed to do some investigation into how viewport widths work. At first glance, it seemed like the viewport width is directly tied to the resolution of the device. This, however, is not the case. The viewport width for a website actually depends on resolution, screen size, pixel density, dppx (dots per pixel unit), on-screen UI elements, and the rendering software of the device. As a result, two devices may have completely different resolutions but render at the same viewport width.

One of the most recognizable examples of this is when Apple’s iOS line of devices switched over to retina displays. The original nonretina iPhone line had a resolution of 320x480, a relatively low pixel density of 163ppi (pixels per inch), and dppx (dots per pixel unit) of 1. The dppx is the key factor here—the dppx of 1 means that there is 1 dot for each CSS pixel unit. In this case, the viewport width actually does match the resolution of 320 because it is a 1:1 ratio.

When the iPhone 4 came out with its new retina display, it had a much higher resolution of 640x960 and almost double the pixel density of 336. Because the physical size of the screen hadn’t changed and the resolution of the display determined how the page rendered, then logically, everything on the iPhone 4 would have been much smaller than on the iPhone 3G (like it would have been on any standard computer monitor at the time).

Comparing them side by side, however, shows the relative size of everything on the screen is exactly the same; the retina display is just sharper. This is because the retina display has a dppx of 2. In other words, for every CSS pixel unit, there are actually 2 physical dots on the retina screen, resulting in a ratio of 2:1. So in this case, even with the resolution width of 640px, the viewport size is still 320px, and the page renders with exactly the same proportions.

Images on the iPhone 3G and iPhone 4 with retina display render the same

So, how did I take all the information above and figure out what to recommend? Really, it was a juggling act—I had to cover the greatest number of viewport sizes and browser versions while avoiding redundancy.

Figuring out all the different viewport sizes for the devices I was considering seemed like it was going to be an annoying task. Luckily, other people had already done a lot of the work for me. For example, the website viewportsizes.com has pretty much every major device listed in a searchable database with their corresponding viewport widths.

Before recommending devices, I decided on a fixed number of devices to commit to based on the timeline of the project and the bandwidth of my team. The goal was to avoid redundant testing. For example, as of today, all iPads have the same viewport width. As long as they have the same OS, the risk that they will render differently is very low. In other words, testing multiple iPad retina devices is a waste of time. The specific device is not what is important; it’s all about the viewport widths and the browser version. As another example, despite the physical size difference in the Note line of phones and the Galaxy S line of phones, most of them have viewport widths of 360. If they have the same OS, testing both of them would really be running the same test twice. It would be like testing IE9 in Windows XP on a Dell computer and an Acer computer: It would render in exactly the same way.

In order to cover the most ground, I mixed OS and viewport widths. Instead of testing a Note 3 (viewport width 360px) and a Samsung Galaxy S4 (viewport width 360px) both with Android 4.4, a better choice would be the Galaxy S4 with Android 4.4 and a Nexus 4 (viewport width 384px) with Android 4.2. This way, with just two devices, I could test for two versions of Android and two different viewport sizes. I went from covering one test scenario on two devices to covering four. To include a broader range of layout views, I could have replaced the Nexus 4 with a Nexus 7 (viewport width of 600px), which would most likely hit a different breakpoint altogether.

Some things have changed in the past three years. Keep in mind that today, some Android devices ship with just the native Android browser as its default browser (e.g., the Samsung Galaxy line), while most other Android devices (e.g., the Nexus line) ship with mobile Chrome as the default browser. Each browser renders pages slightly differently, so it is important to pay attention to that when making choices.

In the end, based on the timing and team size, I decided on five desktop browsers and five device combinations. I maximized the device testing choices, making sure that I covered the largest range of viewport widths without performing redundant tests. Since then, many clients have still requested using specific Android devices with the same version of Android and the same viewport widths, resulting in what would be redundant testing. I have found it helpful to educate everyone on the project about these concepts so we can choose the correct devices to test and reduce risk.

User Comments

1 comment
Savva's picture

Excellent article Adam

August 13, 2015 - 6:37am

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.