The Politics of Accessibility Testing


Web accessibility is often spoken about in terms of design, programming challenges, frameworks, and technical solutions, but there are also personal difficulties for the people involved. This article addresses some of the cases of initial resistance and provides a few practical ideas on how to minimize the challenges.

Digital accessibility refers to software supported by users' assistive technologies as well as accessibility within web browsers.

This concept, that software should be usable by the widest possible audience, has been around for more than twenty years, yet it remains out of the mainstream of testing and development efforts.

We have also seen diversity and digital inclusion become social priorities. On top of the implied social contract we have explicit legal contracts, such as Section 508 in the US and Canadian provincial legislations (AODA in Ontario, Quebec Standards for Accessibility, and others), which define accessibility standards for government and public sector software. This sets a trending example for the overall market. However, laws and regulations do not define accessibility requirements on their own; they refer to the Web Content Accessibility Guidelines and only dictate the level of compliance, from A to AAA. Note that this web standard is evolving, so the same laws mandate keeping up with the changing requirements.

Web accessibility is often spoken about in terms of technical solutions—design and programming. Less voiced are the challenges of the people involved in the change, conflicts of interests, and people’s perceptions that sometimes create problems bigger than technical challenges.

As a testing practice lead taking on the accessibility testing domain, I was surprised by resistance to the initiative. I couldn’t understand why such a noble goal was not welcomed. I was frustrated by misunderstandings and misconceptions—but I also was responsible for causing some of them.

Here, I relate some situations I’ve experienced and provide some insight on people’s perceptions and reactions that may increase challenges in adopting accessibility.

Challenge #1: It’s Not Business Requirements, It’s the Law

On one project, the development manager saw accessibility as a feature that would create change requests and new business requirements. But the business side insisted accessibility be treated like other system qualities built from the ground up—security, performance, maintainability, etc. Development kept stating that these are new business requirements and should be reflected with a budget increase. The business wasn’t ready for such a turnout, and accessibility was excluded from the release scope, with the resolution to address it next year.

Newly introduced laws and regulations may have a disruptive effect on business goals and budgets. Products that weren’t built with accessibility will require updates and rework, which means extra costs; that is just reality.

When you are the bearer of such “bad news” that the product in the near future might become legally incompliant, both business and development people might feel insecure. The main challenge is not technical work and financing, but how people feel about this serious problem—and about the messenger.

A good strategy is to approach very cautiously, avoid making sudden and dramatic announcements, and be empathetic. It is important to keep the trust. You may suggest conducting an informal assessment first. Argue that before making any decisions, it is worth finding out how big the problem is.

From a technical standpoint, give specific examples where improvement is required. Suggest areas and categories where code changes are most needed.

Breaking down this whole scary legal compliance problem into separate technical points lets you speak the same language your stakeholders are used to. The business side gets a choice to prioritize and budget the improvements, and the development side gets a better understanding of what skills and effort need to be allocated.

Challenge #2: Understanding Accessibility Requirements

I once was asked to review the testing strategy for a project that requested feedback and support from the QA center of excellence. I knew accessibility was in scope, but when I reviewed test coverage, I couldn’t find plans to run these specialized tests. I was told testers would cover accessibility aspects “along the way” while executing functional test cases. I asked whether it might be too complex to do everything altogether, but I got a reply that the testing team is comfortable using keyboard-only operations.

Based on some presentation attended by the test manager, accessibility was perceived as a matter of keyboard support. He thought that as long as testers could tab through the screens, the application was considered tested for accessibility. This is indeed an important requirement, but it is not the only requirement.

Unawareness combined with overconfidence may cause a significant underestimation of risks. The testing team was invited to an accessibility testing workshop, where they learned about the multifaceted requirements and got acquainted with important skills and techniques, such as operating screen readers.

While testing teams may think they’re approaching accessibility testing seriously enough, it might be challenging to ensure the right coverage for legal compliance. Friendly suggested peer reviews and involvement in internal accessibility workshops may help spread understanding across the board.

Challenge #3: Putting Too Much Faith in Tools

Test automation in general is a highly debated topic. Context-driven testing teaches that testing is, foremost, critical thinking, which only humans can do, but testers may and should use tools when applicable. Unfortunately, there are still views on testing as something mechanical, mindlessly repeatable, and finite. This leads to some odd expectations.

On one project, challenged by the need to implement accessibility, the development manager was convinced that a tool could scan the entire application, perform all needed tests, investigate the problems, report them, and even tell the programmers how to fix them. Programmers were expected to learn coding for accessibility by fixing errors. This decision was backed up by a previous successful experience utilizing a security testing tool that scanned the code for vulnerability patterns and was indeed helpful for programmers as a way to learn and avoid such errors.

People make decisions based on their understanding of the technical challenge and habits that have been proven to work. It seemed so straightforward that the testing team wasn’t even included in making the decision.

Initially, the approach seemed to be working. But it became apparent that the tool could only scan the home page and login screen. Wherever user interaction was required, the tool got stuck. Programmers had to manually navigate through the application to perform scans. This suddenly became very time-consuming. At that point they decided to involve the testing team.

Skilled testers tend to question any tools before trusting them. One tester noticed that the tool seemed to be checking whether images had an alternative textual description provided. She created a new data entry with an image of a kitten described as “guard dog.” The tool didn’t report any problem. In fact, as it was discovered with further testing, the tool also would pass on generic texts such as “This is an image” and “This is a link.” As the framework produced these generic textual descriptions, the development team using the tool assumed there were no defects.

The news about the tool didn’t make the development manager happy, and the team was blamed for using the tool incorrectly. But the programmers still struggled with gaining purposeful understanding of the accessibility problems and changes they needed to make.

No one likes to hear that their decisions were wrong or based on invalid assumptions. Suggesting doing a pilot or proof of concept with the tool can be very helpful in setting realistic expectations. Keeping it practical, referring to the facts from the experience, and helping the team to gain such experience helps make well-informed decisions.

Getting the Team On Board

Supporting accessibility is an admirable goal, and people should be willing to subscribe to it—in theory. In practice, though, they may appear resisting. Sometimes this is because supporting accessibility will conflict with interests they deem more important. Sometimes they apply an approach they’re used to without consideration of the specifics of accessibility. And sometimes, it’s just a misunderstanding. After all, accessibility technologies are new for many teams, and they could use help in learning them and adapting their processes.

When helping people gain practical experience in the domain of web accessibility, it’s important to be patient and not expect immediate changes or big wins right away. It’s critical to create and support opportunities for safe trial and learning, like workshops and pilot projects, and demonstrate what real help tools can offer but also what their fundamental limitations are. Skills and experience are the main keys to success.

About the author

StickyMinds is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.