shift-left strategy in accessible product development

ImageImage
Anna_Mekhanova.jpg
авторProject Manager in EPAM’s Accessibility Competency Center

<p>In collaboration with&nbsp;Daria Erzunova, Software Testing Engineer in EPAM’s Accessibility Competency Center, and&nbsp;Roman Pavlov, Data Scientist and Product Owner of EPAM’s Accessibility Testing Automated Platform.</p><ul></ul>

<p>In collaboration with&nbsp;Daria Erzunova, Software Testing Engineer in EPAM’s Accessibility Competency Center, and&nbsp;Roman Pavlov, Data Scientist and Product Owner of EPAM’s Accessibility Testing Automated Platform.</p><ul></ul>

Shift-left strategy is one of the critical components of accessible software development. It allows teams to detect possible standards violations at the very beginning of a project launch, and eliminate them by choosing the appropriate technology stack, validating UI/UX design, and choosing the proper accessible techniques for the development team.

If a software development team uses accessible techniques, any shift-left in testing by automated tools and plugins can provide significant beneficial results. This kind of testing is the foundation of the shift-left approach in software development that relies on principles of early testing, the effective gathering of product requirements, and preventing critical bottlenecks.

Our colleagues share their insights about building an effective development process with a shift-left strategy in mind. They also share their tips for creating an accessible product, options for automated code verification, and some handy automated testing tools that our Accessibility Support team uses daily.

Please, welcome:

  • Anna Mekhanova, Project Manager in EPAM’s Accessibility Competency Center
  • Daria Erzunova, Software Testing Engineer in EPAM’s Accessibility Competency Center
  • Roman Pavlov, Data Scientist and Product Owner of EPAM’s Accessibility Testing Automated Platform.

Let’s get started!

Shift-left meaning

In the case of accessible product development, engaging only the QA Team at earlier stages of software development won't offer efficient results. The shift-left approach involves engaging the accessibility specialists at early development stages to clarify the accessibility requirements for all Teams, including the QA Team that will be conducting testing later following these specific requirements.

Why does DevOps recommend “shift-left” testing principles?

The key aspects of shift-left testing include the detection of bugs at early development stages, continuous testing best practices, and the automation of processes that will speed up product time-to-market. The shift-left approach perfectly suits the DevOps philosophy that encourages automation, continuous automation, and continuous deployment practices.

Now let’s see how the shift-to-the-left paradigm correlates with accessible product development.

What a development process looks like for accessible product development

Several major compliance standards define the operating systems, browsers, and assistive technologies that a software product must support:

  • WCAG, version 2.0, or the more recent version 2.1. This standard is mainly used at a compliance level of AA. The WCAG applies to web products and mobile applications.
  • Section 508. Federally funded organizations in the US use this standard. It covers more than just web products or applications, but it refers to WCAG 2.0 level AA regarding web products.

Other accessibility standards are used locally and may be a part of product requirements. Overall, the WCAG is one of the most popular standards applied in the development of accessible websites and mobile apps.

To avoid WCAG violations as early as possible and reduce the number of issues you have to deal with at the QA stage, you’ll need to incorporate accessibility at every stage of the development cycle.

"First, almost half of the 50 WCAG 2.1 level AA SC can fail at the design and prototyping stage. On average, it takes three times more effort to fix accessibility issues like technical debt if accessibility requirements are not part of the initial product design, including stories’ technical descriptions. Fixing this technical debt may include changes in product design and business logic which may trigger the rebuilding of considerable parts of the project. Existing automation tools checking for accessibility compliance catch no more than 30% of the issues, which doesn't guarantee your product’s compliance. An accessible product needs strict and thoughtful accessibility specifications or requirements in the same way as it needs business requirements. The product’s UI/UX has to comply with the standard, and developers should follow the accessible design patterns outlined in the specifications. You can't neglect manual testing with actual assistive technologies like screen readers. At the end of product development, fill in the VPAT (Voluntary Product Accessibility Template) that states the level of accessibility support and notes any exceptions, some of which may have been made at the client’s specific request."

Anna_Mekhanova.jpg
Anna MekhanovaProject Manager in EPAM’s Accessibility Competency Center

<p>In collaboration with&nbsp;Daria Erzunova, Software Testing Engineer in EPAM’s Accessibility Competency Center, and&nbsp;Roman Pavlov, Data Scientist and Product Owner of EPAM’s Accessibility Testing Automated Platform.</p><ul></ul>

The typical accessible development cycle includes the following stages:

  • Identification of requirements. At this stage, a team identifies functional specifications and design concepts, potential gaps and risks, requirements, and optimization opportunities for a software product.
  • Accessibility specification. This stage involves creating an accessibility specification – a document that helps optimize UI/UX prototyping, development, and QA stages. It also helps deliver a quality product compliant with the most up-to-date practices. It generally includes the following components: the supported standard and clearly identified inapplicable parts, color schemes, accessibility requirements for common elements and pages structure, a focus order, elements requiring alternative descriptions, and desktop and mobile expected behavior.
  • Design. It's important to pursue product design with accessibility in mind and follow content architecture and maintenance procedures. Fundamental aspects of accessible UI/UX design include color contrast, visible and highly distinguishable focus, meaningful elements and helpful labels, clear error messages, consistency in layouts, navigation patterns, business logic, etc.
  • Engineering. This stage involves coding and assembling a software product. Accessibility specialists must pass on proper accessibility requirements to a development team. A good rule of thumb is to avoid vague wording and stick to current accessible techniques, WAI-ARIA standards, and accessible patterns published by WAI (web accessibility initiative) group on the w3.org website.
  • Operation. Accessibility support verification and VPAT compliance.
  • Optimization. This final stage includes software revision and updating.

"Start eliminating possible issues at the very beginning by choosing an optimal technological stack. For example, design your front-end as a separate application rather than using existing templates – not all of those are accessible. In my experience, React is one of the best Javascript libraries to use, because it is extremely popular and it has great community support. However, if you need to build a mobile app, React Native would be a poor choice since it does not currently support some very important accessibility features. Perhaps several years from now, they will. So for now, I’d recommend native development for accessible apps."

Anna_Mekhanova.jpg
Anna MekhanovaProject Manager in EPAM’s Accessibility Competency Center

<p>In collaboration with&nbsp;Daria Erzunova, Software Testing Engineer in EPAM’s Accessibility Competency Center, and&nbsp;Roman Pavlov, Data Scientist and Product Owner of EPAM’s Accessibility Testing Automated Platform.</p><ul></ul>

Automated code checks in a shift-left approach

Automated code checks can be a part of your shift-left approach. For example, we recommend the axe-core engine (jest-axe plugin for jest).

Unfortunately, there are only a few rules for accessibility feature verification in existing linters and plugins. For example, Stylelint has only three rules, Eslint for the web has 34, and React Native has 13 rules. The actual number of accessible techniques and examples published on the w3.org website, however, is more than 500. As a result, existing linters and plugins can not be expected to provide a sufficient level of code compliance to the standard.

Accessibility linting and unit testing with existing tools are effective when:

  • Native HTML elements are used
  • ARIA roles are added to non-semantic elements
  • ARIA attributes are used
  • Accessibility features are added (React Native and others)

Remember, even if you use these tools properly, they will help catch, at most, 30% of the problematic issues.

On the other hand, these tools will return zero issues found (meaning that they are not useful at all) if:

  • Non-semantic HTML elements are used
  • No accessibility features were added to the code

Automated testing in a shift-left strategy

Like functional testing, accessibility testing can be performed automatically and manually. Roughly 30% of total accessibility requirements can be tested automatically. Partial automated accessibility testing includes things like HTML and CSS mark-up; correctness of the added ARIA roles and attributes; color contrast in case the colors can be determined within the code and are not implemented as gradients; text resizing; and the page structure. Automated accessibility tests are an essential part of a shift-left strategy. They help you catch a great deal of easily preventable errors and unintended mistakes.

Most of the automated tools can be divided into three groups: Browser plug-ins, Online tools, and bookmarklets.

Browser plug-ins work as extensions for checking the web page directly within the browser. They include:

  • axe (Chrome, FF, Edge)
  • WAVE (Chrome, FF)
  • Accessibility Insights for Web (Chrome, Edge)
  • ARC Toolkit (Chrome)

The next group is online tools. As a rule, they represent websites where you can enter the web address to get it checked. Here's a list of popular online automation tools:

  • Nu HTML Checker (W3C)
  • CSS Validation Service (W3C)
  • PowerMapper
  • WebAIM online contrast checker

Bookmarklets are bookmarks stored in a web browser that contains JavaScript commands that add new features to the browser. For instance, they may show the total number of headings within the web page.

Some automated tools require a vast knowledge of accessibility or code, others provide more simplified explanations. All of them are subject to false positive and false negative results, which means that their outputs must be reviewed manually. The best one to choose depends on the product and personal preferences.

"I am a huge fan of bookmarklets, and I have dozens of them bookmarked for every possible occasion. Some provide visualizations of accessibility features of web pages like landmarks, images, links, and form controls; others highlight roles, states, and properties of elements on the page. In addition, there are tools that change the appearance of a page showing it in text-only or in a grayscale mode. You should remember, however, that almost all bookmarklets are experimental products and may give faulty results."

Daria_Erzunova.jpg
Daria ErzunovaSoftware Testing Engineer in EPAM’s Accessibility Competency Center

What elements can't be auto-tested

Unfortunately, bookmarklets or browser extensions aren't a silver bullet to building a robust software product. There are limitations to keep in mind when using automated testing tools. One issue is that there are various ways to implement elements. For example, if an interactive element is created using a non-semantic element like <div> without proper attributes, there's no way for that to be detected automatically. Also, automated testing can't be applied when you need to test the following elements and cases:

  • Interactive elements created without proper attributes
  • Captions matching the content
  • Consistent navigation patterns across the entire product
  • Whether an image is decorative based on the page context
  • If the alternative texts correspond to the meaning of the element
  • And there are other situations in which automated testing is not useful

ATAP – EPAM's accessibility testing automated platform

Our EPAM team has been building expertise in accessibility development for a while. As a result, we've developed our own accessibility testing automated platform (ATAP) that helps us speed up the process of checking web content for accessibility, and reduce the need for manual efforts. The platform's concept also matches a shift-left DevOps paradigm that strives for maximum automation.

Here are the key features of EPAM’s ATAP platform:

  • Pipeline with ML models (mainly CV and NLP models)
  • Checking conformance with WCAG 2.1 and best practices
  • Audit report and 4 VPAT types support
  • Verification support across success criteria like axe, Wave, Lighthouse, and other open-source automated tools.​
  • Interactive content testing support (like slide-by-slide activities)
  • Integration of Accessibility Compliance checks as quality gates in existing CI/CD processes (plugin for Jenkins).
  • Scheduler

Machine learning evangelist Andrew Ng states that if an average person can perform a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future. Inspired by this idea and thousands of proofs of this rule in other areas, we decided to test whether we can automate more accessibility checks than typical solutions do now. It turned out that it was possible to do so.

We managed to automate new checks for situations in which it is difficult to formalize technical requirements and solve them in a classical programming way based on defined rules. To make a product valuable as a test automation tool, it wasn't enough to increase the coverage of accessibility checks alone.

As a result, we added extra functionality and integrated axe-core tests to perform the basic checks, a web interface with an automatic generator of Audit and VPAT reports, and editable versions to provide a comprehensive analysis of conformance to accessibility standards. To detect accessibility issues as early as possible in the process, we enabled the integration of the accessibility testing into existing CI/CD pipelines. Finally, we automated the creation of accessibility issues in bug tracking systems, such as Jira.

In addition to the Deque Axe-core accessibility engine integrated into the product, we designed and implemented about 70 tests. 30 tests are capable of checking the interactive elements in addition to static code analysis, while a dozen tests use machine learning models. Most of the tests we designed are related to NLP.

Useful links to check

For additional information on this topic that might be of interest, check some of the links from our technical EPAM community. You'll find talks and articles on the accessibility topic:

Anna_Mekhanova.jpg
авторProject Manager in EPAM’s Accessibility Competency Center

<p>In collaboration with&nbsp;Daria Erzunova, Software Testing Engineer in EPAM’s Accessibility Competency Center, and&nbsp;Roman Pavlov, Data Scientist and Product Owner of EPAM’s Accessibility Testing Automated Platform.</p><ul></ul>

<p>In collaboration with&nbsp;Daria Erzunova, Software Testing Engineer in EPAM’s Accessibility Competency Center, and&nbsp;Roman Pavlov, Data Scientist and Product Owner of EPAM’s Accessibility Testing Automated Platform.</p><ul></ul>