How to incorporate accessibility testing early in the SDLC
Picture the scene: your team has spent months building a global, user-facing application. The major features are all ready and you’re eager to get the application into production. Then you discover the accessibility score is too low and the application wouldn’t be usable for an extensive set of users.
Boom! We can’t release the app and the go-to-market timeline takes a big hit.
This is actually a really common problem when we don’t give enough importance to non-functional requirements (NFRs) such as accessibility. Let’s look at how we can shift left accessibility testing to the unit level, by leveraging some good practice and widely available tools.
ByRole query in React testing library
We often try to use an escape hatch by adding test IDs to the HTML elements and using getByTestID query to get hold of elements in the unit tests. For example, we might have an inaccessible form where inputs aren’t tied with label:
Since we are adding testIDs, it’s very easy to add the unit test for this inaccessible form:
Even the div with click listener, which behaves like a button to submit the form, works as expected but is totally inaccessible.
To overcome this problem, React Testing Library does provide byRole queries to get the elements by their accessible role. This means that the same form with proper accessible role should use the accessible role to get hold of the element.
The test will fail if we:
- Remove the htmlfor attribute from label, or event ids from the input elements
- Use any other element other than a button with the name submit to submit the form
- Change the heading level of the welcome message
Taking these steps makes the form more accessible by checking it at the unit level only.
UserEvent catches what fireEvent misses
User-event is a companion library for the testing library that simulates user interactions by dispatching the events that would happen if the interaction took place in a browser.
The browser usually does more than just trigger one event for one interaction. For example, when a user types into a text box, the element has to be focused, and then keyboard and input events are fired and the selection and value on the element are manipulated as they type.
fireEvent dispatches DOM events, whereas user-event simulates full interactions, which may fire multiple events and do additional checks along the way. It therefore adds visibility and interactivity checks along the way – manipulating the DOM in the same way that a user interaction in a browser would.
An interesting example of an accessibility error caught by userEvent that’s missed by fireEvent is pointer-events: none CSS property. If you try to click a button that has this CSS property using userEvents, the test will fail – but this would not be the case when using fireEvent.click.
Use packages to add accessibility assertions
Plugins like jest-axe or vite-axe (a custom Jest matcher for testing accessibility with axe) help to run automated tests at unit level. Simply wrap the container inside the axe utility provided by the package to check for any accessibility issues.
If the input doesn’t have a label, the tests will produce an error.
What else could we do to check accessibility?
It’s important to remember that around 30% of access barriers are missed by accessibility tools. We need to do more than simply rely on automated tests. Some other things to consider:
- Include accessibility checks in automation pipelines and use the lighthouse score for accessibility as a check with every build
- Test your interface with assistive technologies like Voice Over in Mac etc.
- Always test for keyboard access and manual accessibility testing
- Involve people with disabilities in your accessibility testing
Accessibility is a very important part of any successful product. We need to make sure we’re testing for it early in the software development lifecycle, rather than making accessibility something we only think about at the end of the process.
If you’re involved in design (and even if you’re not) you probably know about the Double Diamond process. But, is that the end of the story? This article expands the perspective and completes the model by taking you from ambition to market in a repeatable entrepreneurial cycle.
Divergence and convergence
Back in 1996, Hungarian-American linguist Béla H. Bánáthy made obvious something that was hiding in plain sight, and as a linguist he expressed it in beautifully simple terms. He described the thinking process in terms of two key phases, the first being divergence and the second convergence. During divergence we consider a situation and create a range of choices arising from that situation. Convergence then follows and we narrow those choices down to a single decision point.
For example, you’ve bought a new painting for your home, and now it needs to be displayed. Maybe you already have a space for it, but more likely, you now examine your rooms and build up in your mind a list of places where it would look great. This is divergence.
Next, you imagine the painting in each of these places. Perhaps you get a friend or partner to hold it up so that you can get a feel for how it’ll look. Slowly you narrow down the field of options until you’re left with just one – the perfect spot. This is convergence.
Divergence involves expanding your understanding and increasing your options without judgement. To judge and reject too early is to do so without adequate knowledge. Far too many great ideas end up on the cutting room floor (or voted down in “brainstorming” sessions) because they’re unfamiliar or poorly understood. This is why divergence has to be allowed to happen before any attempt is made at convergence.
Convergence is reduction of options through elimination, and most importantly elimination through experimentation rather than through opinion or “gut feel”. You’d be amazed at just how many “obvious” things turn out to be wrong when put to the test in the real world. For example, we all know that warm water freezes faster than cold water, don’t we. That’s so obvious we really don’t need to waste time testing it? It’s a safe assumption, right? If you agree, I recommend you read up on the Mpemba Effect – a timely reminder that there is no such thing as a safe assumption.
A tale of two diamonds
Divergence followed by convergence can be pictured as a diamond, starting from a single decision point, and expanding out like a tree, growing freely. Once the tree is fully grown its branches are then pruned back until only the strongest remains. This is your new decision point from which you grow your next tree.
You can apply this pattern to most situations that require thought and an outcome. Choosing a holiday; picking a route to work; designing a new mode of transport; even painting that picture. In just two words, Béla brought to life the idea that problem solving was not a linear process that created certainty out of chaos, but instead a journey that created chaos in order to get to the next point of certainty.
This is a concept that appeals to the creative mindset, and so it comes as no surprise that the British Design Council took this concept and popularised it in the form of the Double Diamond design process model.
As the name suggests, the model sees the design process as two consecutive divergence-convergence activities, the first consisting of discover and define, and the second of develop and deliver. The first phase allows you to discover a problem domain to gain insight, and then narrow down that insight to define the problem to be solved. The second uses that problem definition to develop a range of potential solutions, tested for viability, and then deliver these solutions to an audience and use their feedback to converge on a single preferred solution.
The double diamond model has become almost ubiquitous in the world of design, and there may well be as many renderings of the illustration above as there are designers. However, as I mentioned earlier, the divergence-convergence model can be applied to most situations requiring thought and therefore to most meaningful undertakings. It was only a matter of time before the Double Diamond concept was extended.
Two’s company, three’s a crowd
It’s at this point in our story that the Triple Diamond Model enters the fray. There are many versions of this model and they all vary in the terminology used, but the one thing the majority have in common is the direction in which they extend the model. The new diamond is placed in front of the Double Diamond model and its two stages are often referred to as “opportunity” and “strategy”.
The intention of this extra diamond is to reflect the process that gets us to the awareness that there is a particular problem worth solving. First explore the market to find a range of opportunities (divergence), then formulate a strategy that targets the first of these opportunities in the form of a proposition (convergence). In the world of software development, this is where the product strategy emerges.
Some would say this is just taking a good model and ruining it with unnecessary embellishment. It’s a common pattern with models that people spend too little time understanding them before adding complexity to cover aspects already encompassed by the original concept. A minimalist’s nightmare.
Others would suggest that the Double Diamond is specific to one particular viewpoint, and requires extension to cover a broader range of perspectives. I’m an advocate for the second view – the double diamond is an excellent start, but it’s not the full story. The real question is how far does that story stretch? Is three diamonds enough? Four? Is there even a limit?
For those of you who hate spoilers, look away now. For the rest of you, (the impatient ones), the answer is five.
A matter of perspective
To understand the Double Diamond we have to see it in context, and to do this we need to expand our perspective to encompass the full process from start to finish. Designers often (rightly) complain that they are not engaged early enough in the process, and this in itself is a recognition that there is a broader activity to which they could contribute. The same is true for most participants in the creative process. Each sees the process from the perspective of when they are first approached to the point where their role ends. Beyond those points things become more vague and tend to get condensed into “the step before” and “the step after”. What might, from a local perspective, seem like brief steps, are actually a chain of events spanning weeks or even months of investigation and insight.
Also, when considering processes, it is popular to think in terms of straight lines and trends (a beginning, a middle and an end; a start point and a destination). However, many things in life are cyclic, and only look like linear trends if we observe them for a limited duration. Proper understanding requires context. The context for the Double Diamond process is product development, and this is part of a loop I like to refer to as the “Entrepreneurial Cycle”.
As can be seen, this cycle involves five points of focus, and it is from this that the limit of five diamonds is derived. To get from one point to another requires divergence to understand and create options followed by convergence to get to the decision point. But first a quick summary of how the cycle operates.
An entrepreneur operating in a given market observes that market and perceives threats and opportunities emerging as that market evolves. This observation gives rise to an ambition and ambition leads to the development of a range of strategies from which one is chosen. This strategy gives rise to a proposition which in turn is developed into a product that can be deployed into the market. If this product is successful, it will inevitably change the market, leading to new threats and opportunities, and so the cycle repeats itself.
The biggest lesson to draw from this is not just that product development is a cycle, but that the act of developing products feeds the cycle and keeps it going. The very act of introducing a new product into the market changes that market, and so a wise entrepreneur recognises the need to keep moving. In essence, the market for which your product was developed no longer exists once your product is successful.
Turning the wheel
Let’s take a simple example to bring the Entrepreneurial Cycle to life. A company providing an online service has seen a plethora of challengers enter their market. Customer churn is high and brand loyalty is low. Although they’re seeing as much customer gain as loss (and thus maintaining market share), the cost of maintaining this share is becoming uneconomical.
They believe their service is superior and embark on a venture to radically improve customer retention and brand loyalty. They have established their ambition.
They also observe other apparently similar industries adopting subscription models and believe this might be a way to achieve this retention. This approach now forms the basis of their strategy.
The information available to them shows that a monthly subscription model providing customers great value for money through unlimited access to services has improved retention in other industry sectors. They also believe that by adding benefits and exclusive rewards to this model they will achieve the levels of customer retention they desire. Their proposition is starting to take shape.
They now look into their customer base and decide a single subscription won’t work for everyone, so they define a tiered model and develop this into a set of subscriptions options from which the customer can choose (silver, gold and platinum). Each has a different level of service and associated offers, and comes with different levels of customer support ranging from online chat through to personal concierge service. They set out to develop these products and launch them onto the market in place of their current offerings.
If, as they hope, this is a successful venture, they will retain their current customer base and continue to acquire additional market share through the high churn the industry is experiencing. However, the model is easy to duplicate and their competitors will no doubt adapt by introducing similar, or potentially better, products. They will need to prepare in advance to react to this change in the market if they are to stay ahead of the game.
And what if they’re wrong? That’s a significant amount of effort and investment to waste on an unsuccessful venture, not to mention the potential reputational damage. This is the danger of linear thinking.
Diamonds are forever
To address this, each stage in the Entrepreneurial Cycle needs to exploit the elegant wisdom of Béla H. Bánáthy. The earlier diagram, however, fails to convey this. It’s drawn in a way that implies movement from one state to the next is a linear, predictable activity, and the example above includes elements of this linear thinking style. It should be clear from the example that the decision making involves a lot of assumptions and gut feel decisions. There is little room for uncertainty or verification.
So now we need to overlay this cycle with the diamond model to make it more complete and add some rigour to the process of investigation and decision making. For ease of drawing, I’ve stretched it out horizontally, but please remember it’s cyclic:
And for those who prefer text to visuals, here is a further breakdown of each of the five stages of the Entrepreneurial Cycle described in terms of convergence followed by divergence::
- Market to Ambition
Before doing anything you need situational awareness. To achieve this you must observe the market in which you’re operating to understand how it’s evolving and to identify the threats and opportunities. You might even want to draw a map of your territory to better understand it. From a diversification perspective, we’ve developed the Business Evolution Map. For a more personalised picture, you might want to try creating a Wardley Map. Once you understand the space in which you’re operating you can identify the most attractive opportunity (or the most immediate threat) and target an outcome that addresses it. The way this outcome will be achieved is not yet known, but there is now an ambition to achieve it. You have an intended destination. - Ambition to Strategy
Now you know the outcome you want but not how to make it happen. What you need to do is find a course of action that will get you there. The best way to do this is to explore the routes (the strategies) available to you to get to your intended destination. Exploration doesn’t just involve conceiving of the routes – it includes testing out your assumptions in some small way to see how feasible they are. This testing of assumptions is essential if you are to successfully orientate on the strategy most likely to succeed. Never underestimate your ability to choose the wrong thing based on familiarity when you lack the evidence to make a value based decision. - Strategy to Proposition
To turn the intent of your strategy into something with real business potential you need to do some targeted research. What is the true value of the idea (the size of the prize) and what will need to change in your business and in the relationship with your customer to make it real? Once you understand this, you can shape your intent into a proposition that includes a viable business and service model. And always remember, research is not just a numbers game; it’s a hands-on activity where assumptions need to be tested and hypotheses proved.
- Proposition to Product
The first diamond from the Double Diamond Process, in which you discover the needs of the users involved in your proposition, and define the product required to fulfil those needs.
- Product to Market
The second diamond from the Double Diamond Process, in which you design the product and deliver it to the market, iterating on your design as you go based on user feedback and early indicators of success from the market.
In summary
- Product development is a cycle, not a straight line journey to done
The very act of introducing a new product to the market causes a ripple effect of change. Competitors will respond, either through emulation of your change or innovation to defend against it. Customer expectations will also change. What was once fresh and exciting will become commonplace as familiarity potentially breeds contempt. The cycle is self-sustaining. - It starts before an idea has formed or a problem has been identified
Limited perspective often leads people involved in the process to believe they are closer to the beginning than that really are. A new idea rarely emerges from nothing overnight. It is the product of an extended process stretching back weeks or even months. - Divergence expands understanding without judgement
It is tempting, when researching a situation, to filter the information you’re receiving and reject concepts that don’t fit your current understanding of the domain. All too often, the real solution (and definitely the innovative solution) lies in the unknown. This is why suspending judgement is essential to successful divergence. - Convergence chooses an answer by eliminating that which doesn’t work
Good convergence requires you to try things out; to test every assumption before deciding. It’s a hands-on activity, not an academic exercise. Nor is it a democratic process. Voting as a way to choose might be quick, but it relies on gut feel and prior assumptions. To do convergence you need practitioners to try things out and you need decision makers who act on the evidence of those experiments, not the security of familiar ideas.
And finally, if you take only one thing away from this article, make it this:
Limited or untested knowledge leads to bad decisions. To discover what you don’t know, diverge without judgement. To find the answer that works, converge through experimentation.