ByRole and aria-modal

[mauricedb]

• @testing-library/dom version: 7.29
• Testing Framework and version: Cypress 6.1.0
• DOM Environment: Chrome 87.0.4280.88

Relevant code or config:

  it.only('no aria-modal support', () => {
    cy.visit('https://ns2hp.csb.app/');

    cy.findByRole('button', { name: 'Go' }).click();
    cy.findByText('Modal title').should('be.visible');
    cy.wait(500);

    cy.findByRole('button', { name: 'Go' }).click();
    
  // Note: This does produce the expected result
  // cy.get('[aria-modal=true]').findByRole('button', { name: 'Go' }).click();
  });

What you did:

In a Cypress test a Bootstrap modal dialog is opened. Once the modal is opened findByRole() is used to search for a second button inside the modal with the same label as a button outside of the modal.

What happened:

Received an error when running the test:
Timed out retrying: Found multiple elements with the role "button" and name "Go"

Reproduction:

See for the code under test: https://codesandbox.io/s/epic-hooks-ns2hp?file=/index.html

The Cypress test is above

Problem description:

According to the specification using the aria-modal=true attribute means the windows underneath the current dialog are not available for interaction (inert). Therefor the button outside the modal should not have been selected by findByRole().

Note that explicitly searching for the modal root and doing the find on the subtree work's as expected.

Suggested solution:

Check if aria-modal=true is present in the current subtree and if so limit the search to there.

[eps1lon]

Thanks for the report!

I don't know if cypress-testing-library is doing something different. But just from the code you posted, it shouldn't find the second button ever since that is below a aria-hidden element.

Though I do think that we're currently not handling aria-modal due to it's poor support by browsers/screen readers.

Could you include a reproduceable example so that we can debug what is actually happening in your test?

[mauricedb]

This is a simple Bootstrap modal. The aria-hidden=true in the markup is replaced with aria-modal=true when the modal shows up.

The first cy.findByRole('button', { name: 'Go' }) is just fine, it ignores the hidden modal as it should. The second time it should only search in the aria-modal=true subtree and only find that button. In that case however it finds both.

Here is a demo Cypress test: https://github.com/mauricedb/dom-testing-library-issue-851
Clone the repo, npm install and npm test to start Cypress

[eps1lon]

The second time it should only search in the aria-modal=true subtree and only find that button.

I think what we should rather add a filter for modal: true. Right now we're including all elements of the a11y tree which includes elements outside of aria-modal. I don't think we should filter by default because aria-modal does not necessarily have a direct effect on the UX. Without JS you could still tab to elements outside of it as far as I know.

So if we decide to ignore elements that are not part of [aria-modal="true"] (if such an element exists) then we should only do so if assistive technology has widespread support. We should be very careful with supporting ARIA semantics if these don't work in actuality.

Also note that by spec aria-modal does not necessarily limit navigation:

Assistive technologies MAY limit navigation to the modal element's contents.

-- https://www.w3.org/TR/wai-aria-1.2/#aria-modal

In summary:

• add modal: boolean to the filter options

Bonus:

1. prepare a minimal page with <section data-testid="page">...some...content...her</section><div role="dialog" aria-modal="true" data-testid="modal">...other...content...here</div>
   • no framework e.g. no bootstrap
   • no JS
   • no CSS
2. Check if content outside of [data-testid="modal"] is accessible i.e. can we navigate to content inside [data-testid="page"]
   • NVDA + firefox
   • VoiceOver + safari (mobile and desktop)
   • JAWS + IE 11
3. If all of these exclude content within [data-testid="modal"] set default for modal to true, otherwise false

[MatanBobi]

@eps1lon I just want to be sure I understand correctly before starting to implement this.
We'll add a modal: true flag to the filter options and if we have an [aria-modal="true"] element we want to ignore all elements that aren't a part of that tree?

[eps1lon]

I don't think we should implement this without further research. Before we implemented some aria-* filters that only consider the actual element not parents. So we need to be sure doing this for aria-modal makes sense at all. It's not clear to me in what capacity aria-modal affects children by spec or with actual user agents + assistive technology combinations.

So far there was only a weak rationale by saying "I want to it to work this way" but no strong foundation.

[MatanBobi]

Thanks @eps1lon..
So in the aria-modal spec it states this:

Assistive technologies MAY limit navigation to the modal element's contents. If focus moves to an element outside the modal element, assistive technologies SHOULD NOT limit navigation to the modal element.

So I'm not sure this is something we want to enforce..

[theKashey]

Created a simple test reproducing a problem - https://codesandbox.io/s/rtl-dialog-example-4dceu7?file=/src/test.spec.tsx

Aria-modal currently:

• hides content outside in Chrome
• doing nothing in Safari
• doing nothing in Firefox