Unable to test ARIA/AAM bindings

[gsnedders]

@halindrome wrote some automation tool here (ATTA); we should have these automated out of the box running under wptrunner.

cc/ @joanmarie

[halindrome]

There are a number of factors that make it complicated... but yes, it is possible. @joanmarie has done most of the recent work on the ATTA end. Obviously the ATTA itself is platform specific. I think all of the current ones are written in Python.

[foolip]

Maybe @alice or @robdodson have ideas about how to test this?

I suspect there's a confusion of acronyms here, since https://wicg.github.io/aom/spec/ and the AAM specs aren't about different layers, so the title should maybe say ARIA/AAM? If so, a separate issue for any difficulties testing AOM would be nice.

[alice]

Yes, I suspect this was meant to be AAM rather than AOM. Not the first time I've seen them collide.

[foolip]

I've changed the title. @gsnedders, can you confirm that's what you intended?

[gsnedders]

Uh, yeah, we should probably have a separate issue for AOM.

[kereliuk]

Are there existing manual tests that need this?

[gsnedders]

@JKereliuk everything in wai-aria, core-aam, svg-aam, dpub-aam, and dpub-aria

[joanmarie]

I'm hoping that once AOM gives us a means to examine the shared/platform-independent accessibility trees of user agents, that will solve ARIA, DPub ARIA, and any other ARIA that's not an AAM spec.

Testing the AAM specs requires an additional tool (the client-side, platform-specific ATTAs). And some of them have dependencies (e.g. in Linux, it turns out that if you don't have the latest versions of ATK and AT-SPI2, you can reliably crash user agents when running table-cell related tests). We also need to handle things like ensuring the assistive technology support is enabled when needed and disabled afterwards. All these are solvable problems, but we don't have the solutions ready to go quite yet. Sorry!!

[cookiecrook]

I think this one can be closed now due to patches in #38758 and #38925, which normalizes the ARIA/AAM testing against each implementation's internal representation of the accessibility tree. This unblocks label testing in the AccName spec, as well as a significant amount of role verification for HTML-AAM, DPUB-ARIA, Graphics-ARIA, SVG-AAM, etc. More at interop-2023-accessibility-testing/issues

Beyond that, I'm not certain how the platform-specific API mappings could be tested in WPT (the other bits of ATTA that are no longer maintained), but IMO, platform-specific API verification may be outside the scope of WPT.

[cookiecrook]

@joanmarie wrote:

I'm hoping that once AOM gives us a means to examine the shared/platform-independent accessibility trees of user agents, that will solve ARIA, DPub ARIA, and any other ARIA that's not an AAM spec.

See web-platform-tests/interop-accessibility#21

[zcorpan]

Agreed, closing.