Testing plan and file locations for Interop 2023 Accessibility Investigation

[cookiecrook]

ARIA has had an issue to convert the Accessible Name and Description Computation algorithm to WPT via web driver: w3c/accname#174 It's one of the more complicated portions of ARIA, likely has a lot of interop bugs between implementations, and has only recently become testable due to availability of computed label and computed role.

AccName fits well into the Interop 2023 Accessibility Investigation along with the others @chrishtr mentioned: core-aam, html-aam, and general role mappings rules (fallbacks, invalid roles, prohibited contexts, etc.) from the ARIA spec. In addition to those I suggested that svg-aam and dpub-aam would be low hanging fruit.

Each one of these will likely need a more detailed Testing Plan, but here's a start.

AccName (Priority 1)

There's an existing /accname directory in wpt that is loaded with vaguely named manual tests. I'd like to move all those manual tests to a sub-directory /accname/manual and start filling out automated tests for AccName. I believe this portion is unblocked in WebKit and Chromium due to the existence of get_computed_label in webdriver. Note: I initially filed a portion of this as WPT PR #38502 but closed and refiled as this issue to ensure a shared understanding and agreement.

ARIA Role Mapping Rules (Priority 1) and possibly Core-AAM (Priority 2)

I'd propose these be in a new /aria/role directory. There are many other parts of ARIA aside from roles that could have their own /aria/* directory. The role tests could cover cases like role selection based on versions (1.0 checkbox vs 1.1 switch for example), invalid (role="foo") or abstract roles (role="structure"), orphaned child roles (e.g. a cell outside a table context), fallback roles (role="foo bar button group" == "button"), etc. I believe this portion is unblocked in WebKit and Chromium due to the existence of get_computed_role in webdriver.

It's possible some aspects of Core-AAM may land in the /aria directory, since it's really part of ARIA, though I'm not sure how much of Core-AAM is testable in WPT due to the platform-specific API nature of that evergreen spec.

HTML-AAM (Priority 1)

I'd propose these be in a new /html-aam directory. There is enough here and it's unique enough to justify its own dir. I believe this portion is unblocked in WebKit and Chromium due to the existence of get_computed_role in webdriver.

SVG-AAM and DPUB-AAM (Priority 2 or 3)

I'd propose these be in new /svg-aam and /dpub-aam directories, respectively. I believe this portion is unblocked in WebKit and Chromium due to the existence of get_computed_role in webdriver.

[cookiecrook]

Note there are other potential spec/infrastructure changes (like accessibilityParent and accessibilityChildren) that would be useful to have as part to the investigation, but I'm limiting the above list to that which can be tested today.

[cookiecrook]

@chrishtr @gsnedders @patrickangle @zcorpan @foolip @jgraham @jelbourn

[cookiecrook]

Specific questions:

1. Is it okay to move all the manual tests in /accname to a sub-directory /accname/manual? If so, I'll file a PR soon.
2. Do people like the idea of including AccName automation tests in this Interop investigation? If so, I'll track it here with a more specific Testing Plan issue. If not, I'll link the ARIA/AccName issues to separate trackers in wpt.

[zcorpan]

1. I'm OK with moving manual tests into a subdirectory. Support files such as images are usually in a directory called resources (not required, but might be nicer if we want to use the same for automated tests).
2. Yes.

[zcorpan]

(By the way, I believe the canonical spelling is "Interop")

[chrishtr]

@cookiecrook regarding your original exposition of the additional testable parts: these are great suggestions! I'm really excited about the amount of progress we'll be able to make testing a11y at scale this year. And in Interop 2024 we can hopefully have a lot of coverage to add for this currently-untestable dimension of the web.

Regarding manual tests in /accname -- are you referring to existing files in the WPT repository?

[cookiecrook]

@chrishtr wrote:

Regarding manual tests in /accname -- are you referring to existing files in the WPT repository?

Yes. I mainly wanted to know if @zcorpan thought that would cause breakage in some downstream ARIA-AT project, or was considered an anti-pattern for this project. Seems like the change is okay, so web-platform-tests/wpt#38544

[cookiecrook]

Cc @joanmarie and @halindrome since the bot auto-assigned them as reviewers on the aforementioned PR.

[chrishtr]

@dtsengchromium do the plans proposed here sound good to you also?

[zcorpan]

ARIA-AT doesn't interact with wpt in any way, currently.

[cookiecrook]

@halindrome I missed that there was already a way-aria dir in wpt that references the ATTA tests that, as I understand it, are no longer maintained. Any objection to making a similar change in that directory to move the _manual.html tests to a new manual directory? I'll keep the top-level tools (unreferenced) and scripts directories where they are.

[halindrome]

No objection.

…

On Mar 2, 2023 at 11:01:55 AM, James Craig ***@***.***> wrote: @halindrome <https://github.com/halindrome> I missed that there was already a way-aria dir in wpt that references the ATTA tests that, as I understand it, are no longer maintained. Any objection to making a similar change in that directory to move the _manual.html tests to a new manual directory? I'll keep the top-level tools (unreferenced) and scripts directories where they are. — Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD7D4H25XYDNI6EEILC6YDW2DHAHANCNFSM6AAAAAAU4PC5I4> . You are receiving this because you were mentioned.Message ID: <web-platform-tests/interop-2023-accessibility-testing/issues/1/1452209607 @github.com>

-- Shane McCarron ***@***.***

[cookiecrook]

I'll also move /aria/role/basic.html (in web-platform-tests/wpt#38758) to /wai-aria/role/basic.html

[cookiecrook]

@cyns b/c she wanted to review

[cookiecrook]

@halindrome wrote:

No objection.

Thanks web-platform-tests/wpt#38795

[cookiecrook]

Closing this issue as completed. Thanks all.