Skip to content

Conversation

@BenHenning
Copy link
Collaborator

@BenHenning BenHenning commented Sep 24, 2025

The basics

The details

Resolves

Fixes #8206
Fixes #8210
Fixes #8213
Fixes #8255
Fixes #8211
Fixes #8212
Fixes #8254
Fixes part of #9301
Fixes part of #9304

Proposed Changes

This PR completes the remaining ARIA roles and properties needed for all core fields. Specifically:

  • ARIA for checkbox fields #8206: A better name needed to be used for the checkbox value, plus there was an ARIA property missing for actually representing the checkbox state. The latter needed to be updated upon toggling the checkbox, as well. These changes bring checkbox fields in compliance with the ARIA checkbox pattern documented here: https://www.w3.org/WAI/ARIA/apg/patterns/checkbox/.
  • ARIA for dropdown fields #8210: This one required a lot of changes in order to adapt to the ARIA combobox pattern documented here: https://www.w3.org/WAI/ARIA/apg/patterns/combobox/. Specifically:
    • Menus needed to have a unique ID that's also exposed in order to link the combobox element to its menu when open.
    • ARIA's activedescendant proved very useful in ensuring that the current dropdown selection is correctly read when the combobox has focus but its menu isn't opened.
    • The default properties available for options (label and value) aren't very good for readout, so a custom ARIA property was added for much clearer option readouts. This is only demonstrated for the math arithmetic block for now.
    • The text element is normally hidden for ARIA but it's useful in conjunction with activedescendant to represent the current value selection.
    • Images have been handled here as well (partly as part of ARIA for image fields #8255) by leveraging their alt text for readouts. This actually seems to work quite well both for current value and selection.
  • ARIA for variable fields #8213: Much of the improvements here come from the combobox (FieldDropdown) improvements explained above. However one additional bit was done to provide an explicit 'Variable ' readout for the purpose of clarity. This demonstrates some contextualization of the value of the field which may be a generally useful pattern to copy in other field contexts.
  • ARIA for image fields #8255: Image fields have been refined since they were redundantly specifying 'image' when an image ARIA role is already being used. Now only the alt text is supplied along with the role context. Note that images need special handling since they can sometimes be navigable (such as when they have click handlers).
  • ARIA for text input fields #8211: Text input fields have had their labeling improved like all other fields, and the field's value is now exposed via its text element since this will show up as a StaticText node in the accessibility tree and automatically be read as part of the field's value.
  • ARIA for number input fields #8212: This gets the same benefits as the previous point since those improvements were included for both text and number input. However, existing valuemin and valuemax ARIA properties have been removed. It seems these are really only useful when introducing a slider mechanism (see https://www.w3.org/WAI/ARIA/apg/patterns/slider/) and from testing seems to not really be utilized for the basic text input that FieldNumber currently uses. It may be the case that this is a better pattern to use in the future, but it's more likely that other custom fields could benefit from more specific patterns like slider rather than FieldNumber being changed in that way.
  • ARIA for label fields #8254 and part of Audit and reduce complexity of accessibility node tree #9304: Field labels have been completely removed from the accessibility node tree since they can never be navigated to (as ARIA for label fields #8254 explains all labels will be included as part of the block's ARIA label itself for readout parity with navigation options).

Note that it doesn't cover external fields (such as those supplied in blockly-samples), nor does it fully set up the infrastructure work for those. Ultimately that work needs to happen as part of #9301.

Beyond the role work above, this PR also introduces some fundamental work for #9301. Specifically:

  • It demonstrates how block definitions could be used to introduce accessibility label customizations (in this case for the options of the arithmetic operator block's drop-down field, plus the block itself).
  • It sets up some central label computation for all fields, though more thought is needed on whether this is sufficient for custom fields outside of core Blockly and on how to properly contextualize labels for field values. Core Blockly's fields are fairly simple for representing values which is why that aspect of Introduce better field labeling and accessibility for screen readers #9301 didn't need to be solved in this PR. Note that the field labeling here is being used to improve all of the fields above, but also it tries to aggressively fall back to the next best label to be used (though it's possible to run out of options which is why fields still need contextually-specific fallbacks).

Reason for Changes

Generally the initial approach for implementing labels is leveraging as specific ARIA roles as exist to directly represent the element. This PR is completing that work for all of core Blockly's built-in fields, and laying some of the groundwork for generalizing this support for custom fields.

Having specific roles does potentially introduce inconsistencies across screen readers (though should improve consistency across sites for a single screen reader), and expectations for behaviors (like shortcuts) that may need to be ignored or only partially supported (#9313 is discussing this).

Test Coverage

Only manual testing has been completed since this is experimental work.

Video demonstrating most of the changes:

Screen.recording.2025-10-01.4.05.35.PM.webm

Documentation

N/A -- Experimental work.

Additional Information

This has only been tested on ChromeVox.

This also provides the basis for generalized field typing and labeling
support for high-quality labels.
@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Sep 24, 2025
@BenHenning BenHenning changed the title feat: Fix aria roles and structures for fields (experimental) feat: Fix ARIA roles and setup for fields (experimental) Sep 24, 2025
@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Sep 24, 2025
@BenHenning
Copy link
Collaborator Author

Note to self: maybe check shortcuts since that could allow #9313 to possibly be addressed, too.

@BenHenning
Copy link
Collaborator Author

Keyboard nav test failure is happening due to https://github.com/google/blockly-keyboard-experimentation/blob/30c5286ee0910c2795c1c93bbf440ed5ab1b3953/test/webdriverio/test/block_comment_test.ts#L39 being a bit fragile with directly using a generated ID that can easily change (such as in this PR since more IDs are being generated). That test will need to be updated outside of this PR.

BenHenning added a commit to RaspberryPiFoundation/blockly-keyboard-experimentation that referenced this pull request Sep 25, 2025
…#745)

RaspberryPiFoundation/blockly#9384 revealed that one of the Block comment tests was a bit fragile in directly validating a specific node ID. Since node IDs can change if new elements on the page require a generated ID, the test could easily fail with changes to core Blockly.

This PR fixes the issue by using a regex check instead of a direct ID match. The 'comment_textarea' part is _probably_ sufficiently unique for the test to be considered correct (though it is possible for the wrong comment to be focused here, but it seems really unlikely with how simple the test is being set up and, if that actually happened, it's sort of incidentally still correct as there would be _some_ comment that has focus).

This PR also includes a number of attempts to improve test stability (particularly in CI):
- It introduces a CI-specific command that adds a 30 second Mocha test timeout to provide a lot more time for tests to fail (since CI tends to run on less performant machines than used for development).
- It introduces a synchronization between the Mocha timeout and the timeout used for WebdriverIO's `waitFor*` functions. Note that this actually will usually be 60 seconds regardless of the configured Mocha timeout due to the way the hook is set up. However, this is still far better than the 5 second default as at least 1 test seemed to fairly regularly fail against that limit.
- It adds a few missing 'wait' calls for helpers that are performing side effects which ensures better synchronization when using `PAUSE_TIME` (and possibly better stability when running in CI, too, though it's unclear of `pause()` actually does anything when the provided timeout is 0).
- See #745 (comment) attempts to further stabilize OS X tests only to yield that there are actual test failures.

This will be merged into the screen reader experimental branch to unblock the core Blockly PR as well.

#746 has been filed to track the discovered test failures.
Copy link
Collaborator Author

@BenHenning BenHenning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review.

@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Sep 25, 2025
@BenHenning BenHenning marked this pull request as ready for review September 25, 2025 23:41
@BenHenning BenHenning requested a review from a team as a code owner September 25, 2025 23:41
@BenHenning BenHenning requested review from maribethb and removed request for a team September 25, 2025 23:41
@maribethb
Copy link
Contributor

Also, if there's any way you could attach a video demo to this PR, that would help a lot, as I find it difficult sometimes to know exactly what's being read out based on the roles and state. Like somewhere the actual value should be read out, which doesn't seem to be explicitly in any of the manually-set roles, so the video would help me make sense of how it all fits together. Or, in lieu of that, writing out a couple of examples.

@github-actions github-actions bot added PR: feature Adds a feature and removed PR: feature Adds a feature labels Oct 1, 2025
Copy link
Collaborator Author

@BenHenning BenHenning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-reviewed latest changes.

Also, if there's any way you could attach a video demo to this PR, that would help a lot, as I find it difficult sometimes to know exactly what's being read out based on the roles and state. Like somewhere the actual value should be read out, which doesn't seem to be explicitly in any of the manually-set roles, so the video would help me make sense of how it all fits together. Or, in lieu of that, writing out a couple of examples.

I've been trying to minimize videos since they can be time consuming (especially doing good ones), but I did add a quickly made one to the PR description outlining most of the changes in the PR which hopefully helps add some context. I realize it's difficult especially for a PR like this that's affecting so much at one time.

@BenHenning
Copy link
Collaborator Author

Everything that was addressed seems largely non-controversial, so I don't think another review pass is needed. The more tricky comments were either closed without further action needed, or tracked in new locations so that they can be addressed outside of this PR.

@BenHenning BenHenning merged commit 3cf834a into RaspberryPiFoundation:add-screen-reader-support-experimental Oct 1, 2025
8 of 9 checks passed
@BenHenning BenHenning deleted the fix-aria-roles-and-structures-for-fields branch October 1, 2025 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: feature Adds a feature

Projects

None yet

3 participants