Which states end up in the accessibility tree? #102
Comments
|
This would make accessibility different than other UA algorithms like find-in-page. |
|
Here is a doc summarizing current implementation choices in Chromium: https://docs.google.com/document/d/1_xqCPxplvI-cF69cwMtEhNvd62UHrCqLvBo79xnvBBQ/edit |
|
Currently, we represent invisible content as nodes in the accessibility tree that don't have layout - in Chromium paralance, Revised proposal given that:
The reason to put If we want to maximize performance for item 1 above, and avoid risk of exposing assistive technology to sites, then we'll have to include In this scenario, element |
Unfortunately I'm unfamiliar with the accessibility code, but activated for some other reason in a non-AX here would mean for example find-in-page can do it. Is there an equivalent for AX? I was under the impression that if it's not in the tree at all, then it wouldn't be findable (and if it's not findable we can't activate it, so it's a bit circular here). Or is the AX equivalent of find in page the same as the non-AX find in page? In which case, we would activate and then the activated nodes would get populated into the AX tree. |
Find-in-page will be OK since it's using the same find-in-page mechanisms and it sends back the result to the accessibility technologies (I think..), but yeah I guess the accessibility technologies can't do anything to things that are not in the AX tree? cc @alice |
|
Many ATs implement their own find-in-page (e.g. NVDA's "search for text" feature) which is separate from the browser's find-in-page. Of course, folks using ATs may also use the browser's find-in-page feature.
Yeah, my understanding (from my discussions with @chrishtr and @rakina) is that in this case it would need to be explicitly activated by the author in response to a user interaction, such a clicking a button. So it would be activatable, but the author, not the user agent, is in control of when it is activated. However, this does get a bit weird when find-in-page can find it. I did get a bit lost in those discussions trying to understand use cases when you'd use |
One example is https://github.com/wicg/virtual-scroller, where I'm not sure about other cases, but I've seen requests/bugs filed for making text within collapsed sections to be findable (and opens the section when navigated to), so I guess there's some interest in that too? The other big use case similar to collapsed sections is tabbed/stacked UI, where clicking a "tab" will unlock the content of that tab and hide the previous tab. |
Ahh yes, I could imagine this being useful for something like the mobile view for Wikipedia, for example. So in that case, could you also give an example for the |
If I remember correctly the |
It's for DOM that is not currently part of the UI show to the user at all. One example is the pre-rendered/cached view of a non-currently visible single-page-app screen that the user might navigate to soon. Therefore this content should definitely not be in the AT. |
Does the accessibility tech somehow make use of |
|
I still think the revised resolution I proposed here is a good way forward. Accessibility tech would still be able to find-in-page virtual scroller content by using the browser's built-in one. And the concern of detection of accessibility users would be resolved. |
|
Hmmm I'm still thinking about this, sorry I didn't get a chance to loop back today. |
I am having trouble understanding the context for this question, so perhaps I've misunderstood. Short answer, if I understand the question, is yes. We don't distinguish between |
Ok, so more or less semantically equivalent to
Makes sense. |
Ok. What I was trying to get at is whether there is a notion of just sending the DOM tree to the accessibility tree as opposed to the Layout tree. The former has no rendering cost in Blink, but the latter of course does. My understanding from what Rakina said in previous docs was that somehow there is a notion of sending the DOM across for content not on screen. I think she said this is how accessibility tools know to "scroll up" or "scroll down". Is that right? And then I asked the same question about "find in page". I think your answer was that the "find in page" feature for tools such as NVDA do in fact use the "DOM" in the accessibility tree if it's there.
Yes, exactly. |
|
Discussed this in a meeting earlier w/ @alice and @vmpstr. We concluded to exclude Might reconsider in the future, but this seems enough for now. |
|
From the virtual scroller POV, we have always said that content would be finable and accessible. So leaving things out of the AX tree when they are skip-viewport-activation drops that. This seems wrong to me. I can understand that it conflicts with current implementations but from a semantics point of content is in 1 of 3 states 1 visible These should each map to a distinct value of the In state 3, regardless of whether it is viewport-activated or not, it seems like the AX tree should run off DOM. Am I missing something that makes this bad? |
It's bad because:
Rather than try to work around this by making a mode where AX users have more insight into offscreen content in a developer-managed virtual-scroller or other UI, we should expect AX users to go through similar UA mechanisms such as a "find-in-page" AX API, or perform gestures or clicks (for the collapsed-section use case). OTOH |
|
I think we agree on the goal of making it possible for an advanced developer to manage visibility purely for performance reasons and the goal is that for all purposes, the user should be entirely unaware that this is happening. So if that's not an agreed goal (at least as a north-star) then the following won't really make sense. This means that on-screen display, find-in-page, tab navigation, AX tree and anything else we can think of should just work like normal content. There may be differences but they should be as unobtrusive as possible. Dropping a potential large chunk of the document out of the AX tree is a very obtrusive difference for AX users. If there are things that make it difficult that's one thing but I think we should come to an agreement on whether that would be a major miss for our goal or whether it's WAI. I think it's a major miss. Do others agree? Assuming it's a major miss, I think we should figure out how to achieve it without compromising privacy. I don't know enough about AX clients to know whether they have a concept of focus or "viewport" but maybe for example we could run off DOM and only materialize content that is in the AX client's viewport/close to it's focus? That seems like it would be computationally undetectable. Maybe v1 doesn't achieve it, fully-managed content is going to be rare, so we have a bit of time to try fix it in post. |
I agree with this goal. However, the goal of the developer having to do no work, or the AX tool author to do no work, to work really well with such content in AX tools, is only a nice-to-have if the downsides otherwise outweigh the upsides. After all, the mode we're discussing is a developer-controlled one.
The difference is as follows. For non-AX users, the current plan is to require the developer to do something in response to activation events for the user to ever see hidden content. In other words, the UA algorithms all have to integrate with these events. I don't think AX tools should be an exception to the need to integrate with activation events. Therefore in v1 I think we should not expose hidden content to such tools, and expect the tools/developers/browsers to work together to integrate UA algorithms into those tools in some reasonable way that doesn't have the privacy or semantic downsides I listed earlier in this issue. In the future, I can imagine AX tools adding features to understand |
One worry I have about making developers do work is not so much about the need to do work, it's fine to make people do some work to get a benefit but there's a bit difference between do some work to set things up vs having to change code or maintain separate code paths for optimized and non-optimized content because the latter makes it really hard for people to drop in 3rd-party optimizing solutions like
So if AX tools already have a concept of focus or of a "visible" range of the document that has been rendered accessibly then and if the browser already knows what that currently is and what it changes then it seems like it should be part of the activation story. I don't know if that exists or not. Or does the AX tree know what parts of the tree have been accessed by the AX tool? If there is already a pattern of incremental access by the AX tools then that can translate to activation events and can avoid the all-at-once computation. One problem I see though is that the current version is still at the implementation level, with no semantics expressed in the document. So there's nothing to distinguish whether something is invisible because it's being actively managed for optimization vs it's invisible because it's conceptually not in the document but is waiting in the wings. You can kinda tell by whether it's got a place-holder size and or not but that's a bit fishy. Or looking at it another way, it's OK to turn off viewport activation because JS is already fully aware of the visible viewport and it's relation to the DOM elements. JS is not at all aware of an AX tool's "rendered viewport" (for whatever that might mean, e.g. what was just read out) and we have an explicit goal of not revealing that an AX tool is in use at all. So it's kind of hard to imagine how we can not put the content into the AX tree but later put it in, what's the signal to put it in? Is it just focus navigation? |
Correct. Activation in these cases is controlled by the developer, and the browser has to assume that the content is not meant to be presented to the user. This is another way in which putting such invisible content into the AX tree won't work, because sometimes it really is not part of the document and it's just there because the developer is doing pre-rendering or measurement on it.
The signal is AX gestures such as scroll, focus navigation, find-in-page, or click on the visible content, just like it would be for someone not using an AX tool. |
I actually think it's a pretty big problem that we have no semantic signal here and the browser cannot tell conceptually in the document but optimized under the control of the developer from conceptually out of the document. If find-in-page can find the content, then AX should also see the content. In fact, I would say that
should all see experience exactly the same content. If we have a mode where one or more of those see different content, then that seems broken to me. Maybe it can ship as a v1 is a different question but the intention to make it so seems like it should be there.
The question is, are the AX tools that present content without scrolling to it? E.g. on Android if I turn on talkback, and start swiping right (to read out more and more of a page), the scroll position also changes so that what is being read is on-screen (at least at the block level, sometimes it's reading stuff off-screen but I assume it's all part of the same div). So talkback on Android is safe but are there other screen readers that will e.g. give you a list of and in the page and read them out to you without moving the the viewport or the focus? If so, they will be broken by this kind of content. |
My understanding is that The example that @vmpstr gave me which helped form my intuition and my support for this initial design is the Wikipedia mobile view, where parts of the page are hidden from view in collapsed sections. Scrolling those sections into view doesn't cause them to be activated: the user must take an explicit action to activate them, unless they are activated by a UA interaction such as find in page. If we exposed those collapsed sections of the page to assistive technologies, it would actually be creating a significantly different experience for AT users than for non-AT users. In that case, AT users would notice no difference between collapsed sections and non-collapsed sections. Quite apart from the privacy implications, that also seems like a poor experience to me.
It seems like this is more describing the case without |
|
@alice There's some context that you're missing. The WP collapsed-section is definitely as you described but there is another very different use case for skip-viewport which is that of the virtual scroller. We do not believe that we can build good enough "virtualization" into the rendering engine, partly because if it's in the rendering engine, it needs to work perfectly for all cases. Instead we want people to be able to build virtual-scrollers in JS that work really well. When they do that, they can set limitations (e.g. that all content must have certain For this kind of content, it should behave exactly as if it is in the doc, it should be find-able, tab-navigable, selectable etc and it should also be just as accessible as if it was normal content. The non-skip viewport invisibility gets you a kind of half-way version of this. The content doesn't cost rendering until it scrolls into view but once it scrolls into view it becomes just like any other content. A virtual-scroller would re-hide content that is gone off-screen so that it won't slow down rendering but again, it's not hidden like a collapsed WP section. To the user it should be indistinguishable from unhidden content. |
|
@fergal Could you illustrate how you would build a virtual scroller using |
|
The scroller uses IntersectionObserver to track viewport moves itself. We're currently shipping an experimental one in Chrome but I'm no longer working on this so it's been a stagnant and e.g. I think has not been updated to use |
|
Can you expand on why you'd use IntersectionObserver in preference to viewport activation? |
|
Because viewport activation will never re-hide something, so performance degrades as more and more content becomes visible. Current JS<->DOM virtual scrollers keep basically a screen-full of content visible so rendering/style is always O(screen size), not O(page size) but find in page, accessibility etc is destroyed. The goal is to allow people to write a virtual scroller that keeps everything in DOM, preserving FIP, accessibility but still keep rendering O(screen size) and to do that you need to re-hide content that has been hidden. Once you need to re-hide, you need to be in the business of tracking scroll position either via IntersectionObservers or other ways (using IntersectionObservers is the result of much discussion and is not really important except for the fact that we are somehow tracking this) and once your JS has an opinion on what should be visible/invisible, you don't really want the browser butting in with it's opinion too, so you just turn off the the browser's one entirely to keep things simpler. I'm available to VC most of this afternoon BTW if you want to get into more details. |
|
Also, https://github.com/fergald/virtual-scroller-demos has instructions on
how to launch chrome with the scroller enabled, you can take the demos for
a spin, it appears that they still work, although there might be some
issues.
… |
|
@fergald Thanks for the extra detail there.
I'll get in touch on another channel. |
We actually changed the behavior of viewport activation to automatically re-hide. See this issue comment resolution. Your other points about IntersectionObservers for all of the locked elements facing scaling pressure for very large content still stand though. |
|
Recapping an earlier in-person conversation with @fergald, as well as replying to some points raised here...
Yes, VoiceOver (built in screen reader on Mac devices) does this. I made a little demo with a Wikipedia page:
In the demo, I use ctrl-option-U to bring up the VoiceOver rotor, then the down arrow to scroll through the list of headings on the page. As I arrow through the headings, VoiceOver highlights them and speaks them (except when I start arrowing too fast for it to keep up; then it makes a gentle clicking noise) but does not scroll through the page. Once I reach a heading that I'm (ostensibly) interested in, I press Enter to select it. At that point, VO scrolls to that heading.
Yes, that behaviour will not work in the virtual scroller case where However, it's worth noting that in the case of a page too big for viewport activation to work out of the box (I understand from @fergald that a virtual scroller for smaller pages works fine with viewport activation on its own), AT will already struggle if not outright choke on the page. For example, trying to load the WHATWG single page HTML spec in Chrome with VoiceOver turned on results in a sad tab after about a minute of "Chrome busy". We have several existing bugs on Chrome specifically about hanging behaviour with large numbers of direct child nodes of a single node; the latest I'm aware of is https://crbug.com/981322 (public issue). Even if browsers can handle that large a number of (child) nodes, it's still not clear that all assistive technologies are equipped to handle such a massive tree effectively. For that reason, I'm still in favour of excluding
One project we should keep an eye on is Accessible Loading and Searching of Content (yes, really). This has the explicit goal of solving a closely related problem: pages which don't send all of their content to the client at any given time (Google Docs/MS Office 360 being typical examples). The current, deliberately very basic iteration is to add an ARIA attribute which exposes the fact that not all of the content exists in the DOM, and hints at a direction to scroll to load more content. This allows the AT to scroll to the end of a container to prompt more loading of content. From this base, I believe the authors of that project are hoping to build out richer two-way communication APIs between ATs and browsers around partial/incomplete content. |
Now both render-subtree: skip-viewport-activation and skip-activation subtrees are not in the AX subtree at all, as discussed in WICG/display-locking#102 (comment) Now only render-subtree:invisible subtrees are in the AX tree, and they are automatically marked as offscreen because the nodes are activated when they are on screen. Bug: 1001930 Change-Id: I1de511672f60078a02b3ddf3d989f2da1c44ae9d Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2028883 Reviewed-by: Alice Boxhall <aboxhall@chromium.org> Reviewed-by: vmpstr <vmpstr@chromium.org> Reviewed-by: Chris Harrelson <chrishtr@chromium.org> Commit-Queue: Rakina Zata Amni <rakina@chromium.org> Cr-Commit-Position: refs/heads/master@{#740531}
… tree" This reverts commit c08274e. Reason for revert: this CL made DumpAccessibilityTreeTest.DisplayLockingViewportActivation flaky on https://ci.chromium.org/p/chromium/builders/ci/Mac%20ASan%2064%20Tests%20%281%29?limit=200 Failures: https://ci.chromium.org/p/chromium/builders/ci/Mac%20ASan%2064%20Tests%20%281%29/60763 https://ci.chromium.org/p/chromium/builders/ci/Mac%20ASan%2064%20Tests%20%281%29/60759 Original change's description: > DL: Don't include skip-viewport-activation subtrees in the AX tree > > Now both render-subtree: skip-viewport-activation and skip-activation > subtrees are not in the AX subtree at all, as discussed in > WICG/display-locking#102 (comment) > > Now only render-subtree:invisible subtrees are in the AX tree, and they > are automatically marked as offscreen because the nodes are activated > when they are on screen. > > Bug: 1001930 > Change-Id: I1de511672f60078a02b3ddf3d989f2da1c44ae9d > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2028883 > Reviewed-by: Alice Boxhall <aboxhall@chromium.org> > Reviewed-by: vmpstr <vmpstr@chromium.org> > Reviewed-by: Chris Harrelson <chrishtr@chromium.org> > Commit-Queue: Rakina Zata Amni <rakina@chromium.org> > Cr-Commit-Position: refs/heads/master@{#740531} TBR=vmpstr@chromium.org,aboxhall@chromium.org,chrishtr@chromium.org,rakina@chromium.org Change-Id: If1d7041ed1fa81a2693480480eaf8163f2606aa8 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 1001930 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2051106 Reviewed-by: Maxim Kolosovskiy <kolos@chromium.org> Commit-Queue: Maxim Kolosovskiy <kolos@chromium.org> Cr-Commit-Position: refs/heads/master@{#740614}
… tree" This is a reland of c08274e The CL was previously reverted because the viewport activation test is flaky, probably due to uncertain timing of near-the-viewport activation. I've updated the test to only test for in-the-viewport activation, I'll investigate further on why it's flaky for near-the-viewport (manual testing shows activation happens consistently). SHERIFFS: please don't revert this CL if it introduces a new flaky test. Instead, please disable them and I'll investigate. Original change's description: > DL: Don't include skip-viewport-activation subtrees in the AX tree > > Now both render-subtree: skip-viewport-activation and skip-activation > subtrees are not in the AX subtree at all, as discussed in > WICG/display-locking#102 (comment) > > Now only render-subtree:invisible subtrees are in the AX tree, and they > are automatically marked as offscreen because the nodes are activated > when they are on screen. > > Bug: 1001930 > Change-Id: I1de511672f60078a02b3ddf3d989f2da1c44ae9d > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2028883 > Reviewed-by: Alice Boxhall <aboxhall@chromium.org> > Reviewed-by: vmpstr <vmpstr@chromium.org> > Reviewed-by: Chris Harrelson <chrishtr@chromium.org> > Commit-Queue: Rakina Zata Amni <rakina@chromium.org> > Cr-Commit-Position: refs/heads/master@{#740531} Bug: 1001930 Change-Id: Icdf43cff86ade7e7e8f2aee76933dfc51e3f022c Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2054649 Reviewed-by: vmpstr <vmpstr@chromium.org> Reviewed-by: Alice Boxhall <aboxhall@chromium.org> Commit-Queue: Rakina Zata Amni <rakina@chromium.org> Cr-Commit-Position: refs/heads/master@{#742520}
…n the AX tree"" This reverts commit 0b25b17. Reason for revert: Findit (https://goo.gl/kROfz5) identified CL at revision 742520 as the culprit for flakes in the build cycles as shown on: https://analysis.chromium.org/p/chromium/flake-portal/analysis/culprit?key=ag9zfmZpbmRpdC1mb3ItbWVyQwsSDEZsYWtlQ3VscHJpdCIxY2hyb21pdW0vMGIyNWIxN2VkYThlMWY0YjRiN2MxMDI0ZmRjZWQ2Mjg0Nzc5Y2RlYQw Sample Failed Build: https://ci.chromium.org/b/8888021964038070960 Sample Failed Step: content_browsertests Sample Flaky Test: All/DumpAccessibilityTreeTest.DisplayLockingViewportActivation/blink Original change's description: > Reland "DL: Don't include skip-viewport-activation subtrees in the AX tree" > > This is a reland of c08274e > > The CL was previously reverted because the viewport activation test is > flaky, probably due to uncertain timing of near-the-viewport activation. > I've updated the test to only test for in-the-viewport activation, > I'll investigate further on why it's flaky for near-the-viewport > (manual testing shows activation happens consistently). > > SHERIFFS: please don't revert this CL if it introduces a new flaky test. > Instead, please disable them and I'll investigate. > > Original change's description: > > DL: Don't include skip-viewport-activation subtrees in the AX tree > > > > Now both render-subtree: skip-viewport-activation and skip-activation > > subtrees are not in the AX subtree at all, as discussed in > > WICG/display-locking#102 (comment) > > > > Now only render-subtree:invisible subtrees are in the AX tree, and they > > are automatically marked as offscreen because the nodes are activated > > when they are on screen. > > > > Bug: 1001930 > > Change-Id: I1de511672f60078a02b3ddf3d989f2da1c44ae9d > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2028883 > > Reviewed-by: Alice Boxhall <aboxhall@chromium.org> > > Reviewed-by: vmpstr <vmpstr@chromium.org> > > Reviewed-by: Chris Harrelson <chrishtr@chromium.org> > > Commit-Queue: Rakina Zata Amni <rakina@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#740531} > > Bug: 1001930 > Change-Id: Icdf43cff86ade7e7e8f2aee76933dfc51e3f022c > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2054649 > Reviewed-by: vmpstr <vmpstr@chromium.org> > Reviewed-by: Alice Boxhall <aboxhall@chromium.org> > Commit-Queue: Rakina Zata Amni <rakina@chromium.org> > Cr-Commit-Position: refs/heads/master@{#742520} Change-Id: I9258a82fa5a36921963dda0d12ce6efde1895e4b No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: 1001930 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2063667 Cr-Commit-Position: refs/heads/master@{#742651}
How about:
rendersubtree: invisible-> Present in accessibility tree while in viewport onlyrendersubtree: invisible skip-viewport-activation-> Not in accessibility tree unless activated for some other reasonrendersubtree: invisible skip-activation-> Not in accessibility tree@rakina
This way, a UI with content offscreen will not have to pay the cost of that offscreen content. Also, content that is on-screen but hidden behind a click (e.g.
<details>element, modal dialog) also won't pay the cost. This makes such content behave to accessibility tools likedisplay:none.Also, this will hopefully reduce the ability of developers to detect whether accessibility tools are being used.
The text was updated successfully, but these errors were encountered: