NVDA/JAWS should announce the placeholder text provided at edit fields in command palette.

[qpetraroia]

Hello, we have been recently testing out our extension for accessibility bugs, and we believe we may have found one directly linked to vscode. The extension we are using is Bridge to Kubernetes.

https://learn.microsoft.com/en-us/visualstudio/bridge/bridge-to-kubernetes-sample

Does this issue occur when all extensions are disabled?: Yes

• VS Code Version: Visual Studio Code Version 1.73.1. (user setup)
• OS Version: Windows 11 Version 21H2 (OS Build 22000.376)

Steps to Reproduce:

1. Turn on NVDA/JAWS.
2. Open Visual studio code application and setup Bridge to Kubernetes extension.
3. Download cluster config file and load 'stats-apo' project.
4. Press 'Ctrl+Shift+P' open command palette in View menu.
5. Select 'Bridge to Kubernetes: Configure' and press enter.
6. Navigate using down arrow key to select 'stats-api' in connect to kubernetes (1/4) stage and press enter.
7. Input '3001' in connect to kubernetes service: stats-api (2/4) stage and press enter.
8. Verify whether NVDA/JAWS is announcing the given placeholder text at edit field or not.

[VSCodeTriageBot]

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.73.1. Please try upgrading to the latest version and checking whether this issue remains.

Happy Coding!

[qpetraroia]

Hi @meganrogge,

The bug still exists with the new vscode version. I edited the original post. Thanks!

[meganrogge]

@TylerLeonhardt I think you just worked on improving the accessibility of the quick pick - thoughts on this?

[kupatkar99]

#A11yMAS; #A11yTCS; #A11ySev2; #DesktopApp; #Win32; #Visual Studio Code Client; #BM-VisualStudioCodeClient-Win32-Aug2022; #WCAG1.3.1; #Element:Placeholdertext; #AILimited; #JAWS; #NVDA;

[meganrogge]

@TylerLeonhardt have you looked into the feasibility of supporting this?

[TylerLeonhardt]

Sorry I'm not sure I totally follow here. Just to make sure, @qpetraroia can you attach a gif?

/gifPlease

[VSCodeTriageBot]

Thanks for reporting this issue! Unfortunately, it's hard for us to understand what issue you're seeing. Please help us out by providing a screen recording showing exactly what isn't working as expected. While we can work with most standard formats, .gif files are preferred as they are displayed inline on GitHub. You may find https://gifcap.dev helpful as a browser-based gif recording tool.

If the issue depends on keyboard input, you can help us by enabling screencast mode for the recording (Developer: Toggle Screencast Mode in the command palette). Lastly, please attach this file via the GitHub web interface as emailed responses will strip files out from the issue.

Happy coding!

[meganrogge]

@TylerLeonhardt from the above description, I believe what @qpetraroia expects is for the placeholder text to be read before the selected item is. From my testing though, that is working already (I tried Tasks: Run task and Terminal: Select Default Profile

[qpetraroia]

Thanks for the comments @TylerLeonhardt and @meganrogge. I will try and get a gif for you folks in the upcoming days!

[qpetraroia]

@meganrogge, @TylerLeonhardt sadly gif does not record audio. I will record a video and send it to you internally.

[TylerLeonhardt]

@qpetraroia video uploads on GitHub issues work now so you can always attach here :)

[qpetraroia]

@TylerLeonhardt, @meganrogge I have attached a video below! Basically, the place holder text should be read with NVDA. The accessibility folks have flagged this as a bug. Thanks!

Bridge.to.Kubernetes.-.accessibility.bugs.-.placeholder.text.not.being.read.mp4

[TylerLeonhardt]

@qpetraroia do you have Accessibility mode turned on? Just checking. You should have gotten a notification about it.

[meganrogge]

you can turn that on with editor.accessibilitySupport: on

[qpetraroia]

Let me test that today @TylerLeonhardt @meganrogge, thanks!

[qpetraroia]

@meganrogge where do I find editor.accessibilitySupport: on ?

[meganrogge]

Ctrl+, and search for editor.accessibilitySupport. The alternative way would be to run the Toggle Screen Reader Accessibility Mode command via the command palette

[qpetraroia]

Thanks for this @meganrogge, it was set to auto. I will turn it to on and try again

[qpetraroia]

@meganrogge, @TylerLeonhardt. It is still not pronouncing the text in the edit field. Would you classify this as a bug or is this just a part of VsCode?

[meganrogge]

I can reproduce the issue - @TylerLeonhardt I'd expect the placeholder text to be read before the selected item is, but that does not happen.

[meganrogge]

We'll try to fix this in February with the following read by the screen reader via @TylerLeonhardt :

• placeholder of input box (note this doesn’t logically have to be the placeholder, it could be an aria label set somewhere)
• number of items in the list
• the label of the highlighted item in the list (aka the first item in the list)

[isidorn]

I just discussed with @TylerLeonhardt and he says

"I need an ARIA expert because I can’t get this to work nicely.
aria-activedecendent is what we use today on the quick pick input box. That points to the focused item in the list. This is needed because true focus always stays in the input box. The problem is that if an element has aria-label AND aria-activedecentdent the aria-label is ignored.

I’ve tried a few other solutions and haven’t had luck, unfortunately. Looking for creative solutions"

@scottaohara might you have suggestions here?

[TylerLeonhardt]

Just to provide an update...

After chewing on the problem yesterday, my best idea is to try to mirror this design pattern as much as possible: https://w3c.github.io/aria-practices/examples/combobox/combobox-autocomplete-list.html

Applying it to the quick pick means making the following changes:

• in a11y mode, we do not focus any of the items in the list. Doing this, we can easily ensure that the placeholder is read out because aria-activedecentant won’t be set yet
• in a11y mode, When the user types something in the input box, aria-activedecendent is removed and we remove the "active" state of the elements (thus focus goes back to the input)

When pressing down the first element is highlighted. aria-activedecendent is set on the inputbox pointing to that item. The item gets read out. Nothing new here.

While this pattern is fine, it's not really what I want because the QuickPick isn't really a combo box (though I'll admit it's close). I’m sad because I would rather not have specific a11y mode behavior here… but this is the best I think we can offer with my ARIA skills.

Tl;dr: I need a solution for reading the placeholder of the quickpick input box first and then read out the first highlighted item in the list (the aria-activedecendent)

[rperez030]

Hi team. I’m reading the thread and I’m probably missing something but I’m concerned that there is perhaps a misunderstanding about how the quick input experience works in general and the particular issue of the Bridge to Kubernetes extension. Perhaps I’m missing the problem, but the quick input list works well for me. Here is an example:

1. Open the command palette by pressing CTRL +Shift +P
   a. NVDA announces the label and placeholder text for the editable field as follows: “Type the name of a command to run.  combo box  has auto complete  editable  Press 'Enter' to confirm your input or 'Escape' to cancel>”.
2. I type git, for example, and select the Git clone command with the arrow keys and Enter.
   a. Upon selecting the command, focus returns to the editable section and NVDA reads the prompt as follows: “Provide repository URL or pick a repository source.  combo box  has auto complete  editable  Press 'Enter' to confirm your input or 'Escape' to cancel  Provide repository URL or pick a repository source.  blank”.
3. I use the arrow keys to select the option to clone from remote sources and press enter on it.
   a. Upon pressing Enter, focus returns again to the editable field and the prompt is read again: “Repository name (type to search)  combo box  has auto complete  editable  Press 'Enter' to confirm your input or 'Escape' to cancel  Repository name (type to search)  blank”.

I wouldn’t expect this to behave any different. The only instance where I have observed a different behavior is with the Bridge to Kubernetes extension where, upon selecting the connect command, for example, screen reader focus seems to jump directly to the list instead of the editable portion which is the standard behavior, maybe because aria-activedescendant is set at a time when it shouldn't?

@TylerLeonhardt why you would like the placeholder text and the first highlighted item to be read together? I understand the placeholder text as the label for the editable field, therefore it is relevant only when the editable field is focused. By focused in this context I mean that it has screen reader focus, not keyboard focus. When we manipulate aria-activedescendant we are communicating the screen reader to focus on a different control, therefore the label of the control with focus shouldn't be read. The label of the list portion is "Quick Input list", and that is actually the hint that we are now exploring the list of options. I fully agree that we should stay away from an A11Y exclusive mode as much as possible.

[TylerLeonhardt]

why you would like the placeholder text and the first highlighted item to be read together?

@qpetraroia because this is the behavior of a sighted users today. They are able to see the placeholder in the input box and the first item in the list is already focused (using aria-activedescendant). This allows a user to be shown the quick pick and be able to hit ENTER to accept the first item in the list. Putting the "quick" in "quick pick".

Today, users of a11y mode via the setting editor.accessibilitySupport have a different experience, which is what you describe... no item is initially focused and a user in a11y mode must hit down arrow to get to the first item in the list.

My wish is that we could have the same experience for users of a11y mode and not... but my research above, albeit not expert level, has given me the perspective that it's just not possible... and that it would be easier to fix the bug you call out which can be simplified to:

"When we chain quick picks we need to remove aria-activedescendant on any item if one exists so that the 2nd quick pick reads the placeholder"

[rperez030]

Thanks @TylerLeonhardt. I think I understand now. Since this is really not a combobox, I would then suggest a change to the way this works from a keyboard and screen reader perspective. Please take this as a general idea which could be significantly refined or completely abandoned.

1. Tab and Shift +tab can be used to move focus between the edit field and the list box. Tab works now, but shift +tab doesn't return focus to the text field. Visual keyboard users can press Enter in the text field to trigger the first element in the list which is selected. Blind screen reader users can tab to the list to hear / read the element selected and press enter from there.
2. Typing text in the text field changes selection visually and updates an aria-live region with the selected item in the list. In a way it is better than manipulating aria-activedescendant because the text is still editable. note: It should be a visually hidden live-region which will only contain the visually highlighted item at a given time.
3. Typing while focus is in the list filters the list as it does now but doesn't update the aria-live region because screen readers are tracking ffocus in the list itself therefore there is no need to announce anything.
4. While focus is in the text field, pressing down arrow changes selection in the list and moves focus to the list itself.
5. if quick pick changes resulting from selecting an option, we reset focus to the edit field so that the placeholder is read. Screen reader users can then tab to the list to hear the selected item / navigate, or start typing in the edit field.
6. We remove the combobox role from the text field because we agree it is not a combobox.

note: As a blind screen reader user myself, I don't know how the experience looks / feels like from a visual perspective. For example, I don't know if sighted keyboard users can edit the text as they type, or if there still an insertion point visible in the text field as ones navigates in the list with the keyboard as it is now.

[TylerLeonhardt]

@rperez030 I think we are on to something here... Thanks for your input. It sounds like, from what I can tell, you are suggesting we drop aria-activedescendant usage from the input box and go with your alternative. I'm intrigued by this idea.

I have a couple of follow ups mostly because my ARIA skills are quite low:

1. quoting you:

Typing text in the text field changes selection visually and updates an aria-live region with the selected item in the list. In a way it is better than manipulating aria-activedescendant because the text is still editable

Could you explain the difference between these two approaches? How are they represented differently via a screen reader? Just to confirm, yes, the inputbox that currently has aria-activedescendant is still editable and should be editable. We want users to be able to continue typing to further filter the list... just like a combobox.

2. you said:

Typing while focus is in the list filters the list as it does now but doesn't update the aria-live region because screen readers are tracking focus in the list itself therefore there is no need to announce anything.

Today, when you type a character, the active item in the list becomes the first item in the list. I would imagine we would still announce via aria-live if that item isn't the one the screen reader user is on?

3. to clarify what you said here:

or if there still an insertion point visible in the text field as ones navigates in the list with the keyboard as it is now.

Yes. Visual users are able to see a cursor in the input box that they can move left to right as they choose and edit the text in the box while an item in the list is active (again, similar to the combobox example I linked above). This works today because true focus is entirely kept by the input box. "fake focus" or what we call "the active list item" never actually gets browser focus, it just appears "focused" or "active" or "highlighted" to the user compared to the items in the list.

[rperez030]

Hi @TylerLeonhardt !
quoting you:

Could you explain the difference between these two approaches? How are they represented differently via a screen reader?

Aria-activedescendant overrides DOM focus. It means that, as soon as we add the property, screen reader focus goes to the list and screen reader no longer track the cursor in the text field. If we use an aria-live region however, changes in selection can be announced while focus is maintained in the input field for the user to edit. In this way if an element is selected in the list by default, we can announce that element and the placeholder text will be read by the screen reader due to the focus event. We may not be able to fully control the order in which both events are read, but that's a lesser problem in my opinion. Does it make sense?

Today, when you type a character, the active item in the list becomes the first item in the list. I would imagine we would still announce via aria-live if that item isn't the one the screen reader user is on?

Yes, that is correct. actually, your visual explanation has helped me understand the entire experience better. I initially thought that there was no cursor in the text field at all while the user was typing.

[TylerLeonhardt]

@rperez030 thanks a lot for your comments! I really want to sit down and play with aria-live some more based on your idea. Considering that this issue is pressing, I've made a small change to get the placeholder to be read out.

[rperez030]

@TylerLeonhardt happy to test that change when it becomes available in an insiders build, or remotely with you in case you want earlier testing. for my idea to work, it would require some changes in keyboard behavior as well, and careful testing so that we don't break current user expectations around a widget which is critical for some customers. feel free to reach out if you need additional input.

[VSCodeTriageBot]

This bug has been fixed in the latest release of VS Code Insiders!

@qpetraroia, you can help us out by commenting /verified if things are now working as expected.

If things still don't seem right, please ensure you're on version 6fe3014 of Insiders (today's or later - you can use Help: About in the command palette to check), and leave a comment letting us know what isn't working as expected.

Happy Coding!

[rperez030]

I did some testing using the latest insiders build and NVDA

Results with screen reader mode on:

• Placeholder text is announced when the widget receives focus.
• When typing in the text field, the number of results is announced by NVDA but not the highlighted option. This can be a productivity killer. I believe the announcement of the highlighted option is the most important piece of information while typing. the number of results is irrelevant at this point. It becomes important when the user starts to navigate with the arrow keys, and at that time the screen reader takes care of it.

Screen reader mode off:

• When the widget receives focus, the number of results is communicated by NVDA followed by the announcement of the quick input list and the highlighted element, but the placeholder text is not announced, likely because the "combobox" portion doesn't have focus.
• when typing, NVDA announces changes in the highlighted option in Braille and speech as expected.

Version: 1.77.0-insider (user setup)
Commit: 6fe3014
Date: 2023-03-23T10:21:45.873Z
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: 10.2.154.26-electron.0
OS: Windows_NT x64 10.0.22621
Sandboxed: Yes

[kupatkar99]

#Closed
Verified the issue in latest environment:
Version: 1.78.0-insider (user setup)
Issue is fixed. Hence closing the bug. please find added attachment for reference.

166920_Fixed.mp4