UIA Formatted Text Navigation

[carlos-zamora]

Description of the new feature/enhancement

Text Ranges can be explored via a number of ways:

• Character
• Format
• Word
• Line
• Paragraph
• Page
• Document

We don't necessarily have to support all of them (refer to #3161 for us adding word navigation). But it would be interesting if we supported "format".

I envision us being able to jump between runs of formatted text. Today, the main formatting I can think of is colored text composed of foreground and background color. Maybe we could just see if any have changed and, if that's the case, that would be a breakpoint for formatted text navigation.

Proposed technical implementation details (optional)

Coincidentally, I just did a little bit of work in the AttrRowIterator. If that's actually taking a look at the text attributes, this would be an integral part of this design.

[carlos-zamora]

More info from link:

TextUnit_Format is used to position the boundary of a text range based on the formatting attributes of the text. For example, if a text range is currently positioned on a single character of a word, specifying TextUnit_Format in a call to IUIAutomationTextRange::ExpandToEnclosingUnit expands the text range to include all text that shares all the same attributes as the single character. The resulting text range might or might not include the entire word. Also, using the format text unit will not expand a text range across the boundary of an embedded object such as an image or hyperlink.

Unlike the other text units, which include the text units that are smaller than themselves, TextUnit_Format can be smaller or larger than the other units. For example, if an entire document shares the same text attributes and contains no embedded objects, expanding a text range by TextUnit_Format would create a new range that encompasses the entire document, while expanding the text range by TextUnit_Word would create a smaller range.

[codeofdusk]

Are there plans to close this in time for 21H1?

[DHowett-MSFT]

This one isn't on our radar for 21H1; we need to figure out how this works in a world where text-mode applications can print complicated UIs and things that aren't basic formatted text. There's a spec floating around in the Terminal Working Group as to how to best handle sub-channel accessibility information (like alt text for textual screen elements), but I can't find it right now -- that might have to be part of a comprehensive solution here.

[codeofdusk]

As an aside, does the console even expose colours to UIA? I'm not quite sure that it does.

[DHowett-MSFT]

It doesn't currently.

[codeofdusk]

Oh, I just found #2161. Thanks.

[codeofdusk]

Are there any plans to add this? It seems much more straightforward after #10336.