Consider revamping accessibility focus system

[chunhtai]

b/193413757

If a new route is pushed or removed from the screen, the existing accessibility focus will be shifted to a new semantics node in the new page. Currently Flutter leaves it to the native OS to decide which node it should focus. This can sometimes be unpredictable.

Example:
#97522
#93926
#115808

and various feature request due to this issue
#94523
#59594

This is even more tricker that app built with native components usually performed correctly compares to custom view with virtual accessibility trees, such as Flutter application.

We should probably think about whether the Flutter should manage accessibility focus internally

[chunhtai]

@goderbauer @dnfield
I wonder what do you think about the idea that flutter manage the focus itself

Although the Android does not encourage the use of private API, we can still use it internally as long as we don't expose it. Worst comes to worst, the API no longer works and things would just go back to the behavior we have now.

Some idea I have is that Flutter framework remember which node is last focused in a scopesRoute, maybe caching it in somewhere in the scopesRoute semantics node. it would then up to the embedding to decide what to do with this information. The iOS and Android accessibility bridge will then try to refocus that node if it detects scope route structure has changed.

This may also solve the issue where the focus will be reset when a semantics node is swapped with another node if we caches the entire ancestor chain of a focused semantics node.

[goderbauer]

This is even more tricker that app built with native components usually performed correctly compares to custom view with virtual accessibility trees, such as Flutter application.

What is the "correct" behavior as defined by native components? And how do they achieve that behavior? Are there maybe certain events we are not sending or not sending correctly?

[chunhtai]

What is the "correct" behavior as defined by native components?

A native Android and IOS app will remember previous focused item in a page and attempt to refocus that when a user revisits that page after a pop.

And how do they achieve that behavior? Are there maybe certain events we are not sending or not sending correctly?

The iOS firing screen change event with the focus target. They may be storing the state somewhere, but i don't have access to their source code.

For android, I don't think it depends events. talkBack will focus different thing in different situations even if flutter only set the view accessibility title (it is the equivalence of screen change event )

Our current android behavior is weird. If we send a new accessibility tree when user push a new page with new the accessibility ids that talkBack never seen before, the talkBack will sometimes attempt to keep the foucs at the same location even if we only change the view accessibility title without sending other event.

[goderbauer]

A native Android and IOS app will remember previous focused item in a page and attempt to refocus that when a user revisits that page after a pop.

What mechanism is used on Android to achieve that? Can we hook into that?

I was just trying this out in the iOS settings app and there the focus also moves somewhat unpredictable when you go back one screen. Sometimes it goes back to the setting you were just coming out of. Sometimes it goes back to the title of the page.

[chunhtai]

What mechanism is used on Android to achieve that? Can we hook into that?

This only works correctly for native apps with different activities or fragments to represent pages. let me look into their source code.

I was just trying this out in the iOS settings app and there the focus also moves somewhat unpredictable when you go back one screen. Sometimes it goes back to the setting you were just coming out of. Sometimes it goes back to the title of the page.

yes, I also found the same behavior. sometimes it jump to title. maybe our behavior isn't as pressing as Android until iOS fixes it.

[luomo-pro]

I should add that in iOS settings, if you click the back button in the top left corner to return to the page, the focus will not focus exactly where it was before.
However, if you use VoiceOver's two-finger rubbing gesture to return to the page, the focus will be accurately focused on the previous position.
Here is a video of how I did it:

demo.mp4

One more thing is that in #93926, I tested that the accessibility events triggered by Android's native software and flutter when jumping between pages are different, these accessibility events I can't determine whether they are sent by the app itself or talk back, hope this helps you guys.

[daiweiqiang6688]

I agree with what @luomo-pro said in #94523.
flutter should want to better manage accessibility focus between page navigation, and should also add an API for setting accessibility focus.
These two things are not in conflict.

[chunhtai]

let's discuss the API for explicit control in the #94523. For this issue, we should focus on how or whether we should work around the android behavior.