[Android] Using AutomationId prevents TalkBack screenreader accessibility

[tomgilder]

Description

When using the AutomationId property on Android, apps end up almost completely inaccessible to users who use Android's TalkBack screenreader.

Xamarin.Forms "borrows" the ContentDescription property on Android for Automation IDs. These IDs polute Android's TalkBack accessibility tree, making apps almost impossible to navigate.

This means you can support test automation or accessibility, not both. Our app needs to support both 😃

I can't find any easy way of working around this restriction, short of walking the Android view tree and setting ContentDescription to null. Being unable to update AutomationId once set also adds to the difficulty of working around this issue.

Potential solutions

1. Use another system for setting AutomationIDs

Probably a non-starter for backwards compatibility?

2. Monitor AccessibilityManager to see if accessibility is enabled

Assuming there’s no alternative property to use for setting IDs, one solution would be for Xamarin.Forms to use Android's AccessibilityManager and check the IsTouchExplorationEnabled property.

This is true when the TalkBack screen-reader is being used, and Forms could ignore AutomationId and not set ContentDescription. Forms should then also use an ITouchExplorationStateChangeListener to monitor changes to this state.

Steps to Reproduce

1. Create any element with an AutomationId set
2. Go to that element on an Android device with TalkBack enabled

Expected Behavior

Element is ignored, or its actual content is read.

Actual Behavior

Element's AutomationId is read to the user.

Basic Information

• Version with issue: 2.5.0.121934
• Last known good version: none. Always happened since AutomationId supported.

[nexxuno]

Hello,
I'm having this problem too, this is how I workaround it while waiting for an improved xamarin version.

In xaml production code:
<Button AutomationProperties.Name="{international:BMapTranslate goLabel}" MinimumWidthRequest="150" Text="{international:BMapTranslate goLabel}" ...

In Xamarin Coded UI testing:
app.Tap(c => c.Button(tr.GetTranslation("goLabel")));

In other words I'm using the same translator class and resource files I use in production as helpers in the UItests. I know it's not optimal but it works for now...

[codingL3gend]

@tomgilder

i was able to find a work around to this issue by creating a customrenderer and respective custom component to allow for overriding the native android method(s) that get triggered when accessibility events are fired. you will need to create some bindable properties on your custom component that you can access in the custom renderer to allow for setting the content description value to what you want but that's simple enough.

this method gets triggered in the control/custom renderer whenever an accessibility event is fired

public override bool OnRequestSendAccessibilityEvent(Android.Views.View child, AccessibilityEvent e)
  {
         if (AccessibilityHandler.IsAccessibilityEnabled(_context) && child != null)
         {
                if (!string.IsNullOrEmpty(_automationId) && _automationId.Equals(child.ContentDescription))
                {
                    child.ContentDescription = $"{_automationName} {_helpText}";
                }
         }

         return base.OnRequestSendAccessibilityEvent(child, e);
}

then you can set the contentDescription value of the control/custom renderer back to what the automationId value was originally when the control/custom renderer is detached from the view.

protected override void OnDetachedFromWindow()
  {
        base.OnDetachedFromWindow();

        if (!string.IsNullOrEmpty(_automationId))
        {
            Control.ContentDescription = _automationId;
        }
  }

helper class

public static class AccessibilityHandler
    {
        public static bool IsAccessibilityEnabled(Context context)
        {
            var accessibility = (AccessibilityManager)context.GetSystemService(MainActivity.AccessibilityService);

            return accessibility?.GetEnabledAccessibilityServiceList(Android.AccessibilityServices.FeedbackFlags.Spoken)?.Count > 0;
        }
    }

[hminaya]

@jimbobbennett have you run into this before? any thoughts?

[claudiosanchez]

@brminnick and @davidortinau are aware of this.

[claudiosanchez]

@codingL3gend How many renderers did you have to create? Have you had any additional issues with the approach you suggested?

[codingL3gend]

@claudiosanchez i think your question may have been meant to be directed toward me. We already had renderers created for other reasons outside of needed ada and automation support; so I just leveraged those implementations. However for any controls where we need this support that we don't have renderers for already, we will create new ones for those. So far we haven't had any issues with this approach thus far and it is working as expected with the controls we have tested it against.

[claudiosanchez]

@codingL3gend You are correct. I fixed the mention :). We implemented your workaround and so far is working like a charm for a Button Renderer, and your answer is on point. I wanted to know what had been the experience with this workaround so far. We are going to propose this solution to the development team 💯. cc: @brminnick

[codingL3gend]

@claudiosanchez glad to hear that I could be of assistance with the workaround. currently our application is still in the development phase and have only gone through local testing so we don't know if any major issues will arise from it if any. We've also ran it against some automated testing and its performing as expected. Hopefully the implementation will help others in the future and perhaps help with an overall platform solution soon enough.

[claudiosanchez]

@codingL3gend Happy to report we have implemented a button solution based on your guidance. The hope is to submit back to the code base at some point. What other renderers did you have implement to support both Accessibility and UI Test Automation?

[codingL3gend]

@claudiosanchez that's great news to hear; happy I could help in some capacity with providing a solution. the other renderers implemented were around the default xamarin forms controls built on top of the native controls such as Label, Switch, DateEntry, LinkButton, and Entry. these are the more common controls that are being used; although custom controls have been created for ListView, ScrollView, and TableView. those renderers haven't needed any specific Accessibility or UI Test Automation configuration.

[codingL3gend]

@claudiosanchez also want to add a quick touch point. we noticed that on android when adding automationId to elements on the xamarin forms side (not sure if this is a result of our custom renderers or just how the controls would get created in general) the android resulting control would be wrapped in an additional ViewGroup. meaning that instead of say a button being wrapped in a single ViewGroup parent container and then the resulting Button element as its child, it is instead 2 Parent ViewGroup containers and then the resulting Button element as the inner most child. as a result the direct Parent ViewGroup of the Button then gets its ContentDescription value assigned by default to the child's or element's (i.e Button) AutomationId property plus _Container such as Button_Container. this in turn causes these container elements to be read when the top most parent element is clicked on a View when the Accessibility tree is triggered. even using the FastRenderers (though in beta) does not flatten the elements enough to not include the direct ViewGroup parent of the Button element. this is the next challenge that will be tackled from a workaround standpoint to get either those elements to not read the ContentDescription of Button_Container or ideally to not have the additional ViewGroup to be created at all. if there is a known workaround for this, i'd appreciate the direction.

hopefully that all made sense. I can provide a screen shot if needed.