Skip to content

Commit

Permalink
TalkBack support for ScrollView accessibility announcements (list and…
Browse files Browse the repository at this point in the history 
… grid) - Javascript Only Changes (#33180)

Summary:
This is the Javascript-only changes from D34518929 (dd6325b), split out for push safety. Original summary and test plan below:

This issue fixes [30977][17] . The Pull Request was previously published by [intergalacticspacehighway][13] with [31666][19].
The solution consists of:
1. Adding Javascript logic in the [FlatList][14], SectionList, VirtualizedList components to provide accessibility information (row and column position) for each cell in the method [renderItem][20] as a fourth parameter [accessibilityCollectionItem][21]. The information is saved on the native side in the AccessibilityNodeInfo and announced by TalkBack when changing row, column, or page ([video example][12]). The prop accessibilityCollectionItem is available in the View component which wraps each FlatList cell.
2. Adding Java logic in [ReactScrollView.java][16] and HorizontalScrollView to announce pages with TalkBack when scrolling up/down. The missing AOSP logic in [ScrollView.java][10] (see also the [GridView][11] example) is responsible for announcing Page Scrolling with TalkBack.

Relevant Links:
x [Additional notes on this PR][18]
x [discussion on the additional container View around each FlatList cell][22]
x [commit adding prop getCellsInItemCount to VirtualizedList][23]

## Changelog

[Android] [Added] - Accessibility announcement for list and grid in FlatList

Pull Request resolved: #33180

Test Plan:
[1]. TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer ([link][1])
[2]. TalkBack announces pages and cells with Vertical Flatlist in the Paper Renderer ([link][2])
[3]. `FlatList numColumns={undefined}` Should not trigger Runtime Error NoSuchKey exception columnCount when enabling TalkBack. ([link][3])
[4]. TalkBack announces pages and cells with Nested Horizontal Flatlist in the rn-tester app ([link][4])

[1]: fabOnReact/react-native-notes#6 (comment)
[2]: fabOnReact/react-native-notes#6 (comment)
[3]: fabOnReact/react-native-notes#6 (comment)
[4]: fabOnReact/react-native-notes#6 (comment)
[10]:https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/AdapterView.java#L1027-L1029 "GridView.java method responsible for calling setFromIndex and setToIndex"
[11]:fabOnReact/react-native-notes#6 (comment) "test case on Android GridView"
[12]:fabOnReact/react-native-notes#6 (comment) "TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer"
[13]:https://github.com/intergalacticspacehighway "github intergalacticspacehighway"
[14]:https://github.com/fabriziobertoglio1987/react-native/blob/80acf523a4410adac8005d5c9472fb87f78e12ee/Libraries/Lists/FlatList.js#L617-L636 "FlatList accessibilityCollectionItem"
[16]:https://github.com/fabriziobertoglio1987/react-native/blob/5706bd7d3ee35dca48f85322a2bdcaec0bce2c85/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L183-L184 "logic added to ReactScrollView.java"
[17]: #30977
[18]: fabOnReact/react-native-notes#6
[19]: #31666
[20]: https://reactnative.dev/docs/next/flatlist#required-renderitem "FlatList renderItem documentation"
[21]: fabOnReact@7514735 "commit that introduces fourth param accessibilityCollectionItem in callback renderItem"
[22]: #33180 (comment) "discussion on the additional container View around each FlatList cell"
[23]: fabOnReact@d50fd1a "commit adding prop getCellsInItemCount to VirtualizedList"

Reviewed By: kacieb

Differential Revision: D37189197

Pulled By: blavalla

fbshipit-source-id: 3765213c5d8bfde56e0e5f155cdd899c368512e7
  • Loading branch information
@fabOnReact @facebook-github-bot
fabOnReact authored and facebook-github-bot committed Jun 18, 2022
1 parent 71da212 commit 2d58821
Show file tree
Hide file tree
Showing 14 changed files with 824 additions and 31 deletions.
1 change: 1 addition & 0 deletions Libraries/Components/View/ViewAccessibility.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ export type AccessibilityRole =
| 'tablist'
| 'timer'
| 'list'
| 'grid'
| 'toolbar';

// the info associated with an accessibility action
Expand Down
17 changes: 17 additions & 0 deletions Libraries/Components/View/ViewPropTypes.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,23 @@ export type ViewProps = $ReadOnly<{|
*/
accessibilityActions?: ?$ReadOnlyArray<AccessibilityActionInfo>,

/**
*
* Node Information of a FlatList, VirtualizedList or SectionList collection item.
* A collection item starts at a given row and column in the collection, and spans one or more rows and columns.
*
* @platform android
*
*/
accessibilityCollectionItem?: ?{
rowIndex: number,
rowSpan: number,
columnIndex: number,
columnSpan: number,
heading: boolean,
itemIndex: number,
},

/**
* Specifies the nativeID of the associated label text. When the assistive technology focuses on the component with this props, the text is read aloud.
*
Expand Down
10 changes: 9 additions & 1 deletion Libraries/Lists/FlatList.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -624,10 +624,17 @@ class FlatList<ItemT> extends React.PureComponent<Props<ItemT>, void> {
return (
<View style={StyleSheet.compose(styles.row, columnWrapperStyle)}>
{item.map((it, kk) => {
const itemIndex = index * cols + kk;
const accessibilityCollectionItem = {
...info.accessibilityCollectionItem,
columnIndex: itemIndex % cols,
itemIndex: itemIndex,
};
const element = renderer({
item: it,
index: index * cols + kk,
index: itemIndex,
separators: info.separators,
accessibilityCollectionItem,
});
return element != null ? (
<React.Fragment key={kk}>{element}</React.Fragment>
Expand Down Expand Up @@ -658,6 +665,7 @@ class FlatList<ItemT> extends React.PureComponent<Props<ItemT>, void> {
return (
<VirtualizedList
{...restProps}
numColumns={numColumns}
getItem={this._getItem}
getItemCount={this._getItemCount}
keyExtractor={this._keyExtractor}
Expand Down
73 changes: 70 additions & 3 deletions Libraries/Lists/VirtualizedList.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ const ScrollView = require('../Components/ScrollView/ScrollView');
const View = require('../Components/View/View');
const Batchinator = require('../Interaction/Batchinator');
const ReactNative = require('../Renderer/shims/ReactNative');
const Platform = require('../Utilities/Platform');
const flattenStyle = require('../StyleSheet/flattenStyle');
const StyleSheet = require('../StyleSheet/StyleSheet');
const infoLog = require('../Utilities/infoLog');
Expand All @@ -53,10 +54,20 @@ export type Separators = {
...
};

export type AccessibilityCollectionItem = {
itemIndex: number,
rowIndex: number,
rowSpan: number,
columnIndex: number,
columnSpan: number,
heading: boolean,
};

export type RenderItemProps<ItemT> = {
item: ItemT,
index: number,
separators: Separators,
accessibilityCollectionItem: AccessibilityCollectionItem,
...
};

Expand Down Expand Up @@ -85,9 +96,19 @@ type RequiredProps = {|
*/
getItem: (data: any, index: number) => ?Item,
/**
* Determines how many items are in the data blob.
* Determines how many items (rows) are in the data blob.
*/
getItemCount: (data: any) => number,
/**
* Determines how many cells are in the data blob
* see https://bit.ly/35RKX7H
*/
getCellsInItemCount?: (data: any) => number,
/**
* The number of columns used in FlatList.
* The default of 1 is used in other components to calculate the accessibilityCollection prop.
*/
numColumns?: ?number,
|};
type OptionalProps = {|
renderItem?: ?RenderItemType<Item>,
Expand Down Expand Up @@ -308,6 +329,10 @@ type Props = {|
...OptionalProps,
|};

function numColumnsOrDefault(numColumns: ?number) {
return numColumns ?? 1;
}

let _usedIndexForKey = false;
let _keylessItemComponentName: string = '';

Expand Down Expand Up @@ -1253,8 +1278,33 @@ class VirtualizedList extends React.PureComponent<Props, State> {
);
}

_getCellsInItemCount = props => {
const {getCellsInItemCount, data} = props;
if (getCellsInItemCount) {
return getCellsInItemCount(data);
}
if (Array.isArray(data)) {
return data.length;
}
return 0;
};

_defaultRenderScrollComponent = props => {
const {getItemCount, data} = props;
const onRefresh = props.onRefresh;
const numColumns = numColumnsOrDefault(props.numColumns);
const accessibilityRole = Platform.select({
android: numColumns > 1 ? 'grid' : 'list',
});
const rowCount = getItemCount(data);
const accessibilityCollection = {
// over-ride _getCellsInItemCount to handle Objects or other data formats
// see https://bit.ly/35RKX7H
itemCount: this._getCellsInItemCount(props),
rowCount,
columnCount: numColumns,
hierarchical: false,
};
if (this._isNestedWithSameOrientation()) {
// $FlowFixMe[prop-missing] - Typing ReactNativeComponent revealed errors
return <View {...props} />;
Expand All @@ -1269,6 +1319,8 @@ class VirtualizedList extends React.PureComponent<Props, State> {
// $FlowFixMe[prop-missing] Invalid prop usage
<ScrollView
{...props}
accessibilityRole={accessibilityRole}
accessibilityCollection={accessibilityCollection}
refreshControl={
props.refreshControl == null ? (
<RefreshControl
Expand All @@ -1284,8 +1336,14 @@ class VirtualizedList extends React.PureComponent<Props, State> {
/>
);
} else {
// $FlowFixMe[prop-missing] Invalid prop usage
return <ScrollView {...props} />;
return (
// $FlowFixMe[prop-missing] Invalid prop usage
<ScrollView
{...props}
accessibilityRole={accessibilityRole}
accessibilityCollection={accessibilityCollection}
/>
);
}
};

Expand Down Expand Up @@ -2056,10 +2114,19 @@ class CellRenderer extends React.Component<
}

if (renderItem) {
const accessibilityCollectionItem = {
itemIndex: index,
rowIndex: index,
rowSpan: 1,
columnIndex: 0,
columnSpan: 1,
heading: false,
};
return renderItem({
item,
index,
separators: this._separators,
accessibilityCollectionItem,
});
}

Expand Down
17 changes: 15 additions & 2 deletions Libraries/Lists/VirtualizedSectionList.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
*/

import type {ViewToken} from './ViewabilityHelper';

import type {AccessibilityCollectionItem} from './VirtualizedList';
import {keyExtractor as defaultKeyExtractor} from './VirtualizeUtils';
import invariant from 'invariant';
import * as React from 'react';
Expand Down Expand Up @@ -341,7 +341,16 @@ class VirtualizedSectionList<
_renderItem =
(listItemCount: number) =>
// eslint-disable-next-line react/no-unstable-nested-components
({item, index}: {item: Item, index: number, ...}) => {
({
item,
index,
accessibilityCollectionItem,
}: {
item: Item,
index: number,
accessibilityCollectionItem: AccessibilityCollectionItem,
...
}) => {
const info = this._subExtractor(index);
if (!info) {
return null;
Expand Down Expand Up @@ -370,6 +379,7 @@ class VirtualizedSectionList<
LeadingSeparatorComponent={
infoIndex === 0 ? this.props.SectionSeparatorComponent : undefined
}
accessibilityCollectionItem={accessibilityCollectionItem}
cellKey={info.key}
index={infoIndex}
item={item}
Expand Down Expand Up @@ -482,6 +492,7 @@ type ItemWithSeparatorProps = $ReadOnly<{|
updatePropsFor: (prevCellKey: string, value: Object) => void,
renderItem: Function,
inverted: boolean,
accessibilityCollectionItem: AccessibilityCollectionItem,
|}>;

function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
Expand All @@ -499,6 +510,7 @@ function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
index,
section,
inverted,
accessibilityCollectionItem,
} = props;

const [leadingSeparatorHiglighted, setLeadingSeparatorHighlighted] =
Expand Down Expand Up @@ -572,6 +584,7 @@ function ItemWithSeparator(props: ItemWithSeparatorProps): React.Node {
index,
section,
separators,
accessibilityCollectionItem,
});
const leadingSeparator = LeadingSeparatorComponent != null && (
<LeadingSeparatorComponent
Expand Down
59 changes: 59 additions & 0 deletions Libraries/Lists/__tests__/__snapshots__/FlatList-test.js.snap
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@ exports[`FlatList renders all the bells and whistles 1`] = `
ListEmptyComponent={[Function]}
ListFooterComponent={[Function]}
ListHeaderComponent={[Function]}
accessibilityCollection={
Object {
"columnCount": 2,
"hierarchical": false,
"itemCount": 5,
"rowCount": 3,
}
}
data={
Array [
Object {
Expand All @@ -29,6 +37,7 @@ exports[`FlatList renders all the bells and whistles 1`] = `
getItemCount={[Function]}
getItemLayout={[Function]}
keyExtractor={[Function]}
numColumns={2}
onContentSizeChange={[Function]}
onLayout={[Function]}
onMomentumScrollBegin={[Function]}
Expand Down Expand Up @@ -121,6 +130,14 @@ exports[`FlatList renders all the bells and whistles 1`] = `

exports[`FlatList renders empty list 1`] = `
<RCTScrollView
accessibilityCollection={
Object {
"columnCount": 1,
"hierarchical": false,
"itemCount": 0,
"rowCount": 0,
}
}
data={Array []}
getItem={[Function]}
getItemCount={[Function]}
Expand All @@ -144,6 +161,14 @@ exports[`FlatList renders empty list 1`] = `

exports[`FlatList renders null list 1`] = `
<RCTScrollView
accessibilityCollection={
Object {
"columnCount": 1,
"hierarchical": false,
"itemCount": 0,
"rowCount": 0,
}
}
getItem={[Function]}
getItemCount={[Function]}
keyExtractor={[Function]}
Expand All @@ -166,6 +191,14 @@ exports[`FlatList renders null list 1`] = `

exports[`FlatList renders simple list (multiple columns) 1`] = `
<RCTScrollView
accessibilityCollection={
Object {
"columnCount": 2,
"hierarchical": false,
"itemCount": 3,
"rowCount": 2,
}
}
data={
Array [
Object {
Expand All @@ -182,6 +215,7 @@ exports[`FlatList renders simple list (multiple columns) 1`] = `
getItem={[Function]}
getItemCount={[Function]}
keyExtractor={[Function]}
numColumns={2}
onContentSizeChange={[Function]}
onLayout={[Function]}
onMomentumScrollBegin={[Function]}
Expand Down Expand Up @@ -237,6 +271,14 @@ exports[`FlatList renders simple list (multiple columns) 1`] = `

exports[`FlatList renders simple list 1`] = `
<RCTScrollView
accessibilityCollection={
Object {
"columnCount": 1,
"hierarchical": false,
"itemCount": 3,
"rowCount": 3,
}
}
data={
Array [
Object {
Expand Down Expand Up @@ -298,6 +340,14 @@ exports[`FlatList renders simple list 1`] = `
exports[`FlatList renders simple list using ListItemComponent (multiple columns) 1`] = `
<RCTScrollView
ListItemComponent={[Function]}
accessibilityCollection={
Object {
"columnCount": 2,
"hierarchical": false,
"itemCount": 3,
"rowCount": 2,
}
}
data={
Array [
Object {
Expand All @@ -314,6 +364,7 @@ exports[`FlatList renders simple list using ListItemComponent (multiple columns)
getItem={[Function]}
getItemCount={[Function]}
keyExtractor={[Function]}
numColumns={2}
onContentSizeChange={[Function]}
onLayout={[Function]}
onMomentumScrollBegin={[Function]}
Expand Down Expand Up @@ -369,6 +420,14 @@ exports[`FlatList renders simple list using ListItemComponent (multiple columns)
exports[`FlatList renders simple list using ListItemComponent 1`] = `
<RCTScrollView
ListItemComponent={[Function]}
accessibilityCollection={
Object {
"columnCount": 1,
"hierarchical": false,
"itemCount": 3,
"rowCount": 3,
}
}
data={
Array [
Object {
Expand Down
Loading

0 comments on commit 2d58821

Please sign in to comment.