Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

[ios][ios17]fix auto correction highlight on top left corner #44779

Merged
merged 4 commits into from
Aug 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2449,18 +2449,32 @@ - (void)takeKeyboardScreenshotAndDisplay {
}

- (void)setEditableSizeAndTransform:(NSDictionary*)dictionary {
[_activeView setEditableTransform:dictionary[@"transform"]];
NSArray* transform = dictionary[@"transform"];
[_activeView setEditableTransform:transform];
const int leftIndex = 12;
const int topIndex = 13;
if ([_activeView isScribbleAvailable]) {
// This is necessary to set up where the scribble interactable element will be.
int leftIndex = 12;
int topIndex = 13;
_inputHider.frame =
CGRectMake([dictionary[@"transform"][leftIndex] intValue],
[dictionary[@"transform"][topIndex] intValue], [dictionary[@"width"] intValue],
[dictionary[@"height"] intValue]);
CGRectMake([transform[leftIndex] intValue], [transform[topIndex] intValue],
[dictionary[@"width"] intValue], [dictionary[@"height"] intValue]);
_activeView.frame =
CGRectMake(0, 0, [dictionary[@"width"] intValue], [dictionary[@"height"] intValue]);
_activeView.tintColor = [UIColor clearColor];
} else {
// TODO(hellohuanlin): Also need to handle iOS 16 case, where the auto-correction highlight does
// not match the size of text.
// See https://github.com/flutter/flutter/issues/131695
if (@available(iOS 17, *)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a TODO to simplify all of this logic. It looks like post cherry-pick we can probably just make the _inputHider.frame logic unconditional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The iOS 16 behavior is different - the highlight region does not match the size of text (see screenshot in flutter/flutter#131695 for iOS 16). But will add a TODO to explain this.

// Move auto-correction highlight to overlap with the actual text.
// This is to fix an issue where the system auto-correction highlight is displayed at
// the top left corner of the screen on iOS 17+.
// This problem also happens on iOS 16, but the size of highlight does not match the text.
// See https://github.com/flutter/flutter/issues/131695
// TODO(hellohuanlin): Investigate if we can use non-zero size.
_inputHider.frame =
CGRectMake([transform[leftIndex] intValue], [transform[topIndex] intValue], 0, 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if you pass the real width and height here like in the scribble case above instead of 0,0?
[dictionary[@"width"] intValue], [dictionary[@"height"] intValue]

Copy link
Contributor Author

@hellohuanlin hellohuanlin Aug 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried that and got the same result in my tests. Nothing seems to break. But wanted to be safe and keep the same zero size as before.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This is one of the things I was thinking would be part of unifying the these two codepaths in a follow-up, master-only PR.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a todo there.

}
}
}

Expand Down Expand Up @@ -2488,7 +2502,22 @@ - (void)setSelectionRects:(NSArray*)encodedRects {
? NSWritingDirectionLeftToRight
: NSWritingDirectionRightToLeft]];
}

BOOL shouldNotifyTextChange = NO;
if (@available(iOS 17, *)) {
// Force UIKit to query the selectionRects again on iOS 17+
// This is to fix a bug on iOS 17+ where UIKit queries the outdated selectionRects after
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to above, I think this bug exists on all versions of iOS, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in this case - in iOS 16, we do not use system highlight, so no such bug.

I also tried to notify the change on iOS 16 and nothing seems to break, but just to be safe here.

// entering a character, resulting in auto-correction highlight region missing the last
// character.
shouldNotifyTextChange = YES;
}
if (shouldNotifyTextChange) {
[_activeView.inputDelegate textWillChange:_activeView];
}
_activeView.selectionRects = rectsAsRect;
if (shouldNotifyTextChange) {
[_activeView.inputDelegate textDidChange:_activeView];
}
}

- (void)startLiveTextInput {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ @interface FlutterSecureTextInputView : FlutterTextInputView

@interface FlutterTextInputPlugin ()
@property(nonatomic, assign) FlutterTextInputView* activeView;
@property(nonatomic, readonly) UIView* inputHider;
@property(nonatomic, readonly) UIView* keyboardViewContainer;
@property(nonatomic, readonly) UIView* keyboardView;
@property(nonatomic, assign) UIView* cachedFirstResponder;
Expand Down Expand Up @@ -422,6 +423,72 @@ - (void)testAutocorrectionPromptRectDoesNotAppearDuringScribble {
}
}

- (void)testInputHiderOverlapWithTextWhenScribbleIsDisabledAfterIOS17AndDoesNotOverlapBeforeIOS17 {
FlutterTextInputPlugin* myInputPlugin =
[[FlutterTextInputPlugin alloc] initWithDelegate:OCMClassMock([FlutterEngine class])];

FlutterMethodCall* setClientCall =
[FlutterMethodCall methodCallWithMethodName:@"TextInput.setClient"
arguments:@[ @(123), self.mutableTemplateCopy ]];
[myInputPlugin handleMethodCall:setClientCall
result:^(id _Nullable result){
}];

FlutterTextInputView* mockInputView = OCMPartialMock(myInputPlugin.activeView);
OCMStub([mockInputView isScribbleAvailable]).andReturn(NO);

// yOffset = 200.
NSArray* yOffsetMatrix = @[ @1, @0, @0, @0, @0, @1, @0, @0, @0, @0, @1, @0, @0, @200, @0, @1 ];

FlutterMethodCall* setPlatformViewClientCall =
[FlutterMethodCall methodCallWithMethodName:@"TextInput.setEditableSizeAndTransform"
arguments:@{@"transform" : yOffsetMatrix}];
[myInputPlugin handleMethodCall:setPlatformViewClientCall
result:^(id _Nullable result){
}];

if (@available(iOS 17, *)) {
XCTAssert(CGRectEqualToRect(myInputPlugin.inputHider.frame, CGRectMake(0, 200, 0, 0)),
@"The input hider should overlap with the text on and after iOS 17");

} else {
XCTAssert(CGRectEqualToRect(myInputPlugin.inputHider.frame, CGRectZero),
@"The input hider should be on the origin of screen on and before iOS 16.");
}
}

- (void)testSetSelectionRectsNotifiesTextChangeAfterIOS17AndDoesNotNotifyBeforeIOS17 {
FlutterTextInputPlugin* myInputPlugin =
[[FlutterTextInputPlugin alloc] initWithDelegate:OCMClassMock([FlutterEngine class])];

FlutterMethodCall* setClientCall =
[FlutterMethodCall methodCallWithMethodName:@"TextInput.setClient"
arguments:@[ @(123), self.mutableTemplateCopy ]];
[myInputPlugin handleMethodCall:setClientCall
result:^(id _Nullable result){
}];

id mockInputDelegate = OCMProtocolMock(@protocol(UITextInputDelegate));
myInputPlugin.activeView.inputDelegate = mockInputDelegate;

NSArray<NSNumber*>* selectionRect = [NSArray arrayWithObjects:@0, @0, @100, @100, @0, @1, nil];
NSArray* selectionRects = [NSArray arrayWithObjects:selectionRect, nil];
FlutterMethodCall* methodCall =
[FlutterMethodCall methodCallWithMethodName:@"Scribble.setSelectionRects"
arguments:selectionRects];
[myInputPlugin handleMethodCall:methodCall
result:^(id _Nullable result){
}];

if (@available(iOS 17.0, *)) {
OCMVerify([mockInputDelegate textWillChange:myInputPlugin.activeView]);
OCMVerify([mockInputDelegate textDidChange:myInputPlugin.activeView]);
} else {
OCMVerify(never(), [mockInputDelegate textWillChange:myInputPlugin.activeView]);
OCMVerify(never(), [mockInputDelegate textDidChange:myInputPlugin.activeView]);
}
}

- (void)testTextRangeFromPositionMatchesUITextViewBehavior {
FlutterTextInputView* inputView = [[FlutterTextInputView alloc] initWithOwner:textInputPlugin];
FlutterTextPosition* fromPosition = [FlutterTextPosition positionWithIndex:2];
Expand Down