Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

[iOS] Supported rendering platform views without merging the raster thread. #53826

Merged
merged 55 commits into from
Aug 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
02556ae
++
jonahwilliams Jul 11, 2024
b3f6670
move all state updates to end of compositing.
jonahwilliams Jul 11, 2024
b31aae6
working but unsynchronized.
jonahwilliams Jul 11, 2024
75f931a
hey it almost works.
jonahwilliams Jul 12, 2024
4525bd8
disable partial repaint.
jonahwilliams Jul 12, 2024
6ce3aa0
cleanups.
jonahwilliams Jul 12, 2024
be03cf8
set presentsWithTransaction.
jonahwilliams Jul 12, 2024
6770376
sanity refactor.
jonahwilliams Jul 12, 2024
912cd40
linting.
jonahwilliams Jul 12, 2024
1f65cc3
skip post message if there are no platform views.
jonahwilliams Jul 12, 2024
53adc81
linting.
jonahwilliams Jul 12, 2024
870b950
dispose views in callback.
jonahwilliams Jul 15, 2024
f8267bd
++
jonahwilliams Jul 15, 2024
80565d5
Merge branch 'main' of github.com:flutter/engine into three_phase_render
jonahwilliams Jul 16, 2024
2b9a825
tear down.
jonahwilliams Jul 16, 2024
0a7dc5a
reserve.
jonahwilliams Jul 17, 2024
2745e88
++
jonahwilliams Jul 17, 2024
6002aa0
++
jonahwilliams Jul 17, 2024
e7a832f
++
jonahwilliams Jul 17, 2024
f099d74
come on big money no whammies.
jonahwilliams Jul 17, 2024
e2e6a2a
okay one more.
jonahwilliams Jul 17, 2024
44d5b5b
debug printf
jonahwilliams Jul 17, 2024
b53063a
++
jonahwilliams Jul 19, 2024
9a84539
make test work.
jonahwilliams Jul 23, 2024
65a5210
++
jonahwilliams Jul 23, 2024
735ce22
testing fixes.
jonahwilliams Jul 24, 2024
a4f523b
Merge branch 'main' of github.com:flutter/engine into three_phase_render
jonahwilliams Jul 24, 2024
b6c3345
merge in view slicer.
jonahwilliams Jul 24, 2024
a4b771e
Merge branch 'main' of github.com:flutter/engine into three_phase_render
jonahwilliams Jul 24, 2024
cd8e1de
++
jonahwilliams Jul 25, 2024
6603d18
minor cleanups.
jonahwilliams Jul 25, 2024
27c0daa
formatting.
jonahwilliams Jul 25, 2024
d794211
add back clear
jonahwilliams Jul 26, 2024
2f4314b
++
jonahwilliams Jul 26, 2024
25a4e06
++
jonahwilliams Jul 26, 2024
ea2e66d
merge threads on SIM.
jonahwilliams Jul 26, 2024
d220885
++
jonahwilliams Jul 26, 2024
9b534cd
Update shell/platform/darwin/ios/ios_external_view_embedder.mm
jonahwilliams Jul 26, 2024
970e13a
++
jonahwilliams Jul 26, 2024
63ebe66
Merge branch 'three_phase_render' of github.com:jonahwilliams/engine …
jonahwilliams Jul 26, 2024
ff4802a
Merge branch 'main' of github.com:flutter/engine into three_phase_render
jonahwilliams Jul 26, 2024
e3fc731
does it blend?
jonahwilliams Jul 26, 2024
c0fab3b
++
jonahwilliams Jul 26, 2024
d73671e
test fixes and clang tidy.
jonahwilliams Jul 27, 2024
7285907
simplify and clean up thread access for UIViews.
jonahwilliams Jul 27, 2024
bcd8519
reset before clearning composition order.
jonahwilliams Jul 27, 2024
b25a742
Merge branch 'main' of github.com:flutter/engine into three_phase_render
jonahwilliams Jul 30, 2024
5f6a8d4
switch to SurfaceFrame API.
jonahwilliams Jul 30, 2024
be7853d
++
jonahwilliams Jul 30, 2024
b19b4d8
Remove extra include.
jonahwilliams Jul 30, 2024
ab08739
clang tidy
jonahwilliams Jul 30, 2024
6e6e5a2
bracken feedback
jonahwilliams Jul 30, 2024
355ab30
more bracken review.
jonahwilliams Aug 1, 2024
8cd1c74
++
jonahwilliams Aug 1, 2024
30e7b97
review comments and remove dead code.
jonahwilliams Aug 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions flow/surface_frame.h
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,19 @@ class SurfaceFrame {
//
// Defaults to true, which is generally a safe value.
bool frame_boundary = true;

// Whether this surface presents with a CATransaction on Apple platforms.
//
// When there are platform views in the scene, the drawable needs to be
// presented in the same CATransaction as the one created for platform view
// mutations.
//
// If the drawables are being presented from the raster thread, we cannot
// use a transaction as it will dirty the UIViews being presented. If there
// is a non-Flutter UIView active, such as in add2app or a
// presentViewController page transition, then this will cause CoreAnimation
// assertion errors and exit the app.
bool present_with_transaction = false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is necessary for things to correctly work with FlutterMetalLayer disabled?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FlutterMetalLayer is always presenting with a CATransaction, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it always makes the jump to platform thread (can't set CALayer content on background thread).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I ported this over from the existing logic, but once we remove the old metal layer usage we could remove this as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

};

bool Encode();
Expand Down
8 changes: 8 additions & 0 deletions impeller/renderer/backend/metal/surface_mtl.h
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,13 @@ class SurfaceMTL final : public Surface {
// Returns a Rect defining the area of the surface in device pixels
IRect coverage() const;

/// Mark this surface as presenting with a transaction.
///
/// If true, [Present] will block on the scheduling of a command buffer.
void PresentWithTransaction(bool present_with_transaction) {
present_with_transaction_ = present_with_transaction;
}

// |Surface|
bool Present() const override;

Expand All @@ -74,6 +81,7 @@ class SurfaceMTL final : public Surface {
bool requires_blit_ = false;
std::optional<IRect> clip_rect_;
bool frame_boundary_ = false;
bool present_with_transaction_ = false;

static bool ShouldPerformPartialRepaint(std::optional<IRect> damage_rect);

Expand Down
2 changes: 1 addition & 1 deletion impeller/renderer/backend/metal/surface_mtl.mm
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ - (void)flutterPrepareForPresent:(nonnull id<MTLCommandBuffer>)commandBuffer;
// If the threads have been merged, or there is a pending frame capture,
// then block on cmd buffer scheduling to ensure that the
// transaction/capture work correctly.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above comment isn't adding much and rather than making it even longer to cover the new case, we could probably just yank out a local and the code would do the exact same amount of documenting as the comment currently does:

BOOL threads_merged = [[NSThread currentThread] isMainThread];

Or maybe extract the whole blobby conditional to a bool named transactions_enabled or something.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I can move this to the surface_frame doc comment?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

if ([[NSThread currentThread] isMainThread] ||
if (present_with_transaction_ || [[NSThread currentThread] isMainThread] ||
[[MTLCaptureManager sharedCaptureManager] isCapturing] ||
alwaysWaitForScheduling) {
TRACE_EVENT0("flutter", "waitUntilScheduled");
Expand Down
2 changes: 2 additions & 0 deletions shell/gpu/gpu_surface_metal_delegate.h
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ class GPUSurfaceMetalDelegate {
///
virtual bool PresentDrawable(GrMTLHandle drawable) const = 0;

virtual bool PreparePresent(GrMTLHandle drawable) const { return true; }

//------------------------------------------------------------------------------
/// @brief Returns the handle to the MTLTexture to render to. This is only
/// called when the specified render target type is `kMTLTexture`.
Expand Down
8 changes: 7 additions & 1 deletion shell/gpu/gpu_surface_metal_impeller.mm
Original file line number Diff line number Diff line change
Expand Up @@ -109,8 +109,11 @@
disable_partial_repaint = disable_partial_repaint_, //
aiks_context = aiks_context_, //
drawable, //
last_texture //
last_texture, //
mtl_layer //
](SurfaceFrame& surface_frame, DlCanvas* canvas) mutable -> bool {
mtl_layer.presentsWithTransaction = surface_frame.submit_info().present_with_transaction;

if (!aiks_context) {
return false;
}
Expand Down Expand Up @@ -151,6 +154,7 @@
if (!surface) {
return false;
}
surface->PresentWithTransaction(surface_frame.submit_info().present_with_transaction);

if (clip_rect && clip_rect->IsEmpty()) {
surface_frame.set_user_data(std::move(surface));
Expand Down Expand Up @@ -271,6 +275,8 @@
auto surface = impeller::SurfaceMTL::MakeFromTexture(aiks_context->GetContext(),
mtl_texture, clip_rect);

surface->PresentWithTransaction(surface_frame.submit_info().present_with_transaction);

if (clip_rect && clip_rect->IsEmpty()) {
return surface->Present();
}
Expand Down
8 changes: 5 additions & 3 deletions shell/gpu/gpu_surface_metal_skia.mm
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,9 @@
return nullptr;
}

auto encode_callback = [this, drawable](const SurfaceFrame& surface_frame,
DlCanvas* canvas) -> bool {
SurfaceFrame::EncodeCallback encode_callback =
[this, drawable, mtl_layer](const SurfaceFrame& surface_frame, DlCanvas* canvas) -> bool {
mtl_layer.presentsWithTransaction = surface_frame.submit_info().present_with_transaction;
if (canvas == nullptr) {
FML_DLOG(ERROR) << "Canvas not available.";
return false;
Expand Down Expand Up @@ -186,7 +187,8 @@
return true;
};

auto submit_callback = [this, drawable](const SurfaceFrame& surface_frame) -> bool {
SurfaceFrame::SubmitCallback submit_callback =
[this, drawable](const SurfaceFrame& surface_frame) -> bool {
TRACE_EVENT0("flutter", "GPUSurfaceMetal::Submit");
return delegate_->PresentDrawable(drawable);
};
Expand Down
5 changes: 4 additions & 1 deletion shell/platform/darwin/ios/framework/Source/FlutterEngine.mm
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,6 @@ - (instancetype)initWithName:(NSString*)labelPrefix
_pluginPublications = [[NSMutableDictionary alloc] init];
_registrars = [[NSMutableDictionary alloc] init];
[self recreatePlatformViewController];

_binaryMessenger = [[FlutterBinaryMessengerRelay alloc] initWithParent:self];
_textureRegistry = [[FlutterTextureRegistryRelay alloc] initWithParent:self];
_connections.reset(new flutter::ConnectionCollection());
Expand Down Expand Up @@ -869,6 +868,8 @@ - (BOOL)createShell:(NSString*)entrypoint
flutter::Shell::CreateCallback<flutter::PlatformView> on_create_platform_view =
[self](flutter::Shell& shell) {
[self recreatePlatformViewController];
self->_platformViewsController->SetTaskRunner(
shell.GetTaskRunners().GetPlatformTaskRunner());
return std::make_unique<flutter::PlatformViewIOS>(
shell, self->_renderingApi, self->_platformViewsController, shell.GetTaskRunners(),
shell.GetConcurrentWorkerTaskRunner(), shell.GetIsGpuDisabledSyncSwitch());
Expand Down Expand Up @@ -1468,6 +1469,8 @@ - (FlutterEngine*)spawnWithEntrypoint:(/*nullable*/ NSString*)entrypoint
flutter::Shell::CreateCallback<flutter::PlatformView> on_create_platform_view =
[result, context](flutter::Shell& shell) {
[result recreatePlatformViewController];
result->_platformViewsController->SetTaskRunner(
shell.GetTaskRunners().GetPlatformTaskRunner());
return std::make_unique<flutter::PlatformViewIOS>(
shell, context, result->_platformViewsController, shell.GetTaskRunners());
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@
FLUTTER_ASSERT_ARC

// This is mostly a duplication of FlutterView.
// TODO(amirh): once GL support is in evaluate if we can merge this with FlutterView.
@implementation FlutterOverlayView {
fml::CFRef<CGColorSpaceRef> _colorSpaceRef;
}
Expand Down
Loading