Skip to content

[Android] Add a way to request new Surfaces from SurfaceProducer and avoid SurfaceProducer returning invalid Surface #169899

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jun 20, 2025

Conversation

camsim99
Copy link
Contributor

@camsim99 camsim99 commented Jun 2, 2025

Note

For anyone reviewing this PR, see flutter/packages#9360 for how I'd use the getSurface(boolean forceNewSurface) method.

Note

For anyone coming across this PR post-landing, in the code review process, I renamed getSurface(boolean forceNewSurface) to getForcedNewSurface().

What this does

(1) Adds method getSurface(boolean forceNewSurface) to SurfaceProducer that will force the creation of a new Surface when SurfaceProducer.getSurface() is called.
(2) Fixes SurfaceProducer to avoid returning invalid Surfaces when SurfaceProducer.getSurface()/getSurface(boolean forceNewSurface) is called.

Why we should...

(1) Add getSurface(boolean forceNewSurface)

My motivation for adding this is directly tied to #155294. The camera_android_camerax plugin supports a camera preview use case that requires providing a Surface to in order to render the preview. It does so via SurfaceProducer; we provide a Surface retrieved from SurfaceProducer.getSurface() to the CameraX library to render a camera preview, and when the camera preview is done, a callback that we provide is called, reporting that Surface as used and that it now should be released/invalidated (see SurfaceRequest, SurfaceRequest.Result). However, the CameraX library makes no guarantees about when requests for new Surfaces are made, so the following race condition can happen:

  1. CameraX requests a Surface
  2. We provide Surface A from SurfaceProducer.getSurface()
  3. The camera preview was paused; CameraX requests a new Surface
  4. The camera preview is resumed
  5. We provide Surface A from SurfaceProducer.getSurface() because it is still technically valid
  6. CameraX calls our callback and now Surface A is released/invalid

I believe SurfaceProducer currently has no way to handle the level of complexity of Surface handling required for use cases like the camera_android_camerax camera preview, where a Surface may be invalidated between calls to SurfaceProducer.getSurface(). So, getSurface(boolean forceNewSurface) can support these use cases.*

getSurface(boolean forceNewSurface) simply will force SurfaceProducer to return a new Surface when SurfaceProducer.getSurface is called versus potentially returning the same Surface as the previous invocation.

For camera_android_camerax, see flutter/packages#9360 for an example of how it would be used.

*I'd like to note that I also played around with creating new SurfaceProducers to solve the camera problem, which was successful and probably could be generally for that use case, but I think it's cleaner to support this functionality from within the same SurfaceProducer since we can 🤷‍♀️ It also helps avoid the user from needing to create/manage multiple Flutter Textures, from my understanding.

(2) Fix SurfaceProducer.getSurface()/getSurface(boolean forceNewSurface) to never return an invalid Surfaces

I noticed that SurfaceProducer.getSurface()/getSurface(boolean forceNewSurface) does not check for Surface validity before returning them while I was working on (1). Honestly, this just feels right? We should ensure that a Surface is valid and can be used before returning it and potentially confusing the user.

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@github-actions github-actions bot added platform-android Android applications specifically engine flutter/engine repository. See also e: labels. labels Jun 2, 2025
Copy link
Member

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jmagman

This comment was marked as resolved.

@camsim99

This comment was marked as outdated.

@camsim99 camsim99 changed the title [WIP] Add a way to request new Surface from SurfaceProducer and avoid SurfaceProducer returning invalid Surface [Android] Add a way to request new Surfaces from SurfaceProducer and avoid SurfaceProducer returning invalid Surface Jun 16, 2025
@camsim99 camsim99 marked this pull request as ready for review June 18, 2025 15:34
@camsim99 camsim99 requested review from matanlurey and a team June 18, 2025 15:35
}

@Override
public Surface getSurface(boolean forceNewSurface) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of this API, versus possible alternatives?

  • void invalidateSurface() (surface = null), which is called before getSurface to force a new surface.
  • Surface getForceNewSurface(), identical this one, but without using a boolean parameter.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, you suggesting we provide two different ways to force getting a new surface, i.e. a user could call invalidateSurface then getSurface or just call getForceNewSurface?

void invalidateSurface() (surface = null), which is called before getSurface to force a new surface.

Also, as an aside, I think we should avoid the invalidateSurface name because we aren't actually invalidating the Surface. Maybe requestNewSurface or something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I just know if getSurface(true) really ... screams anything, like I wouldn't notice it in a code review.

Copy link
Contributor Author

@camsim99 camsim99 Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hahaha no that's such a good point. I'm thinking we do Surface getForceNewSurface() then so it's clearer.

Edit: Chose getForcedNewSurface so "forced new" is kinda like a descriptor for the surface.

@camsim99 camsim99 requested a review from matanlurey June 18, 2025 20:50
@camsim99
Copy link
Contributor Author

Gonna go ahead and land this! I'll monitor the dashboard too in case I need to revert in case folks aren't around.

@camsim99 camsim99 added the autosubmit Merge PR when tree becomes green via auto submit App label Jun 20, 2025
@auto-submit auto-submit bot added this pull request to the merge queue Jun 20, 2025
Merged via the queue into flutter:master with commit c7362b4 Jun 20, 2025
174 checks passed
@flutter-dashboard flutter-dashboard bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jun 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jun 20, 2025
…Producer` and avoid `SurfaceProducer` returning invalid `Surface` (flutter/flutter#169899)
auto-submit bot pushed a commit to flutter/packages that referenced this pull request Jun 20, 2025
flutter/flutter@85a9b4f...c7362b4

2025-06-20 43054281+camsim99@users.noreply.github.com [Android] Add a way to request new `Surface`s from `SurfaceProducer` and avoid `SurfaceProducer` returning invalid `Surface` (flutter/flutter#169899)
2025-06-20 engine-flutter-autoroll@skia.org Roll Dart SDK from cc2f604acb05 to a554bdd0a2cc (1 revision) (flutter/flutter#170912)
2025-06-20 robert.ancell@canonical.com Set the background in the Linux template (flutter/flutter#170841)
2025-06-20 engine-flutter-autoroll@skia.org Roll Skia from 2fc58cffe69b to f1e68950ea7b (2 revisions) (flutter/flutter#170914)
2025-06-20 engine-flutter-autoroll@skia.org Roll Skia from 8a710de2a1d3 to 2fc58cffe69b (2 revisions) (flutter/flutter#170906)
2025-06-20 137456488+flutter-pub-roller-bot@users.noreply.github.com Roll pub packages (flutter/flutter#170907)
2025-06-20 engine-flutter-autoroll@skia.org Roll Skia from 2ac2476b0bcb to 8a710de2a1d3 (2 revisions) (flutter/flutter#170900)
2025-06-20 fluttergithubbot@gmail.com Marks Linux_pixel_7pro dynamic_path_stroke_tessellation_perf__timeline_summary to be flaky (flutter/flutter#170785)
2025-06-20 engine-flutter-autoroll@skia.org Roll Dart SDK from 6a4b11760906 to cc2f604acb05 (1 revision) (flutter/flutter#170895)
2025-06-19 engine-flutter-autoroll@skia.org Roll Skia from 61c89daa3888 to 2ac2476b0bcb (4 revisions) (flutter/flutter#170890)
2025-06-19 engine-flutter-autoroll@skia.org Roll Dart SDK from bce9abe03175 to 6a4b11760906 (1 revision) (flutter/flutter#170884)
2025-06-19 bkonyi@google.com [ Tool ] Roll DDS 5.0.3 (flutter/flutter#170880)
2025-06-19 engine-flutter-autoroll@skia.org Roll Packages from 715a0a5 to 0ec4053 (8 revisions) (flutter/flutter#170879)
2025-06-19 engine-flutter-autoroll@skia.org Roll Skia from 794936b23812 to 61c89daa3888 (7 revisions) (flutter/flutter#170878)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC stuartmorgan@google.com on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
feduke-nukem pushed a commit to Yobari-Timeliners/golub that referenced this pull request Jun 21, 2025
flutter/flutter@85a9b4f...c7362b4

2025-06-20 43054281+camsim99@users.noreply.github.com [Android] Add a way to request new `Surface`s from `SurfaceProducer` and avoid `SurfaceProducer` returning invalid `Surface` (flutter/flutter#169899)
2025-06-20 engine-flutter-autoroll@skia.org Roll Dart SDK from cc2f604acb05 to a554bdd0a2cc (1 revision) (flutter/flutter#170912)
2025-06-20 robert.ancell@canonical.com Set the background in the Linux template (flutter/flutter#170841)
2025-06-20 engine-flutter-autoroll@skia.org Roll Skia from 2fc58cffe69b to f1e68950ea7b (2 revisions) (flutter/flutter#170914)
2025-06-20 engine-flutter-autoroll@skia.org Roll Skia from 8a710de2a1d3 to 2fc58cffe69b (2 revisions) (flutter/flutter#170906)
2025-06-20 137456488+flutter-pub-roller-bot@users.noreply.github.com Roll pub packages (flutter/flutter#170907)
2025-06-20 engine-flutter-autoroll@skia.org Roll Skia from 2ac2476b0bcb to 8a710de2a1d3 (2 revisions) (flutter/flutter#170900)
2025-06-20 fluttergithubbot@gmail.com Marks Linux_pixel_7pro dynamic_path_stroke_tessellation_perf__timeline_summary to be flaky (flutter/flutter#170785)
2025-06-20 engine-flutter-autoroll@skia.org Roll Dart SDK from 6a4b11760906 to cc2f604acb05 (1 revision) (flutter/flutter#170895)
2025-06-19 engine-flutter-autoroll@skia.org Roll Skia from 61c89daa3888 to 2ac2476b0bcb (4 revisions) (flutter/flutter#170890)
2025-06-19 engine-flutter-autoroll@skia.org Roll Dart SDK from bce9abe03175 to 6a4b11760906 (1 revision) (flutter/flutter#170884)
2025-06-19 bkonyi@google.com [ Tool ] Roll DDS 5.0.3 (flutter/flutter#170880)
2025-06-19 engine-flutter-autoroll@skia.org Roll Packages from 715a0a5c4fdd to e3b2898 (8 revisions) (flutter/flutter#170879)
2025-06-19 engine-flutter-autoroll@skia.org Roll Skia from 794936b23812 to 61c89daa3888 (7 revisions) (flutter/flutter#170878)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC stuartmorgan@google.com on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
engine flutter/engine repository. See also e: labels. platform-android Android applications specifically
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants