Windows: Add Direct3D texture interoperability support

[jnschulze]

This PR adds support for Direct3D texture interop using ANGLE's ANGLE_d3d_share_handle_client_buffer extension (see https://www.khronos.org/registry/EGL/extensions/ANGLE/EGL_ANGLE_d3d_share_handle_client_buffer.txt)

It adds a new texture variant, flutter::GpuSurfaceTexture, as an addition to the existing flutter::PixelBufferTexture.

See https://github.com/Microsoft/angle/wiki/Interop-with-other-DirectX-code for an example of how to create compatible Direct3D textures.

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene wiki page, which explains my responsibilities.
• I read and followed the Flutter Style Guide and the C++, Objective-C, Java style guides.
• I listed at least one issue that this PR fixes in the description above.
• I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See testing the engine for instructions on
  writing and running engine tests.
• I updated/added relevant documentation (doc comments with ///).
• I signed the CLA.
• All existing and new tests are passing.
• The reviewer has submitted any presubmit flakes in this PR using the engine presubmit flakes form before re-triggering the failure.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[chinmaygarde]

cc @cbracken @clarkezone. The approach looks fine to me. But the presubmit failures are real though (compilation failures). Can you please address those in the meantime?

[jnschulze]

@chinmaygarde

But the presubmit failures are real though (compilation failures). Can you please address those in the meantime?

done.

[clarkezone]

I'll take a look over the weekend

[clarkezone]

Took a quick look, looks good based on that.

[chinmaygarde]

cc @clarkezone Care to approve the patch 😄

[clarkezone]

@jnschulze do you have a test app I can run against your branch?

[jnschulze]

@clarkezone
Yes, this webview_windows branch should work: https://github.com/jnschulze/flutter-webview-windows/commits/features/gpu-surface-texture

[chinmaygarde]

@clarkezone Do you have the cycles to look over the sample?

[clarkezone]

Appols.. I have not had a chance to look and I'm heading out on a vacation for a week. I have cycles on 7/26 that I could carve out to look but appreciate that is a ways out there.

[knopp]

@jnschulze, I tried the gpu-surface-texture webview branch, and I have a question. The fallback texture bridge seems to work fine, but when enabling the gpu surface (use_gpu_surface=true in WebviewBridge constructor, the surface seem to have scaling issues (it doesn't fill the entire window). How much of window it fills seems to depend on window size. Are you aware of this?

[jnschulze]

@knopp
Thanks, I will take a look this week.

[chinmaygarde]

@jnschulze Any updates?

[jnschulze]

@chinmaygarde
Sorry, I haven't had a chance to see what's wrong with the example yet.

[chinmaygarde]

I'm going to add the WIP tag in the meantime.

[clarkezone]

@jnschulze, I tried the gpu-surface-texture webview branch, and I have a question. The fallback texture bridge seems to work fine, but when enabling the gpu surface (use_gpu_surface=true in WebviewBridge constructor, the surface seem to have scaling issues (it doesn't fill the entire window). How much of window it fills seems to depend on window size. Are you aware of this?

What DPI / scaling factor are you running at?
Does this repo if you set scaling to 100%?

@knopp

[jnschulze]

I've just pushed an update.

Changes since the previous commit

• kFlutterDesktopGpuSurfaceTypeDxgi was renamed to kFlutterDesktopGpuSurfaceTypeDxgiSharedHandle
• kFlutterDesktopGpuSurfaceTypeD3d11Texture2D was added
• release_callback and release_context were added to FlutterDesktopGpuSurfaceDescriptor
• struct_size was added to FlutterDesktopGpuSurfaceDescriptor in order to safely support future changes.
• Tests were added

[jnschulze]

Why not exposing OpenGL and instead going straight for D3D interop though ?

Because Flutter effectively uses DirectX on Windows. The fact that it uses ANGLE for translating OpenGL to D3D on Windows is an implementation detail which might change in the future.
Besides, exposing OpenGL would require additional APIs for context sharing etc.

[oddko]

Because Flutter effectively uses DirectX on Windows. The fact that it uses ANGLE for translating OpenGL to D3D on Windows is an implementation detail which might change in the future.

True but so is the use of DirectX on Windows which is itself an implementation detail of ANGLE on Windows. My comment was more on the fact that you have (as it seems from the PR) to go through ANGLE anyway for the interop, so OpenGLES seemed like an easier target. But given you used this to embed native views (like webviews) that makes sense you used D3D.

[alexmercerind]

Greetings!

I need a little help with this. So, I'm trying to render stuff using open-gl APIs with this texture API in Flutter. By using ANGLE & using shared HANDLE after creating ID3D11Device & ID3D11Texture2D. But, everything is just blank. Am I doing something completely meaningless? Is this not how it's meant to be used?

I just stitched together various code I could find related to ANGLE, Direct3D or this new Direct3D texture API in Flutter. There's no boilerplate, everything is written very procedurally to have a minimal running example.

I've configured everything in the repository & a single flutter run will work. Your help/guidance will be appreciated.

Thankyou.

Repository: https://github.com/alexmercerind/flutter_windows_angle_d3d_texture
Dart: https://github.com/alexmercerind/flutter_windows_angle_d3d_texture/blob/5651c198b0ab6aeb7475d5da820d7bd1bebbf44d/example/lib/main.dart#L25-L35
C++ Plugin Interface: https://github.com/alexmercerind/flutter_windows_angle_d3d_texture/blob/5651c198b0ab6aeb7475d5da820d7bd1bebbf44d/windows/flutter_windows_angle_d3d_texture_plugin.cpp#L102-L314

Please copy these DLLs near your .exe before running ⚠️⚠️⚠️.

Used References

https://github.com/Microsoft/angle/wiki/Interop-with-other-DirectX-code
https://github.com/google/angle/blob/main/samples/hello_triangle/HelloTriangle.cpp
https://github.com/clarkezone/anglehosting

[alexmercerind]

I updated repository with fix & minor intro: https://github.com/alexmercerind/flutter-windows-ANGLE-OpenGL-Direct3D-Interop. Hope this serves as a good example to a new-comer.

---

UPDATE: My happiness is immeasurable.