External WebGPU surfaces / BYOW

[littledivy]

Goals:

• De-facto windowing solution.
• BYOW: Bring your own window.
• Deno should not link to native windowing systems like X11.
• Deno should not have to be forked for windowing support.

/* any windowing library */
import { WindowBuilder } from "https://deno.land/x/sdl2/mod.ts";
const win = new WindowBuilder("Hello WGPU", 512, 512).build();

/* [string, Deno.PointerValue, Deno.PointerValue] */
const [system, windowHandle, displayHandle] = win.rawHandle();

const surface = new Deno.UnsafeWindowSurface(system, windowHandle, displayHandle);

/* GPUCanvasContext */
const context = surface.getContext();
context.configure({ /* ... */ });

for (const events of win.events()) {
  const texture = context.getCurrentTexture();
  // ...
  surface.present();
}

Proposed: Add Deno.UnsafeWindowSurface API:

1. Creates a raw_window_handle::RawWindowHandle implementation from windowHandle and displayHandle
2. Internally calls instance->create_surface(&win)
3. Surface is returned wrapped in as a GPUCanvasContext (@crowlKats has already implemented presentation APIs)

class UnsafeWindowSurface {
  constructor(
    system: "cocoa" | "x11" | "win32",
    winHandle: Deno.PointerValue,
    displayHandle: Deno.PointerValue | null
  );
  
  getContext(type: "webgpu"): GPUCanvasContext;

  present(): void;
}

Pros:

• Single binary. No extensions or Deno forks.
• Does not link CLI to windowing systems.
• Plug and play FFI windowing libraries with Deno WebGPU.
• deno compile-able.

Cons:

• Very low level
• displayHandle and winHandle can be invalid handles. Creating them requires --allow-ffi --unstable

I have a working implementation and a few demos I will share shortly.

cc-ing for visibility: @DjDeveloperr @load1n9 @eliassjogreen (also @aapoalas since this touches FFI too)

[littledivy]

Some demos using deno_sdl2:

1. Shaderplay, a WGSL live playground: https://github.com/littledivy/shaderplay

2. Conway's game of life (compute + render pass example): https://github.com/littledivy/deno_sdl2/blob/webgpu_external_surface/webgpu-examples/gameoflife.ts

3. Orillusion, a WebGPU game engine example scene: https://github.com/littledivy/deno_sdl2/blob/webgpu_external_surface/webgpu-examples/orillusion.ts

4. brendan-duncan/gpucraft: Minecraft-like game using WebGPU.
   Gameplay in Deno: https://youtu.be/vTJvjBG-GLs

[lino-levan]

This makes a ton of sense. Massive +1 from my side.

Deno should not have to be forked for windowing support.

This is definitely the most important point for me. Maintaining a fork just doesn't make sense in the long run.

[crowlKats]

This is definitely the most important point for me. Maintaining a fork just doesn't make sense in the long run.

just as an FYI: Forking hasnt been necessary for a long time, and could be achieved with the deno crates.

[abflow]

Is it something that could be paired with Skia, or CanvasKit with WebAssembly ?
https://skia.org/docs/user/modules/canvaskit/

[crowlKats]

No, since skia is it own rendering system, which is unrelated to WebGPU

[abflow]

Thanks, I thought that as skia can render graphics to WebGL (at least concerning the WebAssembly CanvasKit module), it could have rendered to WebGPU as well

[littledivy]

Updated the proposal. Renamed API to Deno.UnsafeWindowSurface