Support `PixmapMut<'_>` and stride in `vello_cpu`?

[madsmtm]

Hey! I'm working on improving Softbuffer, which you use in vello_cpu's examples to render to a window. I believe that this combination of winit, softbuffer and either tiny-skia or vello_cpu is a good match for simple demos and such, and I'll be recommending using your crates in our documentation in rust-windowing/softbuffer#325.

Softbuffer works by creating special shared memory that can be efficiently sent to the compositor, so for it to be zero-copy, it has to be in control of the buffer (it cannot just create its buffers from a Vec<u8>). Additionally, for it to be zero-copy on Android and macOS, the buffer needs to have a stride, see rust-windowing/softbuffer#315. Both of these are a bit unfortunate, because it means that integration with vello_cpu requires rendering into a separate buffer first.

Would it be possible (and desirable) for vello_cpu to support something like tiny-skia's PixmapMut<'_>, and to support passing a byte stride != width * 4 in its creation method?

This is the complementary issue to linebender/tiny-skia#168, see that for a code example of how I imagine an integration would look like in the future.

[LaurenzV]

You can already use the render_to_buffer method, which should have the same functonality as a PixmapMut.

I'm not sure what you mean by byte stride != width * 4, you mean that only every xth pixel is used for storing the result? So for example if you have a stride of 4 and you want to store the RGBA value (255, 128, 128, 128), that it should be stored as [255, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0]?

[madsmtm]

You can already use the render_to_buffer method, which should have the same functonality as a PixmapMut.

Ah, nice, I should've investigated that more thoroughly beforehand!

I'm not sure what you mean by byte stride != width * 4, you mean that only every xth pixel is used for storing the result? So for example if you have a stride of 4 and you want to store the RGBA value (255, 128, 128, 128), that it should be stored as [255, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0, 128, 0, 0, 0]?

No, I mean that each row is padded such that it is a multiple of whatever the platform requires (on macOS Apple Silicon, byte_stride % 128 == 0). So the buffer is logically of size 4 * width * height bytes, but internally it's byte_stride * height bytes. I guess you could also call it row_width_in_bytes if that's more descriptive for you?

As an example, a width = 3, height = 2, byte_stride = 16 buffer would be represented as:

[
    R0, G0, B0, A0,   R1, G1, B1, A1,   R2, G2, B2, A2,   P, P, P, P,
    R3, G3, B3, A3,   R4, G4, B4, A4,   R5, G5, B5, A5,   P, P, P, P,
]

Where P is "padding" byte, i.e. a byte that can be read from and written to, but won't matter for the output, they're simply there to align each row.

You can almost treat it as-if width = byte_stride / 4, this works with e.g. fill_solid, since it doesn't matter if you end up filling the padding bytes too, but it would go wrong if you used fill_gradient, as that would chop the last few pixels off the gradient at the right.

[madsmtm]

Maybe it would be enough if render_to_buffer allowed RenderContext.width <= buffer_width (and RenderContext.height <= buffer_height)?

[LaurenzV]

Oh! I see. I need to double check the implementation, but it probably should be doable.

[nicoburns]

We probably ought to support this, but I would note that in my testing with current softbuffer (on macOS - not tested on other platforms) rendering directly into the softbuffer buffer (and then pixel-swizzling RGBA to BGRA in-place) is slower than rendering into a separate buffer and swizzling while copying. Not sure why, it may well be possible to fix this. And it would be a big memory usage win if we can.

[madsmtm]

We probably ought to support this, but I would note that in my testing with current softbuffer (on macOS - not tested on other platforms) rendering directly into the softbuffer buffer (and then pixel-swizzling RGBA to BGRA in-place) is slower than rendering into a separate buffer and swizzling while copying. Not sure why, it may well be possible to fix this. And it would be a big memory usage win if we can.

Hmm, interesting! I suspect that macOS might be one of the platforms that isn't going to support RGBA (as IOSurface might not support it IIUC), but pretty sure that pixel-swizzling RGBA -> BGRA can be done a lot faster if you use SIMD. A simple way to check would be to try out vImagePermuteChannels_ARGB8888.

EDIT: I also found image_swizzle which if nothing else could serve as a useful basis to build upon.