Add custom cursors

[eero-lehtinen]

Objective

• Add custom images as cursors
• Fixes Add first-party asset-based cursor icons to Bevy #9557

Solution

• Change cursor type to accommodate both native and image cursors
• I don't really like this solution because I couldn't use Handle<Image> directly. I would need to import bevy_assets and that causes a circular dependency. Alternatively we could use winit's CustomCursor smart pointers, but that seems hard because the event loop is needed to create those and is not easily accessable for users. So now I need to copy around rgba buffers which is sad.
• I use a cache because especially on the web creating cursor images is really slow
• Sorry to WIP custom cursors #14196 for yoinking, I just wanted to make a quick solution for myself and thought that I should probably share it too.

Update:

• Now uses Handle<Image>, reads rgba data in bevy_render and uses resources to send the data to bevy_winit, where the final cursors are created.

Testing

• Added example which works fine at least on Linux Wayland (winit side has been tested with all platforms).
• I haven't tested if the url cursor works.

Migration Guide

• CursorIcon is no longer a field in Window, but a separate component can be inserted to a window entity. It has been changed to an enum that can hold custom images in addition to system icons.
• Cursor is renamed to CursorOptions and cursor field of Window is renamed to cursor_options
• CursorIcon is renamed to SystemCursorIcon

[Hexorg]

Would it be more clear to have SystemCursor and SpriteCursor?

[alice-i-cecile]

I really want this feature, and I'm excited to see that there's a path forward.

That said, I really dislike the custom image buffer :( That's really unpleasant to use, and hard to maintain. Can you say more about the circular dependency? What exactly is going on here: can we refactor things to get a nice API?

[eero-lehtinen]

That said, I really dislike the custom image buffer :( That's really unpleasant to use, and hard to maintain. Can you say more about the circular dependency? What exactly is going on here: can we refactor things to get a nice API?

bevy_render contains Image and depends on bevy_window, so bevy_window can't depend on bevy_render and so can't use Image in the Window component.

We also can't use Handle in bevy_winit because bevy_asset depends on bevy_winit. It's only used with a little bit with android specific stuff, so it could maybe be removed. Then we could use a separate component from bevy_winit (not included in Window) to store a cursor Handle<Image>.

[eero-lehtinen]

Here is a graph of bevy workspace dependencies if someone wants to think about it. It doesn't show bevy_asset depending on bevy_winit because that is only on android.

Basically it would be best if both bevy_window and bevy_winit could depend on bevy_render, but at least bevy_window is deeply used in bevy_render, so is probably impossible. Maybe just inverting the (not shown in the graph) dependency between bevy_asset and bevy_winit is enough, because it would allow bevy_winit to depend on bevy_render and bevy_asset and then could use both Handle and Image. Then we need to just live with the fact that cursors will not be included in the Window component.

[Hexorg]

Is moving Image into bevy_asset an option?

[alice-i-cecile]

I'd really prefer not to move that into bevy_asset. Could we move Image into its own crate?

For reference, the same problem is encountered with #1031: we really want to feed normal image data (loaded via the asset system) back into windowing.

[eero-lehtinen]

Having Image in a separate crate could fix this as long as it's made to not depend on bevy_render and then we would also need to reverse/fix the android dependency.

[alice-i-cecile]

Great. Can you link to the Android stuff? We should pull this out into its own issue; I think this is a pretty reasonable plan of attack.

[eero-lehtinen]

bevy/crates/bevy_asset/src/io/android.rs

Line 20 in e512cb6

let asset_manager = bevy_winit::ANDROID_APP

On android, we get the asset path from winit. It should be possible to just reverse the dependency and smuggle the paths with resources or call functions the other way. (Or I guess its a whole asset manager, so might be more difficult)

[tychedelia]

Really great change, thank you for doing this! It would be awesome to have an example demonstrating the new feature, but this looks great as is.

[eero-lehtinen]

Really great change, thank you for doing this! It would be awesome to have an example demonstrating the new feature, but this looks great as is.

The window_settings example is modified to include an image cursor.

[tychedelia]

Really great change, thank you for doing this! It would be awesome to have an example demonstrating the new feature, but this looks great as is.

The window_settings example is modified to include an image cursor.

Ah great :) I just saw the system cursors and missed the custom. Perfect!

[janhohenheim]

Great API, good work!

[alice-i-cecile]

For folks looking in the future, #14351 talks about the web of dependencies and why this implementation was chosen.