Specialized UI transform

[ickshonpe]

Objective

Add specialized UI transform Components and fix some related problems:

• Animating UI elements by modifying the Transform component of UI nodes doesn't work very well because ui_layout_system overwrites the translations each frame. The overflow_debug example uses a horrible hack where it copies the transform into the position that'll likely cause a panic if any users naively copy it.
• Picking ignores rotation and scaling and assumes UI nodes are always axis aligned.
• The clipping geometry stored in CalculatedClip is wrong for rotated and scaled elements.
• Transform propagation is unnecessary for the UI, the transforms can be updated during layout updates.
• The UI internals use both object-centered and top-left-corner-based coordinates systems for UI nodes. Depending on the context you have to add or subtract the half-size sometimes before transforming between coordinate spaces. We should just use one system consistantly so that the transform can always be directly applied.
• Transform doesn't support responsive coordinates.

Solution

• Unrequire Transform from Node.
• New components UiTransform, UiGlobalTransform:
  • Node requires UiTransform, UiTransform requires UiGlobalTransform
  • UiTransform is a 2d-only equivalent of Transform with a translation in Vals.
  • UiGlobalTransform newtypes Affine2 and is updated in ui_layout_system.
• New helper functions on ComputedNode for mapping between viewport and local node space.
• The cursor position is transformed to local node space during picking so that it respects rotations and scalings.
• To check if the cursor hovers a node recursively walk up the tree to the root checking if any of the ancestor nodes clip the point at the cursor. If the point is clipped the interaction is ignored.
• Use object-centered coordinates for UI nodes.
• RelativeCursorPosition's coordinates are now object-centered with (0,0) at the the center of the node and the corners at (±0.5, ±0.5).
• Replaced the normalized_visible_node_rect: Rect field of RelativeCursorPosition with cursor_over: bool, which is set to true when the cursor is over an unclipped point on the node. The visible area of the node is not necessarily a rectangle, so the previous implementation didn't work.

This should fix all the logical bugs with non-axis aligned interactions and clipping. Rendering still needs changes but they are far outside the scope of this PR.

Tried and abandoned two other approaches:

• New transform field on Node, require GlobalTransform on Node, and unrequire Transform on Node. Unrequiring Transform opts out of transform propagation so there is then no conflict with updating the GlobalTransform in ui_layout_system. This was a nice change in its simplicity but potentially confusing for users I think, all the GlobalTransform docs mention Transform and having special rules for how it's updated just for the UI is unpleasently surprising.
• New transform field on Node. Unrequire Transform on Node. New transform: Affine2 field on ComputedNode.
  This was okay but I think most users want a separate specialized UI transform components. The fat ComputedNode doesn't work well with change detection.

Fixes #18929, #18930

Testing

There is an example you can look at:

cargo run --example ui_transform

Sometimes in the example if you press the rotate button couple of times the first glyph from the top label disappears , I'm not sure what's causing it yet but I don't think it's related to this PR.

Migration Guide

New specialized 2D UI transform components UiTransform and UiGlobalTransform. UiTransform is a 2d-only equivalent of Transform with a translation in Vals. UiGlobalTransform newtypes Affine2 and is updated in ui_layout_system.
Node now requires UiTransform instead of Transform. UiTransform requires UiGlobalTransform.

In previous versions of Bevy ui_layout_system would overwrite UI node's Transform::translation each frame. UiTransforms aren't overwritten and there is no longer any need for systems that cache and rewrite the transform for translated UI elements.

RelativeCursorPosition's coordinates are now object-centered with (0,0) at the the center of the node and the corners at (±0.5, ±0.5). Its normalized_visible_node_rect field has been removed and replaced with a new cursor_over: bool field which is set to true when the cursor is hovering an unclipped area of the UI node.

[alice-i-cecile]

I like this as an incremental move forward. We shouldn't be storing a full Transform here, but that's easy to migrate from this PR.

Should this be on ComputedNode instead though? I don't think this is intended to be user-modifiable.

[github-actions]

It looks like your PR is a breaking change, but you didn't provide a migration guide.

Could you add some context on what users should update when this change get released in a new version of Bevy?
It will be used to help writing the migration guide for the version. Putting it after a ## Migration Guide will help it get automatically picked up by our tooling.

[NthTensor]

This does seem like the least-bad option. But I thought there was code in bevy_transform to specifically filter out ui elements from propagation. Could you check if that's still there and remove it if it is?

I believe this should fix the issues big_space had with the use of Transform by ui elements, which is nice.

[ickshonpe]

I like this as an incremental move forward. We shouldn't be storing a full Transform here, but that's easy to migrate from this PR.

Should this be on ComputedNode instead though? I don't think this is intended to be user-modifiable.

The new transform field on Node is intended to be user-modifiable. It's meant to be simar to the CSS transform property and allows users to add simple UI animations or whatever without weird hacks to get around the conflicts between the transformation propagation and UI layout update systems.

I made another branch first just before this one that removed both Transform and GlobalTransform from UI nodes and instead added the transform to ComputedNode but it felt like it would be difficult to get through review as it required lots of changes to extraction and focus and new a UI specific visibility system. Performance is much better than main with that approach though and would allow for some nice helper methods on Node like contains_point etc which would simplify a lot of the systems.

[ickshonpe]

This does seem like the least-bad option. But I thought there was code in bevy_transform to specifically filter out ui elements from propagation. Could you check if that's still there and remove it if it is?

I believe this should fix the issues big_space had with the use of Transform by ui elements, which is nice.

I haven't seen anything like this. Afaik the only way to opt-out of transform propagation is by removing Transform and there's no UI specific filter.

[ickshonpe]

Starting a review. From just the readme, this seems fantastic. Exactly what I was looking for in basically every way.

thanks that's great, I was just about to go and beg on discord for someone to review this

[eckz]

Just an idea. You could make Node depend only on UiGlobalTransform, making UiTransform an optional component.

The only difference would be using Option<&UiTransform> instead of &UiTransform in ui_layout_system, saving a call to compute_affine when the component is not present.

[NthTensor]

I think we may want to consider folding GlobalZIndex and ZIndex into the UI transform eventually. Would appreciate your thoughts on the plan sketched out here.

[viridia]

The cursor position is transformed to local node space during picking so that it respects rotations and scalings.

I'm not sure this solves the problem I have been having with pointer interactions for multi-part widgets. Consider a slider with a draggable thumb: you click on the thumb entity, so the picking event you get has the coordinates relative to the thumb. But what you need for the drag calculation is the coordinates relative to the slider as a whole.

What I'd like is for the observer function to have the option to transform the picking coordinates into local coordinates for any arbitrary ancestor of the picked element. The easiest way to do this, I think, is to have the event contain global coordinates, and then have the observer do a conversion to local coordinates using the global transform component on the ancestor.