Specialized UI transform #16615

ickshonpe · 2024-12-02T20:04:19Z

Objective

Add specialized UI transform Components and fix some related problems:

Animating UI elements by modifying the Transform component of UI nodes doesn't work very well because ui_layout_system overwrites the translations each frame. The overflow_debug example uses a horrible hack where it copies the transform into the position that'll likely cause a panic if any users naively copy it.
Picking ignores rotation and scaling and assumes UI nodes are always axis aligned.
The clipping geometry stored in CalculatedClip is wrong for rotated and scaled elements.
Transform propagation is unnecessary for the UI, the transforms can be updated during layout updates.
The UI internals use both object-centered and top-left-corner-based coordinates systems for UI nodes. Depending on the context you have to add or subtract the half-size sometimes before transforming between coordinate spaces. We should just use one system consistantly so that the transform can always be directly applied.
Transform doesn't support responsive coordinates.

Solution

Unrequire Transform from Node.
New components UiTransform, UiGlobalTransform:
- Node requires UiTransform, UiTransform requires UiGlobalTransform
- UiTransform is a 2d-only equivalent of Transform with a translation in Vals.
- UiGlobalTransform newtypes Affine2 and is updated in ui_layout_system.
New helper functions on ComputedNode for mapping between viewport and local node space.
The cursor position is transformed to local node space during picking so that it respects rotations and scalings.
To check if the cursor hovers a node recursively walk up the tree to the root checking if any of the ancestor nodes clip the point at the cursor. If the point is clipped the interaction is ignored.
Use object-centered coordinates for UI nodes.
RelativeCursorPosition's coordinates are now object-centered with (0,0) at the the center of the node and the corners at (±0.5, ±0.5).
Replaced the normalized_visible_node_rect: Rect field of RelativeCursorPosition with cursor_over: bool, which is set to true when the cursor is over an unclipped point on the node. The visible area of the node is not necessarily a rectangle, so the previous implementation didn't work.

This should fix all the logical bugs with non-axis aligned interactions and clipping. Rendering still needs changes but they are far outside the scope of this PR.

Tried and abandoned two other approaches:

New transform field on Node, require GlobalTransform on Node, and unrequire Transform on Node. Unrequiring Transform opts out of transform propagation so there is then no conflict with updating the GlobalTransform in ui_layout_system. This was a nice change in its simplicity but potentially confusing for users I think, all the GlobalTransform docs mention Transform and having special rules for how it's updated just for the UI is unpleasently surprising.
New transform field on Node. Unrequire Transform on Node. New transform: Affine2 field on ComputedNode.
This was okay but I think most users want a separate specialized UI transform components. The fat ComputedNode doesn't work well with change detection.

Fixes #18929, #18930

Testing

There is an example you can look at:

cargo run --example ui_transform

Sometimes in the example if you press the rotate button couple of times the first glyph from the top label disappears , I'm not sure what's causing it yet but I don't think it's related to this PR.

Migration Guide

New specialized 2D UI transform components UiTransform and UiGlobalTransform. UiTransform is a 2d-only equivalent of Transform with a translation in Vals. UiGlobalTransform newtypes Affine2 and is updated in ui_layout_system.
Node now requires UiTransform instead of Transform. UiTransform requires UiGlobalTransform.

In previous versions of Bevy ui_layout_system would overwrite UI node's Transform::translation each frame. UiTransforms aren't overwritten and there is no longer any need for systems that cache and rewrite the transform for translated UI elements.

RelativeCursorPosition's coordinates are now object-centered with (0,0) at the the center of the node and the corners at (±0.5, ±0.5). Its normalized_visible_node_rect field has been removed and replaced with a new cursor_over: bool field which is set to true when the cursor is hovering an unclipped area of the UI node.

* Required `GlobalTransform` instead of `Transform` on `Node`. * Update node's global transforms during layout updates.

…bevy into node-global-transform

alice-i-cecile · 2024-12-02T20:42:15Z

I like this as an incremental move forward. We shouldn't be storing a full Transform here, but that's easy to migrate from this PR.

Should this be on ComputedNode instead though? I don't think this is intended to be user-modifiable.

github-actions · 2024-12-02T20:43:04Z

It looks like your PR is a breaking change, but you didn't provide a migration guide.

Could you add some context on what users should update when this change get released in a new version of Bevy?
It will be used to help writing the migration guide for the version. Putting it after a ## Migration Guide will help it get automatically picked up by our tooling.

NthTensor · 2024-12-02T20:55:24Z

This does seem like the least-bad option. But I thought there was code in bevy_transform to specifically filter out ui elements from propagation. Could you check if that's still there and remove it if it is?

I believe this should fix the issues big_space had with the use of Transform by ui elements, which is nice.

ickshonpe · 2024-12-02T21:00:05Z

I like this as an incremental move forward. We shouldn't be storing a full Transform here, but that's easy to migrate from this PR.

Should this be on ComputedNode instead though? I don't think this is intended to be user-modifiable.

The new transform field on Node is intended to be user-modifiable. It's meant to be simar to the CSS transform property and allows users to add simple UI animations or whatever without weird hacks to get around the conflicts between the transformation propagation and UI layout update systems.

I made another branch first just before this one that removed both Transform and GlobalTransform from UI nodes and instead added the transform to ComputedNode but it felt like it would be difficult to get through review as it required lots of changes to extraction and focus and new a UI specific visibility system. Performance is much better than main with that approach though and would allow for some nice helper methods on Node like contains_point etc which would simplify a lot of the systems.

ickshonpe · 2024-12-02T21:05:19Z

This does seem like the least-bad option. But I thought there was code in bevy_transform to specifically filter out ui elements from propagation. Could you check if that's still there and remove it if it is?

I believe this should fix the issues big_space had with the use of Transform by ui elements, which is nice.

I haven't seen anything like this. Afaik the only way to opt-out of transform propagation is by removing Transform and there's no UI specific filter.

…lue in physical pixels. This required adding an extra parameter for the scale factor.

ickshonpe · 2025-04-28T23:18:30Z

Should this PR move to Draft? Was said to be nearly done, then got 12 commits merged. Also designed completely changed from initial version. I'm not sure as a reviewer when it's safe to spend time reviewing again. Thanks

Yeah my fault keep changing my mind as to how this should be implemented. I don't intend to make any more changes now though.

NthTensor · 2025-05-28T11:55:17Z

Starting a review. From just the readme, this seems fantastic. Exactly what I was looking for in basically every way.

NthTensor

The code looks great. This is a smaller change than I was expecting tbh. Have not run any of the examples, I'll check it out later today and approve if I don't find any bugs.

NthTensor · 2025-05-28T12:06:38Z

crates/bevy_ui/src/ui_transform.rs

+        Self::ZERO
+    }
+}
+


Suggested change

pub trait IntoVal2 {

fn px(self) -> Val2;

fn percent(self) -> Val2;

fn vw(self) -> Val2;

fn vh(self) -> Val2;

fn vmin(self) -> Val2;

fn vmax(self) -> Val2;

}

impl IntoVal2 for Vec2 {

fn px(self) -> Val2 {

Val2 { x: Val::Px(self.x), y: Val::Px(self.y) }

}

fn percent(self) -> Val2 {

Val2 { x: Val::Percent(self.x), y: Val::Percent(self.y) }

}

fn vw(self) -> Val2 {

Val2 { x: Val::Vw(self.x), y: Val::vw(self.y) }

}

fn vh(self) -> Val2 {

Val2 { x: Val::Vh(self.x), y: Val::Vh(self.y) }

}

fn vmin(self) -> Val2 {

Val2 { x: Val::Vmin(self.x), y: Val::Vmin(self.y) }

}

fn vmax(self) -> Val2 {

Val2 { x: Val::Vmax(self.x), y: Val::Vmax(self.y) }

}

}

I'm not sure but implementing these on Vec2 might be a bit controversial. I could just have Val2's px and percent methods take Vec2s instead and add more methods for the other variants instead?

Lets leave it as it is then, and I'll open up a PR adding this as follow up. I don't think it will be that controversial, and I want to add a similar pattern to some other types for conversion into Val anyway.

crates/bevy_ui/src/ui_transform.rs

NthTensor · 2025-05-28T12:08:32Z

crates/bevy_ui/src/ui_transform.rs

+    /// Translate the node.
+    pub translation: Val2,
+    /// Scale the node. A negative value reflects the node in that axis.
+    pub scale: Vec2,


Non-uniformscale my old enemy. This may create issues with shear just like we have in 3d. It's probably fine. Non-blocking, just a call-out.

Yep only went with scaling in both axis because it was supported with Transform/GlobalTransform and I wanted to avoid too much controversy with this PR.

NthTensor · 2025-05-28T12:09:45Z

crates/bevy_ui/src/focus.rs

+/// A component storing the position of the mouse relative to the node, (0., 0.) being the center and (0.5, 0.5) being the bottom-right
+/// If the mouse is not over the node, the value will go beyond the range of (-0.5, -0.5) to (0.5, 0.5)


I don't think you justified this change anywhere. I'm not against it, but why?

It's part of the changes to replace top-left-aligned coordinates and consistantly use only object-centered coordinates.

Definitely some users find having the origin at the top-left more intuitive though, if people really hate it I'm not against changing it back in this case.

ickshonpe · 2025-05-28T12:18:55Z

Starting a review. From just the readme, this seems fantastic. Exactly what I was looking for in basically every way.

thanks that's great, I was just about to go and beg on discord for someone to review this

Added `compute_affine` function

…bevy into node-global-transform

… a Rot2

ickshonpe added 6 commits December 2, 2024 19:19

* Added a transform: Transform field to Node.

76c0480

* Required `GlobalTransform` instead of `Transform` on `Node`. * Update node's global transforms during layout updates.

Fixed overflow_debug example

6847c51

Use logical units for Node's translation

ff39bd4

Merge branch 'main' into node-global-transform

96465d1

fix for ambiguity detection

f6a3dc8

Merge branch 'node-global-transform' of https://github.com/ickshonpe/…

447cea6

…bevy into node-global-transform

alice-i-cecile requested a review from NthTensor December 2, 2024 20:41

alice-i-cecile added the M-Needs-Migration-Guide A breaking change to Bevy's public API that needs to be noted in a migration guide label Dec 2, 2024

alice-i-cecile added the A-Animation Make things move and change over time label Dec 2, 2024

alice-i-cecile requested review from mockersf and djeedai December 2, 2024 21:49

BenjaminBrienen added S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Jan 22, 2025

ickshonpe added 2 commits February 17, 2025 13:17

Merge branch 'main' into node-global-transform

3dc5c2a

Fixes for merge

67e8315

ickshonpe mentioned this pull request Feb 17, 2025

Anonymous ghost nodes #17908

Draft

ickshonpe added 3 commits March 5, 2025 21:23

Changed the Val::resolve method to always return always return a va…

10833bf

…lue in physical pixels. This required adding an extra parameter for the scale factor.

Merge branch 'main' into node-global-transform

e5a59f1

Merge branch 'main' into node-global-transform

78ddbf3

alice-i-cecile removed the S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged label Apr 23, 2025

ickshonpe added 5 commits April 28, 2025 17:52

Renamed UiVec to Val2, added doc comment.

93f3ce8

Fixed unresolved doc links

94714e6

Removed default on unit struct

5882696

Merge branch 'main' into node-global-transform

84b1663

Merge branch 'main' into node-global-transform

c5afa14

ickshonpe mentioned this pull request Apr 29, 2025

Replace Transform in the UI with NodeTransform #8240

Closed

ickshonpe added 2 commits May 20, 2025 12:04

Merge branch 'main' into node-global-transform

67b4735

Removed useless conversion

05e874a

jannik4 mentioned this pull request May 20, 2025

UI picking doesn't support rotated and scaled elements #18930

Open

ickshonpe added 4 commits May 28, 2025 12:04

Merge branch 'main' into node-global-transform

7ede048

Fixed unused imports left over after merge.

b4469ea

Changed rect_size to a Vec2 in prepare_uinodes`

1218ad9

Removed unused import.

e4c1f44

NthTensor reviewed May 28, 2025

View reviewed changes

ickshonpe added 12 commits May 28, 2025 14:27

Fixed unused imports left over after merge.

0c76276

Updated gradients module

be00de2

Changed UiTransform::rotation to Rot2

c261a07

Added `compute_affine` function

Fixed compute_affine method

3d84a36

Merge branch 'node-global-transform' of https://github.com/ickshonpe/…

280cb3e

…bevy into node-global-transform

Fixed full_ui example

ee33460

Fixed backticks in comment.

0c5ca2a

Replaced from_angle fns with a single from rotation function taking…

92d3b1a

… a Rot2

Updated ui_transform example

f0a582e

Updated migration guide

6b633e3

Added migration guide for relative cursor position

ffd6914

Fixes for migration guide lints

7eddd04

cart added this to @cart's attention May 28, 2025

+pub trait IntoVal2 {
+    fn px(self) -> Val2;
+    fn percent(self) -> Val2;
+    fn vw(self) -> Val2;
+    fn vh(self) -> Val2;
+    fn vmin(self) -> Val2;
+    fn vmax(self) -> Val2;
+}
+impl IntoVal2 for Vec2 {
+    fn px(self) -> Val2 {
+        Val2 { x: Val::Px(self.x), y: Val::Px(self.y) }
+    }
+    fn percent(self) -> Val2 {
+        Val2 { x: Val::Percent(self.x), y: Val::Percent(self.y) }
+    }
+    fn vw(self) -> Val2 {
+        Val2 { x: Val::Vw(self.x), y: Val::vw(self.y) }
+    }
+    fn vh(self) -> Val2 {
+        Val2 { x: Val::Vh(self.x), y: Val::Vh(self.y) }
+    }
+    fn vmin(self) -> Val2 {
+        Val2 { x: Val::Vmin(self.x), y: Val::Vmin(self.y) }
+    }
+    fn vmax(self) -> Val2 {
+        Val2 { x: Val::Vmax(self.x), y: Val::Vmax(self.y) }
+    }
+}

		/// A component storing the position of the mouse relative to the node, (0., 0.) being the center and (0.5, 0.5) being the bottom-right
		/// If the mouse is not over the node, the value will go beyond the range of (-0.5, -0.5) to (0.5, 0.5)

Uh oh!

Specialized UI transform #16615

Are you sure you want to change the base?

Specialized UI transform #16615

Conversation

ickshonpe commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Objective

Solution

Testing

Migration Guide

Uh oh!

alice-i-cecile commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 2, 2024

Uh oh!

NthTensor commented Dec 2, 2024

Uh oh!

ickshonpe commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ickshonpe commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ickshonpe commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NthTensor commented May 28, 2025

Uh oh!

NthTensor left a comment

Choose a reason for hiding this comment

Uh oh!

NthTensor May 28, 2025

Choose a reason for hiding this comment

Uh oh!

ickshonpe May 28, 2025

Choose a reason for hiding this comment

Uh oh!

NthTensor May 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NthTensor May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ickshonpe May 28, 2025

Choose a reason for hiding this comment

Uh oh!

NthTensor May 28, 2025

Choose a reason for hiding this comment

Uh oh!

ickshonpe May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ickshonpe commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ickshonpe commented Dec 2, 2024 •

edited

Loading

alice-i-cecile commented Dec 2, 2024 •

edited

Loading

ickshonpe commented Dec 2, 2024 •

edited

Loading

ickshonpe commented Dec 2, 2024 •

edited

Loading

ickshonpe commented Apr 28, 2025 •

edited

Loading

NthTensor May 28, 2025 •

edited

Loading

ickshonpe May 28, 2025 •

edited

Loading

ickshonpe commented May 28, 2025 •

edited

Loading