`Transform3D.rotate_local` does not rotate as expected

[derkork]

Tested versions

v4.3.stable.mono.official [77dcf97]

System information

Godot v4.3.stable.mono - Windows 10.0.22631 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 4080 (NVIDIA; 32.0.15.5599) - AMD Ryzen 9 7950X3D 16-Core Processor (32 Threads)

Issue description

I have a 3D transform and I want to apply two rotations locally to it in this order:

1. rotate it around the local Y axis by 45 degrees.
2. rotate it around the local X axis by 45 degrees (the local x axis as it is after the first rotation)

From the description the rotate_local function would do what I want, so I wrote the following code:

var t3d = Transform3D().translated(Vector3(0, 2, 0))
print("Rotate local")
print(t3d)
t3d = t3d.rotated_local(t3d.basis.y, deg_to_rad(45))
print(t3d)
t3d = t3d.rotated_local(t3d.basis.x, deg_to_rad(45))
print(t3d)

This outputs:

Rotate local
[X: (1, 0, 0), Y: (0, 1, 0), Z: (0, 0, 1), O: (0, 2, 0)]
[X: (0.707107, 0, -0.707107), Y: (0, 1, 0), Z: (0.707107, 0, 0.707107), O: (0, 2, 0)]
[X: (0.5, -0.5, -0.707107), Y: (0.707107, 0.707107, 0), Z: (0.5, -0.5, 0.707107), O: (0, 2, 0)]

The first rotation looks OK, we rotated around the Y axis, so the Y axis stays as is and X and Z are rotated by 45 degrees in the X-Z plane around the local origin, so they get the 0.7-ish values which are sin/cos of 45 degrees. So far so good. Now the second rotation looks wrong. We rotate around the local X axis, so the basis.x value in the output should stay the same, but it doesn't.

Now I repeated the same code with a direct basis rotation:

# same with basis.rotated
t3d = Transform3D().translated(Vector3(0,2,0))
print("Rotate basis directly")
	
print(t3d)
t3d.basis = t3d.basis.rotated(t3d.basis.y, deg_to_rad(45))
print(t3d)
t3d.basis = t3d.basis.rotated(t3d.basis.x, deg_to_rad(45))
print(t3d)

This outputs:

Rotate basis directly
[X: (1, 0, 0), Y: (0, 1, 0), Z: (0, 0, 1), O: (0, 2, 0)]
[X: (0.707107, 0, -0.707107), Y: (0, 1, 0), Z: (0.707107, 0, 0.707107), O: (0, 2, 0)]
[X: (0.707107, 0, -0.707107), Y: (0.5, 0.707107, 0.5), Z: (0.5, -0.707107, 0.5), O: (0, 2, 0)]

This is more in line with what I would expect. The second rotation is around the local X axis so basis.x stays the same.

When visualizing the outputs, it is apparent that the results differ:

Now I see two explanations for this:

1. most likely: I misunderstood how rotate_local is supposed to work and the results it produces are actually correct. In this case maybe the documentation could be improved to state how this function works and maybe what an appropriate use case for this function would be.
2. The output is indeed not what is expected and in this case the implementation should be changed, e.g. maybe to:

Transform3D Transform3D::rotated_local(const Vector3 &p_axis, real_t p_angle) const {
	return Transform3D(basis.rotated(p_axis, p_angle), origin);
}

Steps to reproduce

In the attached reproducer project, open node_3d.tscn to see the visualization of the output. To run the printing, run this scene.

Minimal reproduction project (MRP)

rotate_local_reproducer.zip

[rburing]

The input you're passing to rotated_local is in the wrong coordinate system. If you want to rotate around the local Y and X axes you should pass in Vector3.UP and Vector3.RIGHT respectively.

[derkork]

That seems indeed to be the case, so I indeed misunderstood how this function is supposed to be used. Maybe a sentence could be added to the documentation like:

The axis is expected to be in the local coordinate system of the transform, e.g. to rotate around the local X, Y and Z-axes, use Vector3.RIGHT, Vector3.UP and Vector3.FORWARD.

[furkanCalik7]

Can I contribute to this issue as my first issue?