SoftBody3D behaves incorrectly if transform is adjusted after it has been added to a scene

[simpkins]

Tested versions

• Reproducible in 4.4.1 and master

System information

Windows 11

Issue description

Changing the transform on a SoftBody3D object behaves incorrectly once it has been added to a scene.

If you change the transform of a SoftBody3D that is currently part of a scene, it then renders at the wrong location.
The transform appears like it might be getting applied twice somehow, and the physics simulation is now being done incorrect relative to the rendered location. For instance, setting body.global_transform to a value of (0, 1, 0) causes the mesh to instead be rendered at a height of 2, and when it falls via gravity it stops at y=1 rather than y=0. If you apply a transform that has a rotation, the mesh now appears to fall at an angle instead of falling downwards.

It appears like it is somewhat possible to adjust the transform by:

1. remove the SoftBody3D from the scene tree
2. change the SoftBody3D transform
3. change the SoftBody3d mesh (this does reset the body vertices to their original mesh positions, so you cannot preserve the current mesh state)
4. re-add the SoftBody3D back to the scene tree

However, even with this procedure there is still a graphical glitch where the mesh is rendered at the wrong location for 1 frame when it is first re-added to the scene. The SoftBody3D code has a simulation_started boolean flag intended to prevent this behavior, but it only works the very first time that the mesh is added to the scene, and is not reset when the mesh is removed from the scene.

Note that these transform bugs appear to occur with both Jolt and Godot physics engines.

Steps to reproduce

Load the attached MRP, and use the "Restore Transform", "Reset Mesh & Transform Out-of-scene", and "Raise Body" buttons to play around with changing the transform on the body.

Minimal reproduction project (MRP)

soft-body-updates.zip

[simpkins]

When using Jolt physics, it seems like the behavior also differs based on whether or not the soft body is sleeping. The rendered object is incorrect in either case, but if the soft body was sleeping it does not get woken up so that physics apply to its new location.
For instance, if you click the "Raise Body" button in the MRP the body will fall downwards due to gravity if it was still awake, but if it had already settled into a sleeping state then it won't fall due to gravity after "Raise Body" is clicked.

Changing JoltSoftBody3D::set_transform() to also call wake_up() appears to fix that problem.

[simpkins]

It looks like the bug is that SoftBody3D disables transform notifications when attempting to reset the transform:

godot/scene/3d/physics/soft_body_3d.cpp

Lines 383 to 386 in e1b4101

	set_notify_transform(false);
	set_as_top_level(true);
	set_transform(Transform3D());
	set_notify_transform(true);

I believe the intent here is to prevent it from receiving notifications about its own change, and processing this in a loop.
However, this also causes the code in the parent VisualInstance3D class to be skipped, so the RenderingServer never gets told about the transform being reset to 0:

godot/scene/3d/visual_instance_3d.cpp

Line 94 in e1b4101

RenderingServer::get_singleton()->instance_set_transform(instance, get_global_transform());

[lawnjelly]

See https://chat.godotengine.org/channel/physics/thread/i9vYNaP4iuQYd427g .

As far as I can make out, SoftBody is using a nasty hack to set the transform, and it only works in the particular situation of setting the soft body up the first time (from the editor).

The soft_body_set_transform() API is currently acting as a relative API. If you call it twice, it will xform the softbody by that amount twice. If you think about it, setting the transform of a gloopy mess is a somewhat undefined operation (because you may get e.g. stray verts affecting centre of mass). The only highly "defined" operation probably is a relative xform, but that isn't easy to set up via the editor's regular systems, hence this hack I believe.

I cover some possible approaches in the chat there, like calculating a centre of mass and allowing an absolute set_transform. But currently the behaviour is crazy (and also broke the assumptions of #105901 ).

I prepared some fixes to softbody after #106863 (which caused some dubious changes to non-interpolated behaviour which I pointed out there), but I didn't make a PR in the end partly because I don't want to end up maintaining soft body (you touch it, you maintain it 😁 ).

I think rather than diving into PRs, we really need a proposal to agree on changing the behaviour of SoftBody to something sensible. It's possible it will have to be compat breaking (as the current behaviour is bonkers).

You were correct to mention _set_use_identity_transform() as a possibility, which I added a while back to handle some similar cases where global xforms were used in the visual server, it was much more sensible than the hacks previously used for some cases like particles, and was necessary for interpolation to work correctly. (toplevel is similarish concept, but they work slightly differently.)

[lawnjelly]

However, this also causes the code in the parent VisualInstance3D class to be skipped, so the RenderingServer never gets told about the transform being reset to 0:

I'm not sure the RenderingServer ever gets told about instance xforms, hence it is always at the origin. But _set_use_identity_transform() is usually a better way of achieving this (it was added after SoftBody).

[simpkins]

I'm not sure the RenderingServer ever gets told about instance xforms

I'm pretty sure VisualInstance3D is telling the RenderingServer about the transform here

For SoftBody3D here, I believe what is happening is that:

1. User code updates the transform, causing the node to add itself to the SceneTree xform_change_list to be notified
2. The scene tree flushes notifications, and uses Object::_notification_forwardv() which will run VisualInstance3D::_notification() first, followed by SoftBody3D::_notification() second.
3. VisualInstance3D therefore runs first, informing the RenderingServer about the real requested transform change.
4. The SoftBody3D notification handler then runs, and resets the transform back to the identity. However, it suppresses notifications for this, causing VisualInstance3D to never see the second update, and never reset the RenderingServer transform back to 0.

As far as I can make out, SoftBody is using a nasty hack to set the transform

I agree that this is a wacky hack. It appears to pre-date the existence of _set_use_identity_transform(), and I agree that updating it to use _set_use_identity_transform() would be a much nicer fix.

I have thoughts about how to make SoftBody3D have a somewhat sane and explainable behavior here. I can go ahead and work on writing up a proposal. At a high level:

1. I do agree that SoftBody3D should use _set_use_identity_transform() to always render at the origin, regardless of its transform.
2. I think that the SoftBody3D transform should only be used to specify the initial starting location of the mesh when it is first added to the scene and converted from its static geometry to a dynamic mesh.
3. If the transform is modified when a soft body is already in the scene, we apply the relative transform between the old and new global transform to the dynamic body. (Alternatively, I'd also be okay if we simply ignored the change, and the new change only took effect the next time set_mesh() was called. A bunch of other soft body properties like pressure, stiffness, etc, also only take effect when the mesh changes.)

Note that I think this behavior would make sense with my PhysicsServer3D API proposal at godotengine/godot-proposals#12670, where I am proposing we deprecate PhysicsServer3D::soft_body_set_mesh(), and replace it with a soft_body_set_settings() API that behaves a bit more like Jolt's SoftBodySharedSettings class. With this API you can create many SoftBody3D instances using the same settings, having different initial transforms for different instances would allow them to be placed at different locations in the world.

With that said, I think SoftBody3D probably needs some other fixes before this can be one. e.g., I think #108061 is important, because otherwise SoftBody3D goes through some slightly weird intermediate states between when the mesh is set and when it actually tells the physics server and then when it actually receives the first vertex updates and has correct mesh data in the right world transform.