Add physics step interpolation in 3D (implemented in 2D since 4.3)

[reduz]

Describe the project you are working on

Godot

Describe the problem or limitation you are having in your project

The following are common scenarios where users often find trouble in Godot:

• When using monitors with high refresh rate, code that depends on physics remains fixed, does not interpolate. This results in these to have less frame rate and, in the worst case, may lead to jitter.
• When working on multiplayer games, in order to send less information, games often run at a lower tick rate (say 10 or 15fps) in order to maximize bandwidth. When using dedicated server hosting, this also allows to reduce the game CPU usage. This requires a more coarse interpolation.
• In mobile games, running at a lower tick rate also helps reduce CPU usage, but this again requires better interpolation.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Interpolation of code that runs in the physics step is generally desired, but implementing this is not easy. The main limitation is that, when using interpolation, one must discern between motion and teleporting. Failure to specify this will result in the wrong behavior.

Additionally, "automating" things too much (make everything interpolation by default) can also be troublesome and lead to unexpected results.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

After through analysis of the use cases, it is clear that the places where actual interpolation is desired are as follows:

• Rigid Bodies. Their motion (when coming from the physics server) is always interpolated.
• KinematicBody move_and_slide. When using this function, physics interpolation is always expected to happen.
• Physics objects moved via AnimationPlayer (as an example, a moving platform), requires animation to operate in the physics tick.
• Character animation with Root Motion. This is a bit of a more complex case, because animation has no problem being processed in the regular idle tick, but the physics tick is currently required for root motion.
• For some reason, moving something via code in the physics tick that may require interpolation (though this is rare).

With this in mind, we can probably do some engine changes to make this process more or less transparent to the use. This would begin by implementing two new methods:

Node3D::set_transform_interpolated(const Transform& p_transform)
CanvasItem::set_transform_interpolated(const Transform2D& p_transform)

(we can also add translate interpolate, rotate interpolate and scale interpolate, which would call this method).

Calling these methods outside of the physics ticks will result in an error (in case user misunderstand what these are for).

In any case, calling these will work not only on the current node but also on the sub-tree, so motion of children nodes will also be interpolated. Basically, all these will call set_transform for RenderingServer 3D and 2D instances with an extra flag telling them that its interpolated, then the RenderingServer will internally perform the interpolation. Additionally, Camera3D and Camera2D will have this method too in the rendering server, since their motion can be interpolated.

The idea is that move_and_slide, RigidBody, etc. will internally all use this method so by default the user does not need to do anything and everything will just work. Camera2D and Camera3D are generally children of the character in Godot, so they will naturally know when their transform was interpolated, for which this requires no changes from the user.

So, yes, the idea is that this is implemented as transparent and possible, so existing games don't need to call the interpolated transform, and games just work, and complexity to develop by having to discern between motion and teleport does not happen.

FAQ

Q: So, I can interpolate manually by calling set_interpolated_transform, but I want to teleport an object, how do I do this?
A: By default nothing is interpolated except the cases mentioned above, so setting position manually is equivalent to teleporting.

Q: Does this solve jitter in high refresh rate monitors?
A: While this mostly works now, this new implementation will out of the box improve the quality and physics motion in high refresh rate monitors.

Q: Does using this have known shortcomings?
A: Traditionally, the physics steps happens left often than the refresh, which makes it have a small amount of latency, so using the new position as target interpolation will add a physics frame of latency. This is unavoidable. The alternative would be to instead set a linear and angular velocity, but when running your game at low physics FPS (for networking or saving mobile GPU), overshooting becomes visible, so its not viable in this case.

Q: Interpolation is done in the RenderingServer, why not in the node?
A:: This is desired for performance, its much cheaper to do the interpolation in the rendering server. The only shortcoming is that, in 3D, interpolation happens in global space rather in local space, so objects rotating around a pivot will introduce some motion aliasing (mostly noticeable in 3D games with very lock physics FPS. In 2D pivot information is retained in the server, so this is not a problem). That said, this should not be a big problem, as when running at low physics tick rate, motion aliasing is present almost everywhere. If you use low physics FPS you should be aware of its effects.

Q: Will cubic/hermite/etc interpolation be supported?
A: These interpolations are troublesome in many ways. Mostly because when rotating, velocity is not constant (objects can accelerate/decelerate), and trajectories can easily overshoot (go past, then go back). Additionally as an extra sample is needed, an extra physics frame of lag is added. So, in general this will most likely not be supported.

Q: Will I need to change my game too much to take advantage of it?
A: I believe with the proposed changes, most games should be able to take advantage of this out of the box.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Is there a reason why this should be core and not an add-on in the asset library?

There are some scripts to do this, but its clearly not ideal, needs to be core.

[nonunknown]

@lawnjelly made a solution for this! https://github.com/lawnjelly/smoothing-addon the only downside of his plugin, is that you need to reestructure the way you compose your scene, maybe a improved version of it, whithout requiring changing the scene structure? this will make easy for porting old versions of the game!

[Seubmarine]

Could this pr solve #961 or be able to add this feature more easily ?

[reduz]

@nonunknown this proposal is so you can achieve the same without doing absolutely anything.

[reduz]

@Seubmarine At some point I think I fixed this specifically, so I am surprised it remains broken. Will have to give it a check. This is unrelated, though.

[GarbajYT]

I usually deal with physics jitter with a just a few lines of code https://github.com/GarbajYT/godot_updated_fps_controller/blob/main/FPS_controller_3.3/FPS.gd but I think there is some value in having an "it just works" button

[theraot]

What about move_and_slide_with_snap and move_and_collide? Which are not mentioned at the time of writing.

I guess it makes sense that move_and_slide_with_snap also interpolates by default the same as move_and_slide.

Will move_and_collide be considered a teleport? Considering that it is already different from move_and_slide in that it does not take a velocity. That would mean remembering to set the interpolation when using it…

Which brings another question: Do I need to know the final transform before hand to set the interpolation?

If that is the case, I'd appreciate an improved test_move that tells me the distance it can move, not only if it collides. So I can call test_move to get the distance and then set the interpolation for that distance.

[AndreaCatania]

This seems great, and maybe we will be able to introduce some classes to customize it and allow much sophisticated interpolation mechanism, later on.

Regarding the interpolation, once the transform is set using set_global_transform_interpolated / set_transform_interpolated, if the function get_global_transform is executed to that node or one of the child nodes, what transform it returns? The interpolated one or the "static" one?
This is important for networking, that functions should always return the static transform, as is now.

I like the idea to put this inside the renderer directly, this will speedup the interpolation a lot especially because we will likely need a class that calculates the interpolation speed, and if this is done directly on the rendering side we can do it once for all the nodes, which is great!
The above class is needed because the rendering framerate is dynamic and its oscillations will introduce some delays. So we need a class that sometimes speedup and slowdown the interpolation depending on the frame rate, so the interpolation is always smooth and uptodate.
Here, I wrote something #671 (comment), for sure you can do better and more universal, though here the link anyway. The class is: FrameInterpolator.

Nice, I like this feature and how it's turning out.

[SaracenOne]

This seems like a good proposal. We already make use of Lawnjelly's own interpolated node addon, and I think he also wanted to provide a more native implementation of the same functionality. It makes a ton of sense particularly for VR where you have wildly varying refresh rates. The latency is unfortunate, but feels like a necessary tradeoff.

[reduz]

@AndreaCatania

If the function get_global_transform is executed to that node or one of the child nodes, what transform it returns? The interpolated one or the "static" one?

Indeed, it should return the static one, the scene nodes will not do anything interpolation related other than mark the rendering server that the transform includes interpolation.

[jknightdoeswork]

https://github.com/jknightdoeswork/goodoh/blob/master/nodes/InterpolatedTransform.gd

Above is the Script I have used to interpolate transforms. In my opinion, it is simpler than Lawn Kelly's add-on.

An alternative API for this proposal is to have a boolean on a node (is_interpolated) which, when enabled causes a node to be rendered between its current position and it's last renderered position.

This should default to true for RigidBodies, and default to false otherwise.

In this design, you do not need to call set_interpolate_transform with a transform. The destination transform is always the current transform, and the other transform is always the last rendered transform.

This way, the same API is used for setting the transform on an interpolated or non interpolated node. Turning on and off interpolation is done completely by toggleing the is_interpolated flag.

A function could be added which disables interpolation for a single frame (reset_interpolation) which can be used for teleports.

To summarise my proposed API is this:

Spatial.is_interpolated(bool)
Spatial.reset_interpolation()

(And similar for 2D)

[jknightdoeswork]

In our networking system, we also produce a different interpolation fraction, as we are not using physics_process to do our fixed update. Perhaps a Engine.set_interpolation_fraction(float) can be used to override this value. Additionally, if Engine.set_interpolation_fraction(-1) could restore it to use the automatic, physics_process based fraction.

This would be very customizable and extensible.

[Calinou]

In our networking system, we also produce a different interpolation fraction, as we are not using physics_process to do our fixed update.

See also #2020.