Implement Retarget feature in the importer

[TokageItLab]

Describe the project you are working on

Godot Animation System for 3D Model

Describe the problem or limitation you are having in your project

It is difficult to share animations across multiple models.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

It greatly improves the workflow of developing games with assets such as character models and animations.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Retargeting was discussed by #3379. As a result, I implemented godotengine/godot#56902 as a prototype, but the discussion led to the decision that the importer should do the most of the work. Of course, I agree with that.

But, in that prototype, I had implemented Global/Local(RestRelative)/Absolute animations, each of which could be blended, but such real-time retargeting was rejected by @reduz as it being over-tech for most users. I did not agree with it, but as a compromise, I agreed with him that it could be easily implemented as a custom-module. This means that some APIs need to be modularized, exposed, or virtualized so that the animation applying process can be overridden.

Based on the above, I will add an implementation for retargeting in the importer as shown in the following mock-up image:

First of all, prerequisite is the need to unify the roles of the bones. This is accomplished as a bone mapping.

The retargeting process must then be performed after that. It is important to understand that retargeting is just a name given to multiple processes as workflows. In other words, the processes within a retarget can be completely separated into several processes. And I believe that it makes sense to modularize them as plug-ins.

With modularization, when more advanced processing is needed, add-on features such as custom modules can allow it to be highly integrated with the built-in processes in the importer. This allows us to create plug-ins outside of core such as "do real-time retargeting", "reproduce RestRelative animations like Godot 3" and "import original bone information before renaming or re-resting as dictionary".

So this proposal can be separated into several PRs, as follows:

---

(Basic feature)
1. Implement Bone Mapper Done in godotengine/godot#61610
2. Implement Bone Renamer Done in godotengine/godot#61610
3. Implement Rest Pose Fixer Done in godotengine/godot#62939
4. Implement Position Track Optimizer (Adjust for root motion) Done in godotengine/godot#63138

---

(Advance feature)
5. Change Animation API to expose the processes for custom-modules (Make RealtimeRetarget Extention as custom-module) API has been changed in godotengine/godot#63138
6. Improve Mapping GUI (Add Pulldown "Profile Bone" Selector & Highlight Bone Position In Scene) Added BoneTreeDialog in godotengine/godot#63854
7. Implement Mapping Estimation Done in godotengine/godot#63854

---

My plan is to try to implement 1 and 2 at the same time first, and if that goes well, we can work on the other PRs as well.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No.

Is there a reason why this should be core and not an add-on in the asset library?

A simple retargeting system is needed by most light users. It also needs to be changed to allow the Animation process API to be exposed for advanced users.

[SaracenOne]

I like this, it looks very sensible and should cover most of what people should need out of import-time retargeting.

[reduz]

This looks really good to me, I suppose there are a couple of things that come to mind:

• Option to keep some existing bones that are children of retargeted bones (like for tails, hair, etc).
• For T-Posing I think it should be selectable whether you want a standard T-Pose, or use one of the animations for it (or an external resource i guess?). At least I recall this was something you needed.

[TokageItLab]

Option to keep some existing bones that are children of retargeted bones (like for tails, hair, etc).

It was planned that no processing would be done on unmapped bones, meaning that the animation would be imported as is. However indeed, it would be better to be able to choose whether they are imported or not, to reduce the size of the sharable animations.

BTW, since errors are always printed for tracks that are not currently found when playing animation, it may be necessary to reduce the output warning.

For T-Posing I think it should be selectable whether you want a standard T-Pose, or use one of the animations for it (or an external resource i guess?). At least I recall this was something you needed.

@reduz I was thinking that reference pose is a resource that can be output from SkeletonEditor or anyway, would it be better to use gltf?

[TokageItLab]

I was thinking that reference pose is a resource that can be output from SkeletonEditor or anyway, would it be better to use gltf?

In that case, I think we need something to take out only the poses from gltf, since we need to avoid using an importer in the importer, so we will discuss this with @fire when we implement the T-poser.

[Ansraer]

Could you explain a bit how this proposal differs from #4296?

My naive understanding is that AnimationLibraries should do the trick as long as both the animation source and target use the exact same skeleton, while this proposal would be useful when one of the skeletons has additional bones or the target and source have different sizes.

Assuming my assumption is correct, would the animation retarget imported create a new AnimationLibrary I can simply apply to my target model?

[Zireael07]

My naive understanding is that AnimationLibraries should do the trick as long as both the animation source and target use the exact same skeleton, while this proposal would be useful when one of the skeletons has additional bones or the target and source have different sizes.

Not just different sizes. I have several meshes from the same author that were in different packs, so the bones are differently named and this is enough to confuse Godot.

[TokageItLab]

The purpose of the AnimationLibrary implementation is to import animations separately from the model; even with using the AnimationLibrary, it is not possible to share animations between models with different bone names and rests. The retarget feature will solve that.

[Zireael07]

@WiredDreamcaster: This is a proposal repo. The previous PR was linked in the OP. This proposal is yet to be implemented.

[Calinou]

@WiredDreamcaster Animation retargeting is already being worked on as a built-in feature for Godot 4.0. It relies on backwards-incompatible changes, so it can't be backported to Godot 3.x though.