Animation Retargeting Woes

[emberlightstudios]

What problem does this solve or what need does it fill?

Animation retargeting is useful for sharing animation clips among various models. Otherwise each model may hold duplicate animations, e.g. a walk cycle for every character in a game is pretty redundant and takes up extra space if it could be shared among all of them. In my case I am constructing the mesh and skeleton in code, so I cannot import the model with animations. Retargeting is the only way to get them animated.

What solution would you like?

I'm having a couple problems trying to get animation retargeting working. Please see pictures for reference.

1. The animation clips appear to have position tracks even after I removed them all in blender. I believe this is the reason the baby model for example is stretched out to the same height as all the others. Or maybe it's just putting all the bones into the animation clip's rest pose rather than the scene model's rest pose before rotating them? I don't know. The joint gizmos look correct when not animated. It would be nice to be able to have a clip which only has rotation tracks and it would just work with all of my humanoid models. In Godot, there is also a property on their skeleton3d node for adjusting position tracks called motion_scale, which scales the position deltas to help with position track issues during retargeting.

2. Bone twist angles seem all wrong. I don't know how godot handles this but I know it was not an issue there. This may also be due to the difference in rest pose between the procedurally constructed skeleton and the animation file glb skeleton.

3. Since joints have to be assigned UUIDs to match up with the animation clip curves from the file, this limits your ability to share animations from multiple files unless all of them use the same skeleton with the same paths, and therefore the same UUIDs. I think we need an intermediary abstract, generic humanoid skeleton to remap imported clips onto, which we can then map our other models onto also. More control over the animation clip import process would be useful.

What alternative(s) have you considered?

None

Additional context

Unanimated: https://i.imghippo.com/files/LsTT21727931519.png
Animatied: https://i.imghippo.com/files/VZ09i1727931405.png

Working on bringing makehuman character creator into bevy

[pcwalton]

I personally have animation retargeting working great in Bevy.

I think we need an intermediary abstract, generic humanoid skeleton to remap imported clips onto, which we can then map our other models onto also.

I'm skeptical of this because it leads to Mecanim (Unity Humanoid) which is not well-loved. I don't see much of a point of having Bevy choose one set of bone names as the "blessed" one when each project can just as well choose one for themselves.

[emberlightstudios]

I personally have animation retargeting working great in Bevy.

Hmm. I'm curious what your rig looks like Are you using the exact same rig, just from different files or are there any differences? Do you have any idea why I'm seeing these artifacts that I'm seeing? Or how I can resolve them?

I'm skeptical of this because it leads to Mecanim (Unity Humanoid) which is not well-loved. I don't see much of a point of having Bevy choose one set of bone names as the "blessed" one when each project can just as well choose one for themselves.

Mecanim is pretty terrible. I agree. Godot and Unreal both have much better retargeting tools, especially unreal which allows per joint adjustments. I used godot's humanoid default skeleton without any of the issues I had in Mecanim. We need at least some way to map joints during animation clip import. There doesn't have to be a "blessed" set of bones. It just speeds things up. But it would be very helpful if the user can define whatever they need.

[pcwalton]

I had problems until I made the rig exactly match the Mixamo one, including things like bone roll (which Mixamo rigs unfortunately use). I agree that it's easy to get wrong. I feel like it may be easiest to experiment out of tree with animation "fixup" crates -- I'm not sure what Godot and Unreal do here.

[emberlightstudios]

There need to be transformations done on the curves to align the rotation values within a new coordinate system at each joint.

[emberlightstudios]

I didn't mean to close the issue. That's annoying.

[brokolja]

Anyone aware of a guide related to Animation retargeting in bevy? I think this is a feature every 3D Engine must have and has for good reason. Nobody wants to store the same animations in every model again and again, this leads to massive filesizes and bad workflows…