How to Parsing SMPL-X Parameters in ActorsHQ Dataset?

[yiyuzhuang]

Dear Authors,

I hope you are doing well. I appreciate your work on the HumanRF project, and I am currently experimenting with the ActorsHQ dataset. However, I am facing some difficulties in parsing the SMPL-X parameters and obtaining an unreasonable posed mesh. Please check the figures below.

Parameters: The provided parameters have a 'poses' shape of [1, 87]. I am parsing it into ['body_poses' (indices [0:63]), 'right_hand_pose' (indices [63:75]), and 'left_hand_pose' (indices [75:87])].

Implementation: I first parse the 'poses' of the parameters. For the other parameters, I use the following mapping: {'shapes' to 'betas', 'expression' to 'expression', 'Rh' to 'global_orient', 'Th' to 'translation'}. For the missing values like ["jaw_pose", "leye_pose", "reye_pose"], I fill them with zero vectors. Then, I forward the parsed results to the SMPLX model of 'SMPLX_MALE.npz' using the "smplx" Python module.

I would appreciate your guidance on the following questions:

1. Are there any issues or pitfalls in my implementation? If so, can you suggest solutions or improvements?
2. Are the provided SMPL-X parameters correct? Could you provide some explanation to the data format of the saved parameters? Or could you provide a script to parse them?
3. Have you estimated the missing values? Could you provide them? I noticed some expression changing between different frames.

[qianqiuzzz]

@yiyuzhuang 我注意到您展示的图有人物的mesh，请问这个mesh是数据集提供的msehes.abc.xz文件吗，您是怎么解压这个压缩包的呢

[Pbb-land]

@yiyuzhuang @isikmustafa Hi, I was having the same problem, have you been able to solve it?

[Zielon]

Probably your problem is that the poses vector in the JSON file has the first three components set to zero. They usually represent global rotation, however, in this case, it is put to the Rh field instead.

[
    {
        "id": 0,
        "Rh": [
          [0.04361098, 0.07433257, -0.02178280]
        ],
        "Th": [
          [0.00391468, 1.32414436, 0.03250297]
        ],
        "poses": [
          [**0.00000000, 0.00000000, 0.00000000**, -0.11908470, -0.03610654, 0.05601613, ...]
        ],
        "expression": [
          [-0.00000144, -0.00000581, 0.00000272, -0.00000236, ...]
        ],
        "shapes": [
          [-0.32091987, 0.09722190, 0.04891836, ...]
        ]
    }
]

Then depending on which implementation of SMPL you are using you need to take care of the correct mapping poses and Rh. The fix np.concatenate([smplx["Rh"], smplx["poses"][3:]], axis=-1) passed as pose can help.

[chunjins]

Where do you find those SMPL parameters?

[MontaEllis]

Where is the json file of SMPL? I can't find it in the downloaded files.

[chunjins]

@MontaEllis Have you found the JSON files?

[MontaEllis]

@MontaEllis Have you found the JSON files?

No.

[chunjins]

@MontaEllis Have you found the JSON files?

No.

I found that the current version includes the SMPL fittings. The download link can be found in the 'actorshq_access_xx.yaml' file.

[MontaEllis]

@MontaEllis Have you found the JSON files?

No.

I found that the current version includes the SMPL fittings. The download link can be found in the 'actorshq_access_xx.yaml' file.

Can you view these samples? Do they correspond to the 3D scan?

[gwang-kim]

Hi @Zielon, I'm also having problems using SMPLx paramters. Body poses are weird. Would you provide a sample code for utilizing smplx?

---

I found that they are easymocap-smplx params, not original smplx params

[Zielon]

Using class SMPLlayer(nn.Module):from EasyMocap:

poses = batch["poses"]
shapes = batch["shapes"]
expression = batch["expression"]
Rh = batch["Rh"]
Th = batch["Th"]

self.lbs_module(poses=poses, shapes=shapes, Rh=Rh, Th=Th, expression=expression, return_verts=False)

It also depends on whether the root rotation was used in the kinematic chain instead of applied as global rotation after LBS. This is the case in EasyMocap (ActorsHQ provides poses obtained using that software). Thus, the root rotation is zero, and Rh is used after LBS. You always need to check which implementation was used for the tracking.

[gwang-kim]

from easymocap.smplmodel import load_model

body_model = load_model(gender='neutral', model_type='smplx', model_path='path/to/smplx', use_cuda=True)
smplx_path = 'path/to/smplx_param.json'

with open(smplx_path) as json_file:
    smplx_param = json.load(json_file)[0]
for key in ['Rh', 'Th', 'poses', 'shapes']:
    smplx_param[key] = np.array(smplx_param[key])

joints = body_model(return_verts=False, return_tensor=True, return_smpl_joints=False, **smplx_param)

Thanks for your answer, @Zielon.
I'm using the code like this for getting joint. Is this have any problem related to the your comment?