How do I render a MotionFix npy sample file into an mp4 file?

[JK3WN]

I was able to get some npy sample files out of the trained ckpt file, but I have no idea how to visualize/render these into a video.
It looked like your team was able to make a video out of the sample, so I want to know how your team did this.
Is there an easy way or an easy tool to make a video out of it?
Since I am new to this field, if you could explain the process in an easy way, it would be appreciated.

[atnikos]

Hello, I will release a demo to do this. But here is a standalone script with some instructions. Can you try and tell me?

from src.render.mesh_viz import render_motion
from src.model.utils.tools import pack_to_render
from src.render.video import get_offscreen_renderer
import os 

os.system("Xvfb :12 -screen 1 640x480x24 &")
os.environ['DISPLAY'] = ":12"

# GET A RENDERER
r = get_offscreen_renderer(smpl_p) # this path if you followed my setup should be data/body_models
# Load the npy and get the translation aka trans_can and the rotations aka rots_can
# CREATE A DICT TO GIVE TO RENDERING FUNCTION
smpl_params = pack_to_render(trans=trans_can, rots=rots_can, pose_repr='aa')
# RENDER A MOTION AND SAVE VIDEO!
render_motion(r, smpl_params, pose_repr='aa',
                               filename='/some/path/you/want')

# If you want to render them overlaid motion1 motion2 you can do:
smpl_params1 = pack_to_render(trans=trans_can1, rots=rots_can1, pose_repr='aa')
smpl_params2 = pack_to_render(trans=trans_can2, rots=rots_can2, pose_repr='aa')

# RENDER A MOTION AND SAVE VIDEO!
render_motion(r, [smpl_params1, smpl_params2], pose_repr='aa', colors = [color1, color2]
                                filename='/some/path/you/want', text_for_vid='if you want to put some text')

# to give the correct colors look at src/utils/art_utils.py

[JK3WN]

After typing
python3.10 -m pip install aitviewer

I tried to run the code, but it shows the error above. I'm pretty sure I'm forgetting some arguments.
Can you show the example about how you're supposed to use this code?

[JK3WN]

Update to the situation:
Figured out I was installing aitviewer in a wrong way, so I installed using
pip install aitviewer
in another conda environment.
Still got the same error, so I removed line 25 and 26, which is
render_motion(r, [smpl_params1, smpl_params2], pose_repr='aa', colors = [color1, color2]
filename='/some/path/you/want', text_for_vid='if you want to put some text')
Now it shows this error.

[atnikos]

@JK3WN can you try this?
python visualize_sample.py --path experiments/tmed/3way_steps_300_motionfix_noise_last/ld_txt-2.0_ld_mot-2.0/000136.npy
I just pushed a dirty script. Feel free to adapt it and iterate over multiple ids (000136.npy is one saved sample).

[atnikos]

I hope this fixes the issue and helps you render!
@JK3WN @zhangjingze21 lmk.

[FieldFarmer]

Update to the situation: Figured out I was installing aitviewer in a wrong way, so I installed using pip install aitviewer in another conda environment. Still got the same error, so I removed line 25 and 26, which is render_motion(r, [smpl_params1, smpl_params2], pose_repr='aa', colors = [color1, color2] filename='/some/path/you/want', text_for_vid='if you want to put some text') Now it shows this error.

Hello, I met the same problem as you, may I know how you solved it？

[atnikos]

Hey @FieldFarmer ,
can you check my attached script? There are some environment variable you need to check that are included in the visualize sample script. Can you check that?

Best,
Nikos