Final result after render

[Iliceth]

I have run:

python -m train \
    --gin_configs=configs/zipnerf/360.gin \
    --gin_bindings="Config.data_dir = '${DATA_DIR}'" \
    --gin_bindings="Config.checkpoint_dir = '${CHECKPOINT_DIR}'"

python -m eval \
    --gin_configs=configs/zipnerf/360.gin \
    --gin_bindings="Config.data_dir = '${DATA_DIR}'" \
    --gin_bindings="Config.checkpoint_dir = '${CHECKPOINT_DIR}'"

python -m render \
    --gin_configs=configs/zipnerf/360.gin \
    --gin_bindings="Config.data_dir = '${DATA_DIR}'" \
    --gin_bindings="Config.checkpoint_dir = '${CHECKPOINT_DIR}'" \
    --gin_bindings="Config.render_dir = '${CHECKPOINT_DIR}/render/'" \
    --gin_bindings="Config.render_path = True" \
    --gin_bindings="Config.render_path_frames = 240" \
    --gin_bindings="Config.render_video_fps = 30"

python scripts/zipnerf/generate_tables_360.py 

Now I have 4 video's in the render folder that was created.

The output says there are 2 videos missing as certain files were not created during the process.

I0122 20:01:14.943336 140617853252672 render.py:110] Rendered in 11.274s
I0122 20:01:15.526105 140617853252672 render.py:138] Creating videos.
I0122 20:01:15.769567 140617853252672 videos_utils.py:95] Video shape is (840, 1297)
I0122 20:01:15.769668 140617853252672 videos_utils.py:134] Making video /home/user/home/user/Documents/workspace/camp_zipnerf/checkpoints/360_v2_garden_1297x840//render/path_renders_step_200000/videos/360_v2_garden_1297x840_checkpoints_path_renders_step_200000_color.mp4...
I0122 20:01:29.177273 140617853252672 videos_utils.py:132] Images missing for tag normals
I0122 20:01:29.177362 140617853252672 videos_utils.py:132] Images missing for tag normals_rectified
I0122 20:01:29.177408 140617853252672 videos_utils.py:134] Making video /home/user/home/user/Documents/workspace/camp_zipnerf/checkpoints/360_v2_garden_1297x840//render/path_renders_step_200000/videos/360_v2_garden_1297x840_checkpoints_path_renders_step_200000_acc.mp4...
I0122 20:01:32.633051 140617853252672 videos_utils.py:134] Making video /home/user/home/user/Documents/workspace/camp_zipnerf/checkpoints/360_v2_garden_1297x840//render/path_renders_step_200000/videos/360_v2_garden_1297x840_checkpoints_path_renders_step_200000_distance_mean.mp4...
I0122 20:01:49.162443 140617853252672 videos_utils.py:134] Making video /home/user/home/user/Documents/workspace/camp_zipnerf/checkpoints/360_v2_garden_1297x840//render/path_renders_step_200000/videos/360_v2_garden_1297x840_checkpoints_path_renders_step_200000_distance_median.mp4...

All 4 videos are just blank back screens for a few seconds, files seem corrupt according to Github as you can see below. The images in the render folder show stuff, but also probably not as desired. I don't know what went wrong. The dataset and colmap output has been used thoroughly with other repo's without issues.

360_v2_garden_1297x840_checkpoints_path_renders_step_200000_distance_mean.mp4

[jonbarron]

My guess would be that your camera poses are incorrect, or are in a different format from what the code expects.

[jonbarron]

Ignore that warning about normals not being rendered, that is to be expected.

[Iliceth]

My guess would be that your camera poses are incorrect, or are in a different format from what the code expects.

These are the cameras Colmap gave me and that is the set I used for this test.

[jonbarron]

Yep those look like good cameras! Weird.

[Iliceth]

I can gladly say I found the culprit, an error on my side: Config.factor = 4 in 360.gin. There was a mismatch as I had used the full size images for colmap. So setting 4 to 1 did the trick.

Despite the images in the path_renders_step_200000-folder look perfect as above, all 4 video files are still plain black when played back with VLC.

[ichsan2895]

really sharp and crisp result. 👍

[jonbarron]

That looks right to me!

[jonbarron]

I'd guess your issue with the videos is in your FFMPEG or mediapy installation.