convert optical flow and depth back to world-units

[DelinQu]

Hi!

Thanks so much for your awesome project! It really helps me! I met a problem with how to convert the optical flow and depth files in .png/tffi format to world units. I followed the annotations-and-format tutorial and run a demo with this script:

docker run --rm --interactive \
  --user $(id -u):$(id -g)    \
  --volume "$(pwd):/kubric"   \
  kubricdockerhub/kubruntu    \
  /usr/bin/python3 challenges/movi/movi_def_worker.py \
  --camera=linear_movement
  --max_motion_blur=2.0

The generation result include:

• rgba image: rgba_0000.png
• depth image: depth_0000.tiff
• optical flow image: forward/backward_flow_0000.png
• object_coordinated image: object_coordinated_0000.png
• segmentation images: segmentation_0000.png
• some JSON files: data_range.json, events.json, metadata.json

By following the annotations-and-format tutorial, we need accquire some min and max values through the keys forward_flow_range, backward_flow_range and depth_range. I guess the JSON file must contain all the dictionaries we need:

• forward_flow: (s, h, w, 2) [uint16]
  Forward optical flow in the form (delta_row, delta_column). The values are stored as uint16. To convert them back to pixels use:
  

(Here I have a question: are the 'backward_flow': {'max': 18.546600341796875, 'min': -5.85162353515625} the global min-max values across all the optical flow image or a single optical flow image? Scale all the optical images by the same min-max value must be a mistake if the min-max values are about the single one).

• "depth": (s, h, w, 1) [uint16]
  Distance of each pixel from the centre of the camera. (Note this is different from the z-value sometimes used, which measures the distance to the camera plane.)

However, I found that the values are not stored as uint16 but float32 ! and all the JSON files don't contain the key ["depth_range"]. So, does this mean that the depth_0000.tiff already is the real depth in the world without the need to rescale back?

Could you please share with me some suggestions?

Thanks!

---

The followings are the images and Jupiter file:

[files]

[Qwlouse]

Hey,

• flow value ranges are indeed global (over a single scene/video) so can be reused for every frame. Note though, that the units here are pixels and not meters or other scene-units. So a value of 18.5 indicates a movement by 18.5 pixels.
• the tiff format can store floats directly so no need to post-process. So the depth values are in the right range already, and are measured in distance (so meters or whatever scene-units are used).

[DelinQu]

Many thanks!