Question about depth scale and confidence value

[bring728]

hello. First of all, thank you for sharing this wonderful research result.

I have a Synthetic Dataset with Ground Truth Depth.

First of all, my image is 640, 480, but in cds-mvsnet, H and W must be a multiple of 64, so I doubled the image first and ran cds-mvsnet to get the Depth value and compare it with GT.

However, the depth value seemed to be twice as small as that of the GT. My guess is that you divide the intrinsic matrix by 4 in your code, and that seems to be what caused this to happen.

In other words, the image was doubled, but the intrinsic became 4x smaller and the depth seemed to be 2x smaller.

So I multiplied the depth value I got by 2. When I want to get the actual depth value, is my method correct?

The problem was that when I compared the depth value obtained in that way with the GT, there was always an error value of about 1 to 2 meters on average.

I'm not familiar with the depth estimation field, so is this error trivial? Can't I get a more accurate depth value? I think an error of 1 meter is quite large.

For reference, I use depth estimation with 9 images as input, and the layout is a 3 X 3 Array, all facing the same direction.

[TruongKhang]

Hiiii @bring728 ,

First of all, my image is 640, 480, but in cds-mvsnet, H and W must be a multiple of 64, so I doubled the image first and ran cds-mvsnet to get the Depth value and compare it with GT

You can run with your current size by using the option --no_refinement similar to my script for Tanks&Temples. In this case, H and W only need to be a multiple of 32.
Moreover, when you double your image size, the intrinsic matrix should be doubled as well. After that, you don't need to change depth values, or my code, just use your doubled-size image and doubled-scale intrinsic matrix as inputs.

The problem was that when I compared the depth value obtained in that way with the GT, there was always an error value of about 1 to 2 meters on average.

About the depth error, do you prepare your dataset with the same format as DTU or Tanks&Temples provided by Yao Yao? Make sure about your data preparation step. I ran my code with some indoor datasets as in the demo video, the results are acceptable.
Also, when the range value of your depth is large, some tips for increasing performance include

1. increase the number of depth hypothesis planes with --numdepth
2. change the number of input views (--numview) depending on the overlap between your multi-view input images

[bring728]

hello! Thank you for your quick reply. I solved the problem. The cause of the problem, ironically, seems to be the depth scale of the GT. Your code worked fine without any problems. Thanks to your answer, I also found that depth values are more accurate without refinement and resizing. Thanks again!