about test on co3d with pre-trained model

[boxuLibrary]

Hi, I tested the pre-trained model on the co3d dataset. However, the results seem very bad. I checked 1: the intrinsic and extrinsic parameters of the input with the epipolar model. 2 I checked the reshaped images for 256 * 256. 3: I adjusted the depth of the near and far carefully. I wonder if is it because of the generalizability of the pre-trained model？ Thank you so much.

[boxuLibrary]

More rendered pic can be found：

[donydchen]

Hi @boxuLibrary, I think it's probably because the baseline between the input views is too wide. When training on RE10K, we assume there are enough overlaps between the input source views, which we achieve by constraining the frame distance.

Below is a typical example of the overlap between the RE10K input views. The second column shows the regions that overlap with the other input view.

You can test on two other input views with larger overlaps, similar to the one we chose for the DTU testing. That should work better.

You can also visualize the overlaps between the input by using the code snippets from the pixelSplat project at here.

Still, these are interesting findings, although I do expect that the released model might not perform well on object-centric scenes since it is trained on RE10K. I might also find time to look into the CO3D dataset, but no guarantee...