Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify data loaders and eval scripts; allow arbitrary image size #57

Merged
merged 13 commits into from
Dec 2, 2021

Conversation

anmatako
Copy link

@FangjinhuaWang in the interest of expedience I'm including all remaining changes in a single PR. Outlining here the main changes in the repo:

  • Unified the data loaders so now there's only one MVSDataset needed. I kept dtu_yao.py for backwards compatibility so the training can still work in the existing way. That's just legacy now and could be removed in subsequent PR.
  • Intrinsics and extrinsics are provided separately (not as a projection matrix) from the data loader and only for the desired image size. The parameters needed for each stage are now calculated internally in the module. This simplifies the loader and also makes it easier to handle arbitrary image sizes.
  • Module can now work for any image size (not just multiples of 8). Inside the module the image is stretched (slightly) to the nearest multiple of 8 and then after the PatchMatchNet process is complete the output maps are brought back to the original image size. This also means that the most recent change in data_io can be reverted and use a single max_dim instead of tuple.
  • As a consequence of handling arbitrary image sizes the model can now also work for multi-camera setups with different intrinsics and/or orientation.
  • Unified the evaluation scripts to use the new data loader so there's no need for a new eval for each of eth, dtu, tanks, and custom
  • Cleanup and updates of the train.py and eval.py scripts to use unified data loader. Existing training script renamed to train_dtu.py for backwards compatibility; still uses the dtu_yao data loader. This could be removed in subsequent PR.
  • Added support scripts to convert datasets and make them compatible with unified data loader (convert_dtu_dataset.py and convert_eth_dataset.py); need to be applied once in the DTU and ETH3D data before using the new train and eval scripts. The converted datasets could replace the existing ones distributed with the repo
  • Added support script to export PatchMatchNet output as a colmap workspace to be used for fusion (colmap_output.py). Also did minor updates in colmap_input.py script.
  • Updated the list text files (used in train and eval) to be compatible with the new format of the data loader.
  • Updated the checkpoints that are now one for parameters only (checkpoints/params_000007.ckpt used in Python) and one for the TorchScript module (checkpoints/module_000007.pt used in colmap and python)

Note that all these changes have been tested in both training and evaluation with the DTU dataset and also the evaluation has been tested with custom colmap datasets with single and multi-camera setups.

@FangjinhuaWang
Copy link
Owner

@anmatako Thanks!

@FangjinhuaWang FangjinhuaWang merged commit e7ecbdd into FangjinhuaWang:main Dec 2, 2021
@anmatako anmatako mentioned this pull request Dec 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants