Skip to content

Commandline running process inside Docker #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
0db6ab0
add Dockerfile and confirm docker build successfully finished.
MasahiroOgawa Jun 3, 2025
f215ce8
add docker compose file and confimed it works well
MasahiroOgawa Jun 3, 2025
db084af
fix asset file not found error
MasahiroOgawa Jun 3, 2025
a3f9001
update readme to use docker compose
MasahiroOgawa Jun 3, 2025
55fbbde
add command line EDGS python func, and let Dockerfile can choose grad…
MasahiroOgawa Jun 3, 2025
56aed12
fix to runnable
MasahiroOgawa Jun 3, 2025
3fd1a7c
debug fit_model_to_scene_full.py by creating util functions.
MasahiroOgawa Jun 3, 2025
9fc32c2
fix Namespace object has no attribute num_ref_views error
MasahiroOgawa Jun 4, 2025
6d7a5c0
fix error comes from wrong argument name gradio_obj
MasahiroOgawa Jun 6, 2025
c8f1af8
move script files to script/
MasahiroOgawa Jun 6, 2025
cb17a25
fix import in gradio_demo.py
MasahiroOgawa Jun 6, 2025
6f45cef
fix process colmap error
MasahiroOgawa Jun 6, 2025
fa2feac
fix relative directory in gradle_demo.py
MasahiroOgawa Jun 6, 2025
88b356a
debug gradle_demoo to pass accessible copied result.
MasahiroOgawa Jun 6, 2025
4b6bade
fix process_input() to process_input_for_colmap()
MasahiroOgawa Jun 6, 2025
dfd3bb4
fix using abs path to be able to run fit_model_to_scene_full.py scrip…
MasahiroOgawa Jun 6, 2025
ebf2590
let it be able to open jupyter notebook from docker
MasahiroOgawa Jun 10, 2025
221d8b9
let it be able to run A,B,C options in README
MasahiroOgawa Jun 10, 2025
d66a1ba
update config to change name
MasahiroOgawa Jun 17, 2025
5eac9d5
update README to run option A
MasahiroOgawa Jun 17, 2025
de7b415
revert config/train
MasahiroOgawa Jun 17, 2025
b120ed9
fix readme optionC
MasahiroOgawa Jun 17, 2025
c08c9e4
disable wandb if config/train.yaml specify it.
MasahiroOgawa Jun 17, 2025
ca39f9b
fix docker-compose config to configs
MasahiroOgawa Jun 18, 2025
cd4c30e
fix No such file or directory error for output models
MasahiroOgawa Jun 18, 2025
f64d37f
fix could not recongnize scene type error, by adding source_path for …
MasahiroOgawa Jun 18, 2025
6fa2dde
fix You must call wandb.init() before wandb.log error by setting logw…
MasahiroOgawa Jun 18, 2025
94e9db7
fix no cuda gpus are available error
MasahiroOgawa Jun 18, 2025
16526eb
remove wandb edit part from readme
MasahiroOgawa Jun 19, 2025
1019c23
remove unnecessary parameter change inside python file.
MasahiroOgawa Jun 19, 2025
da1317d
set similar iteration with gradio_demo to fit_model_to_scene_full.py
MasahiroOgawa Jun 19, 2025
9ab2a76
add same with gradio demo setting option
MasahiroOgawa Jun 19, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Ignore the following files and directories when building the Docker image
*.pyc
__pycache__/
*.ipynb_checkpoints
*.log
*.csv
*.tsv
*.h5
*.pth
*.pt
*.zip
*.tar.gz
*.egg-info/
dist/
build/
.env
venv/
.env.local
*.DS_Store
*.egg
*.whl
*.pkl
*.json
*.yaml
*.yml
submodules/
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,6 @@ dmypy.json
# Pyre type checker
.pyre/
learnableearthparser/fast_sampler/_sampler.c

# data
data/
52 changes: 52 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
FROM nvidia/cuda:12.1.1-devel-ubuntu22.04

# Set the working directory
WORKDIR /EDGS

# Install system dependencies first, including git, build-essential, and cmake
RUN apt-get update && apt-get install -y \
git \
wget \
build-essential \
cmake \
ninja-build \
libgl1-mesa-glx \
libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*

# Copy only essential files for cloning submodules first (e.g., .gitmodules)
# Or, if submodules are public, you might not need to copy anything specific for this step
# For simplicity, we'll copy everything, but this could be optimized
COPY . .

# Initialize and update submodules
RUN git submodule init && git submodule update --recursive

# Install Miniconda
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O /tmp/miniconda.sh && \
bash /tmp/miniconda.sh -b -p /opt/conda && \
rm /tmp/miniconda.sh
ENV PATH="/opt/conda/bin:${PATH}"

# Create the conda environment and install dependencies
RUN conda create -y -n edgs python=3.10 pip && \
conda clean -afy && \
echo "source activate edgs" > ~/.bashrc

# Set CUDA architectures to compile for
ENV TORCH_CUDA_ARCH_LIST="7.5;8.0;8.6;8.9;9.0+PTX"

# Activate the environment and install Python dependencies
RUN /bin/bash -c "source activate edgs && \
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 && \
pip install -e ./submodules/gaussian-splatting/submodules/diff-gaussian-rasterization && \
pip install -e ./submodules/gaussian-splatting/submodules/simple-knn && \
pip install pycolmap wandb hydra-core tqdm torchmetrics lpips matplotlib rich plyfile imageio imageio-ffmpeg && \
pip install -e ./submodules/RoMa && \
pip install gradio plotly scikit-learn moviepy==2.1.1 ffmpeg open3d jupyterlab matplotlib"

# Expose the port for Gradio
EXPOSE 7862

# Keep the container running in detached mode
CMD ["tail", "-f", "/dev/null"]
69 changes: 34 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,45 +69,14 @@ Alternatively, check our [Colab notebook](https://colab.research.google.com/gith
<a id="sec-install"></a>
## 🛠️ Installation

You can either run `install.sh` or manually install using the following:
You can install it just:

```bash
git clone git@github.com:CompVis/EDGS.git --recursive
cd EDGS
git submodule update --init --recursive

conda create -y -n edgs python=3.10 pip
conda activate edgs

# Set up path to your CUDA. In our experience similar versions like 12.2 also work well
export CUDA_HOME=/usr/local/cuda-12.1
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH
export PATH=$CUDA_HOME/bin:$PATH

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia -y
conda install nvidia/label/cuda-12.1.0::cuda-toolkit -y

pip install -e submodules/gaussian-splatting/submodules/diff-gaussian-rasterization
pip install -e submodules/gaussian-splatting/submodules/simple-knn

# For COLMAP and pycolmap
# Optionally install original colmap but probably pycolmap suffices
# conda install conda-forge/label/colmap_dev::colmap
pip install pycolmap


pip install wandb hydra-core tqdm torchmetrics lpips matplotlib rich plyfile imageio imageio-ffmpeg
conda install numpy=1.26.4 -y -c conda-forge --override-channels

pip install -e submodules/RoMa
conda install anaconda::jupyter --yes

# Stuff necessary for gradio and visualizations
pip install gradio
pip install plotly scikit-learn moviepy==2.1.1 ffmpeg
pip install open3d
docker compose up -d
```

or you can install with running `script/install.sh`.

<a id="sec-data"></a>
## 📦 Data

Expand All @@ -118,6 +87,36 @@ We evaluated on the following datasets:

### Using Your Own Dataset

#### Option A
Use gradle demo.
After running `docker compose up -d`,
```
docker compose exec edgs-app bash
python script/gradio_demo.py --port 7862
```

#### Option B
From command line.
```
docker compose exec edgs-app bash
python script/fit_model_to_scene_full.py --video_path <your mp4 video> [--processed_scenes_dir <output directory>]
```

#### Option C
Using Jupyter lab.
```
docker compose exec edgs-app bash
```
And in the terminal in the docker container,
```
jupyter lab --ip=0.0.0.0 --port=8888 --no-browser --allow-root --notebook-dir=notebooks
```
After JupyterLab starts, it will print URLs to the terminal. Look for a URL containing a token, like:
`http://127.0.0.1:8888/lab?token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx`
Open `http://localhost:8888` (or `http://127.0.0.1:8888`) in your host browser.
When prompted for a "Password or token", paste the `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx` part from the URL in step 4 into the field and log in. Alternatively, you can paste the full URL from step 4 directly into your browser.

#### Option D
You can use the same data format as the [3DGS project](https://github.com/graphdeco-inria/gaussian-splatting?tab=readme-ov-file#processing-your-own-scenes). Please follow their guide to prepare your scene.

Expected folder structure:
Expand Down
18 changes: 8 additions & 10 deletions configs/train.yaml
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
defaults:
- gs: base
- _self_
- _self_

seed: 228

wandb:
mode: "online" # "disabled" for no logging
mode: "disabled" # "online" or "disabled"
entity: "3dcorrespondence"
project: "Adv3DGS"
group: null
name: null
tag: "debug"

train:
gs_epochs: 0 # number of 3dgs iterations
reduce_opacity: True
gs_epochs: 1000 # number of 3dgs iterations
reduce_opacity: True
no_densify: False # if True, the model will not be densified
max_lr: True
max_lr: True

load:
gs: null #path to 3dgs checkpoint
Expand All @@ -28,11 +28,9 @@ verbose: true
init_wC:
use: True # use EDGS
matches_per_ref: 15_000 # number of matches per reference
num_refs: 180 # number of reference images
num_refs: 18 # number of reference images
nns_per_ref: 3 # number of nearest neighbors per reference
scaling_factor: 0.001
proj_err_tolerance: 0.01
roma_model: "outdoors" # you can change this to "indoors" or "outdoors"
add_SfM_init : False


add_SfM_init: False
25 changes: 25 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
services:
edgs-app:
build: . # Instructs Docker Compose to build using the Dockerfile in the current directory
image: edgs-app # This is the name of the image you built
ports:
- "7862:7862" # For Gradio, if you still use it
- "8888:8888" # Map port 8888 for JupyterLab
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all # Use all available GPUs
capabilities: [gpu] # Request GPU capabilities
environment:
- NVIDIA_VISIBLE_DEVICES=all
volumes:
- ./data:/EDGS/data # Example: map a local 'data' folder to '/EDGS/data' in the container
- ./outputs:/EDGS/outputs # Example: map a local 'output' folder
- ./script:/EDGS/script # Example: map a local 'scripts' folder
- ./source:/EDGS/source # Example: map a local 'sources' folder
- ./configs:/EDGS/configs # Example: map a local 'config' folder
- ./notebooks:/EDGS/notebooks # Map a local 'notebooks' folder for JupyterLab
stdin_open: true # Keep STDIN open for interactive processes
tty: true # Allocate a TTY
Loading