Skip to content

Two mode models #409

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
__pycache__/
venv/
.idea/
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
## Changelog
### 0.4.6
* Support for [Depth Anything](https://github.com/LiheYoung/Depth-Anything).
### 0.4.5
* Preliminary support for [Marigold](https://marigoldmonodepth.github.io). [PR #385](https://github.com/thygate/stable-diffusion-webui-depthmap-script/pull/385).
### 0.4.4
* Compatibility with stable-diffusion-webui 1.6.0
### 0.4.3 video processing tab
Expand Down
30 changes: 28 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# High Resolution Depth Maps for Stable Diffusion WebUI
This program is an addon for [AUTOMATIC1111's Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) that creates depth maps. Using either generated or custom depth maps, it can also create 3D stereo image pairs (side-by-side or anaglyph), normalmaps and 3D meshes. The outputs of the script can be viewed directly or used as an asset for a 3D engine. Please see [wiki](https://github.com/thygate/stable-diffusion-webui-depthmap-script/wiki/Viewing-Results) to learn more. The program has integration with [Rembg](https://github.com/danielgatis/rembg). It also supports batch processing, processing of videos, and can also be run in standalone mode, without Stable Diffusion WebUI.

To generate realistic depth maps from individual images, this script uses code and models from the [MiDaS](https://github.com/isl-org/MiDaS) and [ZoeDepth](https://github.com/isl-org/ZoeDepth) repositories by Intel ISL, or LeReS from the [AdelaiDepth](https://github.com/aim-uofa/AdelaiDepth) repository by Advanced Intelligent Machines. Multi-resolution merging as implemented by [BoostingMonocularDepth](https://github.com/compphoto/BoostingMonocularDepth) is used to generate high resolution depth maps.
To generate realistic depth maps from individual images, this script uses code and models from the [Marigold](https://github.com/prs-eth/Marigold/) repository, from the [MiDaS](https://github.com/isl-org/MiDaS) and [ZoeDepth](https://github.com/isl-org/ZoeDepth) repositories by Intel ISL, or LeReS from the [AdelaiDepth](https://github.com/aim-uofa/AdelaiDepth) repository by Advanced Intelligent Machines. Multi-resolution merging as implemented by [BoostingMonocularDepth](https://github.com/compphoto/BoostingMonocularDepth) is used to generate high resolution depth maps.

Stereoscopic images are created using a custom-written algorithm.

Expand Down Expand Up @@ -94,7 +94,7 @@ Feel free to comment and share in the discussions and submit issues.

## Acknowledgements

This project relies on code and information from following papers :
This project relies on code and information from the following papers :

MiDaS :

Expand Down Expand Up @@ -198,3 +198,29 @@ ZoeDepth :
copyright = {arXiv.org perpetual, non-exclusive license}
}
```

Marigold - Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation:

```
@misc{ke2023repurposing,
title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation},
author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler},
year={2023},
eprint={2312.02145},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```

Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data

```
@misc{yang2024depth,
title={Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data},
author={Lihe Yang and Bingyi Kang and Zilong Huang and Xiaogang Xu and Jiashi Feng and Hengshuang Zhao},
year={2024},
eprint={2401.10891},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
3 changes: 3 additions & 0 deletions bundled_sources.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,6 @@ https://github.com/aim-uofa/AdelaiDepth/tree/main/LeReS/Minist_Test/lib/

pix2pix
https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/

Marigold
https://github.com/prs-eth/Marigold/tree/22437a
22 changes: 22 additions & 0 deletions install.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Installs dependencies

import launch
import platform
import sys
import importlib.metadata

# TODO: some dependencies apparently being reinstalled on every run. Investigate and fix.

Expand Down Expand Up @@ -38,6 +41,8 @@ def ensure(module_name, min_version=None):
launch.run_pip('install "moviepy==1.0.2"', "moviepy requirement for depthmap script")
ensure('transforms3d', '0.4.1')

ensure('diffusers', '0.20.1') # For Merigold

ensure('imageio') # 2.4.1
try: # Dirty hack to not reinstall every time
importlib_metadata.version('imageio-ffmpeg')
Expand All @@ -52,3 +57,20 @@ def ensure(module_name, min_version=None):

if platform.system() == 'Darwin':
ensure('pyqt6')

# Depth Anything
def get_installed_version(package: str):
try:
return importlib.metadata.version(package)
except Exception:
return None
def try_install_from_wheel(pkg_name: str, wheel_url: str):
if get_installed_version(pkg_name) is not None:
return
try:
launch.run_pip(f"install {wheel_url}", f" {pkg_name} requirement for depthmap script")
except Exception as e:
print('Failed to install wheel for Depth Anything support. It won\'t work.')
try_install_from_wheel(
"depth_anything",
"https://github.com/huchenlei/Depth-Anything/releases/download/v1.0.0/depth_anything-2024.1.22.0-py2.py3-none-any.whl")
1 change: 1 addition & 0 deletions marigold/marigold/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from .marigold_pipeline import MarigoldPipeline, MarigoldDepthOutput
Loading