You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+20-9Lines changed: 20 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,12 +11,15 @@ This repository contains:
11
11
12
12
## 1. 🚀 Quick start
13
13
14
-
### 📃 Read up
14
+
### 🧱 Learn the basics
15
15
16
-
Check these [slides](https://docs.google.com/presentation/d/1jUO9jZLtUGoK7kgg0kurBgDwDsNOLybrYKU-O2y98xM/edit?usp=sharing) for a short introduction about the idea behind diffusion models and stable diffusion.
16
+
Check these [slides](https://docs.google.com/presentation/d/1jUO9jZLtUGoK7kgg0kurBgDwDsNOLybrYKU-O2y98xM/edit?usp=sharing) for a short introduction about the basic idea behind diffusion models and stable diffusion.
17
17
18
18
### 💻 Play with notebooks
19
19
20
+
> [!WARNING]
21
+
> Since the libraries used in the notebooks below are often updated, the notebooks might stop working at some point. Please, open an issue if you encounter problems and bugs.
22
+
20
23
To try out **Stable Diffusion** by running run one of the Colab notebooks below.
21
24
22
25
- Text to image [](https://colab.research.google.com/drive/1MyESLsR8D5l_EBqumwxL0eMzNmd3uqs6?usp=sharing)
@@ -26,31 +29,36 @@ To try out **Stable Diffusion 2**, you can run one of the Colab notebooks below.
26
29
- Text to image [](https://colab.research.google.com/drive/1e8MV75aD66It0WXo5-ddUEjfvhVG8URv?usp=sharing)
27
30
- Impainting [](https://colab.research.google.com/drive/1JuTzAY0NzojvSU-rB4BNI9F8z3tcv9iX?usp=sharing)
28
31
- Super-resolution [](https://colab.research.google.com/drive/1qnKCWs4IB-n1xfCjeNgbaSu-mFwp7bLv?usp=sharing)
29
-
- Depth-to-image [](https://colab.research.google.com/drive/1wG2L3viHBb3vyPy1-l29bwTGePEA4gLJ?usp=sharing)
32
+
- Depth-to-image [](https://colab.research.google.com/drive/1dS9E8lBJywuoCmZiZAVXo6IaGw4Te5B0?usp=sharing)
30
33
31
-
### ⚒ Understand the theory and learn to build pipelines
34
+
### 🤓 Understand the theory and learn to build pipelines
32
35
33
36
- Understand the theory behind stable diffusion models and learn how to code a simple diffusion model from scratch in [this notebook](https://nbviewer.org/github/FilippoMB/Diffusion_models_tutorial/blob/main/diffusion_from_scratch.ipynb).
34
37
- Become familiar with the stable diffusion pipeline and the diffusers 🧨 library in [this notebook](https://nbviewer.org/github/FilippoMB/Diffusion_models_tutorial/blob/main/Diffusers_library.ipynb).
35
38
36
-
To run the notebooks you need to have several libraries installed. You can do that by installing Anaconda (or [Miniconda](https://docs.anaconda.com/free/miniconda/miniconda-install/)) and then create the environment using the provided env files.
39
+
To run the notebooks you need to have several libraries installed. You can do that by installing Anaconda (or [Miniconda](https://docs.anaconda.com/free/miniconda/miniconda-install/)) and then creating the environment using the provided env files.
37
40
38
-
First, try to create the environment using [environment.yml](https://github.com/FilippoMB/Diffusion_models_tutorial/blob/main/environment.yml):
41
+
First, try to create the environment using [environment.yml](https://github.com/FilippoMB/Diffusion_models_tutorial/blob/main/environment.yml):
39
42
40
43
````bash
41
44
conda env create -f environment.yml
42
45
````
43
46
44
-
If it doesn't work out, try with [env_flex.yml](https://github.com/FilippoMB/Diffusion_models_tutorial/blob/main/env_flex.yml) that allows for a more flexible installation.
47
+
If that doesn't work, try with [env_flex.yml](https://github.com/FilippoMB/Diffusion_models_tutorial/blob/main/env_flex.yml) that allows for a more flexible installation.
45
48
46
49
````bash
47
50
conda env create -f env_flex.yml
48
51
````
52
+
49
53
The risk here is that it will install more recent versions of the software packages and the notebooks might give some errors.
50
54
You might need to this more flexible install also if you are on Windows.
55
+
If you are on MacOS, instead, use [env_mac.yml](https://github.com/FilippoMB/Diffusion_models_tutorial/blob/main/env_mac.yml):
51
56
57
+
````bash
58
+
conda env create -f env_flex.yml
59
+
````
52
60
53
-
## 2. 💡 Prompt engineering guide
61
+
## 2. 👷🏻 Prompt engineering guide
54
62
Let's say you want to draw an image of ``lion``. The raw promt, ``lion`` will give you images that are usually a bit chaotic or worse quality.
55
63
56
64
@@ -90,6 +98,9 @@ To see more examples of prompts and get inspirations, check [here](https://lexic
90
98
- A web-interface with tons of advanced features that runs locally - [WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui).
91
99
- A WebUI extension to generate videos - [Deforum WebUI](https://github.com/deforum-art/sd-webui-deforum)
92
100
101
+
> [!WARNING]
102
+
> Since the libraries used in the notebooks below are often updated, the notebooks might stop working at some point. Please, open an issue if you encounter problems and bugs.
103
+
93
104
**Colab notebooks (demo)**
94
105
- text2img and img2img with advanced features [](https://colab.research.google.com/drive/1AfAmwLMd_Vx33O9IwY2TmO9wKZ8ABRRa)
95
106
- Generate video animations [](https://colab.research.google.com/github/deforum/stable-diffusion/blob/main/Deforum_Stable_Diffusion.ipynb) (you need to download the weights from [here](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original) and upload them to your Google Drive)
@@ -114,4 +125,4 @@ To see more examples of prompts and get inspirations, check [here](https://lexic
114
125
115
126
-[[2](https://arxiv.org/abs/2006.11239)] Ho, Jonathan, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models." Advances in Neural Information Processing Systems, 2020.
116
127
117
-
-[[3](https://arxiv.org/abs/1907.05600)] Song, Yang, and Stefano Ermon. "Generative modeling by estimating gradients of the data distribution." Advances in Neural Information Processing Systems, 2019.
128
+
-[[3](https://arxiv.org/abs/1907.05600)] Song, Yang, and Stefano Ermon. "Generative modeling by estimating gradients of the data distribution." Advances in Neural Information Processing Systems, 2019.
0 commit comments