Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v21.5.7 #699

Merged
merged 69 commits into from
May 1, 2023
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
27ffd9f
feat: support wandb logging
p1atdev Apr 19, 2023
8090dac
fix: wandb not working without logging_dir
p1atdev Apr 19, 2023
12567f5
chore: add wandb to gitignore
p1atdev Apr 19, 2023
a69b24a
fix: tensorboard not working
p1atdev Apr 19, 2023
e746829
おそらくlibgtk2がインストールされていない環境でcv2.waitKey() および cv2.destroyAllWindows() …
tsukimiya Apr 19, 2023
9f8f27f
Merge pull request #429 from tsukimiya/hotfix/debug_dataset_linux_sup…
kohya-ss Apr 21, 2023
c430cf4
Merge pull request #428 from p1atdev/dev
kohya-ss Apr 22, 2023
2204362
some minor fixes
kohya-ss Apr 22, 2023
884e6bf
fix face_crop_aug not working on finetune method, prepare upscaler
kohya-ss Apr 22, 2023
23b261d
update readme
kohya-ss Apr 22, 2023
f256660
Merge pull request #431 from kohya-ss/dev
kohya-ss Apr 22, 2023
dc37fd2
fix no logging command line arg
sALTaccount Apr 22, 2023
ed15f68
Merge pull request #433 from sALTaccount/main
kohya-ss Apr 22, 2023
66edc5a
invert condition for checking log_with
kohya-ss Apr 22, 2023
700af1c
Merge pull request #434 from kohya-ss/dev
kohya-ss Apr 22, 2023
ae3965a
feat: add arguments to set \--wandb_api_key\ before training
Linaqruf Apr 22, 2023
e9a641b
Merge branch 'main' of https://github.com/Linaqruf/sd-scripts
Linaqruf Apr 22, 2023
ed2eef1
Merge pull request #435 from Linaqruf/main
kohya-ss Apr 22, 2023
9ba4c3e
update readme/comment
kohya-ss Apr 22, 2023
b824bbf
Merge pull request #436 from kohya-ss/dev
kohya-ss Apr 22, 2023
46cbae0
fix to log with logging_dir without log_with
kohya-ss Apr 23, 2023
05c57b9
update readme
kohya-ss Apr 23, 2023
25c8279
Merge pull request #437 from kohya-ss/dev
kohya-ss Apr 23, 2023
852481e
Update README.md
kohya-ss Apr 24, 2023
74008ce
add `save_every_n_steps` option
kohya-ss Apr 24, 2023
551fdf3
Merge branch 'main' into dev
kohya-ss Apr 24, 2023
9bb52ac
update readme
kohya-ss Apr 24, 2023
8e833ec
Fix merge path issue
bmaltais Apr 24, 2023
6c6e8be
Fix merge path issue
bmaltais Apr 24, 2023
844b4fd
Update readme
bmaltais Apr 24, 2023
6bf994c
Merge branch 'main' of https://github.com/kohya-ss/sd-scripts into dev
bmaltais Apr 24, 2023
1890535
enable `cache_latents` when `_to_disk` #438
kohya-ss Apr 24, 2023
a85fcfe
fix latent upscale not working if bs>1
kohya-ss Apr 24, 2023
c3768aa
update readme
kohya-ss Apr 24, 2023
c817862
update readme
kohya-ss Apr 24, 2023
ac4935b
Merge pull request #444 from kohya-ss/dev
kohya-ss Apr 24, 2023
dd50514
Create train_README-zh.md
tomj2ee Apr 25, 2023
57bc2ab
update parser format to match a global pattern
mio7690 Apr 26, 2023
5de91b9
Merge pull request #445 from tomj2ee/main
kohya-ss Apr 26, 2023
8810f8a
Merge pull request #453 from mio2333/main
kohya-ss Apr 26, 2023
3e8d389
Merge branch 'main' into dev
kohya-ss Apr 26, 2023
abedbc7
update readme
kohya-ss Apr 26, 2023
c72f8ac
Merge pull request #454 from kohya-ss/dev
kohya-ss Apr 26, 2023
18f171d
add link in change history
kohya-ss Apr 26, 2023
4f4b92d
fixed some error
tomj2ee Apr 27, 2023
dff3872
Update train_README-zh.md
tomj2ee Apr 27, 2023
7f2ac58
Update train_README-zh.md
tomj2ee Apr 27, 2023
c2b51fb
Update train_README-zh.md
tomj2ee Apr 27, 2023
bf53d0f
fix: use python 3.10 prior to avoid use python higher than 3.10 mista…
Apr 27, 2023
d52c524
Merge pull request #457 from tomj2ee/main
kohya-ss Apr 27, 2023
5f5bd4d
Merge pull request #672 from Whyjsee/fix/python3.10_prior
bmaltais Apr 27, 2023
7bd1cb9
Merge branch 'main' of https://github.com/kohya-ss/sd-scripts into dev
bmaltais Apr 27, 2023
19b7b41
Merge branch 'dev' of https://github.com/bmaltais/kohya_ss into dev
bmaltais Apr 27, 2023
ad43627
made a very high quality comprehensive tutorial
FurkanGozukara Apr 29, 2023
8dd64cc
Merge pull request #681 from FurkanGozukara/master
bmaltais Apr 29, 2023
f80303e
Update setup.bat
bmaltais Apr 30, 2023
82fce11
Add docker support
Trojaner May 1, 2023
e4e6140
Docker: separate requirements.txt and depds update commands
Trojaner May 1, 2023
c1216e8
Fix conflicting packages
Trojaner May 1, 2023
2f22a30
Update README
Trojaner May 1, 2023
57ea3ee
Docker: comment out hugging face import
Trojaner May 1, 2023
5f8aa9e
Docker: add related issue link to hotfix
Trojaner May 1, 2023
cba5e46
Docker: remove cache.json
Trojaner May 1, 2023
6a7ca53
Docker: update cached directories
Trojaner May 1, 2023
1daafcc
Docker: fix missing libnvinfer7
Trojaner May 1, 2023
328d260
Update README
Trojaner May 1, 2023
cac5e12
Docker: use xformers 0.0.17
Trojaner May 1, 2023
ffac916
Merge pull request #697 from Trojaner/feature/docker-support
bmaltais May 1, 2023
817cdf3
Update README
bmaltais May 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.cache/
cudnn_windows/
bitsandbytes_windows/
dataset/
__pycache__/
venv/
40 changes: 40 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
FROM nvcr.io/nvidia/pytorch:23.04-py3 as base
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=Europe/London

RUN apt-get update && \
apt-get install -y git curl python3-venv python3-tk libgl1 libglib2.0-0 libgoogle-perftools-dev && \
rm -rf /var/lib/apt/lists/*

RUN useradd -m -s /bin/bash appuser
USER appuser

WORKDIR /app

RUN python3 -m venv ./venv && . ./venv/bin/activate && \
python3 -m pip install wheel

# Install requirements
COPY requirements.txt setup.py .
RUN . ./venv/bin/activate && \
python3 -m pip install --use-pep517 -U -r requirements.txt

# Upgrade to Torch 2.0
RUN . ./venv/bin/activate && \
python3 -m pip install --use-pep517 --no-deps -U triton==2.0.0 torch>=2.0.0+cu121 xformers==0.0.17 \
--extra-index-url https://download.pytorch.org/whl/cu121

# Fix missing libnvinfer7
USER root
RUN ln -s /usr/lib/x86_64-linux-gnu/libnvinfer.so /usr/lib/x86_64-linux-gnu/libnvinfer.so.7 && \
ln -s /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.7

USER appuser
COPY --chown=appuser . .

RUN sed -i 's/import library.huggingface_util/# import library.huggingface_util/g' train_network.py && \
sed -i 's/import library.huggingface_util/# import library.huggingface_util/g' library/train_util.py

STOPSIGNAL SIGINT
ENV LD_PRELOAD=libtcmalloc.so
CMD . ./venv/bin/activate && python3 "./kohya_gui.py" ${CLI_ARGS} --listen 0.0.0.0 --server_port 7860
28 changes: 25 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,13 @@

This repository provides a Windows-focused Gradio GUI for [Kohya's Stable Diffusion trainers](https://github.com/kohya-ss/sd-scripts). The GUI allows you to set the training parameters and generate and run the required CLI commands to train the model.

If you run on Linux and would like to use the GUI, there is now a port of it as a docker container. You can find the project [here](https://github.com/P2Enjoy/kohya_ss-docker).

### Table of Contents

- [Tutorials](#tutorials)
- [Required Dependencies](#required-dependencies)
- [Linux/macOS](#linux-and-macos-dependencies)
- [Installation](#installation)
- [Docker](#docker)
- [Linux/macOS](#linux-and-macos)
- [Default Install Locations](#install-location)
- [Windows](#windows)
Expand Down Expand Up @@ -61,10 +60,33 @@ These dependencies are taken care of via `setup.sh` in the installation section.
### Runpod
Follow the instructions found in this discussion: https://github.com/bmaltais/kohya_ss/discussions/379

### Docker
Docker is supported on Windows and Linux distributions. However this method currently only supports Nvidia GPUs.
Run the following commands in your OS shell after installing [git](https://git-scm.com/download/) and [docker](https://www.docker.com/products/docker-desktop/):
```bash
git clone https://github.com/bmaltais/kohya_ss.git
cd kohya_ss
docker compose up --build
```

This will take a while (up to 20 minutes) on the first run.

The following limitations apply:
* All training data must be added to the `dataset` subdirectory, the docker container cannot access any other files
* The file picker does not work
* Cannot select folders, folder path must be set manually like e.g. /dataset/my_lora/img
* Cannot select config file, it must be loaded via path instead like e.g. /dataset/my_config.json
* Dialogs do not work
* Make sure your file names are unique as this happens when asking if an existing file should be overridden
* No auto-update support. Must run update scripts outside docker manually and then rebuild with `docker compose build`.


If you run on Linux, there is an alternative docker container port with less limitations. You can find the project [here](https://github.com/P2Enjoy/kohya_ss-docker).

### Linux and macOS
In the terminal, run

```
```bash
git clone https://github.com/bmaltais/kohya_ss.git
cd kohya_ss
# May need to chmod +x ./setup.sh if you're on a machine with stricter security.
Expand Down
30 changes: 30 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
version: "3.8"
services:
kohya-ss-gui:
container_name: kohya-ss-gui
image: kohya-ss-gui:latest
build:
context: .
ports:
- 127.0.0.1:7860:7860
tty: true
ipc: host
environment:
CLI_ARGS: ""
SAFETENSORS_FAST_GPU: 1
tmpfs:
- /tmp
volumes:
- ./dataset:/dataset
- ./.cache/user:/home/appuser/.cache
- ./.cache/triton:/home/appuser/.triton
- ./.cache/config:/app/appuser/.config
- ./.cache/nv:/home/appuser/.nv
- ./.cache/keras:/home/appuser/.keras
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: ['0']
capabilities: [gpu]