Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comfy #146

Merged
merged 135 commits into from
May 14, 2023
Merged

Comfy #146

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
149478e
onboard logguru
db0 Mar 28, 2023
b9644bb
model manager wrapper
db0 Mar 28, 2023
ad3dd06
working poc
db0 Mar 28, 2023
9848101
Expand proof of concept:
jug-dev Mar 29, 2023
491342b
Lint fixes
jug-dev Mar 29, 2023
88f5329
Expand comfy proof of concept. Image saving node.
jug-dev Mar 30, 2023
ccbea1f
MM
db0 Apr 2, 2023
9528adf
ported MM for model downloads
db0 Apr 2, 2023
0a20f68
pip
db0 Apr 2, 2023
c5361aa
pip
db0 Apr 2, 2023
0011630
wip
db0 Apr 2, 2023
5f80f0a
fix?
db0 Apr 2, 2023
b340de3
req
db0 Apr 2, 2023
e3c4970
hacky but works!
db0 Apr 2, 2023
2c34fcf
rm mm
db0 Apr 2, 2023
f19cd2d
workingMM
db0 Apr 2, 2023
6e60b2e
clip
db0 Apr 3, 2023
49f7e12
added_csam_filter
db0 Apr 3, 2023
b7ac8ae
still_bnroken
db0 Apr 4, 2023
0a5410a
fix: environment build oddities, enable safety_checker (#159)
tazlin Apr 5, 2023
e190e05
merge_ruff
db0 Apr 5, 2023
96322a9
merge_ruff
db0 Apr 5, 2023
03fc994
censor
db0 Apr 5, 2023
43e3600
working safety checker
db0 Apr 5, 2023
72f0c29
PP work
db0 Apr 6, 2023
19585de
switch_to_generic_generator
db0 Apr 8, 2023
68e93b2
is_image_nsfw
db0 Apr 8, 2023
d824419
post-processing
db0 Apr 8, 2023
1a9cc6e
POST_PROCESSORS_HORDELIB_MODELS
db0 Apr 9, 2023
fbcda27
unnecessary imports
db0 Apr 9, 2023
51d0612
removed img_source and img_mask
db0 Apr 9, 2023
dc2522c
removed_old_stuff
db0 Apr 9, 2023
f6afaf7
inpainting refactoring
db0 Apr 9, 2023
16a96a9
fixes
db0 Apr 9, 2023
0e764bd
fix_for_retaining_transparency
db0 Apr 9, 2023
dc21db0
facefixers
db0 Apr 10, 2023
b0258be
added catch for None image
db0 Apr 11, 2023
55e32b2
prevents ignoring facefixer PPs
db0 Apr 11, 2023
a320df4
index
db0 Apr 11, 2023
a2894b4
stip background
db0 Apr 11, 2023
c277b9d
somwe_stylefix
db0 Apr 11, 2023
479b813
fix
db0 Apr 11, 2023
19aa804
switch to pypi hordelib
db0 Apr 12, 2023
d20edbd
ensure correct torch installed
db0 Apr 12, 2023
362ebcf
fix: remove model dir env var override
jug-dev Apr 13, 2023
a9c967c
requirements
db0 Apr 13, 2023
a216922
Merge branch 'comfy' of github.com:db0/AI-Horde-Worker into comfy
db0 Apr 13, 2023
1e38fc1
switch alchemist to comfy
db0 Apr 13, 2023
9e3ff18
ensure_nataili_uninstalled
db0 Apr 13, 2023
89dc72b
later cuda
db0 Apr 13, 2023
649b1d4
fix: cudatoolkit to 11.8 (#161)
tazlin Apr 13, 2023
a1e0d66
version_bump
db0 Apr 14, 2023
709564a
Merge branch 'comfy' of github.com:db0/AI-Horde-Worker into comfy
db0 Apr 14, 2023
2528fa4
feat: adds hordelib version to top right of terminal ui
jug-dev Apr 15, 2023
f6d7fab
fix: remove obsolete config settings
jug-dev Apr 16, 2023
148b9ee
fix: load safety checker on CPU only (#165)
tazlin Apr 16, 2023
cdd192b
Merge pull request #164 from db0/comfy-settings
jug-dev Apr 16, 2023
bd0974b
re-enable esrgan 2x
db0 Apr 17, 2023
307c3a3
fix: don't modify a list we're iterating over in another thread
jug-dev Apr 19, 2023
a7336b5
stye: Bring up to date with ruff/black (#167)
tazlin Apr 19, 2023
46e4cf3
fix: avoid unloading dynamic models in queued jobs
db0 Apr 19, 2023
07aaea4
fix: default to hordelib stable config
jug-dev Apr 22, 2023
3ff35a6
feat: now attempts to preload all controlnet annotators (#168)
tazlin Apr 22, 2023
9b27101
Ignore models_to_skip for TOP n models (#169)
Cubox Apr 22, 2023
d6131fb
use round() instead of int() for parsed RAM values (#160)
JodanJodan Apr 10, 2023
336a308
fix: try not to send jobs for models we unloaded
jug-dev Apr 24, 2023
1bd48fd
feat: adds benchmark command
jug-dev Apr 24, 2023
2dfa539
fix: important version bump for hordelib
jug-dev Apr 24, 2023
6a6d4e4
fix: allow maintenance mode for workers with short names
jug-dev Apr 25, 2023
6895976
Change bridgeData to new domain name (#171)
Cubox Apr 25, 2023
dfd3b8d
feat: add new resource usage settings to default config template
jug-dev Apr 25, 2023
a878095
Revert to cuda 11.7 as linux has trouble finding the libraries with 1…
db0 Apr 27, 2023
b7c08b6
back_to_118
db0 Apr 27, 2023
bc03ad6
bad link
db0 Apr 27, 2023
908347e
Bump hordelib version
jug-dev Apr 29, 2023
b6c7573
fix: don't suggest running update-runtime as admin.
jug-dev Apr 29, 2023
d9a8f3b
fix: make the terminal ui a bit more robust
jug-dev Apr 29, 2023
65cf98e
fix: add pix2pix to models to skip
jug-dev Apr 29, 2023
29ad830
feat: support new hordelib settings
jug-dev Apr 29, 2023
b9bd2df
fix: pin hordelib
jug-dev Apr 29, 2023
2900407
Adds scribe worker
db0 Apr 29, 2023
68dcd3c
Unpin hordelib
jug-dev Apr 29, 2023
f74682d
fix: Import TerminalUI only when used
db0 Apr 29, 2023
feec2b2
avoid terminal ui on scribe
db0 Apr 29, 2023
8c7967b
fix: prevent KAI Client down crashing worker
db0 Apr 29, 2023
0c4990e
fix: terminal ui should report compvis models
jug-dev Apr 30, 2023
44abe3a
fix: change kudos per hour to be an actual measurement
jug-dev Apr 30, 2023
689a7b4
fix: respect stats output config time
jug-dev Apr 30, 2023
566deea
fix: kudos displays 0 after 1 hour
jug-dev May 1, 2023
e522a42
feat: add (r)eset stats option to ui
jug-dev May 1, 2023
49cc75d
feat: adds 5 min job pop time to ui, lint fixes
jug-dev May 1, 2023
608aeb5
fix: defers cn annotator check until after config is loaded (#176)
tazlin May 1, 2023
deed160
Merge branch 'comfy' of https://github.com/db0/AI-Horde-Worker into c…
jug-dev May 1, 2023
5ce4872
feat: Can now set individual worker type names in bridgeData
db0 May 1, 2023
5af253e
fix: wrong chec
db0 May 1, 2023
d7c1102
fix: avoid showing full debug payload
db0 May 1, 2023
a66f5a8
chore: version bump hordelib
jug-dev May 1, 2023
e7a9e3b
chore: version bump hordelib
jug-dev May 1, 2023
ea8d6af
branded_model
db0 May 2, 2023
4fc1008
Merge branch 'comfy' of github.com:db0/AI-Horde-Worker into comfy
db0 May 2, 2023
4c100c9
feat: added --hordelib arg to update-runtime
db0 May 4, 2023
82c6861
Create a Dockerfile for building horde worker with defaults, added do…
guspuffygit May 4, 2023
ffcfa9d
Merge branch 'comfy' into docker-worker
db0 May 4, 2023
998f6a5
Merge pull request #183 from guspuffygit/docker-worker
db0 May 4, 2023
d1087dc
merge
guspuffygit May 5, 2023
e16ed0a
no changelog
db0 May 5, 2023
91df2b9
feat: add --scribe
db0 May 5, 2023
ff37479
feat: adds feature to save kudos model training data
jug-dev May 6, 2023
bf0ed23
fix: don't assume post_processing exists in payload
jug-dev May 6, 2023
13c6ad9
fix: elif
db0 May 6, 2023
edfab15
Merge branch 'comfy' of github.com:db0/AI-Horde-Worker into comfy
db0 May 6, 2023
e81bd74
fix: adds long prompt into kudos training data
jug-dev May 7, 2023
75991e2
chore: bump hordelib version
jug-dev May 7, 2023
23195b2
fix: remove obsolete GPUInfo
jug-dev May 7, 2023
5df0863
feat: tweaks to worker debug
jug-dev May 7, 2023
3765f24
fix: don't enable kudos calculation always
jug-dev May 7, 2023
6325abe
fix: add missing kudos calculator import
jug-dev May 7, 2023
88939c8
feat: simplify kudos model
jug-dev May 7, 2023
e792fab
feat: simulate 0.5% kudos bonus per model
jug-dev May 7, 2023
c89124b
fix: simulate kudos even when not saving training data
jug-dev May 7, 2023
47b1004
fix: comment out a bunch of noisey debugs
jug-dev May 8, 2023
099f3b5
fix: make jobs per hour calculation more accurate
jug-dev May 8, 2023
e58757b
chore: version bump hordelib
jug-dev May 8, 2023
b142a43
feat: update kudos model v20.66
jug-dev May 8, 2023
41a91ef
feat: activate kudos simulation
jug-dev May 8, 2023
c336bf2
feat: remove per model bonus from simulation
jug-dev May 8, 2023
387700e
style: Pre-release format catch-up (#186)
tazlin May 9, 2023
a16f20c
v22
db0 May 10, 2023
acd1dd8
chore: bump hordelib version
jug-dev May 12, 2023
91b5dd1
chore: bump hordelib version
jug-dev May 13, 2023
6795b7e
fix: change the display of free ram in the terminal ui
jug-dev May 13, 2023
002d95a
fix: annoying UX at startup in terminal ui
jug-dev May 13, 2023
a516e29
fix: stop simulating kudos locally
jug-dev May 13, 2023
ef21381
chore: bump hordelib
jug-dev May 13, 2023
dfde6fc
fix: terminal ui is now not case sensitive on key presses
jug-dev May 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
POST_PROCESSORS_HORDELIB_MODELS
  • Loading branch information
db0 committed Apr 9, 2023
commit 1a9cc6e3c88b534d3ad48df67f9d6b66396ff776
4 changes: 2 additions & 2 deletions worker/bridge_data/interrogation.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

from worker.argparser.interrogation import args
from worker.bridge_data.framework import BridgeDataTemplate
from worker.consts import POST_PROCESSORS_NATAILI_MODELS
from worker.consts import POST_PROCESSORS_HORDELIB_MODELS
from worker.logger import logger


Expand Down Expand Up @@ -31,7 +31,7 @@ def reload_data(self):
if "interrogation" in self.forms and "ViT-L/14" not in self.model_names:
self.model_names.append("ViT-L/14")
if "post-process" in self.forms:
self.model_names += list(POST_PROCESSORS_NATAILI_MODELS)
self.model_names += list(POST_PROCESSORS_HORDELIB_MODELS)
if (not self.initialized and not self.models_reloading) or previous_url != self.horde_url:
logger.init(
(
Expand Down
6 changes: 3 additions & 3 deletions worker/bridge_data/stable_diffusion.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

from worker.argparser.stable_diffusion import args
from worker.bridge_data.framework import BridgeDataTemplate
from worker.consts import KNOWN_INTERROGATORS, POST_PROCESSORS_NATAILI_MODELS
from worker.consts import KNOWN_INTERROGATORS, POST_PROCESSORS_HORDELIB_MODELS

# from nataili import enable_ray_alternative
from worker.logger import logger
Expand Down Expand Up @@ -123,7 +123,7 @@ def reload_data(self):
self.model_names.insert(0,"safety_checker")
self.model_names.insert(0, "ViT-L/14")
if self.allow_post_processing:
self.model_names += list(POST_PROCESSORS_NATAILI_MODELS)
self.model_names += list(POST_PROCESSORS_HORDELIB_MODELS)
if (not self.initialized and not self.models_reloading) or previous_url != self.horde_url:
logger.init(
(
Expand All @@ -141,7 +141,7 @@ def _is_valid_stable_diffusion_model(self, model_name):
if model_name in ["safety_checker", "LDSR"]:
return False

if model_name in POST_PROCESSORS_NATAILI_MODELS or model_name in KNOWN_INTERROGATORS:
if model_name in POST_PROCESSORS_HORDELIB_MODELS or model_name in KNOWN_INTERROGATORS:
return False

return model_name not in self.models_to_skip
Expand Down
4 changes: 2 additions & 2 deletions worker/consts.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
"GFPGAN",
"CodeFormers",
}
POST_PROCESSORS_NATAILI_MODELS = KNOWN_UPSCALERS | KNOWN_FACE_FIXERS
KNOWN_POST_PROCESSORS = POST_PROCESSORS_NATAILI_MODELS | {
POST_PROCESSORS_HORDELIB_MODELS = KNOWN_UPSCALERS | KNOWN_FACE_FIXERS
KNOWN_POST_PROCESSORS = POST_PROCESSORS_HORDELIB_MODELS | {
"strip_background",
}
KNOWN_INTERROGATORS = {
Expand Down
4 changes: 2 additions & 2 deletions worker/jobs/poppers.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import requests
from PIL import Image, UnidentifiedImageError

from worker.consts import BRIDGE_VERSION, KNOWN_INTERROGATORS, KNOWN_POST_PROCESSORS, POST_PROCESSORS_NATAILI_MODELS
from worker.consts import BRIDGE_VERSION, KNOWN_INTERROGATORS, KNOWN_POST_PROCESSORS, POST_PROCESSORS_HORDELIB_MODELS
from worker.logger import logger


Expand Down Expand Up @@ -125,7 +125,7 @@ def __init__(self, mm, bd):
self.available_models = self.model_manager.get_loaded_models_names()
for util_model in (
list(KNOWN_INTERROGATORS)
+ list(POST_PROCESSORS_NATAILI_MODELS)
+ list(POST_PROCESSORS_HORDELIB_MODELS)
+ [
"LDSR",
"safety_checker",
Expand Down
4 changes: 2 additions & 2 deletions worker/jobs/stable_diffusion.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from hordelib.horde import HordeLib
from hordelib.safety_checker import is_image_nsfw
from worker import csam
from worker.consts import KNOWN_INTERROGATORS, POST_PROCESSORS_NATAILI_MODELS
from worker.consts import KNOWN_INTERROGATORS, POST_PROCESSORS_HORDELIB_MODELS
from worker.enums import JobStatus
from worker.jobs.framework import HordeJobFramework
from worker.logger import logger
Expand Down Expand Up @@ -164,7 +164,7 @@ def start_job(self):
for available_model in self.available_models:
if (
"_inpainting" not in available_model
and available_model not in POST_PROCESSORS_NATAILI_MODELS | KNOWN_INTERROGATORS
and available_model not in POST_PROCESSORS_HORDELIB_MODELS | KNOWN_INTERROGATORS
and available_model in self.model_manager.compvis.get_loaded_models_names()
):
logger.debug(
Expand Down