Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add metadata to images #1940

Merged
merged 71 commits into from
Feb 26, 2024
Merged
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
c5a15c7
feat: add metadata logging for images
mashb1t Jan 15, 2024
8d56318
feat: add config and checkbox for save_metadata_to_images
mashb1t Jan 15, 2024
493e484
feat: add argument disable_metadata
mashb1t Jan 15, 2024
191f814
feat: add support for A1111 metadata schema
mashb1t Jan 15, 2024
f7489cc
feat: add model hash support for a1111
mashb1t Jan 15, 2024
1a52367
feat: use resolved prompts with included expansion and styles for a11…
mashb1t Jan 15, 2024
6662381
fix: code cleanup and resolved prompt fixes
mashb1t Jan 15, 2024
7b9deb1
feat: add config metadata_created_by
mashb1t Jan 15, 2024
cd65f21
fix: use stting isntead of quote wrap for A1111 created_by
mashb1t Jan 15, 2024
a2153db
fix: correctlyy hide/show metadata schema on app start
mashb1t Jan 15, 2024
80ad0d0
fix: do not generate hashes when arg --disable-metadata is used
mashb1t Jan 15, 2024
ba5d0b6
refactor: rename metadata_schema to metadata_scheme
mashb1t Jan 15, 2024
3812228
Merge branch 'main_upstream' into feature/add-metadata-to-files
mashb1t Jan 25, 2024
510b587
fix: use pnginfo "parameters" insteadf of "Comments"
mashb1t Jan 25, 2024
20b7978
feat: add resolved prompts to metadata
mashb1t Jan 25, 2024
051faf7
fix: use correct default value in metadata check for created_by
mashb1t Jan 25, 2024
f301031
wip: add metadata mapping, reading and writing
mashb1t Jan 27, 2024
ee21c2b
feat: rename metadata tab and import button label
mashb1t Jan 28, 2024
e19596c
feat: map basic information for scheme A1111
mashb1t Jan 28, 2024
7ddd4e5
wip: optimize handling for metadata in Gradio calls
mashb1t Jan 28, 2024
cbc63eb
feat: add enums for Performance, Steps and StepsUOV
mashb1t Jan 28, 2024
5dcb2bc
fix: correctly map resolution, use empty styles for A1111
mashb1t Jan 28, 2024
2362789
chore: code cleanup
mashb1t Jan 28, 2024
5e84a45
feat: add A1111 prompt style detection
mashb1t Jan 28, 2024
f94b96f
wip: add prompt style extraction for A1111 scheme
mashb1t Jan 29, 2024
13d0341
feat: sort styles after metadata import
mashb1t Jan 29, 2024
c3ab9f1
refactor: use central flag for LoRA count
mashb1t Jan 29, 2024
20e5302
refactor: use central flag for ControlNet image count
mashb1t Jan 29, 2024
c80011b
fix: use correct LoRA mapping, add fallback for backwards compatibility
mashb1t Jan 29, 2024
7fefe3a
feat: add created_by again
mashb1t Jan 29, 2024
33d644f
feat: add prefix "Fooocus" to version
mashb1t Jan 29, 2024
e388f6f
wip: code cleanup, update todos
mashb1t Jan 29, 2024
2656356
fix: use correct order to read LoRA in meta parser
mashb1t Jan 29, 2024
e541097
wip: code cleanup, update todos
mashb1t Jan 29, 2024
89c8e3a
feat: make sha256 with length 10 default
mashb1t Jan 29, 2024
78d1ad3
feat: add lora handling to A1111 scheme
mashb1t Jan 29, 2024
dcc4874
feat: override existing LoRA values when importing, would cause image…
mashb1t Jan 29, 2024
6939f79
fix: correctly extract prompt style when only prompt expansion is sel…
mashb1t Jan 29, 2024
5811234
feat: allow model / LoRA loading from subfolders
mashb1t Jan 29, 2024
e93a345
feat: code cleanup, do not queue metadata preview on image upload
mashb1t Jan 29, 2024
7772eb7
refactor: add flag for refiner_swap_method
mashb1t Jan 31, 2024
9bdb65e
feat: add metadata handling for all non-img2img parameters
mashb1t Jan 31, 2024
6b9c0bd
refactor: code cleanup
mashb1t Jan 31, 2024
23ba050
chore: use str as return type in calculate_sha256
mashb1t Feb 2, 2024
bc9b625
feat: add hash cache to metadata
mashb1t Feb 2, 2024
ea6839b
chore: code cleanup
mashb1t Feb 2, 2024
f4afc4a
feat: add method get_scheme to Metadata
mashb1t Feb 2, 2024
796cf3c
fix: align handling for scheme Fooocus by removing lcm lora from json…
mashb1t Feb 2, 2024
e558701
refactor: add step before parsing to set data in parser
mashb1t Feb 2, 2024
f7e24bd
feat: sort metadata attributes before writing to image
mashb1t Feb 2, 2024
934bdb1
feat: add translations and hint for image prompt parameters
mashb1t Feb 2, 2024
b438f7b
chore: check and remove ToDo's
mashb1t Feb 2, 2024
f745d40
refactor: merge metadata.py into meta_parser.py
mashb1t Feb 2, 2024
9aa82aa
fix: add missing refiner in A1111 parse_json
mashb1t Feb 2, 2024
1c3431e
wip: add TODO for ultiline prompt style resolution
mashb1t Feb 2, 2024
349556b
fix: remove sorting for A1111, change performance key position
mashb1t Feb 2, 2024
ed4a958
fix: add workaround for multiline prompts
mashb1t Feb 2, 2024
63403d6
feat: add sampler mapping
mashb1t Feb 2, 2024
1419231
feat: prevent config reset by renaming metadata_scheme to match confi…
mashb1t Feb 3, 2024
8af73e6
chore: remove remaining todos after analysis
mashb1t Feb 3, 2024
c668228
chore: specify too broad exception types
mashb1t Feb 4, 2024
fe33cc7
feat: add mapping for _gpu samplers to cpu samplers
mashb1t Feb 4, 2024
59dd1c2
Merge remote-tracking branch 'upstream/main' into feature/add-metadat…
mashb1t Feb 4, 2024
dfb48fd
feat: add better handling for image import with empty metadata
mashb1t Feb 4, 2024
c104d58
fix: parse adaptive_cfg as float instead of string
mashb1t Feb 4, 2024
832441e
chore: loosen strict type for parse_json, fix indent
mashb1t Feb 4, 2024
36f6715
chore: make steps enums more strict
mashb1t Feb 18, 2024
f93dd6e
feat: only override steps if metadata value is not in steps enum or i…
mashb1t Feb 18, 2024
692a2e4
Merge branch 'feature/add-metadata-to-files' of github.com:mashb1t/Fo…
mashb1t Feb 18, 2024
0b77db9
Merge branch 'develop' into feature/add-metadata-to-files
mashb1t Feb 26, 2024
65dae45
fix: handle empty strings in metadata
mashb1t Feb 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'develop' into feature/add-metadata-to-files
# Conflicts:
#	modules/async_worker.py
#	modules/config.py
#	modules/meta_parser.py
#	modules/private_logger.py
#	modules/util.py
#	webui.py
mashb1t committed Feb 26, 2024
commit 0b77db9051e07b59a68928ae48ce277fc3786607
75 changes: 54 additions & 21 deletions modules/async_worker.py
Original file line number Diff line number Diff line change
@@ -48,7 +48,7 @@ def worker():
from modules.util import remove_empty_str, HWC3, resize_image, \
get_image_shape_ceil, set_image_shape_ceil, get_shape_ceil, resample_image, erode_or_dilate, ordinal_suffix
from modules.upscaler import perform_upscale
from modules.flags import Performance, lora_count
from modules.flags import Performance
from modules.meta_parser import get_metadata_parser, MetadataScheme

pid = os.getpid()
@@ -148,7 +148,7 @@ def handler(async_task):
base_model_name = args.pop()
refiner_model_name = args.pop()
refiner_switch = args.pop()
loras = [[str(args.pop()), float(args.pop())] for _ in range(lora_count)]
loras = apply_enabled_loras([[bool(args.pop()), str(args.pop()), float(args.pop()), ] for _ in range(modules.config.default_max_lora_number)])
input_image_checkbox = args.pop()
current_tab = args.pop()
uov_method = args.pop()
@@ -157,6 +157,44 @@ def handler(async_task):
inpaint_input_image = args.pop()
inpaint_additional_prompt = args.pop()
inpaint_mask_image_upload = args.pop()

disable_preview = args.pop()
disable_intermediate_results = args.pop()
disable_seed_increment = args.pop()
adm_scaler_positive = args.pop()
adm_scaler_negative = args.pop()
adm_scaler_end = args.pop()
adaptive_cfg = args.pop()
sampler_name = args.pop()
scheduler_name = args.pop()
overwrite_step = args.pop()
overwrite_switch = args.pop()
overwrite_width = args.pop()
overwrite_height = args.pop()
overwrite_vary_strength = args.pop()
overwrite_upscale_strength = args.pop()
mixing_image_prompt_and_vary_upscale = args.pop()
mixing_image_prompt_and_inpaint = args.pop()
debugging_cn_preprocessor = args.pop()
skipping_cn_preprocessor = args.pop()
canny_low_threshold = args.pop()
canny_high_threshold = args.pop()
refiner_swap_method = args.pop()
controlnet_softness = args.pop()
freeu_enabled = args.pop()
freeu_b1 = args.pop()
freeu_b2 = args.pop()
freeu_s1 = args.pop()
freeu_s2 = args.pop()
debugging_inpaint_preprocessor = args.pop()
inpaint_disable_initial_latent = args.pop()
inpaint_engine = args.pop()
inpaint_strength = args.pop()
inpaint_respective_field = args.pop()
inpaint_mask_upload_checkbox = args.pop()
invert_mask_checkbox = args.pop()
inpaint_erode_or_dilate = args.pop()

save_metadata_to_images = args.pop() if not args_manager.args.disable_metadata else False
metadata_scheme = MetadataScheme(args.pop()) if not args_manager.args.disable_metadata else MetadataScheme.FOOOCUS

@@ -203,9 +241,9 @@ def handler(async_task):
guidance_scale = 1.0
adaptive_cfg = 1.0
refiner_switch = 1.0
modules.patch.positive_adm_scale = advanced_parameters.adm_scaler_positive = 1.0
modules.patch.negative_adm_scale = advanced_parameters.adm_scaler_negative = 1.0
modules.patch.adm_scaler_end = advanced_parameters.adm_scaler_end = 0.0
adm_scaler_positive = 1.0
adm_scaler_negative = 1.0
adm_scaler_end = 0.0

print(f'[Parameters] Adaptive CFG = {adaptive_cfg}')
print(f'[Parameters] Sharpness = {sharpness}')
@@ -276,7 +314,7 @@ def handler(async_task):
inpaint_image = inpaint_input_image['image']
inpaint_mask = inpaint_input_image['mask'][:, :, 0]

if advanced_parameters.inpaint_mask_upload_checkbox:
if inpaint_mask_upload_checkbox:
if isinstance(inpaint_mask_image_upload, np.ndarray):
if inpaint_mask_image_upload.ndim == 3:
H, W, C = inpaint_image.shape
@@ -791,31 +829,27 @@ def callback(step, x0, x, total_steps, y):
('Guidance Scale', 'guidance_scale', guidance_scale),
('Sharpness', 'sharpness', sharpness),
('ADM Guidance', 'adm_guidance', str((
modules.patch.positive_adm_scale,
modules.patch.negative_adm_scale,
modules.patch.adm_scaler_end))),
modules.patch.patch_settings[pid].positive_adm_scale,
modules.patch.patch_settings[pid].negative_adm_scale,
modules.patch.patch_settings[pid].adm_scaler_end))),
('Base Model', 'base_model', base_model_name),
('Refiner Model', 'refiner_model', refiner_model_name),
('Refiner Switch', 'refiner_switch', refiner_switch)]

if refiner_model_name != 'None':
if advanced_parameters.overwrite_switch > 0:
d.append(('Overwrite Switch', 'overwrite_switch', advanced_parameters.overwrite_switch))
if overwrite_switch > 0:
d.append(('Overwrite Switch', 'overwrite_switch', overwrite_switch))
if refiner_swap_method != flags.refiner_swap_method:
d.append(('Refiner Swap Method', 'refiner_swap_method', refiner_swap_method))
if advanced_parameters.adaptive_cfg != modules.config.default_cfg_tsnr:
d.append(('CFG Mimicking from TSNR', 'adaptive_cfg', advanced_parameters.adaptive_cfg))
if modules.patch.patch_settings[pid].adaptive_cfg != modules.config.default_cfg_tsnr:
d.append(('CFG Mimicking from TSNR', 'adaptive_cfg', modules.patch.patch_settings[pid].adaptive_cfg))

d.append(('Sampler', 'sampler', sampler_name))
d.append(('Scheduler', 'scheduler', scheduler_name))
d.append(('Seed', 'seed', task['task_seed']))

if advanced_parameters.freeu_enabled:
d.append(('FreeU', 'freeu', str((
advanced_parameters.freeu_b1,
advanced_parameters.freeu_b2,
advanced_parameters.freeu_s1,
advanced_parameters.freeu_s2))))
if freeu_enabled:
d.append(('FreeU', 'freeu', str((freeu_b1, freeu_b2, freeu_s1, freeu_s2))))

metadata_parser = None
if save_metadata_to_images:
@@ -829,8 +863,7 @@ def callback(step, x0, x, total_steps, y):
d.append((f'LoRA {li + 1}', f'lora_combined_{li + 1}', f'{n} : {w}'))

d.append(('Version', 'version', 'Fooocus v' + fooocus_version.version))

log(x, d, metadata_parser)
img_paths.append(log(x, d, metadata_parser))

yield_result(async_task, img_paths, do_not_show_finished_images=len(tasks) == 1 or disable_intermediate_results)
except ldm_patched.modules.model_management.InterruptProcessingException as e:
6 changes: 3 additions & 3 deletions modules/config.py
Original file line number Diff line number Diff line change
@@ -7,8 +7,8 @@
import modules.sdxl_styles

from modules.model_loader import load_file_from_url
from modules.util import get_files_from_folder
from modules.flags import Performance, MetadataScheme, lora_count
from modules.util import get_files_from_folder, makedirs_with_log
from modules.flags import Performance, MetadataScheme

config_path = os.path.abspath("./config.txt")
config_example_path = os.path.abspath("config_modification_tutorial.txt")
@@ -387,7 +387,7 @@ def get_config_item_or_set_default(key, default_value, validator, disable_empty_

example_inpaint_prompts = [[x] for x in example_inpaint_prompts]

config_dict["default_loras"] = default_loras = default_loras[:lora_count] + [['None', 1.0] for _ in range(lora_count - len(default_loras))]
config_dict["default_loras"] = default_loras = default_loras[:default_max_lora_number] + [['None', 1.0] for _ in range(default_max_lora_number - len(default_loras))]

possible_preset_keys = [
"default_model",
1 change: 0 additions & 1 deletion modules/flags.py
Original file line number Diff line number Diff line change
@@ -88,7 +88,6 @@ class MetadataScheme(Enum):
]

lora_count = 5
lora_count_with_lcm = lora_count + 1

controlnet_image_count = 4

14 changes: 8 additions & 6 deletions modules/meta_parser.py
Original file line number Diff line number Diff line change
@@ -10,8 +10,8 @@
import modules.config
import modules.sdxl_styles
from modules.flags import MetadataScheme, Performance, Steps
from modules.flags import lora_count, SAMPLERS, CIVITAI_NO_KARRAS
from modules.util import quote, unquote, extract_styles_from_prompt, is_json, calculate_sha256
from modules.flags import SAMPLERS, CIVITAI_NO_KARRAS
from modules.util import quote, unquote, extract_styles_from_prompt, is_json, get_file_from_folder_list, calculate_sha256

re_param_code = r'\s*(\w[\w \-/]+):\s*("(?:\\.|[^\\"])+"|[^,]*)(?:,|$)'
re_param = re.compile(re_param_code)
@@ -56,7 +56,7 @@ def load_parameter_button_click(raw_metadata: dict | str, is_generating: bool):

get_freeu('freeu', 'FreeU', loaded_parameter_dict, results)

for i in range(lora_count):
for i in range(modules.config.default_max_lora_number):
get_lora(f'lora_combined_{i + 1}', f'LoRA {i + 1}', loaded_parameter_dict, results)

return results
@@ -170,9 +170,11 @@ def get_lora(key: str, fallback: str | None, source_dict: dict, results: list):
try:
n, w = source_dict.get(key, source_dict.get(fallback)).split(' : ')
w = float(w)
results.append(True)
results.append(n)
results.append(w)
except:
results.append(True)
results.append('None')
results.append(1)

@@ -219,18 +221,18 @@ def set_data(self, raw_prompt, full_prompt, raw_negative_prompt, full_negative_p
self.steps = steps
self.base_model_name = Path(base_model_name).stem

base_model_path = os.path.join(modules.config.path_checkpoints, base_model_name)
base_model_path = get_file_from_folder_list(base_model_name, modules.config.paths_checkpoints)
self.base_model_hash = get_sha256(base_model_path)

if refiner_model_name not in ['', 'None']:
self.refiner_model_name = Path(refiner_model_name).stem
refiner_model_path = os.path.join(modules.config.path_checkpoints, refiner_model_name)
refiner_model_path = get_file_from_folder_list(refiner_model_name, modules.config.paths_checkpoints)
self.refiner_model_hash = get_sha256(refiner_model_path)

self.loras = []
for (lora_name, lora_weight) in loras:
if lora_name != 'None':
lora_path = os.path.join(modules.config.path_loras, lora_name)
lora_path = get_file_from_folder_list(lora_name, modules.config.paths_loras)
lora_hash = get_sha256(lora_path)
self.loras.append((Path(lora_name).stem, lora_weight, lora_hash))

22 changes: 12 additions & 10 deletions modules/private_logger.py
Original file line number Diff line number Diff line change
@@ -8,7 +8,7 @@
from PIL.PngImagePlugin import PngInfo
from modules.util import generate_temp_filename
from modules.meta_parser import MetadataParser

from tempfile import gettempdir

log_cache = {}

@@ -20,22 +20,24 @@ def get_current_html_path():
return html_name


def log(img, metadata, metadata_parser: MetadataParser | None = None):
if args_manager.args.disable_image_log:
return

date_string, local_temp_filename, only_name = generate_temp_filename(folder=modules.config.path_outputs, extension='png')
def log(img, metadata, metadata_parser: MetadataParser | None = None) -> str:
path_outputs = args_manager.args.temp_path if args_manager.args.disable_image_log else modules.config.path_outputs
date_string, local_temp_filename, only_name = generate_temp_filename(folder=path_outputs, extension='png')
os.makedirs(os.path.dirname(local_temp_filename), exist_ok=True)

pnginfo = None
if metadata_parser is not None:
parsed_parameters = metadata_parser.parse_string(metadata)
parsed_parameters = metadata_parser.parse_string(metadata) if metadata_parser is not None else ''
image = Image.fromarray(img)

if parsed_parameters != '':
pnginfo = PngInfo()
pnginfo.add_text('parameters', parsed_parameters)
pnginfo.add_text('fooocus_scheme', metadata_parser.get_scheme().value)
else:
pnginfo = None
image.save(local_temp_filename, pnginfo=pnginfo)

Image.fromarray(img).save(local_temp_filename, pnginfo=pnginfo)
if args_manager.args.disable_image_log:
return local_temp_filename

html_name = os.path.join(os.path.dirname(local_temp_filename), 'log.html')

22 changes: 21 additions & 1 deletion modules/util.py
Original file line number Diff line number Diff line change
@@ -179,7 +179,7 @@ def get_files_from_folder(folder_path, exensions=None, name_filter=None):
path = os.path.join(relative_path, filename)
filenames.append(path)

return sorted(filenames, key=lambda x: -1 if os.sep in x else 1)
return filenames


def calculate_sha256(filename, length=HASH_SHA256_LENGTH) -> str:
@@ -340,3 +340,23 @@ def is_json(data: str) -> bool:
except (ValueError, AssertionError):
return False
return True


def get_file_from_folder_list(name, folders):
for folder in folders:
filename = os.path.abspath(os.path.realpath(os.path.join(folder, name)))
if os.path.isfile(filename):
return filename

return os.path.abspath(os.path.realpath(os.path.join(folders[0], name)))


def ordinal_suffix(number: int) -> str:
return 'th' if 10 <= number % 100 <= 20 else {1: 'st', 2: 'nd', 3: 'rd'}.get(number % 10, 'th')


def makedirs_with_log(path):
try:
os.makedirs(path, exist_ok=True)
except OSError as error:
print(f'Directory {path} could not be created, reason: {error}')
20 changes: 14 additions & 6 deletions webui.py
Original file line number Diff line number Diff line change
@@ -515,11 +515,10 @@ def dev_mode_checked(r):

def model_refresh_clicked():
modules.config.update_all_model_names()
results = []
results += [gr.update(choices=modules.config.model_filenames), gr.update(choices=['None'] + modules.config.model_filenames)]
for i in range(flags.lora_count):
results += [gr.update(choices=['None'] + modules.config.lora_filenames), gr.update()]
return results
results = [gr.update(choices=modules.config.model_filenames)]
results += [gr.update(choices=['None'] + modules.config.model_filenames)]
for i in range(modules.config.default_max_lora_number):
results += [gr.update(interactive=True), gr.update(choices=['None'] + modules.config.lora_filenames), gr.update()]

model_refresh.click(model_refresh_clicked, [], [base_model, refiner_model] + lora_ctrls,
queue=False, show_progress=False)
@@ -581,10 +580,19 @@ def inpaint_mode_change(mode):
ctrls += [input_image_checkbox, current_tab]
ctrls += [uov_method, uov_input_image]
ctrls += [outpaint_selections, inpaint_input_image, inpaint_additional_prompt, inpaint_mask_image]
ctrls += [disable_preview, disable_intermediate_results, disable_seed_increment]
ctrls += [adm_scaler_positive, adm_scaler_negative, adm_scaler_end, adaptive_cfg]
ctrls += [sampler_name, scheduler_name]
ctrls += [overwrite_step, overwrite_switch, overwrite_width, overwrite_height, overwrite_vary_strength]
ctrls += [overwrite_upscale_strength, mixing_image_prompt_and_vary_upscale, mixing_image_prompt_and_inpaint]
ctrls += [debugging_cn_preprocessor, skipping_cn_preprocessor, canny_low_threshold, canny_high_threshold]
ctrls += [refiner_swap_method, controlnet_softness]
ctrls += freeu_ctrls
ctrls += inpaint_ctrls

if not args_manager.args.disable_metadata:
ctrls += [save_metadata_to_images, metadata_scheme]

ctrls += ip_ctrls

state_is_generating = gr.State(False)
You are viewing a condensed version of this merge commit. You can view the full changes here.