Skip to content

Support 14.3/14.4 export path changes properly #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Morilli wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Support 14.3/14.4 export path changes properly #112

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
98fb99d
Support 14.3/14.4 export path changes properly
Morilli May 31, 2025
1762eb4
use == instead of >=
Morilli Jun 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions cdtb/export.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,11 +196,12 @@ def filter_exporter(self, other):
del self.wads[path]
else:
# compare the sha256 hashes to find the common files
# don't resolve hashes: we just need the sha256
logger.debug(f"filter modified WAD file: {path}")
other_sha256 = {wf.path_hash: wf.sha256 for wf in other_wad.files}
other_wad = {wf.path_hash: wf for wf in other_wad.files}
Copy link
Member

@benoitryder benoitryder Jun 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A simpler approach: compare both sha and path (through the hash, no need to resolve). If I understand your changes correctly, it should work.

# Compare the sha256 and path to find the common files.
logger.debug(f"filter modified WAD file: {path}")
# Checking the path is required for unmodified files (same sha) that are moved
# They will be exported to a different path and cannot be symlinked
other_wads = {(wf.path_hash, wf.sha256) for wf in other_wad.files}
# Change the files from the wad so it only extract these
self_wad.files = [wf for wf in self_wad.files if (wf.path_hash, wf.sha256) in other_wads]

Copy link
Contributor Author

@Morilli Morilli Jun 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the first comment is misleading; the point of this function is to find changed/new files and only export those. So we're looking for all files that either don't exist in other_wad or have changed.

But we do need to check the actual export path to make sure that patch-specific exporter changes are respected. In this case when self is the exporter for 14.3 and other is the exporter for 14.2, the path assets/ux/endofgame/en_us/defeat.dds from UI.en_US.wad.client has the same hash and sha for both, but the export path for other is assets/ux/endofgame/en_us/defeat.png and the one for self is en_us/assets/ux/endofgame/en_us/defeat.png. If only hash and sha are checked the file will not be kept and instead symlinked, which will result in an invalid symlink.

Copy link
Member

@benoitryder benoitryder Jun 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code assumed that the same file with the same path would be exported the same.
Exporting differently depending on the patch version is definitely clunky. :(

I'm not a fan of having to regenerate export paths at this place. But right now I don't have a good solution.

# change the files from the wad so it only extract these
self_wad.files = [wf for wf in self_wad.files if wf.sha256 != other_sha256.get(wf.path_hash)]
# in case sha is the same but export path(s) changes, also keep the file
self_wad.files = [wf for wf in self_wad.files if (other_wf := other_wad.get(wf.path_hash)) is None or wf.sha256 != other_wf.sha256
or list(self.converted_exported_paths(wf.path)) != list(other.converted_exported_paths(other_wf.path))]
if not self_wad.files:
del self.wads[path]

Expand Down Expand Up @@ -462,6 +463,10 @@ def filter_path(exporter, path):
if patch_version == "main" or patch_version >= PatchVersion("14.4"):
if lang_match := re.search(r"\.(.._..)\.wad\.client$", path):
subdir = f"game/{lang_match.group(1).lower()}"
# for patch 14.3, only export localized UI wad into subdirs
elif patch_version == PatchVersion("14.3"):
if ui_lang_match := re.search(r"UI\.(.._..)\.wad\.client$", path):
subdir = f"game/{ui_lang_match.group(1).lower()}"
for wf in wad.files:
wf.path = f"{subdir}/{wf.path}"
wad.files = [wf for wf in wad.files if filter_path(exporter, wf.path)]
Expand Down