-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
generate_emu_config overhaul (a lot of bugfixes, improvements, new features, new file structure, new helper tools) #2
base: dev
Are you sure you want to change the base?
Conversation
…igs, proper ini parsing, better logging and error handling, helper tools: * add `-def1` ... `-def5` arguments, which can be used to generate your preferred custom config if no `-def` argument is provided, `-def1` will be used by default, to automatically copy from the following folders: * `.\_DEFAULT\0` ............... essential emu files, like latest GSE dlls (*steam_api.dll* and *steam_api64.dll*) * `.\_DEFAULT\1` ............... other GSE files and folders, including default ini files * `.\_DEFAULT\<appid>` ... other GSE files and folders, but only for the current `<appid>`, if the folder exists * (Windows only) add some useful helper tools, written in *AutoIt3*: * **gse_acw_helper.exe** - add the required achievements schema db files for *Achievement Watcher*, if .\steam_misc\extra_acw\extra_acw.zip file exists (if generated by `generate_emu_config.exe -acw <appid>`) * **gse_debug_switch.exe** - automatically switch between release and debug versions of the emulator, if *steam_api.7z* / *steam_api64.7z* file exists (or *steamclient.7z* / *steamclient64.7z*, if you use the steamclient version) paths to release and debug files inside 7z, can be customized in .\steam_misc\tools\au3\scripts\gse_debug_switch.ini * **gse_generate_interfaces.exe** - simple x64-x86 launcher for *generate_interfaces.exe* it also writes all found steam interfaces to CODEX *steam_emu.ini* (if generated by `generate_emu_config -cdx <appid>`) make sure to name your original dll to one of these formats, so it can automatically find its interfaces: * `valve_api.dll / valve_api64.dll` * `steam_api.dll.bak / steam_api64.dll.bak` or `steam_api.dll.org / steam_api64.dll.org` * `steam_api.bak / steam_api64.bak` or `steam_api.org / steam_api64.org` * `steam_api_orig.dll / steam_api64_orig.dll` or `steam_api_legit.dll / steam_api64_legit.dll` * **gse_lobby_connect.exe** - simple x64-x86 launcher for *lobby_connect.exe* * new folder structure, compatible with current and future helper tools --- default arguments are `-acw -cdx -clr <appid>` NEVER delete `.\steam_misc\app_backup`, `.\steam_misc\app_info`, `.\steam_misc\tools` and `.\steam_settings` folders MIGHT need `.\steam_misc\extra_acw` and `.\steam_misc\extra_cdx` for compatibility with Achievement Watcher and CODEX * add `-scx` argument to automatically download images / videos for trading cards, backgrounds, badges, emoticons and other tradable items unfortunately I couldn't find any direct steam api method to download the files, so I had to write a rudimentary web scrapper to extract the download links from a third-party website, hence the *scx_gen.py* script might need updating in the future if the website design changes * download screenshots and videos: * download thumbnails for both screenshots and videos, and compress them to `.zip` files * screenshots and videos are now numbered from first to last published, as in the Steam store page * add `-vids_low` / `-vids_max` arguments to download all videos, in low and / or high quality * create / update .\\*top_owners_ids.txt* when .\\*top_owners_ids.html* is present * generate controller action sets txt files for all found controller vdf configs, and zip them inside .\steam_misc\app_backup\app_backup.zip by default, the emu supports only `xboxone` and `xbox360` controller configs, though if the're are any issues with the default supported controller action sets inside .\steam_settings\controller folder, you could try to unpack and overwrite action sets for other unsupported controller configs * (Windows only) add *AdvancedRun* launchers (cmd console + silent) for `.bat` files and `.py` scripts * **[alex47exe]** major overhaul of **migrate_gse** - uses the same `.\_DEFAULT\0` and `.\_DEFAULT\1` folder structure for default configs it can convert old `.txt` format to `.ini` format, minus *branches.json*, which would require using *top_owners_ids.txt* and some login code from **generate_emu_config**, which should actually be used to properly generate the config files, instead of converting from the old `.txt` format * [alex47exe] *generate_interfaces.exe* - find all Steam Interfaces instead of only old ones the emu will ignore the ones it doesn't require, while we'll have the complete list to write it to CODEX *steam_emu.ini* * [alex47exe] *lobby_connect.exe* - improve cmd console text alignment * [alex47exe] `mods_img` instead of `mod_images` (better folder consistency), better example for `mods_img`, minor tweaks to `.ini` and `.md` files
Not sure why it takes so long to build the experimental debug api here. Mine was done in less than 3 minutes: |
Seems like github issue (I could merge without actually waiting to finish all builds btw) will reopen and probably request some changes (mainly in the ini what should be default) not today but tomorrow. |
Sure, any suggestions are welcome! I just copy pasted the custom ini files included in default configuration for generate_emu_config, but forgot to restore the original values for the example ini files. |
:D no worries |
tools/generate_emu_config/generate_emu_config.py_ini.exe |
___ The exe files are basically just stripped down (no interface) AdvancedRun executables plus their required cfg (commented / documented by me), basically a better method to run various scripts and apps with parameters (including python), with or without interface if required (useful when you need to run an app or script silently -no cmd window flash even-, e.g. when we might need to integrate generate_emu_config into any other automated tool). To be honest, I'm surprised Microsoft doesn't include AdvancedRun by default in an app bundle (and properly integrated with context menu), like they do with Sysinternals Suite or PowerToys, 'cause I really find it the best method to easily run apps and scripts with parameters. Anyway, about those exes:
___ If you are referring to steam_api.7z and steam_api64.7z, they include both release and debug versions of the emulator, needed for gse_debug_switch.exe - I think one double-click to run it is a much superior method of changing between release and debug versions when needed, instead of having to manually backup and copy paste them ___ 7za.exe is currently required by gse_debug_switch.exe and gse_acw_helper.exe, and might be required by future tools I currently have some ideas for. I could've used an AutoIt3 UDF script (user defined functions) for zip compression / decompression, but I prefer 7z. Though I used zip compression in generate_emu_config.py for the app_backup folder, screenshots and trailers thumbnails, and parsed download links for SCX scrapper. What I could do is intergrate 7za.exe in a3x script, and unpack it when the script runs, but that would mean every other future a3x script would have to integrate it as well, in the end taking more hdd space than the current method. |
For 7z: For the steam apis 7z: Also probably could make a github action for building the tools rather than having the exe inside there (which I dont like it.) |
Regarding steam apis 7z, the way I handled them was to manually build the emu on windows, then manually create the steam apis 7z containing both release and debug versions, then push the whole github folder using GitExtensions. Sure, I would love to automate this, but unfortunately I am not experienced with github actions. I guess it would basically mean that the generate_emu_config build would have to be delayed after the emu build, so that the compiled steam apis can be copied and archived to steam apis 7z first. |
…ality also corrected Mr_Goldberg's name in some .md files
For familiar people, this PR appears bloated. |
What do you mean 'bloated'? Sure, maybe it should have been split in more commits, as I said above, but other than that, it only improves generate_config_emu and migrate_gse (though currently it can't generate branches.json, so I don't recommend using it - just regenerate the entire config instead), adds much needed helper tools for automation of various tasks, and corrects spelling mistakes / adds minor changes to wording and punctuation to some readme files. That's literally all it does. It's not 'bloated', it's improved - you just haven't tested it thoroughly to understand it. |
Just reopened it to be able to fix conflicts with upstream fork, so I can correctly merge the latest changes into my own |
…itch helper tool - fix configs.main.ini for generate_emu_config, to support latest changes
Sure thing. Haven't really had the time lately to fully and correctly automate all the necessary building and packaging of extra tools I added. There's still some new useful things I'll add into the generate_emu_config tool, like automatically finding and creating shortcuts to the configuration and savegame folders based on data already present in app_details.json (which is part of the reason I decided to include by default some extra json files based on app_details.json) and also scrapped info from PCGamingWiki page for each game - the page link can be obtained automatically through https://www.pcgamingwiki.com/api/appid.php?appid=391220 (example for 'Rise of the Tomb Raider) |
…itch helper tool - disable overlay by default for generate_emu_config (seems it crashes a lot of games now after some recent updates to the way achivement icons are loaded)
2767fec
to
8141643
Compare
1c5a738
to
1d68c3d
Compare
now using steam id3 (correct) instead of steam id64 (wrong)
- added bat scripts to automatically pack latest steam_api dlss and tools for generate_emu_config and migrate_gse - removed previous steam_api dlls and 7z archives from generate_emu_config and migrate_gse - removed previous generate_interfaces and lobby_connect 7z archives from generate_emu_config and migrate_gse - moved some default required files for generate_emu_config from _DEFAULT\1 to _DEFAULT\0 folder, thus eliminating the need to duplicate them for every other default configuration folder you might create (_DEFAULT\2 - _DEFAULT\5)
b7d713f
to
9c18c68
Compare
replace 'client_id' error with simple 'wrong username and / or password' message, asking for correct username and / or password add pcgw_page script to download PCGamingWiki page (source code only), not yet enabled in main script generate steam_emu.ini for CODEX and RUNE versions, update gse_generate_interfaces
copy _DEFAULT folders for Linux too
replaced %outdir% with $outdir for linux build scripts
should now be correctly parsed from 'app::general' instead of 'main::general'
add top owners_ids.txt to Linux build too
[alex47exe] major overhaul of generate_emu_config - custom configs, proper ini parsing, better logging and error handling, helper tools:
add
-def1
...-def5
arguments, which can be used to generate your preferred custom configif no
-def
argument is provided,-def1
will be used by default, to automatically copy from the following folders:.\_DEFAULT\0
............... essential emu files, like latest GSE dlls (steam_api.dll and steam_api64.dll).\_DEFAULT\1
............... other GSE files and folders, including default ini files.\_DEFAULT\<appid>
... other GSE files and folders, but only for the current<appid>
, if the folder exists(Windows only) add some useful helper tools, written in AutoIt3:
generate_emu_config.exe -acw <appid>
)paths to release and debug files inside 7z, can be customized in .\steam_misc\tools\au3\scripts\gse_debug_switch.ini
it also writes all found steam interfaces to CODEX steam_emu.ini (if generated by
generate_emu_config -cdx <appid>
)make sure to name your original dll to one of these formats, so it can automatically find its interfaces:
valve_api.dll / valve_api64.dll
steam_api.dll.bak / steam_api64.dll.bak
orsteam_api.dll.org / steam_api64.dll.org
steam_api.bak / steam_api64.bak
orsteam_api.org / steam_api64.org
steam_api_orig.dll / steam_api64_orig.dll
orsteam_api_legit.dll / steam_api64_legit.dll
new folder structure, compatible with current and future helper tools --- default arguments are
-acw -cdx -clr <appid>
NEVER delete
.\steam_misc\app_backup
,.\steam_misc\app_info
,.\steam_misc\tools
and.\steam_settings
foldersMIGHT need
.\steam_misc\extra_acw
and.\steam_misc\extra_cdx
for compatibility with Achievement Watcher and CODEXadd
-scx
argument to automatically download images / videos for trading cards, backgrounds, badges, emoticons and other tradable itemsunfortunately I couldn't find any direct steam api method to download the files, so I had to write a rudimentary web scrapper to extract the download links from a third-party website, hence the scx_gen.py script might need updating in the future if the website design changes
download screenshots and videos:
.zip
files-vids_low
/-vids_max
arguments to download all videos, in low and / or high qualitycreate / update .\top_owners_ids.txt when .\top_owners_ids.html is present
generate controller action sets txt files for all found controller vdf configs, and zip them inside .\steam_misc\app_backup\app_backup.zip
by default, the emu supports only
xboxone
andxbox360
controller configs, though if the're are any issues with the default supported controller action sets inside .\steam_settings\controller folder, you could try to unpack and overwrite action sets for other unsupported controller configs(Windows only) add AdvancedRun launchers (cmd console + silent) for
.bat
files and.py
scripts[alex47exe] major overhaul of migrate_gse - uses the same
.\_DEFAULT\0
and.\_DEFAULT\1
folder structure for default configsit can convert old
.txt
format to.ini
format, minus branches.json, which would require using top_owners_ids.txt and some login code from generate_emu_config, which should actually be used to properly generate the config files, instead of converting from the old.txt
format[alex47exe] generate_interfaces.exe - find all Steam Interfaces instead of only old ones
the emu will ignore the ones it doesn't require, while we'll have the complete list to write it to CODEX steam_emu.ini
[alex47exe] lobby_connect.exe - improve cmd console text alignment
[alex47exe]
mods_img
instead ofmod_images
(better folder consistency), better example formods_img
, minor tweaks to.ini
and.md
files