Recreating pyinstaller exe after obfuscation

[olumide-x]

My team already uses pyinstaller to obfuscate and package an application to customers. The problem is that because our application uses a lot of large packages, e.g. scipy and numpy, the start time up time is slow because pyinstaller decompresses these packages these large packages at startup time.

To speed things up, we want to move away from packing and unpacking at runtime. We already have a spec file that collects our sources and all our dependencies into one directory. The spec file looks like something like this:

# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_data_files

datas = []
datas += collect_data_files('app_module', include_py_files=True)
datas += collect_data_files('core_module', include_py_files=True)


block_cipher = None


a = Analysis(
    ['Application.py'],
    hiddenimports=['pyarmor_runtime_006080'],
    pathex=[],
    binaries=[],
    datas=datas,
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=['sphinx', 'jinja2', 'jedi', 'pygments', 'sqlalchemy', 'pytest', 'sqlite3', 'sqlmodel'],
    win_no_prefer_redirects=False,
    win_private_assemblies=False,
    cipher=block_cipher,
    noarchive=True,
)
...

The command pyinstaller --clean --noconfirm Application.spec uses this spec file to create a small (about 350kb) exe that launches the application.

We can now obfuscate our sources in place which were copied courtesy of the command argument include_py_files=True. The problem however is that post-obfuscation when I delete the *.pyc files the application no longer runs and complains about a missing module.

However I can still run python path/to/obfuscated_application_entry_module.py. This is not of much help to customers though because pyintstaller does not package python.

Is there a way to rebuild the small exe based on the obfuscated source or rather to simply launch the obfuscated_application_entry_module.py?

[rokm]

My team already uses pyinstaller to obfuscate and package an application to customers.

If you are using PyInstaller for obfuscation, then you are using a wrong tool...

The problem is that because our application uses a lot of large packages, e.g. scipy and numpy, the start time up time is slow because pyinstaller decompresses these packages these large packages at startup time.

If decompression time of a large program is a problem, you should not be using onefile mode.

We can now obfuscate our sources in place which were copied courtesy of the command argument include_py_files=True. The problem however is that post-obfuscation when I delete the *.pyc files the application no longer runs and complains about a missing module.

I'm not sure I follow.

You seem to be using noarchive=True to bypass the PYZ archive altogether and get collected modules written into separate .pyc files - so of course if you remove those .pyc files, you will end up with missing modules?

[rokm]

Also, the onedir build also has PYZ stored in the executable-embedded PKG archive, so if pyarmor is capable of obfuscating collected python modules in a onefile executable post-hoc, it should also be able to do so with onedir executable (unless this approach works because you use noarchive=True).

[olumide-x]

I've tried just about everything that's come to mind.

what .pyc files are you deleting from the frozen application?
The proprietary ones for which I have *.py files, which I subsequently obfuscate.

The problem is that I have a small exe that obviously launches the entry point module. Is it is possible to rebuild this exe and have it launch the obfuscated entry module? As I said, the entry module can be run using python but unfortunately the python dll is the only Pythion binary that is packaged.

[rokm]

I suppose not.

Why don't you create a simple unobfuscated entry-point script for the exe that launches the obfuscated one, then?

But also... if you are doing this obfuscation manually and post-hoc, are you ensuring that the original unobfuscated app_module and core_modules are not ending up in PYZ archive? I imagine you would need to exclude them explicitly, which would then cause additional issues because you would need to manually track their depenencies... (EDIT: nevermind about this part - I forgot the noarchive=True is in effect).

[olumide-x]

I've made some progress by setting noarchive=False and now have a smaller binary, however one of the modules (core/util/SharedData.py) in the core library is not being "collected" even when listed in the spec file as shown in the excerpt below. (I temporarily reset noarchive=True to see which modules are collected and this module is consistently not collected.)

datas += collect_data_files('app')
datas += collect_data_files('core')

a = Analysis(
    ['app/Application.py', 'core/util/SharedData.py'],
    ...
    noarchive=True,
)

What would be the reason for this? More importantly is there a workaround? The obfuscated application does not start because this module is missing.

I had to downgrade to pyinstaller v5.13.0 because it is the oldest version that pyarmor supports.

[rokm]

Because you listed a module as an additional entry-point script. So now the executable runs PyInstaller's bootstrap scripts and modules, runs your app/Application.py, and runs your core/util/SharedData.py as a script. But of course this "script" is not visible as a module to other scripts/modules.

You should add core.util.SharedData to hidden imports. You would also need to ensure that the directory containing core directory is in the python search path, but I suppose if that collect_data_files('core') works for you as expected, this is already the case.