Obfuscating scripts in-place

[olumide-x]

My team already uses pyarmor and pyinstaller to obfuscate and package a commercial application to customers. We have a pyarmor group license.

The problem is that because our application uses a lot of large packages, e.g. scipy and numpy, the start time up time is slow because pyinstaller decompresses these packages these large packages at startup time.

To speed things up, we want to move away from packing and unpacking at runtime. We already have a spec file that collects our sources and all our dependencies into one directory. The spec file looks like something like this (take note of include_py_files=True -- this is how we copy our sources not just pyc files.):

# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_data_files

datas = []
datas += collect_data_files('app_module', include_py_files=True)
datas += collect_data_files('core_module', include_py_files=True)


block_cipher = None


a = Analysis(
    ['Application.py'],
    hiddenimports=['pyarmor_runtime_006080'],
    pathex=[],
    binaries=[],
    datas=datas,
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=['sphinx', 'jinja2', 'jedi', 'pygments', 'sqlalchemy', 'pytest', 'sqlite3', 'sqlmodel'],
    win_no_prefer_redirects=False,
    win_private_assemblies=False,
    cipher=block_cipher,
    noarchive=True,
)
...

Besides being able to run our application runs successfully from this directory, there is the added benefit of being able to manually delete a whole bunch of unnecessary modules and directories that that pyinstaller has copied.

The only problem is that our sources are not obfuscated. This is the problem that we need help with. I have read the docs and tried so many things, too many to mention, but nothing seems to work. Here is my latest, best, attempt. Note that we don't obfuscate third party modules such as scipy and numpy because this is unnecessary and would take too long.

pyinstaller --clean --noconfirm Application.spec  # creates dist\Application directory with ALL sources, modules and DLLs
cd dist\Application
pyarmor gen -r -O ..\..\dist\Application app_dir  # armor only our sources

This indeed armors all our sources in app_dir in place; however when I run the application I get the error:

Traceback (most recent call last):
  File "app_dir/Application.py", line 16, in <module>
  File "C:\repos\project\dist\Application\app_dir\__init__.py", line 2, in <module>
    from pyarmor_runtime_006080 import __pyarmor__
  File "C:\repos\project\dist\Application\pyarmor_runtime_006080\__init__.py", line 2, in <module>
    from .pyarmor_runtime import __pyarmor__
ImportError: DLL load failed while importing pyarmor_runtime: The specified module could not be found.
[11644] Failed to execute script 'Application' due to unhandled exception!

Please advise.

Thanks.

[jondy]

Please refer to
https://pyarmor.readthedocs.io/en/latest/topic/repack.html#packing-obfuscated-scripts-manually
If you have specfile work, it's the right for this case

[olumide-x]

I've been struggling to get the patching to work.

PyInstaller terminates with the RuntimeError "No obfuscated script found". The modification to the spec file finds the unobfuscated entry module, Application.py, listed in the analysis section. Should the analysis section list the obfuscated modules? (As per your instructions I pointed pyi-makespec at the unobfuscated entry module, Application.py.)

[olumide-x]

I've also tried

pyarmor gen --pack onedir -r .\app\Application.py ..\core

The process runs successfully to completion but when I try to launch the application I get the error message

Traceback (most recent call last):
  File "dist\Application.py", line 2, in <module>
  File "PyInstaller\loader\pyimod02_importers.py", line 385, in exec_module
  File "pyarmor_runtime_006080\__init__.py", line 2, in <module>
    from .pyarmor_runtime import __pyarmor__
ImportError: Module use of python310.dll conflicts with this version of Python.
[15520] Failed to execute script 'Application' due to unhandled exception!

The python310.dll conflict message is particularly confusing because I created a Python 3.10 venv to run these commands.

[jondy]

First make sure the obfuscated scripts without pack could work, then enable pack and debug mode to check the log.
It's better to do in a clean directory

[olumide-x]

I am getting better results now. I can now launch an obfuscated script. The problem was that I was running the pyinstaller step in MYSYS instead of Windows CMD or Powershell. I have consistently found that pyarmor does not work in MYSYS. (If MSYS is not supported it would be nice if pyarmor detected an unsupported environments and warned users.)

I can now use pyarmor to patch the exe generated by pyinstaller. Here is an excerpt of the build log:

INFO     obfuscating file AppLauncher.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write .pyarmor\pack\dist\app\AppLauncher.py
INFO     obfuscating file BaseApplication.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write .pyarmor\pack\dist\app\BaseApplication.py
INFO     obfuscating file Logging.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write .pyarmor\pack\dist\app\Logging.py
INFO     obfuscating file Manual.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write .pyarmor\pack\dist\app\Manual.py
INFO     obfuscating file Version.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write .pyarmor\pack\dist\app\Version.py
INFO     obfuscating file WidgetPool.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write .pyarmor\pack\dist\app\WidgetPool.py
INFO     obfuscating file SignalData.py
DEBUG    process inline marker
DEBUG    parse script
DEBUG    process co
DEBUG    patch co
INFO     write .pyarmor\pack\dist\app\common\SignalData.py
...
2024-05-10 13:49:35,484 repacking bundle ".\dist\Application\Application.exe"
2024-05-10 13:49:35,484 obfuscated scripts at ".pyarmor\pack\dist"
2024-05-10 13:49:35,484 entry script name is "Application.py"
2024-05-10 13:49:35,484 runtime package at .pyarmor\pack\dist\pyarmor_runtime_006080
2024-05-10 13:49:35,484 repacking "PYZ-00.pyz"
2024-05-10 13:49:35,485 replace item "pyarmor_runtime_006080"
2024-05-10 13:49:35,485 replace item "pyarmor_runtime_006080"
2024-05-10 13:49:35,499 repacking PKG "PKG-patched"
2024-05-10 13:49:35,507 Compiling python script/module file .pyarmor\pack\dist\Application.py
2024-05-10 13:49:35,511 repacking EXE ".\dist\Application\Application.exe"

I can see that my modules are obfuscated and stored in .pyarmor\pack\dist and the process ends with statements like repacking EXE ".\dist\Application\Application.exe but how can I be sure that my pyc files are obfuscated? Recall that my spec file sets noarchive=True as shown in my original question, excerpt below:

a = Analysis(
    ['Application.py'],
    ...
    noarchive=True,
)

I am not sure that the pyc files are obfuscated because the following works in the dist\Application directory, created by installer, but does NOT work in the .pyarmor\pack\dist directory.

>>> import Logging
>>> dir(Logging)

[jondy]

Please check
https://pyarmor.readthedocs.io/en/latest/topic/repack.html