Skip to content

Latest commit

 

History

History
97 lines (66 loc) · 4.1 KB

README.md

File metadata and controls

97 lines (66 loc) · 4.1 KB
Raw

Testing

These tests ensure that the import hook behaves correctly when installing a wide variety of different crates.

The recommended way to run the tests is to run:

git submodule update ./maturin
python runner.py

This script will create an isolated environment for the tests to run in and produce a html report from the test results (see runner.py --help for options).

Alternatives

The tests can be run from any python environment with the necessary requirements but may leave temporary files on your system and other unwanted side-effects:

  • the tests install and uninstall packages from the current python virtual environment
  • the maturin import hook build cache for the current virtual environment will be cleared and used by the test
    • you can set the MATURIN_BUILD_DIR environment variable to control this
  • the tests build crates and therefore create a target/ directory
    • you can set the CARGO_TARGET_DIR variable to control this
  • if test_import_hook.common.CLEAR_WORKSPACE = False then the temporary files used during the test are not deleted

To ensure even more isolation than the runner script, you can use act to run the CI of this repository locally.

Maintenance

To update maturin:

  • update the submodule to the maturin commit you want to update to
  • re-run the package_resolver to update resolved.json (see package_resolver/README.md for instructions)
  • update requirements.txt to match the packages and versions used by the maturin ci (.github/workflows.test.yml)
    • check the uniffi package version listed in the Cargo.toml of any of the uniffi-* test crates and update uniffi-bindgen in requirements.txt to match.
  • check that no crates have been added to test-crates that should be excluded from the import hook tests. If so, add them to IGNORED_TEST_CRATES in common.py
  • update the version check in the import hook to ensure it allows using the new version

Notes

Debugging

The tests can be tricky to debug because they require spawning one or more python instances. One option is to use remote debugging with debugpy.

First, install debugpy into the test virtualenv:

test_workspace/venv/bin/python -m pip install debugpy

Add the following line in a helper script that you want to debug, or inside maturin_import_hook itself

import debugpy; debugpy.listen(5678); debugpy.wait_for_client(); debugpy.breakpoint()

Run the test you are interested in, either directly or using the test_runner script

Connect to the debugger, eg using vscode

Note: set CLEAR_WORKSPACE = False in common.py if you want to prevent the temporary files generated during the test from being cleared.

Caching

sccache is a tool for caching build artifacts to speed up compilation. Unfortunately, it is currently useless for these tests as it cannot cache cdylib crates

To run with sccache anyway (to check if the situation has improved):

sccache --stop-server  # so the tests use a separate empty sccache
# sccache cannot cache incremental compilation, so disable it: https://github.com/mozilla/sccache/issues/236
RUSTC_WRAPPER=sccache SCCACHE_DIR=/tmp/sccache CARGO_INCREMENTAL=0 python test_runner/test_runner.py <args>
sccache --show-stats

Faster Linking

You can use lld for linking using the --lld argument to the test runner. This usually provides a speed increase but not a huge one as linking is not a huge bottleneck in the testing.

Profiling

you can run the test runner with the --profile <path> argument to run the tests with cProfile. The majority of the total time is spent waiting for subprocesses and isn't very interesting.

To collect profiling stats for a particular invocation of run_python() inside a test, use the profile=<path> argument to that function.

You can view cprofile results using pyprof2calltree or flameprof.