Multi ipc testing #380
Merged
Multi ipc testing #380
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jul 12, 2025
Merged
Such that we can run (opting-in) tests on both TCP and UDS backends and ensure the `reg_addr` fixture and various timeouts are adjusted accordingly. Impl deats, - add a new `tpc_proto` CLI option and fixture to allow choosing which "transport protocol" will be used in the test suites (either globally or contextually). - rm `_reg_addr` instead opting for a `_rando_port` which will only be used for `reg_addr`s which are net-tpt-protos. - rejig `reg_addr` fixture to set a ideally session-unique `testrun_reg_addr` based on the `tpt_proto` setting making appropriate calls to `._addr` APIs as needed. - refine `daemon` fixture a bit with typing, `tpt_proto` timings, and stderr capture. - in `test_discovery` do a ton of type-annots, add `debug_mode` fixture opt ins, augment `spawn_and_check_registry()` with `psutil.Process` passing for introspection (when things go wrong..).
Via a new accumulative `--tpt-proto` arg you can select which
`tpt_protos: list[str]`-fixture protocol keys will be delivered to
opting in tests!
B)
Also includes,
- CLI quote handling/stripping.
- default of 'tcp'.
- only support one selection per session at the moment (until we figure
out how we want to support multiples, either simultaneously or
sequentially).
- draft a (masked) dynamic-`metafunc` parametrization in the
`pytest_generate_tests()` hook.
- first proven and working use in the `test_advanced_faults`-suite (and
thus its underlying
`examples/advanced_faults/ipc_failure_during_stream.py` script)!
|_ actually needed this to prove that the suite only has 2 failures on
'uds' seemingly due to low-level `trio` error semantics translation
differences to do with with calling `socket.close()`..
On a very nearly related topic,
- draft an (also commented out) `set_script_runtime_args()` fixture idea
for a std way of `partial`-ling in runtime args to `examples/`
scripts-as-modules defining a `main()` which would proxy to
`tractor.open_nursery()`.
In `tests/test_advanced_faults.py` that is. Since instead of zero-responses like we'd expect from a network-socket we actually can get a few differences from the OS when "everything IPC is known" XD Namely it's about underlying `trio` exceptions versus how we wrap them and how we expect to box them. A `TransportClosed` boxing improvement is coming in follow up btw to make this all work! B)
Such that the global test-session always (and only) runs against the CLI specified `--tpt-proto=` transport protocol.
Namely while what I was actually trying to solve was why `TransportClosed` was getting raised from `Portal.cancel_actor()` but still useful edge case auditing either way. Also opts into the `debug_mode` fixture with apprope timeout adjustment B)
For now it just boots a server, parametrized over all tpt-protos, sin any actor runtime bootup. Obvi the future todo is ensuring it all works with a client connecting via the equivalent lowlevel `.ipc._chan._connect_chan()` API(s).
Namely any CLI driven runtime-config fixtures such as,
- `--spawn-backend` and `start_method`,
- `--tpdb` and `debug_mode`,
- `--tpt-proto` and `tpt_protos`/`tpt_proto`,
- `reg_addr` as driven by the above.
This moves all fixtures and necessary hook funcs (CLI parsing,
configuring and test-gen) to the `._testing.pytest` module and thus
allows any dependent project to leverage these fixtures in their own
test suites after pointing to that plugin mod using,
```python
# conftest.py
pytest_plugins: tuple[str] = (
"tractor._testing.pytest",
)
```
Also, add a new `._testing.addr` helper mod which now contains
a factored `get_rando_addr()` helper for creating test-sesh unique
tpt-specific registry (or other) IPC endpoint addrs.
We already have the `.ipc` sub-pkg name so it seems a bit redundant/noisy for a namespace path Bp Leave an alias for the `Server` rn since it's already used in a few other internal mods.. will likely rename later if everyone is cool with it..
Like it sounds, verifying that when that param is passed to the runtime startup eps (`.open_root_actor()/.open_nursery()`), the appropriate tpt-protocol is deployed for IPC (both the server and bound endpoints) in both the root and any sub-actors (as passed down from rent to child via the `.msg.types.SpawnSpec`).
For now just as sanity that we're not breaking anything on that transport backend (since just a little while back there were issues with crash handling in subs..) when it comes to crash-REPLing.
goodboy
force-pushed
the
multi_ipc_testing
branch
from
2c7238c to
c9cd810
Compare
Bc why clutter the intro like everyone else.. instead put them just above the install section.
goodboy
force-pushed
the
multi_ipc_testing
branch
from
c9cd810 to
1eb0d78
Compare
goodboy
commented
Jul 13, 2025
| @@ -0,0 +1,95 @@ | |||
| ''' | |||
Owner
Author
There was a problem hiding this comment.
@guilledk mostly it'd be nice to get eyes on this new susys-unit-suite more then anything else 🙏🏼
goodboy
commented
Jul 13, 2025
| return trio.run(main) | ||
|
|
||
| return wrapper | ||
|
|
Owner
Author
There was a problem hiding this comment.
Aforementioned pytest plugin factoring for usage in other tractor dependents using the pointer mentioned in the descr.
goodboy
commented
Jul 13, 2025
| ) | ||
|
|
||
|
|
||
| def get_rando_addr( |
Owner
Author
There was a problem hiding this comment.
Also of (more minor) note for any reviewers; just a helper to generate rando-non-colliding tpt-proto-specific addrs to avoid multi-test-suite collisions running on the same host.
Owner
Author
|
@guilledk think this is ready for a quick check! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow up to #375 adding,
multi-IPC transport parametrization controls to the test harness
via a new
--tpt-protoarga first draft micro-suite of unit-tests for the new
tractor.ipc._serversubsys APIspluginize-ation of various useful (and general) fixtures and CLI
flags for consumption in dependent projects which are built on
tractoras a runtime, previously these only were found inspecific
conftest.pyfiles but now can be included via,any other make CI green.
Follow-through from #375
pytestflags.ipc._server.ipc._serversubsys in unit-style-isolation,Still Todo before landing
small suite to verify
enable_transportsis activated in bothroot and subactors.
enable_transportstest-suite4b21da9, couldn't get it into The Structural Dynamics of Flow: finally a formalized modular transport layer ->
tractor.ipc😎 #375 due to test mod conflicts._servertypes0065162, "Flip a couple more debug scripts to UDS tpt" for sanity.
Follow-up (if not landed here)