Repository navigation
test: make the suite hermetic against inherited PYTHONPATH - #22
Open
jmvbambico wants to merge 1 commit into
Open
jmvbambico wants to merge 1 commit into
jmvbambico wants to merge 1 commit into
Conversation
`bin/og` exports PYTHONPATH=$OG_POLICY_PATH when it launches the host daemon, so every coding agent it runs inherits it. Anyone running pytest from inside an og-managed session therefore executes the suite with PYTHONPATH=~/.omnigent/policies set. PYTHONPATH is applied before any site-packages .pth, so it shadowed the test fixtures' stand-ins. install_pth's verification subprocess imported the developer's real policies module, which imports `omnigent.policies` that the tests' fake venv does not have, and died with SystemExit(1). Two tests failed -- but only for developers who have actually installed og, since CI has no ~/.omnigent and no PYTHONPATH. The product is correct as-is and is untouched: bin/og's export is deliberate and documented, and install_pth checking in the environment the daemon really runs in is more faithful, not less. The contamination was a property of the harness, so the fix belongs here. An autouse fixture strips PYTHONPATH for every test. tests/ test_env_isolation.py asserts it holds in-process, that a subprocess spawned from a test does not see it, and -- the part that matters, since the other two are vacuous without an ambient value -- that a nested pytest run handed a poisoned PYTHONPATH still comes back clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two tests fail for anyone who has actually installed og, and pass in CI. This
makes the suite hermetic instead.
Root cause
bin/oglaunches the host daemon withPYTHONPATH="$OG_POLICY_PATH${PYTHONPATH:+:$PYTHONPATH}"— deliberately, as afallback for when the installer's
.pthgoes stale. Everything that daemonspawns inherits it, including the coding agents it runs. So running pytest
from inside an og-managed session executes the suite with
PYTHONPATH=~/.omnigent/policiesset.PYTHONPATHlands onsys.pathbefore anything a site-packages.pthappends. So
install_pth's verification subprocess imported the developer'sreal 22.6 KB
~/.omnigent/policies/omnigent_local_policies.pyinstead of thefixture's stand-in. That file does
from omnigent.policies.builtins import orchestration, the test's fake venv(
venv.create(..., with_pip=False)) has noomnigent, so the import raisedModuleNotFoundError,die()turned it intoSystemExit(1), and the testfailed.
CI never sees it: no
~/.omnigent, noPYTHONPATH.Measured on
main, no code changes:Worth noting it is not only a red-build problem: on a contaminated machine the
.pthtests could also pass for the wrong reason, with the import theyverify resolving through
PYTHONPATHrather than through the.pthundertest.
The fix
An autouse fixture in the repo-root
conftest.pystripsPYTHONPATHfor everytest. Suite-wide rather than scoped to the two known failures: no test
references
PYTHONPATH, so nothing wants the inherited value, and any testthat spawns a subprocess is exposed whether or not it fails today.
tests/test_env_isolation.pyguards it — in-process, in a spawned subprocess,and (the load-bearing one, since the first two are vacuous when the ambient
value is already unset) by re-execing a nested pytest with a deliberately
poisoned
PYTHONPATHand asserting the inner run still comes back clean.The product is deliberately untouched
Neither
bin/ognorinstall_pthis wrong:bin/og's export is intentional and documented in place.install_pthverifying in the environment the daemon really runs in ismore faithful, not less — its comment says "this is exactly what the server
will see", and that is true.
The contamination is a property of the test harness, so the fix lives there.
Verification
Proven non-vacuous: with the fixture flipped to
autouse=FalseandPYTHONPATHset, all three isolation tests fail and the two originalfailures return.
bash -n bin/og install.shandshellcheck -S error bin/og install.shclean(untouched).
Known residual
The scrub fixes the environment, not this interpreter's own
sys.path: Pythonapplies
PYTHONPATHat boot, so an entry the pytest process started with isalready baked in and
monkeypatch.delenvcannot un-apply it. Retroactivelypurging it would mean mutating a global every test shares. The guarantee that
matters is the environment, because that is what a subprocess launches from —
and that is the direction the
install_pthfailure actually travelled. Theprobe test says so in a comment.
🤖 Generated with Claude Code