Skip to content

Reframe around explainable selection and the decision not to build a bandit; make every test run offline - #2

Open
retinapeg wants to merge 22 commits into
mainfrom
ai-project-reframe
Open

retinapeg wants to merge 22 commits into
mainfrom
ai-project-reframe

Conversation

@retinapeg

Copy link
Copy Markdown
Owner

Summary

Repositions the repository around explainable selection with deterministic signals and a measured decision about when not to use machine learning; the decision not to build a contextual bandit is now the centrepiece. It also makes the extract run. Before this change, a clean clone collected no tests at all, because the private store, classifier and Gmail helpers were missing. With minimal stand-ins added, all 85 tests pass offline with only pytest installed, on Python 3.9 or later, and a synthetic demo runs on the standard library alone. Along the way the review found, and this change closes, a path where the recipient guard failed open.

What changed

Offline runnability

  • pytest.ini and requirements-dev.txt are added.
  • Minimal stand-ins are added for careerops.store, careerops.inventory, careerops.discovery and the two job_cv_agent Gmail helpers. Each docstring says how it differs from the private version.
  • scripts/demo.py seeds a temporary store with invented vacancies. It runs the real census and a dry run of the real digest, and never contacts Gmail.
  • The test capture in docs/images/ is regenerated from a clean-clone run.

Fixes

  • Recipient guard. If the Gmail profile lookup failed and the token named no account, a configured non-owner address could receive the digest. A real send now requires Gmail to confirm the mailbox, unless another recipient is explicitly enabled. The token preflight now also requires the read scope that this check depends on.
  • Scheduled coverage budgets. These are now never below the project defaults. A fresh chain previously wrote a 600-second limit below the 900-second default.
  • The stale-run reaper now clears only positively terminal statuses, where before it cleared anything that was not running.
  • Apply links are printed in full, up to 2,048 characters, where before they were cut at 400.
  • Configured country keys are compared case-insensitively.
  • The counts line now accounts for roles outside the configured locations.
  • The empty-digest text no longer claims that collection is running.

Docs

  • Citations of AGENTS.md keep their original wording, with a note that the file is not included in this repository.
  • Docstring wording that described a personal location preference is made neutral.
  • The architecture diagram shows the stand-ins and the demo. Its render config is committed.
  • The README is rewritten. It keeps the original first-person statements about the bandit decision and the findings.

Tests

Before After
Clean clone, python -m pytest 2 collection errors 85 passed
With PYTHONPATH=src 24 passed, 13 failed, 1 collection error 85 passed
Python 3.9, pytest 8.4 no requirements file 85 passed
python scripts/demo.py not present exit 0

Each behavioural fix has a regression test that was confirmed to fail with the fix reverted.

Decisions for review

  • The bandit figures come from the private store. The README labels them as measured there and not reproducible from this repository.
  • No labelled evaluation is added here. Selection is fixed rules. Scoring those rules against labels written by the same author would measure the labels, not the system, so the tests assert the rules directly instead.

…pers

The digest, the refresh loop and the tests import a job store, a location
classifier, the collector's coverage defaults and two Gmail helpers that
live in the private system. These stand-ins provide only that surface.
Each module's docstring states how it differs from the private version:
no rescore on open, no private default settings, a classifier that matches
configured city names and the word London only, and a Gmail client that
refreshes a stored token and never starts a consent flow.
pytest.ini puts src on the import path and points collection at tests,
so a clean clone needs no environment variables. requirements-dev.txt
pins the pytest version the suite was run with.
The self-only recipient guard failed open. mailbox_address() returns an
empty string when the Gmail profile lookup fails, and resolve_recipient()
then falls back to the token file's account. That field is often empty,
in which case the comparison was skipped and the digest went to whatever
address was configured, even one that is not the authenticated mailbox.

deliver() now refuses a real send when Gmail has not reported the mailbox
address, unless allow_other_recipient has been set to true. Dry runs and
empty digests are unaffected, as nothing is sent in either case. The new
test reproduces the failure: before the fix it did not raise and the
message was handed to the sender.
The digest already counted roles excluded by the location gate, but
neither the command-line counts line nor the empty-digest text showed
that number, so considered roles could not be reconciled with the rest.
Both now include it, and a test checks that every considered role falls
into exactly one of the reported buckets.
'python scripts/demo.py' seeds a temporary store with seven fictional
roles (every employer marked synthetic, every link on example.invalid),
prints a SYNTHETIC banner, and runs the real ingestion census and the
real digest command as a dry run. It needs only the standard library,
never builds a Gmail client, and refuses to write over an existing store.

The tests run it with network access and the Gmail client both blocked,
and check the banner, the census and digest sections, that every link and
employer is invented, and that the counts line accounts for every role.
Several docstrings pointed to SESSION_NOTES.md or AGENTS.md, neither of
which is in this repository; the rules they cited are now stated in place.
Figures measured on the private system's store (in digest.py, signals.py
and refresh.py) are kept but now say where they were measured and that
they cannot be reproduced here. The delivery docstring also notes that
only the private Gmail helper has a consent-flow fallback, and that a
send needs Gmail to confirm the mailbox.
signals-tests.txt is now the output of 'python -m pytest -v' on a clean
clone of this branch, with the interpreter path shortened to .venv/bin/python
and rootdir to '.'. signals-tests.png is rendered from that text.
On a fresh chain scheduled_budget wrote timeout_seconds as the slice (600s),
below the project default of 900s, into the highest-precedence budget slot.
The timeout is now raised to the default when consumed plus slice is smaller.
The existing widen-only test missed this because it pre-seeded 5000s; a new
test runs on a fresh store with no scope budgets. The docstrings now state
exactly what is guaranteed and what is not compared.
reap_stale_runs cleared any status other than running or cancelling, so an
unrecognised status such as queued was treated as finished. It now clears only
completed, failed, cancelled, time_limit, limit_reached and interrupted, as the
docstring already said, and leaves every other status alone.
render() cut every link at 400 characters, which broke long apply links with
tracking parameters. The limit is now 2,048 and control characters are still
removed. Tests cover a 1,000-character link and a link with a CR LF in it.
digest.select built its set of enabled countries from the raw location keys,
while the classifier reports codes in upper case. A location saved as "fr"
therefore matched nothing and its roles were counted as outside configured
locations. The keys are now upper-cased before the comparison.
A real send now asks users.getProfile whose mailbox this is, and that call
needs gmail.readonly, not gmail.send. A send-only token passed preflight and
was then refused on every send with a message that did not mention scopes.
REQUIRED_SCOPES now lists gmail.readonly as well, so preflight rejects such a
token with the missing-scope reason.
requirements-dev.txt pinned pytest 9.1.1, which needs Python 3.10 or later, so
installing it with the python3 that ships with macOS (3.9) failed. The suite
passes on pytest 8.4 as well, so the requirement is now a range.
The digest docstring and a test docstring described the search that was
offered Shanghai in terms of one person's own location preferences. They now
say London plus a few configured countries, and a search limited to configured
locations.
Seven docstrings and comments had their citations of AGENTS.md reworded as
general project rules. The original wording is restored in each place, with
"(not included in this repository)" after the file name so the reference no
longer reads as dangling.
The default fixture title and one set of searched terms matched a real search
field. They now use a logistics planner and logistics terms; each affected test
still checks the same property.
The capture now shows the 85 tests on this branch, taken from a real
'python -m pytest -v' run in a fresh clone with the interpreter path shortened
to .venv/bin/python and rootdir shown as '.'. The PNG is rendered from it.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-23T23:46:41.457388Z d12c759 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant