Make TS adapter spawning degrade gracefully instead of silently or fatally - #996
Make TS adapter spawning degrade gracefully instead of silently or fatally#996alongd wants to merge 1 commit into
Conversation
72010d0 to
071d6e2
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #996 +/- ##
==========================================
- Coverage 64.38% 64.36% -0.03%
==========================================
Files 119 119
Lines 39601 39609 +8
Branches 10269 10271 +2
==========================================
- Hits 25499 25495 -4
- Misses 11121 11130 +9
- Partials 2981 2984 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR improves the robustness and debuggability of TS adapter spawning in Scheduler.spawn_ts_jobs() by (1) avoiding whole-run termination when an optional TS backend is missing, and (2) emitting an explicit warning when no TS adapter is eligible for a reaction (instead of failing silently).
Changes:
- Catch
FileNotFoundErrorfrom TS search job spawning, record the adapter inunsuccessful_methods, and continue to the next adapter. - Warn when the configured
ts_adapterslist has no eligible adapter for a known/unknown RMG family, making the “no jobs spawned” condition visible. - Add regression-style unit tests documenting the two failure modes.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
arc/scheduler.py |
Adds graceful handling for missing optional TS adapter backends and warns when no TS adapter is eligible to spawn. |
arc/scheduler_test.py |
Adds regression tests intended to cover “no eligible adapter” and “missing backend” scenarios. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…f silently or fatally
Two failure modes in ``Scheduler.spawn_ts_jobs``, both found on the same
campaign run, both costing work that had already succeeded.
1. No eligible adapter, silently.
An adapter is spawned only if it is in BOTH the configured ``ts_adapters`` and
``ts_adapters_by_rmg_family[rxn.family]``. When those lists do not intersect the
loop completes without spawning anything and says nothing. Nothing recovers:
``tsg_spawned`` is latched True before the loop, so the reaction is never
revisited, and the TS guess report is written with ``successful_methods: []``
AND ``unsuccessful_methods: []`` -- both empty, because nothing was attempted.
The only user-visible symptom is "TS did not converge", much later, which is
indistinguishable from having tried every adapter and failed.
Hit when ~/.arc/settings.py pinned
ts_adapters = ['heuristics', 'AutoTST', 'GCN', 'xtb_gsm', 'orca_neb']
predating 'linear' joining ARC's own default in arc/settings/settings.py. The
family 1,2_Insertion_CO admits only ['kinbot', 'goflow', 'rits', 'linear'].
Empty intersection, zero jobs, no log line. Because arc/imports.py has the home
settings override repo defaults key-by-key, a stale local file disables whole
families this way.
2. An uninstalled optional adapter, fatally.
Several TS adapters shell out to a separate conda env and repo checkout. KinBot
and AutoTST signal "backend not installed" by raising FileNotFoundError out of
``execute_incore``. That propagated through run_job -> spawn_ts_jobs ->
spawn_post_opt_jobs and terminated the entire ARC run.
Observed cost: 'linear' reported
Linear successfully found 7 TS guesses for s0_CH2O2 <=> s1_H2O + s2_CO.
and then the run died on the next adapter in the list because KINBOT_PYTHON was
unset. All 7 guesses, and every converged species job before them, were lost to
a missing optional dependency.
Record the unavailable adapter in ``unsuccessful_methods`` and continue -- the
remaining adapters are exactly the redundancy that makes a multi-adapter TS
search worth configuring. The catch is narrowed to FileNotFoundError so it
cannot mask real defects.
071d6e2 to
21ae360
Compare
Two failure modes in
Scheduler.spawn_ts_jobs, both found on the same campaign run, both costing work that had already succeeded.1. No eligible adapter — silently
An adapter is spawned only if it is in both the configured
ts_adaptersandts_adapters_by_rmg_family[rxn.family]. When those lists do not intersect, the loop runs to completion, spawns nothing, and logs nothing.Nothing recovers from it:
tsg_spawnedis latchedTruebefore the loop, so the reaction is never revisited;admit_unknown_familyrequiresnot family_known, so a known family with no eligible adapter cannot fall back tots_adapters_for_unknown_unimolecular;successful_methods: []andunsuccessful_methods: []— both empty, because nothing was attempted;TS did not converge, hours later, indistinguishable from having tried every adapter and failed.Hit when
~/.arc/settings.pypinned:predating
linearjoining ARC's own default inarc/settings/settings.py:114. The family1,2_Insertion_COadmits only['kinbot', 'goflow', 'rits', 'linear']:reproducing the observed report exactly:
Worth naming the general hazard:
arc/imports.pydoessettings.update(local_settings_dict), so the home settings file overrides repo defaults key-by-key. Ats_adapterslist written once and left alone silently loses access to every family whose adapters were added later. It fails closed, quietly, per-family.2. An uninstalled optional adapter — fatally
Several TS adapters shell out to a separate conda env and repo checkout. KinBot (
kinbot_ts.py:253) and AutoTST (autotst_ts.py:216) signal "backend not installed" by raisingFileNotFoundErrorout ofexecute_incore. That propagated throughrun_job→spawn_ts_jobs→spawn_post_opt_jobsand terminated the entire ARC run.The observed cost, in consecutive log lines:
All 7 guesses, and every converged species job before them, lost to a missing optional dependency — one the user opted into precisely because a multi-adapter search is supposed to be redundant.
The fix
Warn in case 1, with the three facts needed to act: what was configured, what the family admits, and that no guess will be produced. Record-and-continue in case 2, appending to
unsuccessful_methods.Deliberate choices:
FileNotFoundErroronly, not a bareexcept— it is the documented "backend missing" signal for both adapters that have one, and a broader catch would mask real defects.Verification
The 35 errors are pre-existing on
mainand unrelated —KeyError: 'server1', the test fixture's templateserversdict shadowed by a real~/.arc/settings.py. Same override mechanism as case 1 above, and worth fixing separately.