Delete ESS checkfiles on the server too, not just locally - #1000
Open
alongd wants to merge 1 commit into
Open
Conversation
alongd
commented
Aug 19, 2026
| """ | ||
| path = servers[server].get('path', '').lower() | ||
| path = os.path.join(path, servers[server]['un']) if path else '' | ||
| return os.path.join(path, 'runs', 'ARC_Projects', project) |
Member
Author
There was a problem hiding this comment.
not alwyas running under 'runs', 'ARC_Projects'
alongd
commented
Aug 19, 2026
| command = f'chmod{recursive} {mode} {file_name}' | ||
| self._send_command_to_server(command, remote_path) | ||
|
|
||
| def delete_check_files(self, remote_path: str) -> None: |
Member
Author
There was a problem hiding this comment.
what's the difference between this and arc.common.delete_check_files()?
ARC's `keep_checks` policy was only half implemented: `delete_check_files()` walked the local `calcs` tree, and no code path ever removed the check files ARC had uploaded to a server. They accumulate silently -- measured at 82% of a fully characterised species' remote footprint, against a managed quota. `ARC.clean_check_files()` now owns both sides of the policy: with `keep_checks` false it deletes the local check files as before and then, on every non-local server in `ess_settings`, deletes `*.chk` under that project's own remote directory. `keep_checks=True` keeps them everywhere. Cleanup runs once ARC is done with the science, so a server that cannot be reached at that point is logged and skipped rather than failing the run.
alongd
force-pushed
the
remote-check-cleanup
branch
from
August 19, 2026 22:17
5246c0e to
90d4838
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1000 +/- ##
==========================================
+ Coverage 64.38% 64.47% +0.08%
==========================================
Files 119 119
Lines 39601 39627 +26
Branches 10269 10275 +6
==========================================
+ Hits 25499 25549 +50
+ Misses 11121 11089 -32
- Partials 2981 2989 +8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ARC uploads a Gaussian checkpoint as a wavefunction guess for each job and downloads it again afterwards. When a run terminates it deletes those files — but only on the local machine:
delete_check_files(project_directory)walks<project_directory>/calcs;main.pycalls it unconditionally at the end of a run, sincekeep_checksdefaults toFalse;So the documented policy — "they usually take up lots of space and are not needed after ARC terminates" — is half-implemented, and every checkpoint ARC has ever uploaded is still sitting in the remote project directory.
Measured impact. On a quota-managed cluster, 82.3% of a fully characterised species' 61.4 MiB footprint is exactly this class of file — the next largest class is 5.6× smaller — and 6.35 GiB had silently accumulated against a 300 GB quota. Nothing downstream reads the remote copy: thermochemistry is parsed from the primary output log, and the local copy is the one reused as a guess.
What this does. Adds
SSHClient.delete_check_filesand adelete_remote_check_fileshelper, and routes the existing teardown through a newARC.clean_check_files. Design constraints kept deliberately tight:keep_checks=Truekeeps them everywhere, local and remote — the flag's meaning is unchanged;*.chkunder the run's own remote project directory, using the same path constructionjob/adapter.pyalready uses, and no-ops if that directory does not exist;localare skipped; nothing about uploads, the wavefunction-guess mechanism, orkeep_checks's default changes.Tests cover all three: deleted with
keep_checks=False(asserting only.chk, and only under this project's directory), kept withkeep_checks=True, and a run completing normally when the server call raises. Verified as a regression test rather than a tautology — removing just the remote call while leaving the helper in place makestest_check_files_are_deleted_locally_and_remotelyfail.