Skip to content

feat: allow to disable the memory array fudge - #482

Merged
cmeesters merged 14 commits into
mainfrom
feat/array-fudge
Aug 10, 2026
Merged

feat: allow to disable the memory array fudge#482
cmeesters merged 14 commits into
mainfrom
feat/array-fudge

Conversation

@cmeesters

@cmeesters cmeesters commented Aug 10, 2026

Copy link
Copy Markdown
Member

Per default array jobs get a higher memory setting, a "memory fudge". This new PR is based on #477 . It allows disabling the memory fudge setting.

Here, the --slurm-array-memory-fudge flag is renamed in favour of --slurm-disable-memory-fudge. Reasoning: The development will eventually entail pooling of shared memory programs onto one or multiple nodes. For this feature as well of array jobs this extra memory might be required. Disabling it should not carry the array string, as this might introduce a breaking change in a future release.

Summary by CodeRabbit

  • New Features

    • Added configurable memory adjustment for Slurm array jobs.
    • Array-job memory requests now increase by default, with a minimal-memory fallback when needed.
    • Added CLI and profile options to disable automatic memory adjustment.
  • Documentation

    • Documented default memory behavior, fallback handling, and configuration options for array jobs.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 40001ae3-9c8b-4977-aeb9-44fa521f5f0e

📥 Commits

Reviewing files that changed from the base of the PR and between 0b00769 and 456f28e.

📒 Files selected for processing (4)
  • docs/further.md
  • snakemake_executor_plugin_slurm/__init__.py
  • tests/test_array_jobs.py
  • tests/test_cli.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • docs/further.md
  • tests/test_array_jobs.py

Walkthrough

The plugin adds a disable_memory_fudge setting for SLURM array jobs. Array submissions skip automatic memory adjustment when enabled. Tests cover defaults, CLI parsing, and both submission modes. Documentation describes the setting.

Changes

SLURM array-job memory control

Layer / File(s) Summary
Memory setting and submission behavior
snakemake_executor_plugin_slurm/__init__.py
The executor adds disable_memory_fudge, defaults it to False, and skips apply_mem_fudge() for array submissions when enabled.
Memory behavior validation and documentation
tests/test_array_jobs.py, tests/test_cli.py, docs/further.md
Tests cover the default setting, disabled and default-enabled memory arguments, and CLI parsing. Documentation describes the memory adjustment and configuration options.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested reviewers: conchoecia

Poem

A rabbit sets the memory rule,
With a CLI switch as its tool.
Array jobs may skip the fudge,
Or keep the default with a nudge.
Tests and docs make the behavior clear.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: adding an option to disable Slurm array-job memory fudge.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/array-fudge

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/further.md`:
- Around line 348-362: Correct the “Array memory adjustment” documentation to
reflect that disable_memory_fudge defaults to false: use true in the CLI example
and Snakemake profile to disable the adjustment, and state that false preserves
the default memory-fudge behavior.

In `@snakemake_executor_plugin_slurm/__init__.py`:
- Around line 196-207: Update the array-job test fixtures to use the new
disable_memory_fudge setting: replace the obsolete array_memory_fudge field in
_make_executor_stub and update test_array_memory_fudge_can_be_disabled to assign
disable_memory_fudge. Ensure run_array_jobs() receives the setting and the test
verifies the opt-out behavior.

In `@tests/test_array_jobs.py`:
- Line 101: Replace the remaining array_memory_fudge references with
disable_memory_fudge: in tests/test_array_jobs.py lines 101 and 408, use False
for the enabled adjustment case and True for the disabled case; in
tests/test_cli.py lines 77-80, register and parse --slurm-disable-memory-fudge
and assert settings.disable_memory_fudge; update the example/profile text in
docs/further.md lines 358-362 so the documented option maps to the enabled
setting.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5a31a0c5-d0d3-47dc-901c-0692b1e98f08

📥 Commits

Reviewing files that changed from the base of the PR and between 7f91037 and 0b00769.

📒 Files selected for processing (4)
  • docs/further.md
  • snakemake_executor_plugin_slurm/__init__.py
  • tests/test_array_jobs.py
  • tests/test_cli.py

Comment thread docs/further.md Outdated
Comment thread snakemake_executor_plugin_slurm/__init__.py Outdated
Comment thread tests/test_array_jobs.py Outdated
@cmeesters
cmeesters merged commit f4b52e2 into main Aug 10, 2026
6 checks passed
@cmeesters
cmeesters deleted the feat/array-fudge branch August 10, 2026 08:45
cmeesters pushed a commit that referenced this pull request Aug 10, 2026
🤖 I have created a release *beep* *boop*
---


##
[2.8.0](v2.7.1...v2.8.0)
(2026-08-10)


### Features

* add no-requeue option for
[#353](#353)
([#476](#476))
([7f91037](7f91037))
* allow to disable the memory array fudge
([#482](#482))
([f4b52e2](f4b52e2))


### Bug Fixes

* emit job metadata via run_job_pre before Slurm submission
([#478](#478))
([fc7aa48](fc7aa48))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants