Add buffer-size cap and Python/Rust boundary property tests (#82, #90) - #229
Conversation
Extract a pure `checked_buffer_size(i64) -> Result<usize, &str>` from `validate_buffer_size` and introduce an explicit 1 GiB upper cap (MAX_BUFFER_SIZE), so an absurd `buffer_size` is rejected before allocation instead of attempting a huge `Vec`. The PyO3 wrapper maps the stable error message to `PyValueError` as before. Rust (`buffer_size_tests.rs`): proptest that `checked_buffer_size` accepts exactly the positive, in-range, at-or-below-cap values and rejects non-positive, over-cap, and usize-overflowing inputs, with explicit boundary unit tests (1, cap, cap+1, i64::MAX, 0, i64::MIN). Python (`test_rust_streams_boundary_property.py`): Hypothesis properties at the Python/Rust seam proving non-positive and over-cap `buffer_size` values and negative / out-of-i32-range descriptors raise `ValueError`, and that omitting `buffer_size` matches the explicit 65536 default. Regenerate the maturin wheel-manifest snapshot for the new test module. Closes #82 Closes #90 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdd a 1 GiB maximum buffer size, centralize Rust validation, expand Rust and Python boundary tests, update stream documentation, record the new packaged test, and add CodeScene rule overrides. ChangesStream buffer validation
Sequence Diagram(s)sequenceDiagram
participant PythonTests
participant RustStreams
participant BufferValidator
PythonTests->>RustStreams: Call stream helper with buffer_size
RustStreams->>BufferValidator: Validate positive value within 1 GiB
BufferValidator-->>RustStreams: Return usize or validation error
RustStreams-->>PythonTests: Return output/count or ValueError
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 19 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (19 passed)
📋 Issue PlannerLet us write the prompt for your AI agent so you can ship faster (with fewer bugs). View plan for ticket: ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d34b3dada2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cuprum/unittests/test_rust_streams_boundary_property.py`:
- Around line 148-149: Add concise assertion messages to both assertions in the
boundary property test, clearly identifying the explicit-versus-default
comparison and the default-versus-decoded-payload comparison; preserve the
existing assertion conditions and decoding behavior.
- Line 28: The packaged test must not import the unavailable
tests.helpers.stream_pipes module. In
cuprum/unittests/test_rust_streams_boundary_property.py at lines 28-28, move
_safe_close into shipped test support or define the small cleanup helper
locally; in cuprum/unittests/__snapshots__/test_maturin_build.ambr at lines
105-105, regenerate the packaging manifest to include that support module, or
omit test_rust_streams_boundary_property.py from the wheel.
- Around line 38-40: Update the _UNUSED_FD test fixture in
test_rust_streams_boundary_property.py from 0 to -1 so it is deterministically
invalid and cannot refer to stdin; preserve the existing buffer_size validation
assertion and all other test behavior.
In `@rust/cuprum-rust/src/buffer_size_tests.rs`:
- Around line 43-60: Update the matches_bounds property to assert each rejected
result against its exact documented Err message instead of only checking
is_err(). Extract the three-way expected-result logic into a predicate or
helper, then keep the property body focused on comparing
checked_buffer_size(value) with that expected result.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 2a83724c-7376-4d73-a280-b520a07cf8ac
📒 Files selected for processing (4)
cuprum/unittests/__snapshots__/test_maturin_build.ambrcuprum/unittests/test_rust_streams_boundary_property.pyrust/cuprum-rust/src/buffer_size_tests.rsrust/cuprum-rust/src/lib.rs
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/shared-actions(auto-detected)leynos/pylint-pypy-shim(auto-detected)leynos/whitaker(auto-detected)
Suppress two module-level Code Health findings where refactoring would be deleterious, each scoped narrowly and documented in the config: - `rust/cuprum-rust/src/lib.rs` — Primitive Obsession. The raw `i64` file-descriptor and buffer-size arguments are the untyped inputs PyO3 hands over from Python; these functions exist to validate them INTO the crate's domain newtypes (BufferSize, ReaderFd, WriterFd, PlatformFd). Wrapping the raw inputs in further domain types before validation would add ceremony without value. - `**/*tests.rs` — Code Duplication. The EINTR-retry tests for the read seam (`read_raw_fd_with`) and the write seam (`write_all_unix_with`) are deliberately parallel to document that both seams honour the identical retry contract; extracting a shared representation would couple independent module boundaries and obscure which seam each test exercises. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Respond to Codex and CodeRabbit feedback on the buffer-size / boundary PR: - Make the packaged boundary test self-contained: define `_safe_close` locally instead of importing it from `tests/helpers/stream_pipes.py`, which the wheel does not ship, so the module now collects from a clean installed wheel (CodeRabbit). - Use `-1` as the throwaway descriptor instead of `0` (stdin), so a validation-order regression fails loudly rather than blocking on a real fd (CodeRabbit). - Skip the two descriptor properties on Windows: they assert the Unix i32 file-descriptor conversion contract, which does not hold on the Windows handle path (Codex). - Add failure messages to the round-trip assertions (CodeRabbit / path instructions). - Document the new 1 GiB buffer-size ceiling and its `ValueError` in the `cuprum/_streams_rs.py` wrapper docstrings and `docs/cuprum-design.md`, which previously stated only that `buffer_size` must be positive (Codex). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.codescene/code-health-rules.json:
- Around line 12-15: The Code Duplication override currently applies to every
Rust test file via matching_content_path. Update the rule configuration around
matching_content_path to target only the specific documented EINTR and seam test
module paths, using separate exact entries where needed, while preserving the
zero weight and rationale for those modules.
In `@cuprum/unittests/test_rust_streams_boundary_property.py`:
- Around line 160-176: Extend test_default_buffer_matches_explicit to exercise
rust_pump_stream with both omitted and explicit _DEFAULT_BUFFER_SIZE (65536)
buffer_size values, and assert their outputs match. Retain the existing
rust_consume_stream assertions while adding substantive property coverage for
the pump default-equivalence contract.
- Around line 136-157: Refactor _consume_via_pipe to use contextlib.ExitStack
for cleanup, registering both read_fd and write_fd immediately after creating
the pipe. Preserve the explicit write_fd close before calling
rust_consume_stream, and avoid the current write_fd sentinel and try/finally
cleanup logic.
In `@docs/cuprum-design.md`:
- Line 1721: Update the buffer_size validation documentation to state that
ValueError is raised when the value is not a positive integer, in addition to
values below 1 or above the 1 GiB maximum. Ensure the documented behavior
matches the wrapper contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d9d171f8-f59e-4dfe-bab9-5b0d8069ec20
📒 Files selected for processing (7)
.codescene/code-health-rules.jsoncuprum/_streams_rs.pycuprum/unittests/__snapshots__/test_maturin_build.ambrcuprum/unittests/test_rust_streams_boundary_property.pydocs/cuprum-design.mdrust/cuprum-rust/src/buffer_size_tests.rsrust/cuprum-rust/src/lib.rs
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/shared-actions(auto-detected)leynos/pylint-pypy-shim(auto-detected)leynos/whitaker(auto-detected)
- Narrow the Code Duplication CodeScene override from the `**/*tests.rs`
wildcard to the exact file that needs it
(`rust/cuprum-rust/src/io_utils/tests.rs`), so the exception cannot
silently cover unrelated future test modules (CodeRabbit).
- Refactor `_consume_via_pipe` to manage descriptors with
`contextlib.ExitStack` and add a shared `_feed_pipe` helper, replacing the
manual `try`/`finally` sentinel cleanup (CodeRabbit).
- Add `test_pump_default_buffer_matches_explicit` so the default-equivalence
property covers `rust_pump_stream`, not just `rust_consume_stream`
(CodeRabbit).
- Align the `docs/cuprum-design.md` pump `ValueError` wording with the
wrapper contract ("not a positive integer or exceeds the 1 GiB maximum")
(CodeRabbit).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@coderabbitai On the "Out of Scope Changes" pre-merge check for This override is not unrelated to #82 — it is required because of the #82 change. Adding |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- CodeScene override configuration ---'
cat .codescene/code-health-rules.json
printf '%s\n' '--- Relevant Rust declarations ---'
ast-grep outline rust/cuprum-rust/src/lib.rs --items all --type function,constant --match 'checked_buffer_size|validate_buffer_size|MAX_BUFFER_SIZE'Length of output: 2275
The verified configuration scopes Primitive Obsession solely to Withdraw the “Out of Scope Changes” finding. 🐇 ✏️ Learnings added
|
|
@coderabbitai On the "User-Facing Documentation" and "Developer Documentation" pre-merge checks for the buffer-size cap: The new 1 GiB |
|
Tip For best results, initiate chat on the files or code changes.
The 1 GiB limit and its 🐇🔎 ✏️ Learnings added
|
The invalid-reader pump property now passes a genuinely valid writer descriptor (os.devnull) so the ValueError can only originate from the invalid reader, not a coincidentally invalid writer (CodeRabbit pre-merge Testing check). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cuprum/unittests/test_rust_streams_boundary_property.py`:
- Around line 136-141: Replace the manual try/finally around writer_fd in the
rust_pump_stream test with contextlib.ExitStack, registering writer_fd for
cleanup immediately after os.open succeeds. Preserve the existing pytest.raises
assertion and bad_fd behavior while ensuring the descriptor is closed when the
stack exits.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: b87676f9-ec0f-4228-bc16-812191f30e27
📒 Files selected for processing (3)
.codescene/code-health-rules.jsoncuprum/unittests/test_rust_streams_boundary_property.pydocs/cuprum-design.md
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/shared-actions(auto-detected)leynos/pylint-pypy-shim(auto-detected)leynos/whitaker(auto-detected)
Manage the valid writer descriptor with contextlib.ExitStack instead of try/finally, matching the repo convention for resource management (CodeRabbit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
Address the round-4 documentation pre-merge checks: - users-guide: note under the Rust stream sections that buffer_size must be a positive integer no larger than 1 GiB, and that out-of-range values raise ValueError. - developers-guide: add a "Rust stream buffer-size validation" section describing checked_buffer_size / MAX_BUFFER_SIZE, the pure-helper seam, and where the contract is property tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/users-guide.md`:
- Around line 1204-1207: Update the buffer_size documentation for
rust_pump_stream and rust_consume_stream to qualify that ValueError is raised
only after PyO3 converts the argument to i64; document that non-integer values
and Python integers outside the i64 range may raise a different exception
earlier, while preserving the existing positive and 1 GiB validation details.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: fd16065b-41c9-44a0-9b9b-e409f3a5fe10
📒 Files selected for processing (2)
docs/developers-guide.mddocs/users-guide.md
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/shared-actions(auto-detected)leynos/pylint-pypy-shim(auto-detected)leynos/whitaker(auto-detected)
|
@coderabbitai Both documentation checks are already satisfied on the current branch head (commit
Could you re-review against head |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- Current HEAD and requested commit ---'
git rev-parse --short HEAD
git show -s --format='%H%n%s' 9aac6b9
git merge-base --is-ancestor 9aac6b9 HEAD && echo '9aac6b9 is an ancestor of HEAD'
printf '%s\n' '--- User guide: Rust stream pump section and buffer_size contract ---'
rg -n -A12 -B4 'Rust stream pump \(internal\)|rust_pump_stream|rust_consume_stream|buffer_size' docs/users-guide.md
printf '%s\n' '--- Developer guide: buffer-size validation section ---'
rg -n -A24 -B3 'Rust stream buffer-size validation|checked_buffer_size|MAX_BUFFER_SIZE|buffer_size' docs/developers-guide.mdLength of output: 5345
Withdraw the superseded user-facing and developer-documentation findings. |
|
@coderabbitai The pre-merge "Developer Documentation" warning recommends marking roadmap items 6.1.1 and 6.1.2 as done. Please reconsider: this PR intentionally implements only a bounded slice of those items, so leaving them unchecked is correct. Scope of this PR (head
Roadmap 6.1.1 requires more than this slice: "Introduce a Checking 6.1.1/6.1.2 would therefore overstate completion and make the roadmap inaccurate. The enum-based centralized error mapping is tracked as future work. Could you confirm this reasoning and dismiss the warning? |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- HEAD ---'
git rev-parse --short HEAD
git show -s --format='%H%n%s' HEAD
printf '%s\n' '--- Roadmap items 6.1.1 / 6.1.2 ---'
rg -n -A20 -B4 '6\.1\.1|6\.1\.2|RustStreamError|centralized|centralised|error.mapping|error mapping' \
README.md docs rust .github 2>/dev/null || true
printf '%s\n' '--- Rust error-related declarations and conversions ---'
ast-grep outline rust/cuprum-rust/src/lib.rs --items all --type enum,function,constant \
--match 'RustStreamError|checked_buffer_size|validate_buffer_size|PyValueError|PyErr|OSError|io_error'
printf '%s\n' '--- Relevant implementation excerpts ---'
rg -n -A18 -B8 'RustStreamError|checked_buffer_size|validate_buffer_size|PyValueError|PyOSError|PyErr|OSError' \
rust/cuprum-rust/src/lib.rs rust/cuprum-rust/src 2>/dev/null || true
printf '%s\n' '--- Changed paths at PR HEAD relative to merge-base ---'
base="$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main)"
git diff --name-status "$base"...HEADLength of output: 47747
At HEAD Item 6.1.2 also explicitly depends on 6.1.1 and requires the broader generated consume-parity domain, shrinking, and committed regression seeds. The new cap-focused property tests do not satisfy that full completion criterion. Withdraw the “Developer Documentation” warning that recommends marking either roadmap item done. 🐇 ✏️ Learnings added
|
Address two review findings on #229. docs/users-guide.md: the buffer_size paragraph claimed invalid values "raise ValueError" without distinguishing the two failure points. Clarify that ValueError is raised after PyO3 converts the argument to a signed 64-bit integer and Rust validation runs, whereas a non-integer or a Python integer outside the i64 range can fail earlier during PyO3 argument conversion with a different exception. No specific conversion exception type is claimed, since that is not verified for the pinned PyO3 version. buffer_size_tests.rs: replace the branchy matches_bounds body with a reference-oracle helper, expected_buffer_size_result, that re-encodes the checked_buffer_size contract independently, and compare the two directly with prop_assert_eq!. Existing boundary tests are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
Hardens
buffer_sizevalidation with an explicit upper cap and adds property-based coverage on both sides of the Python/Rust boundary.Changes
Rust (#82)
checked_buffer_size(i64) -> Result<usize, &'static str>fromvalidate_buffer_size, and add an explicit 1 GiB cap (MAX_BUFFER_SIZE). An absurdbuffer_sizeis now rejected before allocation instead of attempting a hugeVec. The PyO3 wrapper maps the stable message toPyValueErroras before.buffer_size_tests.rs: proptest thatchecked_buffer_sizeaccepts exactly the positive, in-range, at-or-below-cap values and rejects non-positive, over-cap, andusize-overflowing inputs, with explicit boundary unit tests (1, cap, cap+1,i64::MAX, 0,i64::MIN) and a drift guard tying the test cap constant toMAX_BUFFER_SIZE.Python (#90)
test_rust_streams_boundary_property.py: Hypothesis properties at the seam proving non-positive and over-capbuffer_sizevalues and negative / out-of-i32-range descriptors raiseValueError, and that omittingbuffer_sizematches the explicit 65536 default. (Existing example tests already coverbuffer_size=0and the OSError I/O-failure path.)Regenerated the maturin wheel-manifest snapshot for the new test module.
Verification
make check-fmt lint typecheck markdownlintgreen (clippy, whitaker,cargo doc -D warnings);cargo nextest37/37. The new Python boundary tests were verified against a locally-built native extension (they, like the existingrust_streamssuite, skip when the extension is not installed and run in CI). Two unrelatedtest_folded_summary.pyHypothesis-deadline tests flaked under concurrent machine load and pass in isolation.Closes #82
Closes #90
🤖 Generated with Claude Code