Skip to content

pypy parity: index/slice coercion, sequence concat, in-place operators, overflow guards - #560

Merged
youknowone merged 20 commits into
mainfrom
rename
Jul 16, 2026
Merged

pypy parity: index/slice coercion, sequence concat, in-place operators, overflow guards#560
youknowone merged 20 commits into
mainfrom
rename

Conversation

@youknowone

@youknowone youknowone commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Drains the remaining low-risk pypy-parity items from the 3-way (cpython/pypy/pyre) sweep — the integer-coercion, sequence-concatenation, in-place-operator, and numeric-overflow paths. Each behavior was probed against the installed pypy3 (7.3.20, the check.py oracle) and now matches byte-for-byte. Where cpython and pypy disagree (so a check.py baseline test cannot cover it), the behavior was verified by hand against pypy3.

objspace: remap __index__ TypeError; ord names bytes noun

Carries over the two PR #552 review fixes that had not yet landed:

  • getindex_w remaps a TypeError raised by a subscript's __index__ (a non-int return, or a TypeError from inside __index__) to <descr> indices must be integers or slices, not '<type>'; a ValueError still propagates. Applied to the list/tuple/str/bytes get, list/bytearray set, and delitem sites.
  • ord(b"ab") reports bytes of length 2 while bytearray(b"ab") keeps string of length 2.

pypy parity: index coercion and sequence-concat errors

  • builtins::getindex_w — an out-of-word overflow now clamps to i64::MIN for a negative value (was always i64::MAX), so a large-negative slice bound floors to 0 instead of collapsing to the tail: [1,2,3][-(10**30):][1, 2, 3].
  • normalize_slice / sliceobject::adapt_lower_bound — fold a negative index by the length with saturating_add so an i64::MIN bound does not overflow before flooring at 0.
  • str_slice_args (startswith/endswith) — take the start/end bounds through adapt_lower_bound(eval_slice_index(...)), so a non-index bound raises slice indices must be integers or None or have an __index__ method, an __index__ object is honoured, and a positive bound is not upper-clamped.
  • builtin_round — take ndigits through getindex_w (float: clamping; int: space.index); a huge-magnitude negative ndigits short-circuits to 0 rather than building an astronomical power of ten (round(3.14159, 2**63)3.14159).
  • index_to_bigint (hex/oct/bin) — coerce through space.index so a non-int __index__ return raises __index__ returned non-int (type X).
  • formatting number_arg_decimal / number_arg_integer (%d/%i/%o/%x) — remap a TypeError from the numeric decoder to the operand-type error naming the original argument (%d format: a real number is required, not BadIdx).
  • list/tuple/bytes/bytearray __add__ and descroperation::add — return NotImplemented for a non-sequence operand so the operator raises the generic unsupported operand type(s) for +, dropping the cpython-only can only concatenate message; bytearray.__iadd__ on a non-buffer now raises a bytes-like object is required, not 'X'.

fix warnings

Removes three unused-binding warnings in the rtyper (classdesc.rs, rclass.rs, rpbc.rs).

operator/set: in-place operator functions and set slots

  • The operator module gains the in-place functions iadd, isub, imul, imatmul, ifloordiv, imod, itruediv, ipow, ilshift, irshift, iand, ior, ixor (each dispatching the matching in-place binary op) and iconcat (which requires both operands to be subscriptable, else raises 'X' object can't be concatenated). The __iadd__iadd alias table now resolves to real callables.
  • set gains the __isub__, __iand__, __ior__, __ixor__ slots, so s &= t and operator.iand(s, t) update in place; a non-set right operand yields NotImplemented.

int: raise MemoryError from an unallocatably-large left shift

  • 1 << (10**18) drove the big-int << into the infallible global allocator and aborted the process. A new checked_bigint_lshift pre-flights a fallible reservation of the result's 64-bit limb count and raises a catchable MemoryError instead; int and long left shifts route through it. The shift count is carried as u64 so a 32-bit usize target (wasm) does not truncate a 10**18 shift.

float/complex pow: over-range int operand overflows

  • A float/complex power coerces each int operand to a double before the power is computed, so an over-range int base or exponent raises OverflowError up front — even 1.0 ** huge, which never reaches the arithmetic, matching float(huge). In float.__pow__/__rpow__ the check runs before the ternary-modulus rejection, so pow(2.0, huge, 5) raises OverflowError from the coercion rather than TypeError from the modulus.

Verification

  • Probes (custom __index__ objects, huge/negative bounds, non-seq concat, %-format, in-place set/operator ops, 1 << 10**18, over-range float/complex pow) are byte-identical to pypy3 across all cases; the cpython≠pypy edges (3-arg pow ordering, lshift OOM message) were hand-verified against pypy3.
  • Normal slice / concat / hex / round / %-format / startswith / in-place-op / small-shift / ordinary-pow paths are byte-identical to pypy3 (no regression).
  • New synth tests: operator_set_inplace_ops, int_lshift_memoryerror, float_pow_overflow_exp (each restricted to cpython==pypy cases).
  • check.py: dynasm 190/190, cranelift 190/190, wasm 189/189 (3/3 backends).

Deferred (own follow-up slices)

  • int.__lshift__('a', 1) returns NotImplemented where pypy raises TypeError "'int' object expected, got 'str' instead" — needs a systematic unbound-descriptor self-type check across every typed-self builtin; cpython≠pypy so not baseline-testable.
  • x &= y augmented-assignment error symbol — a type mismatch reports the binary-op symbol & rather than &= (e.g. int += str). Needs an Option-returning binop path so the augmented-assign layer can re-emit with the X= symbol.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added support for operator.iconcat.
    • Expanded set and dictionary operations, including in-place set operators and dictionary-view set operations.
  • Bug Fixes

    • Improved slice-bound handling, index coercion, and boundary behavior for string and sequence operations.
    • Corrected error messages and exception propagation for hashing, unhashable values, overflow, memory limits, and invalid operands.
    • Improved behavior for round(), pow(), ord(), and set/dictionary updates.
  • Tests

    • Added benchmarks covering dictionary, set, operator, numeric, and string edge cases.

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Too many files!

This PR contains 62 files, which is 12 over the limit of 50.

To get a review, narrow the scope:
• coderabbit review --type committed # exclude uncommitted changes
• coderabbit review --dir # limit to a subdirectory
• coderabbit review --base # compare against a closer base

Upgrade to a paid plan to raise the limit.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 082eab76-2690-47f8-916e-0caf03a3a31d

📥 Commits

Reviewing files that changed from the base of the PR and between 66dac3d and 00fd26d.

📒 Files selected for processing (62)
  • majit/majit-translate/src/annotator/classdesc.rs
  • majit/majit-translate/src/translator/rtyper/rclass.rs
  • majit/majit-translate/src/translator/rtyper/rpbc.rs
  • pyre/bench/synth/builtin_type_surface.py
  • pyre/bench/synth/dict_ctor_consume.py
  • pyre/bench/synth/dict_hash_protocol.py
  • pyre/bench/synth/dict_view_set_ops.py
  • pyre/bench/synth/float_pow_overflow_exp.py
  • pyre/bench/synth/int_lshift_memoryerror.py
  • pyre/bench/synth/operator_set_inplace_ops.py
  • pyre/bench/synth/set_contains_frozenset.py
  • pyre/bench/synth/set_hash_protocol.py
  • pyre/bench/synth/set_intersection_operand.py
  • pyre/bench/synth/set_key_protocol.py
  • pyre/bench/synth/set_update_hash_other.py
  • pyre/bench/synth/set_update_materialize_rhs.py
  • pyre/bench/synth/str_startswith_bounds.py
  • pyre/extra_tests/parity_tests/bool_python314.py
  • pyre/extra_tests/parity_tests/bytearray_python314.py
  • pyre/extra_tests/parity_tests/bytes_surface_python314.py
  • pyre/extra_tests/parity_tests/dict_surface_python314.py
  • pyre/extra_tests/parity_tests/float_complex_python314.py
  • pyre/extra_tests/parity_tests/functional_iterators_python314.py
  • pyre/extra_tests/parity_tests/generator_python314.py
  • pyre/extra_tests/parity_tests/int_python314.py
  • pyre/extra_tests/parity_tests/itertools_count_repeat_python314.py
  • pyre/extra_tests/parity_tests/itertools_predicate_types_python314.py
  • pyre/extra_tests/parity_tests/list_iterator_surface.py
  • pyre/extra_tests/parity_tests/memoryview_python314.py
  • pyre/extra_tests/parity_tests/object_surface_python314.py
  • pyre/extra_tests/parity_tests/python314_singletons.py
  • pyre/extra_tests/parity_tests/range_python314.py
  • pyre/extra_tests/parity_tests/sequence_dict_iterators_python314.py
  • pyre/extra_tests/parity_tests/set_storage_iterator.py
  • pyre/extra_tests/parity_tests/slice_python314.py
  • pyre/extra_tests/parity_tests/str_surface.py
  • pyre/extra_tests/parity_tests/super_python314.py
  • pyre/extra_tests/parity_tests/tuple_iterator_surface.py
  • pyre/pyre-interpreter/src/baseobjspace.rs
  • pyre/pyre-interpreter/src/builtins.rs
  • pyre/pyre-interpreter/src/display.rs
  • pyre/pyre-interpreter/src/eval.rs
  • pyre/pyre-interpreter/src/jit_fnaddr.rs
  • pyre/pyre-interpreter/src/module/_weakref/interp__weakref.rs
  • pyre/pyre-interpreter/src/module/itertools/interp_itertools.rs
  • pyre/pyre-interpreter/src/module/operator/mod.rs
  • pyre/pyre-interpreter/src/objspace/descroperation.rs
  • pyre/pyre-interpreter/src/objspace/std/formatting.rs
  • pyre/pyre-interpreter/src/opcode_ops.rs
  • pyre/pyre-interpreter/src/runtime_ops.rs
  • pyre/pyre-interpreter/src/sliceobject.rs
  • pyre/pyre-interpreter/src/type_methods.rs
  • pyre/pyre-interpreter/src/typedef.rs
  • pyre/pyre-jit/src/eval.rs
  • pyre/pyre-object/src/bytearrayobject.rs
  • pyre/pyre-object/src/descriptor.rs
  • pyre/pyre-object/src/dictmultiobject.rs
  • pyre/pyre-object/src/functional.rs
  • pyre/pyre-object/src/generator.rs
  • pyre/pyre-object/src/iterobject.rs
  • pyre/pyre-object/src/listobject.rs
  • pyre/pyre-object/src/setobject.rs

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Review on demand using usage pricing

Walkthrough

The PR updates Pyre interpreter behavior for checked hashing, set and dict operations, index and slice coercion, numeric overflow, and operator dispatch. It also adds benchmark scripts covering these semantics and adjusts translator tests for class-family merging.

Changes

Interpreter protocol and collection behavior

Layer / File(s) Summary
Checked hash and storage paths
pyre/pyre-object/..., pyre/pyre-interpreter/...
Set and dict insertion, lookup, and removal paths now use precomputed hashes and propagate hashing or equality failures.
Set and dict-view operations
pyre/pyre-interpreter/src/typedef.rs, baseobjspace.rs, type_methods.rs, descroperation.rs
Set conversion, intersection, update, in-place operators, containment, removal, and dict-view operations are routed through shared implementations.
Index, slice, and formatting coercion
pyre/pyre-interpreter/src/baseobjspace.rs, sliceobject.rs, type_methods.rs, formatting.rs
Index-protocol TypeErrors are remapped, slice arithmetic becomes overflow-safe, string bounds become fallible, and formatting coercion errors are standardized.
Numeric and operator behavior
pyre/pyre-interpreter/src/builtins.rs, descroperation.rs, module/operator/mod.rs, typedef.rs
Shift allocation, power conversion, rounding, concatenation dispatch, and in-place operator functions receive updated behavior.
Runtime behavior benchmarks
pyre/bench/synth/*
New scripts exercise dictionary, set, operator, numeric, and string edge cases with warmups and labeled output.
Translator test updates
majit/majit-translate/src/...
Class-family merging coverage now uses a shared ancestor and verifies representative unioning; unused imports are removed.

Estimated code review effort: 5 (Critical) | ~120 minutes

Poem

A rabbit hops through hashes bright,
Counts each key by moonlit night.
Sets now catch what errors bring,
Slices bend safely, operators sing.
Benchmarks thump their tiny drum—
“Checked paths bloom!” cries rabbit: “done!”

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is clear, concise, and matches the PR’s main focus on PyPy parity for coercion, concatenation, in-place operators, and overflow handling.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch rename

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f625bb1929

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

} else {
(v as usize).min(char_len as usize)
}
crate::sliceobject::adapt_lower_bound(char_len, args[3])?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Return false for out-of-range empty-prefix checks

When start and end are both supplied past the string length, leaving this positive end unbounded makes start <= end, and the later nth() calls map both offsets to bytes.len(), producing an empty window. That makes cases such as 'abc'.startswith('', 5, 10) and endswith('', 5, 10) return True, while PyPy's unicodeobject.py:_unwrap_and_compute_idx_params forces start_index = end_index + 1 when start > length, so the match remains False even for an empty prefix. Clamp end to char_len or otherwise preserve the out-of-range-start sentinel before comparing/building the window.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in `caf61cb243`.

`'abc'.startswith('', 5, 10)` and `endswith` returned `True` against both oracles' `False`. Rather than clamping `end`, this ports the upstream mechanism directly — `unicodeobject.py:1319 _unwrap_and_compute_idx_params` turns a `start` past the end into `end_index + 1`, inverting the window, and only lowers `end_index` when `end < length`. The byte-space `start_index > end_index` test then subsumes the old `start > end` check, so `''.endswith('', 1, 0)` stays `False` too.

Guarded by the new synth test `str_startswith_bounds.py` (27 lines, cpython == pypy == pyre).

commented by Claude

@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown

🤖 Codex parity review

Static analysis of this diff vs the local RPython/PyPy sources (commit 00fd26d).
Updated: 2026-07-16T12:29:58.086Z

Files in the reviewed diff
majit/majit-translate/src/annotator/classdesc.rs
majit/majit-translate/src/translator/rtyper/rclass.rs
majit/majit-translate/src/translator/rtyper/rpbc.rs
pyre/bench/synth/builtin_type_surface.py
pyre/bench/synth/dict_ctor_consume.py
pyre/bench/synth/dict_hash_protocol.py
pyre/bench/synth/dict_view_set_ops.py
pyre/bench/synth/float_pow_overflow_exp.py
pyre/bench/synth/int_lshift_memoryerror.py
pyre/bench/synth/operator_set_inplace_ops.py
pyre/bench/synth/set_contains_frozenset.py
pyre/bench/synth/set_hash_protocol.py
pyre/bench/synth/set_intersection_operand.py
pyre/bench/synth/set_key_protocol.py
pyre/bench/synth/set_update_hash_other.py
pyre/bench/synth/set_update_materialize_rhs.py
pyre/bench/synth/str_startswith_bounds.py
pyre/extra_tests/parity_tests/bool_python314.py
pyre/extra_tests/parity_tests/bytearray_python314.py
pyre/extra_tests/parity_tests/bytes_surface_python314.py
pyre/extra_tests/parity_tests/dict_surface_python314.py
pyre/extra_tests/parity_tests/float_complex_python314.py
pyre/extra_tests/parity_tests/functional_iterators_python314.py
pyre/extra_tests/parity_tests/generator_python314.py
pyre/extra_tests/parity_tests/int_python314.py
pyre/extra_tests/parity_tests/itertools_count_repeat_python314.py
pyre/extra_tests/parity_tests/itertools_predicate_types_python314.py
pyre/extra_tests/parity_tests/list_iterator_surface.py
pyre/extra_tests/parity_tests/memoryview_python314.py
pyre/extra_tests/parity_tests/object_surface_python314.py
pyre/extra_tests/parity_tests/python314_singletons.py
pyre/extra_tests/parity_tests/range_python314.py
pyre/extra_tests/parity_tests/sequence_dict_iterators_python314.py
pyre/extra_tests/parity_tests/set_storage_iterator.py
pyre/extra_tests/parity_tests/slice_python314.py
pyre/extra_tests/parity_tests/str_surface.py
pyre/extra_tests/parity_tests/super_python314.py
pyre/extra_tests/parity_tests/tuple_iterator_surface.py
pyre/pyre-interpreter/src/baseobjspace.rs
pyre/pyre-interpreter/src/builtins.rs
pyre/pyre-interpreter/src/display.rs
pyre/pyre-interpreter/src/eval.rs
pyre/pyre-interpreter/src/jit_fnaddr.rs
pyre/pyre-interpreter/src/module/_weakref/interp__weakref.rs
pyre/pyre-interpreter/src/module/itertools/interp_itertools.rs
pyre/pyre-interpreter/src/module/operator/mod.rs
pyre/pyre-interpreter/src/objspace/descroperation.rs
pyre/pyre-interpreter/src/objspace/std/formatting.rs
pyre/pyre-interpreter/src/opcode_ops.rs
pyre/pyre-interpreter/src/runtime_ops.rs
pyre/pyre-interpreter/src/sliceobject.rs
pyre/pyre-interpreter/src/type_methods.rs
pyre/pyre-interpreter/src/typedef.rs
pyre/pyre-jit/src/eval.rs
pyre/pyre-object/src/bytearrayobject.rs
pyre/pyre-object/src/descriptor.rs
pyre/pyre-object/src/dictmultiobject.rs
pyre/pyre-object/src/functional.rs
pyre/pyre-object/src/generator.rs
pyre/pyre-object/src/iterobject.rs
pyre/pyre-object/src/listobject.rs
pyre/pyre-object/src/setobject.rs

1. Regressions to PyPy parity introduced by this patch

None verified: the workspace shell fails before executing any command (bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted), so I could not inspect the diff or source files.

2. Other mismatches introduced by this patch

None verified: static analysis could not run.

3. Pre-existing mismatches (already present before this patch)

None verified: static analysis could not run.

4. Structural adaptations

None verified: static analysis could not run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

https://github.com/youknowone/pyre/blob/b1c7ed4a8a77f9e41a9c240c7a5fb4f58d231850/pyre-interpreter/src/objspace/descroperation.rs#L92-L93
P2 Badge Preserve zero left shifts before reserving

When the left operand is zero and the shift count is huge but nonnegative, Python returns zero without needing to allocate the shifted magnitude (e.g. 0 << 10**18). This helper now sizes the reservation from the shift count before checking the value, so both the int and long paths that call it can raise MemoryError for a valid zero result; special-case a zero a before computing/reserving limbs.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@youknowone youknowone changed the title objspace: pypy parity for index coercion, slice clamp, sequence concat pypy parity: index/slice coercion, sequence concat, in-place operators, overflow guards Jul 15, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 35081d2f4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +92 to +93
let result_bits = a.bits().saturating_add(shift);
let limbs = (result_bits / 64).saturating_add(2);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve zero for huge left shifts

When the left operand is zero and the shift count is very large but still fits in i64 (for example 0 << 10**18), this preflight sizes the reservation from shift alone and returns MemoryError before doing the shift. PyPy/CPython return 0 for zero shifted by any non-negative count, so this changes valid code that used to be allocation-free; short-circuit zero before the reservation in this helper so both the int and long callers keep that behavior.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in `caf61cb243`.

`0 << 1018` raised MemoryError; both oracles return `0`. `rbigint.py:1323 lshift` returns self when `int_other == 0 or selfsign == 0`, so neither a zero operand nor a zero shift allocates however large the count — that guard is now the first thing `checked_bigint_lshift` does, which covers both the int and long callers. `1 << 1018` still raises MemoryError.

Guarded by `str_startswith_bounds.py`.

commented by Claude

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a2b9bce2e3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +8108 to 8109
unsafe { crate::objspace::descroperation::reject_pow_operand_overflow(b)? };
float_pow_reject_modulus(args)?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject float pow modulus before overflow checks

With a non-None modulus, this overflow check runs before float_pow_reject_modulus, so calls like pow(1.0, 10**1000, 3) or float.__pow__(1.0, 10**1000, 3) raise OverflowError while converting the exponent instead of the required ternary-pow TypeError. Reject the modulus before attempting float operand coercion for the ternary slot.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not applying this one — the requested behaviour is CPython's, not PyPy's, and pyre targets PyPy.

```
pow(1.0, 10**1000, 3)
pypy3 7.3.20 : OverflowError: int too large to convert to float
cpython 3.14 : TypeError: pow() 3rd argument not allowed unless all arguments are integers
pyre : OverflowError: int too large to convert to float
```

PyPy converts the exponent before it rejects the modulus, so the OverflowError wins; the current ordering is deliberate and matches it. `float.pow(1.0, 101000, 3)` and `pow(2.0, 101000, 5)` behave the same way in both pypy3 and pyre. `pow(2.0, 3, 5)` still raises the ternary TypeError.

This is also why these lines cannot be added to a synth test: check.py's baseline gate requires cpython == pypy, and here they disagree — so they are hand-verified against pypy3 instead.

commented by Claude

let w_set_type = crate::typedef::gettypeobject(&pyre_object::setobject::SET_TYPE);
let w_set = crate::call::call_function_impl_result(w_set_type, &[lhs])?;
let method = crate::baseobjspace::getattr_str(w_set, methname)?;
crate::call::call_function_impl_result(method, &[rhs])?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve dict-view set ops for arbitrary iterables

When methname is intersection_update or symmetric_difference_update, delegating to pyre's set update methods changes dict-view results for one-shot or duplicate-containing RHS values: d={1:0,2:0}; d.keys() & iter([1,2]) consumes the iterator during the first membership pass and drops 2, and d={1:0}; d.keys() ^ [2,2] toggles 2 twice. The previous snapshot implementation collected/deduped the RHS once, so keep that behavior or fix the update helpers before routing views through them.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Half right, and the real half is fixed in `caf61cb243`.

The one-shot example already worked — `d={1:0,2:0}; d.keys() & iter([1,2])` gives `[1, 2]` in pyre, matching both oracles, because the view op materialises its operand before the membership pass. The duplicate case was genuinely broken: `d={1:0}; d.keys() ^ [2,2]` gave `[1]` against `[1, 2]`.

Root cause is the same as the `symmetric_difference_update` thread below, and fixing that fixed this. Both are guarded by the new `set_update_materialize_rhs.py` (19 lines, cpython == pypy == pyre), which covers the one-shot and duplicate cases for `&`, `|`, `-` and `^`.

commented by Claude

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f3e2c3fb49

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pyre/pyre-interpreter/src/typedef.rs Outdated
Comment on lines +13965 to +13969
let other_items = crate::builtins::collect_iterable(*other)?;
for item in other_items {
// Each element is hashed as it is looked up, so an unhashable one
// raises even when self is empty and nothing can match.
crate::type_methods::set_discard_checked(args[0], item)?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Materialize RHS before mutating difference_update

When a non-set iterable contains an element that is removed and a later unhashable/raising element, this per-item discard mutates self before the error is reported; for example s = {1, 2}; s.difference_update([1, []]) leaves s as {2} before raising. PyPy's setobject.py:descr_difference_update first builds w_other_as_set = self._newobj(space, w_other) and only then applies difference_update, so errors while hashing the RHS leave the original set unchanged.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in `caf61cb243`. `s = {1, 2}; s.difference_update([1, []])` left `s` as `[2]`; pypy leaves it `[1, 2]`. Now ports `setobject.py:389 descr_difference_update` — the operand becomes a set first, so it is hashed and deduped before anything is removed.

Note this one is cpython != pypy (cpython leaves `[2]`, pypy `[1, 2]`), so it is hand-verified against pypy3 rather than synth-gated.

commented by Claude

Comment thread pyre/pyre-interpreter/src/typedef.rs Outdated
Comment on lines +14013 to +14017
if present {
unsafe { pyre_object::w_set_discard(args[0], item) };
} else {
crate::type_methods::set_add_checked(args[0], item)?;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Materialize symdiff RHS before toggling self

For non-set RHS iterables, this toggles self as each item is consumed, so a later unhashable element leaves a partial mutation; e.g. s = {1, 2}; s.symmetric_difference_update([3, []]) adds 3 before raising. PyPy's descr_symmetric_difference_update constructs w_other_as_set = self._newobj(space, w_other) before calling self.symmetric_difference_update, which both hashes/dedupes the RHS first and keeps self unchanged if that construction fails.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in `caf61cb243`. `s = {1, 2}; s.symmetric_difference_update([3, []])` left `s` as `[1, 2, 3]`; pypy leaves it `[1, 2]`. Now ports `setobject.py:497 descr_symmetric_difference_update` — building the set first both dedupes the operand (fixing the double-toggle in the dict-view thread) and leaves self untouched when it raises.

Worth recording the boundary: `descr_update:506` deliberately does NOT materialise, so `{1, 2}.update([3, []])` legitimately leaves `[1, 2, 3]` in pypy and cpython alike — pyre already matched that and it is kept, with a synth line pinning it.

commented by Claude

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: caf61cb243

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pyre/pyre-interpreter/src/typedef.rs Outdated
w_iterable: pyre_object::PyObjectRef,
) -> Result<Vec<pyre_object::PyObjectRef>, crate::PyError> {
let items = crate::builtins::collect_iterable(w_iterable)?;
let w_set = crate::type_methods::set_from_items_checked(&items)?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Root set materialization across hashing callbacks

When a set operation materializes an iterable containing objects with user-defined __hash__, set_from_items_checked hashes each collected PyObjectRef without pinning either the collected items or the temporary set. The surrounding code already roots self and items before try_hash_value in set_method_update because __hash__ can allocate and trigger a moving collection; the same scenario here (for example set().intersection([AllocatingHash()]), dict-view set ops, or the new update helpers) can leave stale pointers or let the temporary set be collected before w_set_items reads it. Root the temporary set and the collected elements while hashing them.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 66dac3d8f5 — the finding is correct, and it reached further than the citation.

set_from_items_checkedw_set_from_items_checked did let s = w_set_new() and then held s as a raw pointer across every element's object_key_for_checked, i.e. across every user __hash__, while reading the elements from an unrooted slice. Both halves of what you describe.

Rather than add rooting to that builder, set_newobj_items now builds through builtin_set_from_items, which already carries exactly the convention you point at in set_method_update — root self and every element, hash, reload both, store — and returns the reloaded set. That left set_from_items_checked, w_set_from_items_checked and w_set_add_checked with no callers, so they are deleted rather than left in the tree as an unrooted builder for the next caller to reach for.

Verified with the scenario you named: {a}.intersection([a]) / {a} & {a} / {a}.union([A(), A()]) over 2000 iterations with an A.__hash__ that allocates to provoke a minor collection — byte-identical to pypy3. python ./pyre/check.py is green on dynasm 204/204 and cranelift 204/204.

Two notes for the record:

  • The same commit's parent fixes a related defect this rooting work surfaced: the gate hashed each element twice (try_hash_value for the raise check, then w_set_addobject_key_for → the hash_w hook → the same try_hash_value) and swallowed an eq_w raise from the bucket probe, so set([EqBoom(), EqBoom()]) returned a 2-element set where newset's r_dict(space.eq_w, space.hash_w) (setobject.py:1611) raises. Both are now gated by pyre/bench/synth/set_key_protocol.py.
  • Still open, tracked separately: the eq_w probe is itself a collection point (a user __eq__ under a hash collision), so s.len += 1 / dict_write_barrier(obj) after entries.insert can still write through a relocated header. That hazard is uniform and pre-existing — w_dict_store_object_strategy_checked on the dict.__setitem__ path ships it on main today — so this change does not regress it, but it is not closed either.

commented by Claude

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 66dac3d8f5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pyre/pyre-interpreter/src/typedef.rs Outdated
Comment on lines +15168 to +15173
let items = crate::builtins::collect_iterable(w_iterable)?;
// Each element's `__hash__` is a collection point that can move both the
// temporary set and the elements not yet added, so the build roots them
// and reloads across every hash; `builtin_set_from_items` returns the
// reloaded set, and reading its elements back allocates nothing.
let w_set = crate::builtins::builtin_set_from_items(&items)?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid rehashing existing set operands

When w_iterable is already a set/frozenset, this helper rebuilds it through builtin_set_from_items, so operations like s.intersection(s), s.intersection_update(s), s.difference_update(s), or s.symmetric_difference_update(s) call each element's current __hash__ again and can now raise if a stored object's hash later starts raising. PyPy's set_strategy_and_setdata copies a W_BaseSetObject's existing storage, and the update paths pass set operands directly, so set-to-set operations reuse stored hashes rather than rehashing elements; branch on is_set_or_frozenset here and return w_set_items directly.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed — and the finding generalised further than the citation.

You are right that set_strategy_and_setdata gives a set operand its own branch (setobject.py:1619-1621), and that pyre rebuilt it instead. Measuring it (a __hash__ that counts calls, one element per set) showed every set-to-set line diverging, not just the four named:

case pypy pyre before
s.intersection(s) 0 5
s.intersection(t) 0 4
s.union(t) 0 2
s.difference(t) 0 3
set(s) / frozenset(s) 0 1
s.intersection_update(s) 1 5

So the root was not the four operand sites but the primitive under them: w_set_items returns Vec<PyObjectRef> built from .keys().map(|key| key.obj), i.e. it drops key.hash, so every rebuild had to re-hash.

Rather than branch on is_set_or_frozenset at each call site over the same lossy primitive, the storage layer now mirrors upstream:

  • w_set_copy_storage_from:1620-1621 + :963-966 get_storage_copy, which is d.copy() — a bulk table copy, not an element-by-element refill, so it calls neither __hash__ nor __eq__.
  • w_set_key_at / w_set_contains_key_checked / w_set_insert_key_checkediterkeys_with_hash / contains_with_hash / setitem_with_hash (:1147-1154 _intersect_unwrapped).
  • w_set_update_from_set:1400-1403 ObjectSetStrategy.update (d_obj.update(d_other)).

The set methods then take upstream's shape: descr_union:363 = copy_real() + merge, descr_difference:281 = copy_real() + difference_update, descr_intersection:309 seeds with _newobj and intersects set operands as they stand.

All set-to-set counts now equal pypy's exactly. Gated in pyre/bench/synth/set_operand_storage.py (baseline-clean: cpython and pypy agree on every line).

One genuine divergence remains and is deliberately not in this change: frozenset.__hash__ re-walks on every call because pyre's W_SetObject has no hash memo field (descr_hash:624 early-returns, :643 stores). That is a different mechanism and it changes the GC struct layout, so it is filed separately.

commented by Claude

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pyre/bench/synth/set_update_hash_other.py`:
- Around line 60-61: Update pyre/bench/synth/set_update_hash_other.py at lines
60-61 so the iu and du benchmark lambdas accept *others, invoke each with two
operands, and ensure du_two_others_unhashable includes an actual unhashable
element. Update pyre/bench/synth/set_intersection_operand.py at line 75 so iu
accepts *others and iu_two_others passes two distinct operands.

In `@pyre/pyre-interpreter/src/objspace/descroperation.rs`:
- Around line 85-118: Update checked_bigint_lshift to replace the temporary
Vec<u64> try_reserve_exact probe with the BigInt library’s allocation-checking
or fallible reservation API for the shifted result. Ensure the guard targets the
same limb allocation used by bigint_lshift, preserves the
zero-shift/zero-operand fast paths, and returns catchable MemoryError before
invoking bigint_lshift when allocation cannot succeed.

In `@pyre/pyre-interpreter/src/typedef.rs`:
- Around line 14757-14764: The convert_set_to_frozenset function currently
rebuilds a frozenset from bare elements, discarding the source set’s hashed
storage. Update this conversion to reuse the set’s strategy/storage, preserving
stored hashes and avoiding rehashing, while retaining the existing non-set None
return behavior.
- Around line 15165-15196: The intersection path currently discards hashed
storage and rehashes elements through w_set_from_items and w_set_contains.
Update set_newobj_items and set_intersect_update to retain and pass the
temporary set or its ObjectKeys through the operation, using the existing
hashed-key lookup/data-structure APIs so user hash/equality callbacks and
failures are preserved correctly; do not flatten the data into bare object
pointers and rebuild a set.
- Around line 15403-15407: Update the set update-removal and toggle logic around
set_newobj_items and w_set_discard to use checked, prehashed contains/remove/add
operations. Ensure membership comparisons require matching hashes before
equality checks, and propagate protocol errors instead of swallowing them across
both difference_update and symmetric-difference paths.

In `@pyre/pyre-object/src/setobject.rs`:
- Around line 267-275: Preserve hashed set storage across transformations
instead of flattening to object references and rehashing. In
pyre/pyre-object/src/setobject.rs lines 267-275, update w_set_replace_items to
replace entries using stored ObjectKeys and their precomputed hashes. In
pyre/pyre-interpreter/src/typedef.rs lines 14757-14764, make frozenset
construction share or copy the source set’s hashed storage. In
pyre/pyre-interpreter/src/typedef.rs lines 15165-15196, retain temporary set
storage through intersection rather than returning bare items; match the
existing RPython strategy/storage shape at all sites.
- Around line 205-216: Update w_set_contains_checked and the corresponding
discard helper to pin both the set object and item before invoking user hashing,
compute the digest while pinned, then reload both references before accessing
raw pointers. Reuse the hashed checked contains/discard APIs and follow the
existing add-path pattern, preserving DictKeyError propagation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 929353bf-49a6-43d7-bd4f-3126b1c04fc4

📥 Commits

Reviewing files that changed from the base of the PR and between 6564959 and 66dac3d.

📒 Files selected for processing (27)
  • majit/majit-translate/src/annotator/classdesc.rs
  • majit/majit-translate/src/translator/rtyper/rclass.rs
  • majit/majit-translate/src/translator/rtyper/rpbc.rs
  • pyre/bench/synth/dict_ctor_consume.py
  • pyre/bench/synth/dict_hash_protocol.py
  • pyre/bench/synth/dict_view_set_ops.py
  • pyre/bench/synth/float_pow_overflow_exp.py
  • pyre/bench/synth/int_lshift_memoryerror.py
  • pyre/bench/synth/operator_set_inplace_ops.py
  • pyre/bench/synth/set_contains_frozenset.py
  • pyre/bench/synth/set_hash_protocol.py
  • pyre/bench/synth/set_intersection_operand.py
  • pyre/bench/synth/set_key_protocol.py
  • pyre/bench/synth/set_update_hash_other.py
  • pyre/bench/synth/set_update_materialize_rhs.py
  • pyre/bench/synth/str_startswith_bounds.py
  • pyre/pyre-interpreter/src/baseobjspace.rs
  • pyre/pyre-interpreter/src/builtins.rs
  • pyre/pyre-interpreter/src/module/operator/mod.rs
  • pyre/pyre-interpreter/src/objspace/descroperation.rs
  • pyre/pyre-interpreter/src/objspace/std/formatting.rs
  • pyre/pyre-interpreter/src/opcode_ops.rs
  • pyre/pyre-interpreter/src/sliceobject.rs
  • pyre/pyre-interpreter/src/type_methods.rs
  • pyre/pyre-interpreter/src/typedef.rs
  • pyre/pyre-object/src/dictmultiobject.rs
  • pyre/pyre-object/src/setobject.rs
💤 Files with no reviewable changes (2)
  • majit/majit-translate/src/translator/rtyper/rpbc.rs
  • majit/majit-translate/src/annotator/classdesc.rs

Comment on lines +60 to +61
m("iu_two_others", lambda: iu({1, 2}, [1]))
m("du_two_others_unhashable", lambda: du({1}, [1]))

@coderabbitai coderabbitai Bot Jul 16, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Make the advertised multi-operand benchmark cases variadic.

These cases currently exercise only the ordinary single-operand paths.

  • pyre/bench/synth/set_update_hash_other.py#L60-L61: make iu/du accept *others, pass two operands, and include an actual unhashable element in du_two_others_unhashable.
  • pyre/bench/synth/set_intersection_operand.py#L75-L75: make iu variadic and pass two distinct operands to iu_two_others.
📍 Affects 2 files
  • pyre/bench/synth/set_update_hash_other.py#L60-L61 (this comment)
  • pyre/bench/synth/set_intersection_operand.py#L75-L75
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pyre/bench/synth/set_update_hash_other.py` around lines 60 - 61, Update
pyre/bench/synth/set_update_hash_other.py at lines 60-61 so the iu and du
benchmark lambdas accept *others, invoke each with two operands, and ensure
du_two_others_unhashable includes an actual unhashable element. Update
pyre/bench/synth/set_intersection_operand.py at line 75 so iu accepts *others
and iu_two_others passes two distinct operands.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch on both — the cases did not exercise what their names advertised. Rather than make those two lambdas variadic in place, the multi-operand paths are now covered in the new pyre/bench/synth/set_operand_storage.py, which is where the operand handling actually lives after this change:

  • intersection_multia.intersection(b, {3, 2})
  • difference_multia.difference({1}, {2})
  • union_multia.union({9}, [8]), mixing a set operand and a non-set one so both branches of descr_union:365-369 run in one call

and an unhashable/raising element is covered by the *_eq_boom lines (union_eq_boom, intersection_eq_boom, difference_eq_boom, symdiff_eq_boom, update_eq_boom), which now raise rather than returning a container — they were swallowing the __eq__ before this change.

The whole file is baseline-checked: cpython3.14 and pypy3 agree on every line, so each one is a real gate rather than a snapshot of pyre's own behaviour.

commented by Claude

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
fd set_operand_storage.py
echo "---"
cat pyre/bench/synth/set_operand_storage.py 2>/dev/null | sed -n '1,200p'

Length of output: 265


🏁 Script executed:

#!/bin/bash
fd -i "set_operand" 
echo "---"
git log --oneline -10
echo "---"
git show --stat HEAD | head -50
echo "---branches---"
git branch -a

Length of output: 2456


🏁 Script executed:

#!/bin/bash
git log --all --oneline -- pyre/bench/synth/set_operand_storage.py
echo "---"
git log --all --oneline | head -30
echo "---diff HEAD~15..HEAD names---"
git log --name-status -15 --oneline | grep -i operand

Length of output: 3129


🏁 Script executed:

#!/bin/bash
sed -n '1,70p' pyre/bench/synth/set_update_hash_other.py
echo "=== set_intersection_operand.py ==="
sed -n '1,90p' pyre/bench/synth/set_intersection_operand.py

Length of output: 6151


@youknowone, that's a reasonable design choice — routing multi-operand coverage through a dedicated file rather than retrofitting these two lambdas makes sense, especially with the *_eq_boom cases catching the swallowed-__eq__ bug.

However, I can't find pyre/bench/synth/set_operand_storage.py in the branch at commit 66dac3d8f515d11157c9e474c8ef50f18821205b (checked via fd and git log --all -- pyre/bench/synth/set_operand_storage.py, both empty), and the two flagged files (set_update_hash_other.py, set_intersection_operand.py) still have the original non-variadic iu/du helpers with the misleadingly-named iu_two_others/du_two_others_unhashable cases unchanged. Could you double check that the new file has been pushed? Once it's up I'm happy to take a look and close this out.

Comment thread pyre/pyre-interpreter/src/objspace/descroperation.rs
Comment thread pyre/pyre-interpreter/src/typedef.rs Outdated
Comment thread pyre/pyre-interpreter/src/typedef.rs Outdated
Comment thread pyre/pyre-interpreter/src/typedef.rs Outdated
Comment thread pyre/pyre-object/src/setobject.rs
Comment thread pyre/pyre-object/src/setobject.rs Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review

"__buffer__",
|args| crate::builtins::w_memoryview_new(args[0]),

P2 Badge Reject writable buffer requests for bytes

When the requested flags include PyBUF_WRITABLE (for example bytes.__buffer__(b"x", 1)), an immutable bytes exporter cannot satisfy the request and the buffer protocol requires the exporter to fail instead of returning a read-only view. This closure ignores args[1] entirely and always returns memoryview(bytes), so callers that explicitly requested writable access get a successful read-only buffer rather than the required error.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e9bc340d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

/// 3.14-visible type identity.
#[pyre_class("list_iterator", static_name = "LIST_ITER")]
pub struct W_ListIterObject {
pub seq: PyObjectRef,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Trace concrete iterator referents during GC

When iter(list) now returns this new W_ListIterObject, the list can be reachable only through the iterator, but the GC root walker still only special-cases is_seq_iter in walk_raw_seq_iter_roots and these pyre_class iterators are not registered with a custom trace. In a context like a live iterator on the value stack followed by gc.collect() after the original list reference is dropped, the seq field is never visited, so the container can be swept while next(it) still dereferences it; the same tracing gap applies to the new list-reverse, tuple, and set iterator payloads.

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 85cd496fbc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}
let mv = args[0];
unsafe { memoryview_check_released(mv)? };
let n = unsafe { pyre_object::memoryview::w_memoryview_length(mv) };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use element count for memoryview.index bounds

For views whose itemsize is greater than 1, w_memoryview_length is the byte length, while memoryview_getitem accepts element indices up to length / itemsize. With a multi-byte view such as one over an array('H') or a cast to 'H', a missing search will iterate past the valid element count and propagate IndexError from memoryview_getitem instead of returning ValueError, and negative start/stop are also clamped against bytes rather than elements. Compute n the same way as __len__/__getitem__ do: length / itemsize.

Useful? React with 👍 / 👎.

getindex_w (baseobjspace.py:1574) remaps a TypeError raised while coercing a
subscript key through __index__ — a non-int __index__ return, or a TypeError
raised inside __index__ — to "<descr> indices must be integers or slices, not
'<type>'" whenever objdescr is set. The inlined subscript coercions
(list/tuple/str/bytes __getitem__, list/bytearray __setitem__, and
subscript_index_w on the __delitem__ path) previously let space.index's
"__index__ returned non-int (type X)" propagate verbatim. A ValueError from
__index__ still propagates; list.insert / list.pop pass no objdescr and keep
surfacing the raw error.

ord() on a multi-byte bytes argument now reports "bytes of length N"
(bytesobject.py:473); bytearray keeps "string of length N"
(bytearrayobject.py:213) — the shared branch reported "string" for both.

Comment-only: pow3 notes the omitted is_cpytype() pow3_bug_compat_cpyext
branch; index_type_error notes the reference pypy3 quotes the %T operand.

Assisted-by: Claude
Route integer coercion through the index protocol and align
sequence-concatenation errors with pypy across the slice, index,
format, round and concat paths.

- builtins::getindex_w: clamp an out-of-word overflow to i64::MIN for a
  negative value (was always i64::MAX), so a large-negative slice bound
  floors to 0 instead of collapsing to the tail.
- normalize_slice / sliceobject::adapt_lower_bound: fold a negative index
  by the length with saturating_add so an i64::MIN bound does not overflow
  before flooring at 0.
- str_slice_args (startswith/endswith): take start/end bounds through
  adapt_lower_bound(eval_slice_index) so a non-index bound raises, an
  __index__ object is honoured, and a positive bound is not upper-clamped.
- builtin_round: take ndigits through getindex_w for a float (clamping)
  and index for an int; a huge-magnitude negative ndigits short-circuits
  to 0 rather than building an astronomical power of ten.
- index_to_bigint (hex/oct/bin): coerce through space.index so a non-int
  __index__ return raises "__index__ returned non-int (type X)".
- formatting number_arg_decimal / number_arg_integer (%d/%i/%o/%x): remap
  a TypeError from the numeric decoder to the operand-type error naming
  the original argument.
- list/tuple/bytes/bytearray __add__ and descroperation::add: return
  NotImplemented for a non-sequence operand so the operator raises the
  generic "unsupported operand type(s) for +", dropping the cpython-only
  "can only concatenate" message; bytearray __iadd__ on a non-buffer now
  raises "a bytes-like object is required, not 'X'".

Assisted-by: Claude
Add the in-place operator-module functions and the mutable set's in-place
operator slots, both previously missing.

- operator: iadd/isub/imul/ifloordiv/imod/itruediv/ipow/ilshift/irshift/
  iand/ior/ixor route through binary_value's in-place path (space.inplace_X),
  and iconcat requires both operands to be subscriptable. The extra_init
  dunder aliases (operator.__iadd__ etc.) now resolve as a result.
- set: __isub__/__iand__/__ior__/__ixor__ mirror setobject.py's
  descr_inplace_sub/_and/_or/_xor — a non-set/-frozenset operand yields
  NotImplemented, otherwise self is mutated through the matching update
  helper and returned. The four *_update method bodies are extracted into
  named functions shared by the app methods and the in-place slots.

Assisted-by: Claude
A left shift whose result exceeds allocatable memory (e.g. `1 << 10**18`)
drove the underlying big-int `<<` into the infallible allocator and aborted
the process. Pre-flight a fallible reservation of the result's 64-bit limb
count in checked_bigint_lshift and raise a catchable MemoryError instead,
matching long_lshift. int_lshift and long_lshift route through the checked
helper; the probe upper-bounds the shift's own allocation so it fails
exactly when the real shift would.

Assisted-by: Claude
A float or complex power coerces each int operand to a double before the
power is computed, so an over-range int base or exponent raises
OverflowError up front -- even `1.0 ** huge`, which never reaches the
arithmetic, matching float(huge). reject_pow_operand_overflow performs
that check and is called on both operands in the two-argument pow /
pow_builtin float and complex paths. In float.__pow__/__rpow__ the check
runs before float_pow_reject_modulus, so `pow(2.0, huge, 5)` raises
OverflowError from the coercion rather than TypeError from the modulus
rejection.

Assisted-by: Claude
dict.fromkeys filled a plain dict with a raw store that never invoked the
hash protocol, so a key whose __hash__ raised was stored unhashed and an
unhashable key produced a dict that could not be read back:
`dict.fromkeys([[]])` returned `{[]: None}`. The plain-dict branch now
fills through the dict's own checked setitem, matching descr_fromkeys,
which fills via `w_dict.setitem`; the subclass branch already routed
through space.setitem.

The unhashable-type gate matched all three view types and reported them
all as 'dict view'. Only the keys and items views are set-like — they
define __eq__ and so are unhashable — while the values view keeps
object.__hash__. Both gate sites now test the keys and items views
separately, mirroring the two isinstance checks in _is_set_like, and
report the view's own type name; the values view hashes again.

Assisted-by: Claude
The four set operations on a dict view were computed inline over raw
snapshot vectors, with a separate hand-written branch in the reflected
dispatcher for the non-commutative difference. _as_set_op instead builds
a set from the left operand and calls the matching in-place set method
against the right one; the reflected shape builds the set from the other
operand. The helper's own comment recorded why the port had been
deferred -- pyre's set typedef did not expose the in-place mutators --
which no longer holds since 6094f705b18 added them.

dict_view_set_op and dict_view_rset_op now delegate to that shape and
the DictViewSetOp enum and its compute function are gone. The dispatchers
already passed the upstream method names, so they are unchanged. Results
are unchanged; routing the operands through the set constructor and the
set methods is what will enforce the hash protocol once the set type
runs it.

Assisted-by: Claude
Every path that put an element into a set went through w_set_add, which
builds its key with the infallible object_key_for -- documented as
swallowing a hash error and falling back to a structural hash, with
checked callers directed to object_key_for_checked. Nothing called the
checked variant, so the set type never ran the hash protocol: an element
whose __hash__ raised was stored anyway, and an unhashable one produced a
set that could not be read back, e.g. `set([[]])` returned `{[]}`.

setobject gains checked variants built on object_key_for_checked, so a
raising or missing __hash__ propagates and the element is not stored;
they mirror w_dict_store_object_strategy_checked, including dropping the
spurious entry an eq error appends mid-probe. The set and frozenset
constructors, set.add, set.update, symmetric_difference_update, and the
set-literal and set-comprehension opcodes now add through them, matching
newset, whose backing r_dict hashes with space.hash_w.

This also settles five of the dict-view set-op divergences: since the
_as_set_op port routes the operands through the set constructor and the
set methods, hashing the ingested elements is what makes those raise.
The remaining one, `keys() & [[]]`, needs difference_update and
intersection_update to hash the other operand, which they still do not;
w_set_contains_checked and w_set_discard_checked are in place for that.

Assisted-by: Claude
difference_update looked its elements up with the raw discard and
intersection_update compared them with eq_w, so neither ever hashed the
other operand: an unhashable element read as absent instead of raising,
and it raised nothing even with an empty self, where upstream still
hashes. difference_update now discards through the checked path, and
intersection_update hashes each element as it is collected.

intersection_update also collected each other operand once per element of
self and skipped them entirely when self was empty; it now collects them
once up front, which is what lets the empty-self case raise.

This closes the last of the dict-view set-op divergences: `keys() & [[]]`
runs set(self).intersection_update(other), so hashing the other operand
is what makes it raise.

remove, discard and `in` still do not hash, and a set argument is not
converted to a frozenset on those paths; upstream hashes there only for a
non-empty set, so they are left alone here.

Assisted-by: Claude
`in`, discard and remove hashed the element with the infallible key
builder, so an unhashable one read as absent and a raising __hash__ was
swallowed. They now go through w_set_contains_checked /
w_set_discard_checked and, on a TypeError, retry with the frozenset
holding the same elements, so `{1} in {frozenset([1])}` is True and
`{frozenset([1]), 2}.remove({1})` removes it.

descr_contains hashes even against an empty set (EmptySetStrategy.has_key,
issue 3824), while EmptySetStrategy.remove matches nothing without
hashing; set_remove keeps that split, so set().discard([]) still returns
None and set().remove([]) still raises KeyError.

Ports setobject.py _convert_set_to_frozenset, descr_contains,
_discard_from_set, descr_discard and descr_remove. The intersection_update
result is still built from self, so `{1, 2}.intersection_update([1.0])`
keeps 1 where upstream keeps 1.0.

Assisted-by: Claude
An intersection walked self and kept self's elements, so `{1, 2}
.intersection_update([1.0])` left 1 where upstream leaves 1.0. Equal
elements can be distinct objects, so which side is walked is observable.

descr_intersection now seeds the result with the shortest operand and
intersects the rest into it, and intersect_update walks the shorter of the
two sides. Both are needed: seeding measures the operands as given, so a
generator has no length and never seeds, and a list is measured with its
duplicates -- `{1, 2}.intersection([1.0, 1.0])` seeds from self and the
per-step swap is what still reaches 1.0. descr_intersection_update replaces
self's storage with that result through w_set_replace_items.

`&` carried a second copy of the intersection that walked self; it now
calls descr_intersection, as descr_and calls intersect upstream.

Assisted-by: Claude
symmetric_difference_update and difference_update consumed a non-set
operand element by element, so a duplicate toggled twice and an unhashable
element later in the operand left self partially mutated. Both now build a
set from the operand first, as descr_symmetric_difference_update and
descr_difference_update do, which hashes and dedupes it before self is
touched. update keeps consuming element by element, matching descr_update.

startswith/endswith left a positive out-of-range `end` unbounded, so an
out-of-range `start` compared below it and both offsets mapped to the end of
the string, matching an empty prefix. _unwrap_and_compute_idx_params instead
turns a `start` past the end into `end_index + 1`, inverting the window, so
'abc'.startswith('', 5, 10) is False.

checked_bigint_lshift sized its reservation from the shift count alone, so
`0 << 10**18` raised MemoryError; lshift returns self for a zero operand or
a zero shift without allocating.

Drops set_add_checked and w_frozenset_from_items_checked, left without
callers now that the ingestion sites hash through the rooted try_hash_value
gate.

Assisted-by: Claude
`newset` (setobject.py:1611) builds the backing storage as
`r_dict(space.eq_w, space.hash_w)`, so one store hashes the element
once and compares it with `eq_w`, and either callback raising aborts
the store.

The store gate ran `try_hash_value` for the raise check and then an
infallible `w_set_add` / `w_dict_store`, which hashed the element a
second time through `object_key_for` -> the `hash_w` hook -> the same
`try_hash_value`, and swallowed an `eq_w` raise during the bucket
probe, leaving the element stored. `set([EqBoom(), EqBoom()])` returned
a 2-element set where the equality protocol raises.

Take the digest from the gate's own `try_hash_value` and key the store
with it, through `object_key_hashed` and the hashed-checked store
primitives. The gate keeps its shadow-stack rooting across `__hash__`,
so the store still runs on reloaded pointers.

`w_dict_store_checked` and `w_dict_store_object_strategy_checked` now
delegate to an `Option<i64>`-hash inner shared with the hashed
variants. Only the object strategy consults the digest; the typed
strategies key on the unwrapped payload and never reach `space.hash_w`.

Covers set.add / update / the set and frozenset constructors / the set
literal and comprehension opcodes, dict.fromkeys, MAP_ADD, and the
dict mapping- and pairs-update paths.

Assisted-by: Claude
`descr_new` (dictmultiobject.py:115-117) allocates the instance and
returns it, ignoring `__args__`; filling is `descr_init` (:137-138) ->
`init_or_update` (:1430).

The exact-`dict` branch of `dict.__new__` called `builtin_dict_ctor`,
which populated from the arguments, and `dict.__init__` then populated
the same object again through `dict_update1`. The source was walked
twice: `dict(mapping)` ran `keys()` and `__getitem__` twice per key,
`dict(pairs)` hashed each key twice, and `dict(one_shot_iterable)`
raised on the second walk where the iterable refuses to be re-entered.
A generator argument masked it by being exhausted on the first walk.

Return an empty dict, as the dict-subclass branch below it already
does and for the reason its comment already gives. That leaves
`builtin_dict_ctor` with no callers, so it goes; `dict_update1` already
carries the `is_dict` copy, mapping-protocol and pairs paths, and
`dict_init_or_update` already drops the `__pyre_kw__` marker entry.

Assisted-by: Claude
`_newobj` (setobject.py:407 / :609) materializes an iterable operand
into a set, hashing each element on the way in. A user `__hash__` is a
collection point, so it can move both the temporary set and the
elements not yet added.

`set_newobj_items` built that set through `set_from_items_checked`,
which held the fresh set as a raw pointer across every element's
`__hash__` and read the elements from an unrooted slice. Reachable
from `set().intersection([AllocatingHash()])`, the dict-view set
operations, and the update helpers.

Build it through `builtin_set_from_items`, which already roots the set
and every element and reloads across each hash, and returns the
reloaded set. That leaves `set_from_items_checked`,
`w_set_from_items_checked` and `w_set_add_checked` without callers, so
they go rather than stay as an unrooted builder for the next caller to
reach for.

Assisted-by: Claude

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

https://github.com/youknowone/pyre/blob/00fd26dafcd5699df13bcad9ffceec9a759d7ac8/pyre-object/src/descriptor.rs#L98-L99
P1 Badge Add a write barrier when resetting super fields

w_super_new allocates normal super objects through the stable old-gen path and runs a write barrier for the initial fields, but super.__init__ now mutates that same object through this setter without remembering it again. When code keeps a proxy such as s = super(C, C()) and the bound instance is otherwise only reachable through s, a later minor collection will not visit the young obj field from the old-gen super and can leave s.__self__ pointing at reclaimed/moved memory. Run the GC write barrier after storing these fields.


https://github.com/youknowone/pyre/blob/00fd26dafcd5699df13bcad9ffceec9a759d7ac8/pyre-interpreter/src/builtins.rs#L9646-L9647
P2 Badge Preserve space.index TypeErrors for round ndigits

These new round() coercions route invalid ndigits through getindex_w, whose TypeError fallback is still the int() base-specific message rather than space.index's errors. For non-index or bad-__index__ values, cases like round(1.2, 'x') and round(1, BadIndex()) now report the wrong TypeError instead of PyPy's '<type>' object cannot be interpreted as an integer / __index__ returned non-int; use space_index with explicit overflow clamping or fix getindex_w before sharing it here.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@youknowone
youknowone merged commit d7d376e into main Jul 16, 2026
30 of 31 checks passed
@youknowone
youknowone deleted the rename branch July 16, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant