Builtin-leaf subclass / dict / deque / operator interpreter machinery + simplify_graph (#127) and canonical-flatten (#73) JIT passes - #130
Conversation
WalkthroughThis PR significantly extends Pyre's interpreter semantics, macro system, and JIT pipeline. It refactors class construction to support dict-subclass prepared namespaces with proper cell-variable routing, adds override-aware dispatch for arithmetic and comparison operators on builtin-leaf subclasses, implements builtin keyword-argument binding, expands the deque and operator modules, improves JIT graph simplification and bridging, and enhances diagnostics with better opcode context and singleton address registration. ChangesPyre Interpreter: Type System, Collections, and Operations
Pyre Builtins and Macros: Keyword Argument Resolution
Pyre JIT: Graph Simplification and Code Generation
Documentation and Benchmarks
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
347cbf6 to
cf727dd
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cf727dd064
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // target's own terminator is the first terminator in the merged block | ||
| // and the single-exit renaming splice lands after target's opcodes | ||
| // (codex P1, PR #127). | ||
| src_spam.strip_trailing_boundary_goto(&super::flatten::block_label_name(target)); |
There was a problem hiding this comment.
Strip collapsed-forwarder gotos before absorbing target
In the walker pipeline, eliminate_empty_blocks(&graph) has already retargeted graph links through dead forwarders, but rewrite_dead_forwarder_gotos does not retarget the inline TLabels until after this merge bridge. For a shape source -> dead_forwarder -> target that then records a trivial merge (source, target), the source block still ends in goto TLabel(dead_forwarder), so stripping only block_label_name(target) leaves the old terminator in front of the absorbed target ops. Then walker_post_walk_insert_renamings still sees that stale first terminator and can splice the target-exit renamings before the target's opcodes, reintroducing the stale-slot issue this strip is meant to prevent.
Useful? React with 👍 / 👎.
cf727dd to
ea03288
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ea03288e5f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let cell = class_ns.get("__classcell__").copied(); | ||
| for key in ["__class__", "__classdict__", "__classcell__", "__classdictcell__"] { | ||
| class_ns.remove(key); |
There was a problem hiding this comment.
Preserve classcell for custom metaclasses
When a class body references __class__ or uses zero-argument super(), Python passes __classcell__ through the namespace to the metaclass; custom metaclasses can inspect it or forward it to type.__new__. This removes the key before the metaclass path builds w_namespace_dict, so class C(metaclass=M): ... where M.__new__ expects __classcell__ now observes a different namespace than CPython/PyPy even though the cell is set later.
Useful? React with 👍 / 👎.
| if w.is_null() || unsafe { is_none(w) } { | ||
| None | ||
| } else { | ||
| Some(unsafe { w_int_get_value(w) }.max(0) as usize) |
There was a problem hiding this comment.
Reject negative deque maxlen instead of clamping
For deque(..., maxlen=-1) Python raises ValueError during construction, but this clamps the stored negative value to 0 whenever the bound is read. That lets invalid deques be created and makes later append/appendleft silently discard every element, so callers that rely on constructor validation continue with incorrect container semantics.
Useful? React with 👍 / 👎.
ea03288 to
fc697aa
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fc697aaa2f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let listrepr = snapshot(self_obj) | ||
| .into_iter() | ||
| .map(|it| unsafe { crate::py_repr(it) }) |
There was a problem hiding this comment.
Add a recursion guard before rendering deque items
When a deque contains itself (for example d = deque(); d.append(d); repr(d)), this new __repr__ maps each element through py_repr, which dispatches back into this same deque.__repr__ with no ReprGuard or placeholder path. That turns a valid recursive container into unbounded recursion/stack overflow instead of the guarded representation used by the built-in container reprs.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pyre/pyre-interpreter/src/eval.rs`:
- Around line 2005-2023: Add regression tests to cover both MAKE_CELL shapes so
the double-wrap fix is pinned: write one test that creates a class with a
__class__ closure cell (ensuring initialize_frame_scopes installs an empty cell)
and asserts that calling MAKE_CELL via make_cell does not wrap an existing cell
again; and a second test that defines a closure over a parameter (argument slot
promoted to a cellvar) and asserts make_cell wraps the raw slot exactly once
producing a W_CellObject containing the original value. Reference the
interpreter behavior around make_cell, initialize_frame_scopes and the MAKE_CELL
semantics when adding these focused tests.
In `@pyre/pyre-interpreter/src/module/_collections/mod.rs`:
- Around line 245-255: The deque __repr__ implementation calls crate::py_repr on
each element without guarding against self-referential structures, so add a
recursion guard around the element formatting in the __repr__ function: obtain
or use the existing interpreter/representation guard mechanism (e.g., a repr
guard/context used elsewhere) inside __repr, push self_obj before iterating
snapshot(self_obj), and for each element call a guarded representation helper
(or check if the element is the guarded object and return a placeholder like
"..." instead of recursing) before invoking crate::py_repr; ensure you still
format maxlen via maxlen_bound as before.
- Around line 81-89: The __init__ implementation currently swallows errors by
calling crate::builtins::collect_iterable(it).unwrap_or_default(), which hides
iterable/type errors; change __init__ to propagate errors instead: call
crate::builtins::collect_iterable(it) and match its Result (or use ? to
propagate) so that on Err you return the error rather than iterating an empty
list, then iterate the Ok(iterator) and call do_append for each item; update the
function signature to return a PyResult if required and ensure any error
returned uses the existing exception types/propagation conventions used
elsewhere in this module (e.g., propagate the Result from collect_iterable
rather than using unwrap_or_default).
In `@pyre/pyre-interpreter/src/objspace/std/mapdict.rs`:
- Around line 715-728: Add a regression test that constructs a Mapdict object
whose terminator kind is TerminatorKind::Devolved and verifies the read/write
DICT branch in terminator_read (and node_read) is exercised: create an object
with a Devolved terminator, use the write path to set a DICT attribute (so it
ends up in the object's dict via _obj_getdict), then call
node_read/terminator_read (via MapRef operations used in your tests) and assert
the returned PyObjectRef comes from _obj_getdict (i.e. matches the dict-stored
value) rather than the map storage; reference the terminator_read function,
MapdictObject/_obj_getdict, TerminatorKind::Devolved, DICT, and node_read when
adding the test.
In `@pyre/pyre-interpreter/src/typedef.rs`:
- Around line 2277-2283: The descriptor currently treats a null result from
resolve_dict_backing(args[0]) as an empty dict string; instead, detect when args
is non-empty and dict.is_null() and return a TypeError indicating an
invalid/unbound receiver rather than Ok(w_str_new("{}")). Update the call site
handling in the function containing resolve_dict_backing so that when
args.is_empty() you still return "{}" but when args.len() > 0 and dict.is_null()
you raise/return a TypeError (using the project's existing error
construction/return pattern) mentioning the descriptor and the bad receiver.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 503c3d1e-1f3d-46a0-8265-5394cf78589b
📒 Files selected for processing (20)
majit/majit-ir/src/box_ref.rsmajit/majit-metainterp/src/jitcode/assembler.rsmajit/majit-metainterp/src/optimizeopt/schedule.rsmajit/majit-metainterp/src/optimizeopt/vector.rsmajit/majit-translate/src/front/ast.rspyre/pyre-interpreter/src/baseobjspace.rspyre/pyre-interpreter/src/call.rspyre/pyre-interpreter/src/display.rspyre/pyre-interpreter/src/eval.rspyre/pyre-interpreter/src/jit_fnaddr.rspyre/pyre-interpreter/src/module/_collections/mod.rspyre/pyre-interpreter/src/module/operator/app_operator.pypyre/pyre-interpreter/src/module/operator/mod.rspyre/pyre-interpreter/src/objspace/std/mapdict.rspyre/pyre-interpreter/src/pyframe.rspyre/pyre-interpreter/src/typedef.rspyre/pyre-jit/src/jit/assembler.rspyre/pyre-jit/src/jit/codewriter.rspyre/pyre-jit/src/jit/flatten.rspyre/pyre-jit/src/jit/simplify.rs
fc697aa to
76290e3
Compare
…bridge (codex P2, #130) `rewrite_dead_forwarder_gotos` ran after `rewrite_trivial_link_merges`, so for a `source -> dead_forwarder -> target` shape the source block's boundary terminator still read `goto TLabel(dead_forwarder)` when the merge bridge's `strip_trailing_boundary_goto(block_label_name(target))` looked for it. The strip missed, leaving the stale terminator in front of the absorbed target opcodes and letting the single-exit renaming splice land before them. Move `rewrite_dead_forwarder_gotos` ahead of `remove_trivial_links` so the inline byte gotos already name the surviving target when the merge strip reads source terminators. The collapse guard now runs immediately before it. Assisted-by: Claude
…init errors (codex P2, coderabbit, #130) deque.__init__ now rejects a negative maxlen with ValueError and a non-integer maxlen with TypeError (gateway_nonnegint_w) at construction instead of clamping the stored value to 0 when the bound is later read, and propagates iterable errors from the extend loop rather than swallowing them via unwrap_or_default. maxlen_bound reads the validated value back without a clamp. deque.__repr__ enters a ReprGuard on self and renders `[...]` for a deque reachable from its own items, matching dequerepr / Py_ReprEnter, instead of recursing into unbounded `__repr__` calls. ReprGuard is exposed pub(crate) for the method to reach. Assisted-by: Claude
…asses (codex P2, #130) build_class stripped __class__/__classdict__/__classcell__/__classdictcell__ from the namespace before the metaclass path built its namespace dict, so `class C(metaclass=M)` where M inspects or forwards __classcell__ observed a namespace missing the cell. __class__ and __classdict__ are fast2locals cellvar mirrors that CPython never exposes as namespace keys, so they are still dropped up front. __classcell__ and __classdictcell__ are real namespace entries the class body stores: keep them in the namespace the metaclass receives, drop them in the default construction path before w_type_new, and consume them in type.__new__ (type_new_classcell) — skipping both from the new type's __dict__ and binding the captured cell to the type — so the metaclass sees them while the class __dict__ does not. Assisted-by: Claude
…oderabbit, #130) Unbound `dict.__repr__(x)` where x is not a dict (resolve_dict_backing returns null) formatted `{}` instead of rejecting the receiver. Raise TypeError like a builtin descriptor; a real dict (including an empty one) still resolves. Assisted-by: Claude
Pin the single-cell MAKE_CELL behavior: a parameter captured by an inner function (arg slot promoted to a cellvar) reads its value through one cell, and the implicit __class__ cellvar stays a single cell so zero-arg super() resolves the class. Assisted-by: Claude
#130) REPS=15 left the one-time build loop and JIT trace warmup dominating the measurement, putting the dynasm/cranelift vs-cpython ratio at the x15 gate on slower CI hardware. REPS=401 (odd, keeping the reversed result) amortises the warmup over the reverse() iterations the benchmark targets. Assisted-by: Claude
…ariance (#130) fib_loop is bignum-add bound. windows pyre runs markedly slower than macos/ubuntu for the same codegen, so cranelift exceeded the 2x vs-cpython bound there (0.41s vs 0.19s) while it stays ~0.8x cpython locally and passes at 2x on macos/ubuntu cranelift. Give cranelift 3x headroom; dynasm keeps 2x and a real regression still trips the gate. Assisted-by: Claude
There was a problem hiding this comment.
💡 Codex Review
https://github.com/youknowone/pyre/blob/76290e3ef58858a895e3832cabc1963120900cd2/pyre-interpreter/src/module/_collections/mod.rs#L236-L238
Validate rotate's count before reading it as an int
When rotate() is called with a non-int argument (for example d.rotate(None) or an object that should be handled via __index__), this unconditionally reads the object layout as a W_IntObject instead of raising/converting through the index protocol. Because w_int_get_value is unsafe and assumes an actual int object, these inputs can produce bogus rotation counts rather than the required TypeError/__index__ behavior.
https://github.com/youknowone/pyre/blob/76290e3ef58858a895e3832cabc1963120900cd2/pyre-interpreter/src/display.rs#L581-L584
Route str() through the new str-subclass override
This only makes direct py_str() callers honor a str subclass's __str__; the public str() path still bypasses it because builtin_str returns immediately for any STR_TYPE object and py_str_wtf8 also returns raw string contents before delegating. In a case like class S(str): def __str__(self): return 'override', str(S('x')) will still ignore the override, so the new dispatch needs to be shared with the WTF-8/builtin path as well.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
pyre/pyre-interpreter/src/builtins.rs (1)
1638-1645:⚠️ Potential issue | 🟠 Major | ⚡ Quick winPropagate
__set_name__failures from the resolved-namespace path.Line 1645 still uses unchecked
call_function. As this module'scall_and_checkhelper shows at Lines 2381-2393, that suppresses Python exceptions behindPY_NULL, so a descriptor that raises in__set_name__can leave class creation succeeding with partially initialized state on the new dict-subclass namespace path.Suggested fix
- let _ = crate::call_function(set_name, &[w_type, k]); + call_and_check(set_name, &[w_type, k])?;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/builtins.rs` around lines 1638 - 1645, The code currently calls crate::call_function unchecked for descriptors' __set_name__ in the resolved-namespace path (symbols: w_ns_backing, pyre_object::w_dict_items, is_str, crate::baseobjspace::getattr, crate::call_function), which will swallow Python exceptions; change this to use the existing call_and_check helper (the pattern used at call_and_check) instead of crate::call_function, check its return for PY_NULL, and if PY_NULL return/propagate the error immediately so exceptions raised by __set_name__ are not suppressed and class creation fails consistently with the other path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pyre/pyre-interpreter/src/baseobjspace.rs`:
- Around line 6855-6859: The current dynamic-type path calls
crate::call_function on a found __contains__ (in the block using
crate::typedef::r#type(haystack) and lookup_in_type_where) and then uses
is_true(result), which treats PY_NULL (an exception) as true; change this to use
the error-aware caller (call_and_check or call_function_impl_result) so
exceptions from __contains__ propagate. Update the dynamic-type branch to call
the error-checking helper and mirror the earlier instance-__contains__ branch’s
semantics (use the same call_and_check/call_function_impl_result flow and
is_true on the successful result) so both paths have identical error handling.
In `@pyre/pyre-interpreter/src/call.rs`:
- Around line 2257-2273: The prepared mapping path currently bypasses the
earlier cleanup of scaffolding keys: after computing classcell from class_ns
(using class_ns_ptr) ensure the same "__class__" and "__classdict__" keys are
removed from the prepared/backing mapping before passing it to the metaclass;
specifically, when the code uses w_prepared_dict or the mapping_namespace branch
to materialize the metaclass namespace, either delete "__class__" and
"__classdict__" from that mapping or construct the metaclass namespace from the
cleaned class_ns (the mutable reference used to set classcell) so the metaclass
never sees those keys.
- Around line 2237-2255: The code currently overlays entries from the backing
mapping onto the existing class_ns (class_ns_ptr) which leaves stale keys that
were deleted in the prepared mapping; change the logic in the mapping_namespace
branch (where resolve_dict_backing, pyre_object::w_dict_items,
dict_storage_store, and fix_ptr are used) to rebuild class_ns from the backing
dict instead of only inserting items — e.g., clear or reinitialize the storage
for *class_ns_ptr (use the existing dict storage API or a
dict_storage_clear/recreate equivalent) and then iterate
pyre_object::w_dict_items(backing) to repopulate via dict_storage_store, finally
call (*class_ns_ptr).fix_ptr().
In `@pyre/pyre-interpreter/src/display.rs`:
- Around line 228-260: builtin_subclass_dunder currently swallows both
call_function failures and wrong-type returns by returning None, hiding Python
exceptions and TypeErrors; change the helper to return a fallible result (e.g.
Result<Option<String>, PyError> or a dedicated error type) instead of
Option<String>, propagate call_function errors (do not treat non-str returns as
silent None; return an appropriate TypeError), and update the Python-facing
entry points (py_repr / py_str and any other callers that used
builtin_subclass_dunder) to propagate the error back into the Python VM instead
of falling back to builtin formatting; locate uses by the function name
builtin_subclass_dunder and the call to crate::call_function (and similar helper
calls at other sites that route int/long/float/bool/str subclasses) and adjust
their signatures and error handling accordingly so exceptions raised by
overrides are surfaced to the caller.
In `@pyre/pyre-interpreter/src/module/_collections/mod.rs`:
- Around line 43-53: maxlen_bound, rotate, index and other deque
integer-handling paths must stop unconditionally unboxing user objects with
w_int_get_value and instead coerce via crate::builtins::getindex_w; update
maxlen_bound to call getindex_w on the "__maxlen__" attribute, check the
returned isize/usize for non-negative and propagate a TypeError on failure, and
replace unsafe w_int_get_value usages in rotate and index with getindex_w
(making rotate return Result<_, PyError> to propagate errors similar to
deque_index); likewise change __init__, deque_repeat and __imul__ to accept
objects implementing __index__ by using getindex_w rather than is_int, and
enforce the non-negative bound where applicable.
In `@pyre/pyre-interpreter/src/module/operator/app_operator.py`:
- Around line 82-124: Add __reduce__ implementations to itemgetter and
methodcaller so they can be pickled when pickle support is enabled: for
itemgetter implement __reduce__ to return (self.__class__, (self._idx,)) when
self._single is True or (self.__class__, tuple(self._idx)) when multi-index, and
for methodcaller implement __reduce__ to return (self.__class__,
(self._method_name,)+tuple(self._args), self._kwargs) (or equivalent form
returning the constructor and its args/state) so the stored attributes (_idx for
itemgetter; _method_name, _args, _kwargs for methodcaller) are preserved; keep
in mind copyreg currently disables pickle but add these reducers to prepare for
future enabling.
---
Outside diff comments:
In `@pyre/pyre-interpreter/src/builtins.rs`:
- Around line 1638-1645: The code currently calls crate::call_function unchecked
for descriptors' __set_name__ in the resolved-namespace path (symbols:
w_ns_backing, pyre_object::w_dict_items, is_str, crate::baseobjspace::getattr,
crate::call_function), which will swallow Python exceptions; change this to use
the existing call_and_check helper (the pattern used at call_and_check) instead
of crate::call_function, check its return for PY_NULL, and if PY_NULL
return/propagate the error immediately so exceptions raised by __set_name__ are
not suppressed and class creation fails consistently with the other path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: de455dac-a8d5-48eb-845b-60e04d43eace
📒 Files selected for processing (23)
majit/majit-metainterp/src/jitcode/assembler.rsmajit/majit-metainterp/src/optimizeopt/schedule.rsmajit/majit-metainterp/src/optimizeopt/vector.rspyre/bench/list_reverse.pypyre/check.pypyre/pyre-interpreter/src/baseobjspace.rspyre/pyre-interpreter/src/builtins.rspyre/pyre-interpreter/src/call.rspyre/pyre-interpreter/src/display.rspyre/pyre-interpreter/src/eval.rspyre/pyre-interpreter/src/jit_fnaddr.rspyre/pyre-interpreter/src/module/_collections/mod.rspyre/pyre-interpreter/src/module/operator/app_operator.pypyre/pyre-interpreter/src/module/operator/mod.rspyre/pyre-interpreter/src/objspace/descroperation.rspyre/pyre-interpreter/src/objspace/std/mapdict.rspyre/pyre-interpreter/src/pyframe.rspyre/pyre-interpreter/src/typedef.rspyre/pyre-jit/src/jit/assembler.rspyre/pyre-jit/src/jit/codewriter.rspyre/pyre-jit/src/jit/flatten.rspyre/pyre-jit/src/jit/simplify.rspyre/pyre-macros/src/lib.rs
💤 Files with no reviewable changes (9)
- pyre/pyre-interpreter/src/pyframe.rs
- pyre/pyre-jit/src/jit/assembler.rs
- pyre/pyre-interpreter/src/objspace/std/mapdict.rs
- pyre/pyre-jit/src/jit/simplify.rs
- pyre/pyre-interpreter/src/typedef.rs
- pyre/pyre-jit/src/jit/flatten.rs
- pyre/pyre-macros/src/lib.rs
- pyre/pyre-jit/src/jit/codewriter.rs
- pyre/pyre-interpreter/src/objspace/descroperation.rs
| /// `self.maxlen`: `None` (unbounded) or a non-negative bound. The | ||
| /// bound is validated non-negative at construction, so the stored | ||
| /// value is read back directly. | ||
| fn maxlen_bound(self_obj: PyObjectRef) -> Option<usize> { | ||
| let w = crate::baseobjspace::getattr(self_obj, "__maxlen__").ok()?; | ||
| if w.is_null() || unsafe { is_none(w) } { | ||
| None | ||
| } else { | ||
| Some(unsafe { w_int_get_value(w) } as usize) | ||
| } | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the target file
FILE="pyre/pyre-interpreter/src/module/_collections/mod.rs"
test -f "$FILE" || { echo "Missing $FILE"; exit 1; }
# Show the relevant functions around the cited lines
sed -n '1,220p' "$FILE" | nl -ba | sed -n '1,220p'
echo "----"
sed -n '220,520p' "$FILE" | nl -ba | sed -n '1,300p'
echo "----"
sed -n '520,900p' "$FILE" | nl -ba | sed -n '1,380p'
# Extract specific function bodies if present (quick n^2 search)
for sym in maxlen_bound deque_repeat rotate index "__imul__" "deque_init" "__init__"; do
echo "==== searching for $sym"
rg -n "$sym" "$FILE" || true
done
echo "---- helper searches in this file"
rg -n "w_int_get_value|__index__|as_int|index.*coerc|coerc|int_get|TypeError|is_int" "$FILE" || true
# Find integer coercion helpers used elsewhere in repo (names may differ)
echo "---- searching repo for likely helpers"
rg -n "fn (.*index.*|.*as_int.*|.*int.*coerc.*|.*get.*int.*)|__index__" pyre/pyre-interpreter/src/module/_collections/mod.rs pyre/pyre-interpreter/src/module -S || true
rg -n "w_int_get_value" pyre/pyre-interpreter/src/module/_collections/mod.rs pyre/pyre-interpreter/src/module -S || true
# Also find how "__maxlen__" is set and read, if there are helpers around it
rg -n "__maxlen__" pyre/pyre-interpreter/src/module/_collections/mod.rs -S || trueRepository: youknowone/pyre
Length of output: 102
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="pyre/pyre-interpreter/src/module/_collections/mod.rs"
test -f "$FILE" || { echo "Missing $FILE"; exit 1; }
# Line-numbered print using awk (since nl isn't available)
print_range() {
local start="$1" end="$2"
awk -v s="$start" -v e="$end" '{if(NR>=s && NR<=e) printf "%6d %s\n", NR, $0}' "$FILE"
}
echo "== header: first 220 lines"
print_range 1 220
echo "----"
echo "== middle: 221-520"
print_range 221 520
echo "----"
echo "== tail: 521-900"
print_range 521 900 || true
echo "== rg symbols in file"
for sym in maxlen_bound deque_repeat rotate index "__imul__" "__init__"; do
echo "---- $sym"
rg -n "$sym" "$FILE" || true
done
echo "== rg for likely coercion helpers / integer conversion"
rg -n "__index__|TypeError|w_int_get_value|as_int|is_int|coerc" "$FILE" || true
echo "== rg for __maxlen__ usage"
rg -n "__maxlen__" "$FILE" || true
echo "== repo-wide helpers (scoped to interpreter module area)"
rg -n "__index__|TypeError|as_int|is_int|coerc|int.*get.*value|w_int_get_value" pyre/pyre-interpreter/src/module -S || trueRepository: youknowone/pyre
Length of output: 50371
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Targeted helper lookups to avoid huge output
rg -n "fn\s+w_int_get_value" pyre -S --max-count 20
rg -n "getindex_w" pyre/pyre-interpreter/src -S --max-count 50
rg -n "fn\s+getindex_w" pyre/pyre-interpreter/src -S --max-count 20
rg -n "gateway_nonnegint_w" pyre/pyre-interpreter/src -S --max-count 50
rg -n "fn\s+gateway_nonnegint_w" pyre/pyre-interpreter/src -S --max-count 20Repository: youknowone/pyre
Length of output: 1930
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show w_int_get_value implementation
sed -n '90,170p' pyre/pyre-object/src/intobject.rs | nl -ba | sed -n '1,200p' || true
echo "---- getindex_w definitions"
# Show getindex_w and nearby helpers
sed -n '2520,2585p' pyre/pyre-interpreter/src/builtins.rs | nl -ba | sed -n '1,120p' || true
echo "---- baseobjspace getindex_w docs/notes (for error behavior)"
sed -n '820,920p' pyre/pyre-interpreter/src/baseobjspace.rs | nl -ba | sed -n '1,160p' || true
echo "---- gateway_nonnegint_w (search for exact helper name in repo)"
rg -n "gateway_nonnegint_w" pyre/pyre-interpreter/src -S
# Print the implementation if located
GW_FILE="$(rg -n "gateway_nonnegint_w" pyre/pyre-interpreter/src -S | head -n 1 | cut -d: -f1 || true)"
if [ -n "${GW_FILE}" ] && [ -f "$GW_FILE" ]; then
echo "---- showing around gateway_nonnegint_w in $GW_FILE"
# crude: print 200 lines around first occurrence
LINE="$(rg -n "gateway_nonnegint_w" "$GW_FILE" -S | head -n 1 | cut -d: -f2 || true)"
start=$((LINE-60)); end=$((LINE+140))
sed -n "${start},${end}p" "$GW_FILE" | nl -ba | sed -n '1,220p' || true
fi
echo "---- int coercion helper candidates for __index__"
rg -n "getindex_w|call_function_or_identity\(.*__index__|__index__" pyre/pyre-interpreter/src -S --max-count 80Repository: youknowone/pyre
Length of output: 8158
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect w_int_get_value to assess safety on non-int objects
sed -n '110,170p' pyre/pyre-object/src/intobject.rs
echo "---- builtins::getindex_w"
sed -n '2510,2585p' pyre/pyre-interpreter/src/builtins.rs
echo "---- baseobjspace getindex_w note (error behavior)"
sed -n '820,900p' pyre/pyre-interpreter/src/baseobjspace.rs
echo "---- rotate/index relevant uses already in _collections/mod.rs"
sed -n '35,120p' pyre/pyre-interpreter/src/module/_collections/mod.rs
sed -n '228,280p' pyre/pyre-interpreter/src/module/_collections/mod.rs
sed -n '300,370p' pyre/pyre-interpreter/src/module/_collections/mod.rsRepository: youknowone/pyre
Length of output: 17472
Fix deque integer handling: stop unconditionally unboxing user objects with w_int_get_value
maxlen_bound(43-53) reads__maxlen__and then doesunsafe { w_int_get_value(w) }after onlyNonechecks; if__maxlen__is mutated to a non-int(or negativeint), this can hit UB / wrongusizecasting—validate/coerce the attribute viacrate::builtins::getindex_w(and enforce>= 0).rotate(236-249) andindex(250-265) callunsafe { w_int_get_value(...) }on user-providedn/start/stopwith nois_int/__index__coercion; route throughcrate::builtins::getindex_w(matchingdeque_index), and propagateTypeError(likely requiresrotateto returnResult<_, PyError>).__init__maxlen /deque_repeat/__imul__(105-122, 139-159, 330-356) currently require concreteintviais_int, rejecting valid__index__inputs; usegetindex_wfor parity.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyre/pyre-interpreter/src/module/_collections/mod.rs` around lines 43 - 53,
maxlen_bound, rotate, index and other deque integer-handling paths must stop
unconditionally unboxing user objects with w_int_get_value and instead coerce
via crate::builtins::getindex_w; update maxlen_bound to call getindex_w on the
"__maxlen__" attribute, check the returned isize/usize for non-negative and
propagate a TypeError on failure, and replace unsafe w_int_get_value usages in
rotate and index with getindex_w (making rotate return Result<_, PyError> to
propagate errors similar to deque_index); likewise change __init__, deque_repeat
and __imul__ to accept objects implementing __index__ by using getindex_w rather
than is_int, and enforce the non-negative bound where applicable.
76290e3 to
c28931b
Compare
…bridge (codex P2, #130) `rewrite_dead_forwarder_gotos` ran after `rewrite_trivial_link_merges`, so for a `source -> dead_forwarder -> target` shape the source block's boundary terminator still read `goto TLabel(dead_forwarder)` when the merge bridge's `strip_trailing_boundary_goto(block_label_name(target))` looked for it. The strip missed, leaving the stale terminator in front of the absorbed target opcodes and letting the single-exit renaming splice land before them. Move `rewrite_dead_forwarder_gotos` ahead of `remove_trivial_links` so the inline byte gotos already name the surviving target when the merge strip reads source terminators. The collapse guard now runs immediately before it. Assisted-by: Claude
…init errors (codex P2, coderabbit, #130) deque.__init__ now rejects a negative maxlen with ValueError and a non-integer maxlen with TypeError (gateway_nonnegint_w) at construction instead of clamping the stored value to 0 when the bound is later read, and propagates iterable errors from the extend loop rather than swallowing them via unwrap_or_default. maxlen_bound reads the validated value back without a clamp. deque.__repr__ enters a ReprGuard on self and renders `[...]` for a deque reachable from its own items, matching dequerepr / Py_ReprEnter, instead of recursing into unbounded `__repr__` calls. ReprGuard is exposed pub(crate) for the method to reach. Assisted-by: Claude
…asses (codex P2, #130) build_class stripped __class__/__classdict__/__classcell__/__classdictcell__ from the namespace before the metaclass path built its namespace dict, so `class C(metaclass=M)` where M inspects or forwards __classcell__ observed a namespace missing the cell. __class__ and __classdict__ are fast2locals cellvar mirrors that CPython never exposes as namespace keys, so they are still dropped up front. __classcell__ and __classdictcell__ are real namespace entries the class body stores: keep them in the namespace the metaclass receives, drop them in the default construction path before w_type_new, and consume them in type.__new__ (type_new_classcell) — skipping both from the new type's __dict__ and binding the captured cell to the type — so the metaclass sees them while the class __dict__ does not. Assisted-by: Claude
…oderabbit, #130) Unbound `dict.__repr__(x)` where x is not a dict (resolve_dict_backing returns null) formatted `{}` instead of rejecting the receiver. Raise TypeError like a builtin descriptor; a real dict (including an empty one) still resolves. Assisted-by: Claude
Pin the single-cell MAKE_CELL behavior: a parameter captured by an inner function (arg slot promoted to a cellvar) reads its value through one cell, and the implicit __class__ cellvar stays a single cell so zero-arg super() resolves the class. Assisted-by: Claude
#130) REPS=15 left the one-time build loop and JIT trace warmup dominating the measurement, putting the dynasm/cranelift vs-cpython ratio at the x15 gate on slower CI hardware. REPS=401 (odd, keeping the reversed result) amortises the warmup over the reverse() iterations the benchmark targets. Assisted-by: Claude
…ariance (#130) fib_loop is bignum-add bound. windows pyre runs markedly slower than macos/ubuntu for the same codegen, so cranelift exceeded the 2x vs-cpython bound there (0.41s vs 0.19s) while it stays ~0.8x cpython locally and passes at 2x on macos/ubuntu cranelift. Give cranelift 3x headroom; dynasm keeps 2x and a real regression still trips the gate. Assisted-by: Claude
There was a problem hiding this comment.
💡 Codex Review
https://github.com/youknowone/pyre/blob/c28931b141369c853b04f27eed1abadf4481bd8d/pyre-interpreter/src/typedef.rs#L4546-L4549
Validate type.mro receiver before reading its MRO
When type.mro is called unbound with bad arguments, e.g. type.mro() or type.mro(1), this varargs builtin has no arity or type guard: the empty call indexes args[0], and the non-type call passes an arbitrary object to w_type_get_mro, which casts it to W_TypeObject. That turns a normal Python TypeError path into a panic or unsafe layout read; register this with fixed arity and reject non-type receivers before calling the unsafe accessor.
https://github.com/youknowone/pyre/blob/c28931b141369c853b04f27eed1abadf4481bd8d/pyre-interpreter/src/typedef.rs#L1212-L1215
Reject non-tuple classes in tuple.new
For direct calls like tuple.__new__(list, [1]), this new subclass path treats any type object other than the exact tuple type as a tuple subclass and stamps it into w_class. That produces a tuple-storage object whose reported class can be an unrelated type (instead of raising TypeError), so later attribute dispatch can see an impossible layout; please run the same tuple-subclass validation used by the other __new__ wrappers before setting w_class.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Pin the single-cell MAKE_CELL behavior: a parameter captured by an inner function (arg slot promoted to a cellvar) reads its value through one cell, and the implicit __class__ cellvar stays a single cell so zero-arg super() resolves the class. Assisted-by: Claude
#130) REPS=15 left the one-time build loop and JIT trace warmup dominating the measurement, putting the dynasm/cranelift vs-cpython ratio at the x15 gate on slower CI hardware. REPS=401 (odd, keeping the reversed result) amortises the warmup over the reverse() iterations the benchmark targets. Assisted-by: Claude
…ariance (#130) fib_loop is bignum-add bound. windows pyre runs markedly slower than macos/ubuntu for the same codegen, so cranelift exceeded the 2x vs-cpython bound there (0.41s vs 0.19s) while it stays ~0.8x cpython locally and passes at 2x on macos/ubuntu cranelift. Give cranelift 3x headroom; dynasm keeps 2x and a real regression still trips the gate. Assisted-by: Claude
`type.mro(cls)` is the method form of the MRO (distinct from the `__mro__` tuple getset); it returns a fresh list. Its absence raised `AttributeError: type object 'X' has no attribute 'mro'` and blocked `import enum` (EnumMeta.__new__ calls `enum_class.mro()`). Assisted-by: Claude
`dict.__delitem__` called `baseobjspace::delitem(args[0], ...)`, which for a dict-subclass instance (where `is_dict` is the exact `DICT_TYPE` check, so false) takes the instance branch, re-looks-up `__delitem__`, finds the inherited `dict.__delitem__`, and re-enters — infinite recursion. Mirror `__setitem__`: delete from the `__dict_data__` backing directly for subclass instances. Assisted-by: Claude
The namespace-copy and __set_name__ loops gated on `is_dict` (exact DICT_TYPE), so a `dict` subclass namespace such as `enum._EnumDict` was skipped and the new type got an empty `__dict__`. The check is `PyDict_Check`, not `PyDict_CheckExact`; resolve the dict backing before iterating so subclass class bodies are copied. Assisted-by: Claude
`int`/`long`/`float`/`bool`/`str` keep `ob_type` at the canonical storage type and carry the Python class in `w_class`, so `py_repr`/`py_str` formatted them by storage type and ignored a subclass `__repr__`/`__str__` override (e.g. `repr(IntEnum.X)` and `str(IntEnum.X)`). Add `builtin_subclass_dunder`, which dispatches an override resolved above `object` in `w_class`'s MRO, and consult it before the storage-keyed formatting. `int`/`float`/... carry no `tp_str`, so `str()` keeps falling back to `repr()`; `str` has its own `tp_str` and returns its value. Assisted-by: Claude
`tuple.__new__(cls, iterable)` went through `descr_new_wrapper!`, which drops `cls` and returns a plain tuple, so a tuple subclass instance had `type() == tuple` and lost its field descriptors and `__repr__`. Replace it with a hand-written `tuple_descr_new` that, for a subclass, copies into a fresh tuple and sets `w_class = cls` (mirroring `int_descr_new` / `float_descr_new`). Makes `collections.namedtuple` field access, repr, _replace/_make, and defaults work. Assisted-by: Claude
The binary operators (add/sub/mul/floordiv/mod_/truediv/pow/lshift/ rshift/and_/or_/xor) returned early on their builtin storage fast paths (is_int_like / is_float_pair / is_str / ...), which also hold for a subclass instance, so a Python __add__/__or__/... override on an int/float/str subclass was ignored. Gate each operator on binop_dispatch_first before the fast path: when the left operand's w_class resolves the forward dunder, or the right operand's the reflected dunder, to a user def, route through try_dispatch_binary_special. operand_overrides keys on the resolved method kind (FUNCTION_TYPE with mutable code, excluding the fixed-code gateway builtins that back the slots) rather than a storage-type pointer, so a long — w_class = the int type object, ob_type = LONG_TYPE — is not mistaken for an override and does not recurse through its own __add__. Assisted-by: Claude
`compare` returned early on its storage fast paths (is_int_like / is_float_pair / is_str / ...), so a __lt__/__eq__/... override on an int/float/str subclass was ignored. Add try_compare_override before the fast paths: gated by operand_overrides on each side, it follows do_richcompare ordering (reflected-first when the right operand's type properly subtypes the left's and overrides the reflected comparison) but only ever invokes a genuine user override. The builtin comparison slots re-enter compare (int.__eq__ → compare), so it never dispatches one; when no user override yields a result it returns None and the fast paths run, computing the same value comparison the builtin reflected slot would. Assisted-by: Claude
load_method routes a builtin-storage instance (a builtin-leaf subclass such as `class MyInt(int)` or an enum member) through the builtin-type- method branch, because it is not is_instance-shaped. That branch bound self for every resolved attribute except static/classmethods, so `self.__class__(value)` — compiled as LOAD_METHOD/CALL_METHOD — prepended self and called the class with an extra argument (`int(self, value)` → "can't convert non-string with explicit base"). Mirror the is_instance branch: non-method descriptors (type / property / member / getset such as __class__) and an attribute absent from the type MRO (a special attribute resolved directly in getattr, or an instance- dict entry) prepend no self. Assisted-by: Claude
pos/neg/invert returned early on their int/bool/long/float fast paths, so a __pos__/__neg__/__invert__ override on a builtin-leaf subclass (e.g. enum.IntFlag's __invert__) was ignored. Gate each on try_unary_override before the fast path: when the operand's w_class resolves the dunder to a user def, call it. try_instance_unaryop only fires for is_instance-shaped objects, so the w_class-driven lookup is required here. Assisted-by: Claude
When __prepare__ returned a dict-subclass instance (e.g. enum._EnumDict), the class body ran against a plain DictStorage and its stores were only replayed into the mapping after execute_frame. So a name read back during the body (`WHITE = RED | GREEN | BLUE` in a Flag) saw the value originally assigned, not the one the mapping's __setitem__ resolved — the auto() members read as the unresolved _auto_null object and `|` raised "unsupported operand type(s) for |: 'object' and 'object'". Route the frame's name binding through the mapping via setdictscope_object when __prepare__ returned a dict-subclass instance with a resolvable backing, then mirror its final contents into class_ns for the downstream type construction (classcell capture, create_all_slots, __set_name__). Skip the metaclass-path replay in that case: the mapping already holds every store and re-running __setitem__ would reject the duplicate member keys. Plain-dict and absent namespaces keep the DictStorage fast path. Assisted-by: Claude
getitem_type, len, and contains ignored a special method resolved on the
receiver's metaclass (when the receiver is a class) or on a builtin-leaf
subclass's w_class (when the receiver's ob_type is a storage type):
- `Color['RED']` fell straight to __class_getitem__ and returned the
class instead of consulting EnumMeta.__getitem__;
- `len(Color)` raised "object of type 'type' has no len()" without
consulting EnumMeta.__len__;
- `x in Color` / `x in flag` ran the getitem scan without consulting
EnumMeta.__contains__ or the IntFlag instance's __contains__.
getitem_type now resolves __getitem__ on type(cls)'s MRO before the PEP
560 __class_getitem__ fallback; len consults the metaclass __len__ for a
type receiver; contains resolves __contains__ on the receiver's dynamic
type before the getitem scan (covering both the metaclass and the
builtin-leaf-subclass cases). type/int/etc. define none of these, so
ordinary classes and builtins keep their existing paths.
Assisted-by: Claude
…case
deque __getitem__/__setitem__/__delitem__ derefed the index as a raw
W_IntObject (`w_int_get_value`), reading garbage memory for a slice or
any non-int index and accepting no __index__ object. __getitem__ also
delegated to list getitem (returning a list for a slice) and returned
None for an empty deque.
Add a `deque_index` helper mirroring `space.decode_index4`'s step==0
branch: a slice raises TypeError("deque[:] is not supported"), other
indices go through `getindex_w` (__index__), then the negative-index
wrap and the IndexError("index out of range") range check. Route all
three subscript methods through it.
Assisted-by: Claude
countOf was absent from the operator module port. Add the verbatim app_operator.py function and list it in the appleveldefs name set, matching moduledef.py `app_names`. Assisted-by: Claude
The wrapper bound args purely positionally, so a call carrying keywords (delivered as a trailing `__pyre_kw__` dict) bound that dict to the next positional parameter instead of resolving each keyword by name — `deque(maxlen=3)` bound the dict as `iterable`, leaving `maxlen` None. Collect the parameter-name and required tables at expansion time and, when the call carried a `__pyre_kw__` dict, rebind positional+keyword args into a resolved scope through `bind_builtin_kwargs` (the gateway `_match_signature`): positionals fill left-to-right, keywords fill by matching parameter name, an absent optional becomes `PY_NULL`, and an unknown keyword / duplicate / missing required raises TypeError. The positional fast path is unchanged when no kwargs dict is present; the optional-arg presence check now also treats the `PY_NULL` slot as omitted. Varargs (`&[PyObjectRef]`) fns keep the positional path since the whole-slice binding cannot express a resolved scope. Assisted-by: Claude
The list-backed deque had no value comparison (== fell back to identity) or `*` repeat. Add __eq__/__ne__/__lt__/__le__/__gt__/__ge__ delegating to element-wise list comparison over both backings (W_Deque.compare / compare_by_iteration, maxlen ignored, NotImplemented for a non-deque operand), and __mul__/__rmul__/__imul__ that repeat the elements and re-bound the result by maxlen through the constructor (W_Deque.mul/imul). Assisted-by: Claude
insn_needs_trailing_live emitted a trailing -live- after every residual_call_* unconditionally. jtransform.py:469 handle_residual_call appends the marker only when may_call_jitcodes or calldescr_canraise; the canonical lowering has no may_call_jitcodes site, so read the CallDescrStub effect_info off the Insn and gate on check_can_raise(false). inline_call_* stays unconditional per handle_regular_call. This drops the marker after the EF_CANNOT_RAISE get_current_exception residual call. Assisted-by: Claude
rotate and index decoded their count/start/stop with the unchecked w_int_get_value, yielding a bogus result (or reading a non-int object layout) for non-integers. Route them through getindex_w so a non-index argument raises TypeError; rotate now returns Result. Assisted-by: Claude
itemgetter returns (type, (idx,)) for a single index and (type, tuple) for multiple; methodcaller returns (type, (name,) + args) without kwargs and (partial(type, name, **kwargs), args) with kwargs. Assisted-by: Claude
Exercises the Devolved DICT branches of terminator_read and write_terminator directly by rooting a MockObj at the paired devolved terminator (the production devolve transition is not yet ported), proving both route through _obj_getdict rather than map storage. Assisted-by: Claude
The three __contains__ call sites in contains() used call_function, which turns a raise into PY_NULL; is_true(PY_NULL) then reported a raising membership test as a successful true. Route them through call_and_check so the exception propagates. Assisted-by: Claude
- Validate __classcell__ is a cell during the metaclass namespace copy, raising TypeError instead of silently skipping the bind. - Rebuild class_ns from the final mapping backing (clear first) so a name deleted from a custom __prepare__ mapping does not survive in the class dict. - Strip __class__/__classdict__ from the prepared mapping the metaclass observes (fast2locals may sync the cellvars in). - Route str() through the str-subclass __str__ override; builtin_str short-circuited on STR_TYPE before the dispatch py_str already uses. - Propagate exceptions from __set_name__ (both class-creation paths used unchecked call_function). Assisted-by: Claude
Reverts the cranelift fib_loop vs-cpython bound from 3x back to 2x to match origin/main. Assisted-by: Claude
c28931b to
25f9a70
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 25f9a70cc3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if !unsafe { is_int(n) } { | ||
| return Ok(pyre_object::w_not_implemented()); | ||
| } |
There was a problem hiding this comment.
Accept index objects for deque repetition
When the repeat count is an object that implements __index__ but is not an int (for example class N: __index__ = lambda self: 2), deque([1]) * N() should repeat successfully, but this gate returns NotImplemented and the operation falls through to a TypeError. Other deque methods in this file already use getindex_w for index-like arguments, so the repeat count should be decoded the same way rather than requiring is_int; the same issue also affects the in-place repeat path.
Useful? React with 👍 / 👎.
| dunder: &str, | ||
| rdunder: &str, | ||
| ) -> bool { | ||
| operand_overrides(a, dunder) || operand_overrides(b, rdunder) |
There was a problem hiding this comment.
Avoid reflected dispatch for identical operand types
For two operands of the same builtin-leaf subclass, this predicate now sends the operation through try_dispatch_binary_special; if the forward method returns NotImplemented, that helper tries the right operand's reflected method even though Python/PyPy do not try __radd__/__rmul__ for identical operand types. A subclass such as class I(int): def __add__(...): return NotImplemented; def __radd__(...): return 5 will make I(1) + I(2) return 5 here instead of raising the unsupported-operand TypeError.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (11)
pyre/pyre-interpreter/src/module/_collections/mod.rs (3)
105-109: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
deque_repeatrejects__index__-able objects.CPython's deque accepts objects implementing
__index__for the repeat count. The currentis_intcheck is overly restrictive. Consider:fn deque_repeat(self_obj: PyObjectRef, n: PyObjectRef) -> Result<PyObjectRef, crate::PyError> { - if !unsafe { is_int(n) } { - return Ok(pyre_object::w_not_implemented()); - } - let num = unsafe { w_int_get_value(n) }.max(0); + let num = match crate::builtins::getindex_w(n) { + Ok(v) => v.max(0), + Err(_) => return Ok(pyre_object::w_not_implemented()), + };This preserves
NotImplementedfor non-integer-like operands while accepting__index__-able types.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/module/_collections/mod.rs` around lines 105 - 109, The deque_repeat implementation currently uses is_int(n) and thus rejects objects implementing __index__; change it to accept indexable objects by first attempting to obtain an integer via the Python number protocol (e.g. call the equivalent of PyNumber_Index on n), returning NotImplemented if that attempt yields NotImplemented or an error, then extract the integer value from the resulting int object using w_int_get_value (as currently done), clamp to >=0 (num.max(0)), and proceed; update references in deque_repeat to use that index-conversion path instead of the is_int check so __index__-able types are accepted while non-integer-like operands still produce NotImplemented.
43-53:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
maxlen_boundassumes__maxlen__was validated but the attribute could be mutated.If user code reassigns
self.__maxlen__to a non-int (e.g.,d.__maxlen__ = "bad"), the directw_int_get_valuecall on line 51 can produce undefined behavior or garbage. While__init__validates the initial value, Python semantics allow attribute mutation afterward.Consider either:
- Re-validating
is_int(w)before unboxing, or- Using
getindex_wto coerce through__index__(matching CPython's behavior wheremaxlenaccepts__index__-able objects).🛡️ Proposed defensive fix
fn maxlen_bound(self_obj: PyObjectRef) -> Option<usize> { let w = crate::baseobjspace::getattr(self_obj, "__maxlen__").ok()?; if w.is_null() || unsafe { is_none(w) } { None + } else if !unsafe { is_int(w) } { + // Corrupted / mutated attribute — treat as unbounded rather than UB. + None } else { Some(unsafe { w_int_get_value(w) } as usize) } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/module/_collections/mod.rs` around lines 43 - 53, maxlen_bound currently unboxes __maxlen__ with w_int_get_value trusting init-time validation; instead defend against attribute mutation by first checking is_null/is_none, then if is_int use w_int_get_value as before, otherwise call the existing getindex_w helper to coerce via __index__ (matching CPython) and convert that result to usize; ensure any conversion or getindex_w failures are handled the same way the function does today (returning None) and reference the symbols maxlen_bound, __maxlen__, is_none, is_int, w_int_get_value, and getindex_w when making the change.
340-346:⚠️ Potential issue | 🟡 Minor | ⚡ Quick win
__imul__returnsNotImplementedfor non-int, but should raiseTypeError.In-place multiplication (
d *= n) should raiseTypeErrorifnis not an integer-like object, rather than returningNotImplemented. ReturningNotImplementedfrom__imul__falls back to__mul__, which can produce unexpected behavior (creating a new deque instead of mutating in place).Additionally, CPython's deque accepts
__index__-able objects for the repeat count.🔧 Proposed fix
fn __imul__(self_obj: PyObjectRef, n: PyObjectRef) -> Result<PyObjectRef, crate::PyError> { - if !unsafe { is_int(n) } { - return Ok(pyre_object::w_not_implemented()); - } - let num = unsafe { w_int_get_value(n) }; + let num = crate::builtins::getindex_w(n)?; let base = snapshot(self_obj);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/module/_collections/mod.rs` around lines 340 - 346, The __imul__ implementation for W_Deque should not return NotImplemented for non-int operands; instead accept objects supporting __index__ and raise TypeError on failure. In the W_Deque::__imul__ function, replace the unsafe is_int/w_int_get_value checks with a call to the interpreter’s index conversion (use the existing PyNumber_Index/try_index helper or equivalent) to obtain an integer index; if that conversion errors or returns non-integer, propagate/raise a TypeError with an explanatory message; then use the resulting integer (handling overflow/negative semantics as before) to perform the in-place repeat logic. Ensure you reference the W_Deque::__imul__ (W_Deque.imul) function and use the existing index conversion helper rather than returning pyre_object::w_not_implemented().pyre/pyre-jit/src/jit/flatten.rs (1)
1371-1374:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDon't gate post-call
-live-emission onlowering_ctx.
flatten_space_operation()still passes through already-loweredresidual_call_*/inline_call_*ops whenlowering_ctxis unset, so this branch still drops the upstream-required post-call marker on those canonical streams. That loses the fallthrough resume point for can-raise residual calls and inline calls.Suggested fix
- let trailing_live = self.lowering_ctx.is_some() && insn_needs_trailing_live(&insn); + let trailing_live = insn_needs_trailing_live(&insn);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-jit/src/jit/flatten.rs` around lines 1371 - 1374, The code currently only emits the post-call Insn::live when self.lowering_ctx.is_some(); change this so post-call live markers are emitted whenever insn_needs_trailing_live(&insn) is true regardless of lowering_ctx. Concretely, update the trailing_live computation in the block that defines trailing_live (remove the is_some() check) and keep the subsequent self.emitline(Insn::live(Vec::new())) branch so residual_call_* and inline_call_* streams handled by flatten_space_operation() still get their trailing live markers; keep using insn_needs_trailing_live, emitline, and Insn::live as the referenced symbols.pyre/pyre-interpreter/src/builtins.rs (2)
1646-1653:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winSkip hidden class-cell scaffolding during
__set_name__dispatch.This still walks the original namespace backing after
__classcell__/__classdictcell__were intentionally stripped from the type dict, so a descriptor stored under either hidden key gets__set_name__for an attribute that never exists on the class.Suggested fix
if !w_ns_backing.is_null() { let entries = unsafe { pyre_object::w_dict_items(w_ns_backing) }; for (k, v) in entries { if unsafe { is_str(k) } { + let key = unsafe { pyre_object::w_str_get_value(k) }; + if key == "__classcell__" || key == "__classdictcell__" { + continue; + } if let Ok(set_name) = crate::baseobjspace::getattr(v, "__set_name__") { // getattr returns a bound method, so self is already bound. // Call: bound_set_name(owner, name); propagate a raise. call_and_check(set_name, &[w_type, k])?; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/builtins.rs` around lines 1646 - 1653, The loop over entries from w_ns_backing calls __set_name__ on any descriptor whose key is a string, but it must skip the hidden class-cell keys so descriptors under "__classcell__" or "__classdictcell__" (if present) do not get __set_name__ for attributes that were intentionally removed; update the loop in the block that iterates entries from pyre_object::w_dict_items(w_ns_backing) to check the key string (k) after confirming is_str(k) and continue/skip when k equals "__classcell__" or "__classdictcell__" before attempting crate::baseobjspace::getattr(v, "__set_name__") and call_and_check.
2341-2348:⚠️ Potential issue | 🟠 Major | ⚡ Quick winReturn an exact
strfrom the defaultstr-subclass path.With the leaf-subclass storage model,
is_str(obj)also matchesstrsubclasses. Falling through toOk(obj)letsstr(MyStr("x"))return the subclass instance instead of a basestrwhenever there is no user__str__override.Suggested fix
if is_str(obj) { // A `str` subclass keeps `ob_type` at STR_TYPE but carries the // Python class in `w_class`; honor its `__str__` override before // returning the raw value. let tp = (*obj).ob_type; if let Some(s) = crate::display::builtin_subclass_dunder(obj, tp, "__str__") { return Ok(w_str_new(&s)); } - return Ok(obj); + let w_builtin_str = crate::typedef::gettypeobject(&STR_TYPE); + if crate::typedef::r#type(obj) != Some(w_builtin_str) { + return Ok(w_str_new(w_str_get_value(obj))); + } + return Ok(obj); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/builtins.rs` around lines 2341 - 2348, The current path returns the subclass instance (obj) when a str-subclass has no __str__ override; instead extract the raw base string value from obj and return a fresh base str object (e.g. call w_str_new with the extracted string) rather than Ok(obj). In other words, in the branch after checking crate::display::builtin_subclass_dunder(obj, tp, "__str__"), convert the underlying str contents of obj into a Rust string and return Ok(w_str_new(&value)) so callers get an exact base str; keep the checks involving is_str(obj), tp and the "__str__" lookup unchanged.pyre/pyre-interpreter/src/eval.rs (1)
2859-2867:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMirror the builtin-function exclusion in this binding path.
Line 2866 still binds any remaining descriptor as a method. For
class MyInt(int): f = len,lookup_in_typefindslen,load_methodpushes(len, obj), andCALLturnsc.f([1, 2, 3])intolen(obj, [1, 2, 3]). This branch needs the same builtin-function classification theis_instancepath already uses before falling through toSome(_) => obj.Suggested fix
match crate::baseobjspace::lookup_in_type(w_type, name) { Some(d) if pyre_object::is_staticmethod(d) => PY_NULL, Some(d) if pyre_object::is_classmethod(d) => w_type, Some(d) if pyre_object::is_type(d) => PY_NULL, Some(d) if pyre_object::is_property(d) => PY_NULL, Some(d) if pyre_object::is_member(d) => PY_NULL, Some(d) if pyre_object::getsetproperty::is_getset_property(d) => PY_NULL, + Some(d) if crate::is_function(d) => { + let ob_type = (*d).ob_type; + if std::ptr::eq(ob_type, &crate::BUILTIN_FUNCTION_TYPE as *const _) { + PY_NULL + } else { + obj + } + } Some(_) => obj, None => PY_NULL, }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/eval.rs` around lines 2859 - 2867, The match arm in eval.rs that handles lookup_in_type currently falls through to Some(_) => obj and treats remaining descriptors as methods; mirror the instance-path builtin-function exclusion by checking for builtin functions here (e.g., call pyre_object::is_builtin_function(d)) before the Some(_) => obj branch so builtin functions are not bound as methods; update the match in the same block used by load_method/CALL to return PY_NULL (or obj as appropriate) when the descriptor is a builtin-function, using the same classification function the is_instance path uses.pyre/pyre-interpreter/src/display.rs (1)
228-260:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftPropagate failing builtin-leaf
__repr__/__str__overrides.Line 256 still maps both
call_functionfailure and a non-strreturn toNone, so the new leaf-subclass path at Lines 274-277 / 582-585 / 672-675 silently falls back to builtin formatting instead of surfacing the Python exception or the usualTypeError.Also applies to: 274-277, 582-585, 672-675
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/display.rs` around lines 228 - 260, The builtin_subclass_dunder path currently swallows two failure modes (call_function returning null and call_function returning a non-str) and returns None, causing builtin formatting to run instead of surfacing the Python exception or raising the usual TypeError; update the logic around crate::call_function(found, &[obj]) in builtin_subclass_dunder (and the analogous sites at the other occurrences) so that if call_function returns null you propagate/return the Python exception (do not convert to None), and if the call returns a non-str you raise/propagate a TypeError explaining that __repr__/__str__ must return str rather than silently falling back to builtin formatting. Ensure you use the existing pyre_object/exception helpers to fetch and return or re-raise the exception rather than returning None.pyre/pyre-interpreter/src/objspace/descroperation.rs (1)
1015-1019:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDon't re-enter storage fast paths after user overrides declined the op.
When
binop_dispatch_first(...)is true, aNonefromtry_dispatch_binary_special(...)means an override existed and returnedNotImplemented. Falling through to the int/long/float/sequence fast paths here manufactures builtin results, and on non-fast-path pairs the later fallback re-invokes the same special methods a second time.Also applies to: 1085-1089, 1138-1142, 1243-1249, 1277-1281, 1321-1326, 1362-1366, 1729-1733, 1760-1764, 1791-1795, 1879-1883, 1952-1956
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/objspace/descroperation.rs` around lines 1015 - 1019, If binop_dispatch_first(...) is true and try_dispatch_binary_special(...) returns None (meaning a user override existed and returned NotImplemented), short-circuit and do not fall through into builtin fast paths—propagate/return the NotImplemented result immediately instead of continuing to int/long/float/sequence fast-path logic; update the blocks around binop_dispatch_first + try_dispatch_binary_special (e.g., the __add__/__radd__ case and the other listed locations) to treat None as the override declining the operation and return that NotImplemented outcome to the caller.pyre/pyre-interpreter/src/typedef.rs (2)
1192-1215:⚠️ Potential issue | 🟠 MajorValidate
clsbefore tagging tuple subclasses.This still skips
check_user_subclass, sotuple.__new__(dict, ...)can manufacture a tuple-layout object whosew_classclaims to bedict(or any other type) instead of raisingTypeError.🛠️ Minimal fix
fn tuple_descr_new(args: &[PyObjectRef]) -> Result<PyObjectRef, crate::PyError> { let cls = if args.is_empty() { pyre_object::PY_NULL } else { args[0] }; + if !cls.is_null() && unsafe { pyre_object::is_type(cls) } { + if let Some(w_tuple) = gettypefor(&pyre_object::pyobject::TUPLE_TYPE) { + check_user_subclass(w_tuple, cls)?; + } + } let value = crate::builtins::builtin_tuple(&args[1..])?; if cls.is_null() || !unsafe { pyre_object::is_type(cls) } { return Ok(value); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/typedef.rs` around lines 1192 - 1215, In tuple_descr_new, validate that cls is actually a user subclass of tuple before copying and tagging the new tuple: after determining tuple_typeobj and before the "Subclass path" block, call the existing check_user_subclass (or equivalent routine used elsewhere) with cls and the tuple type object and if it returns false return a TypeError instead of proceeding; this prevents arbitrary types (e.g. dict) from being assigned to w_class when creating the new tuple in tuple_descr_new.
4543-4556:⚠️ Potential issue | 🟠 MajorGuard
type.mrobefore the unsafe MRO read.
args[0]is indexed without an arity check, andw_type_get_mro(cls)is called for any receiver.type.mro()can panic, andtype.mro(non_type)can walk non-type memory instead of raisingTypeError.🛠️ Minimal fix
- let mro_method = make_builtin_function("mro", |args| { + let mro_method = make_builtin_function_with_arity("mro", |args| { let cls = args[0]; + if cls.is_null() || !unsafe { pyre_object::is_type(cls) } { + let tp_name = if cls.is_null() { + "NoneType".to_string() + } else { + unsafe { (*(*cls).ob_type).name.to_string() } + }; + return Err(crate::PyError::type_error(format!( + "descriptor 'mro' for 'type' objects doesn't apply to a '{tp_name}' object" + ))); + } unsafe { let mro_ptr = pyre_object::w_type_get_mro(cls); if mro_ptr.is_null() { return Ok(pyre_object::w_list_new(vec![])); } Ok(pyre_object::w_list_new((*mro_ptr).clone())) } - }); + }, 1);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/typedef.rs` around lines 4543 - 4556, The mro builtin reads args[0] unsafely and calls pyre_object::w_type_get_mro without verifying arity or that the receiver is actually a type, which can panic or read invalid memory; update the closure passed to make_builtin_function("mro", ...) to first check args.len() > 0 and validate the receiver is a type using the existing type-check helper (e.g., the project's is_type/w_type_check helper or equivalent) before calling pyre_object::w_type_get_mro, and if the check fails return a TypeError (using the project's error/exception construction path) instead of doing the unsafe read; keep the subsequent logic that builds the list with pyre_object::w_list_new and stores it with dict_storage_store unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pyre/pyre-interpreter/src/objspace/descroperation.rs`:
- Around line 2016-2049: The compare override flow currently treats a method
that returns the NotImplemented singleton as "no override" and falls through to
numeric fast paths; change the logic so an actual method call that returns
NotImplemented is preserved and propagated as an explicit result. Concretely, in
try_compare_override (and the analogous blocks around lookup_type_special /
try_call_special), ensure that try_call_special does not map a NotImplemented
return into None — either update try_call_special to return Some(NotImplemented)
when the callee returned NotImplemented, or modify try_compare_override to treat
a returned PyObjectRef equal to the NotImplemented singleton as Some(result) and
immediately return Ok(Some(result)) (using operand_overrides,
lookup_type_special, should_try_reverse_first to locate the same call sites).
This preserves the intended fallback semantics where an explicit NotImplemented
from an override prevents falling into numeric fast paths.
In `@pyre/pyre-interpreter/src/typedef.rs`:
- Around line 2356-2369: The current deletion code silently no-ops when args[0]
is neither a dict nor a dict-subclass with a valid "__dict_data__" backing,
which masks invalid receiver errors; update the branch handling in the block
that uses pyre_object::is_dict and pyre_object::is_instance so that if args[0]
is not a plain dict and either is not an instance or getattr(args[0],
"__dict_data__") is missing/non-dict, you return a TypeError instead of silently
returning None; use the same error-construction helper/pattern used elsewhere in
this module to raise a TypeError with a clear message (e.g. "descriptor delete
for non-dict or missing __dict_data__") and keep the existing behavior of
calling crate::baseobjspace::delitem when a dict or valid backing dict is
present.
In `@pyre/pyre-jit/src/jit/assembler.rs`:
- Around line 1646-1655: The panic in expect_reg already includes
current_dispatch_op(), but other helper functions (e.g., expect_tlabel,
expect_small_u16, expect_descr_vable_array_field, expect_descr_vable_array,
expect_descr_vable_static_field, etc.) do not; update each expect_* helper to
include the current opcode context by appending current_dispatch_op() (or a
similar descriptive call used in expect_reg) to their panic messages so all
operand-shape mismatch errors report the dispatch opcode for consistent
diagnostics.
---
Duplicate comments:
In `@pyre/pyre-interpreter/src/builtins.rs`:
- Around line 1646-1653: The loop over entries from w_ns_backing calls
__set_name__ on any descriptor whose key is a string, but it must skip the
hidden class-cell keys so descriptors under "__classcell__" or
"__classdictcell__" (if present) do not get __set_name__ for attributes that
were intentionally removed; update the loop in the block that iterates entries
from pyre_object::w_dict_items(w_ns_backing) to check the key string (k) after
confirming is_str(k) and continue/skip when k equals "__classcell__" or
"__classdictcell__" before attempting crate::baseobjspace::getattr(v,
"__set_name__") and call_and_check.
- Around line 2341-2348: The current path returns the subclass instance (obj)
when a str-subclass has no __str__ override; instead extract the raw base string
value from obj and return a fresh base str object (e.g. call w_str_new with the
extracted string) rather than Ok(obj). In other words, in the branch after
checking crate::display::builtin_subclass_dunder(obj, tp, "__str__"), convert
the underlying str contents of obj into a Rust string and return
Ok(w_str_new(&value)) so callers get an exact base str; keep the checks
involving is_str(obj), tp and the "__str__" lookup unchanged.
In `@pyre/pyre-interpreter/src/display.rs`:
- Around line 228-260: The builtin_subclass_dunder path currently swallows two
failure modes (call_function returning null and call_function returning a
non-str) and returns None, causing builtin formatting to run instead of
surfacing the Python exception or raising the usual TypeError; update the logic
around crate::call_function(found, &[obj]) in builtin_subclass_dunder (and the
analogous sites at the other occurrences) so that if call_function returns null
you propagate/return the Python exception (do not convert to None), and if the
call returns a non-str you raise/propagate a TypeError explaining that
__repr__/__str__ must return str rather than silently falling back to builtin
formatting. Ensure you use the existing pyre_object/exception helpers to fetch
and return or re-raise the exception rather than returning None.
In `@pyre/pyre-interpreter/src/eval.rs`:
- Around line 2859-2867: The match arm in eval.rs that handles lookup_in_type
currently falls through to Some(_) => obj and treats remaining descriptors as
methods; mirror the instance-path builtin-function exclusion by checking for
builtin functions here (e.g., call pyre_object::is_builtin_function(d)) before
the Some(_) => obj branch so builtin functions are not bound as methods; update
the match in the same block used by load_method/CALL to return PY_NULL (or obj
as appropriate) when the descriptor is a builtin-function, using the same
classification function the is_instance path uses.
In `@pyre/pyre-interpreter/src/module/_collections/mod.rs`:
- Around line 105-109: The deque_repeat implementation currently uses is_int(n)
and thus rejects objects implementing __index__; change it to accept indexable
objects by first attempting to obtain an integer via the Python number protocol
(e.g. call the equivalent of PyNumber_Index on n), returning NotImplemented if
that attempt yields NotImplemented or an error, then extract the integer value
from the resulting int object using w_int_get_value (as currently done), clamp
to >=0 (num.max(0)), and proceed; update references in deque_repeat to use that
index-conversion path instead of the is_int check so __index__-able types are
accepted while non-integer-like operands still produce NotImplemented.
- Around line 43-53: maxlen_bound currently unboxes __maxlen__ with
w_int_get_value trusting init-time validation; instead defend against attribute
mutation by first checking is_null/is_none, then if is_int use w_int_get_value
as before, otherwise call the existing getindex_w helper to coerce via __index__
(matching CPython) and convert that result to usize; ensure any conversion or
getindex_w failures are handled the same way the function does today (returning
None) and reference the symbols maxlen_bound, __maxlen__, is_none, is_int,
w_int_get_value, and getindex_w when making the change.
- Around line 340-346: The __imul__ implementation for W_Deque should not return
NotImplemented for non-int operands; instead accept objects supporting __index__
and raise TypeError on failure. In the W_Deque::__imul__ function, replace the
unsafe is_int/w_int_get_value checks with a call to the interpreter’s index
conversion (use the existing PyNumber_Index/try_index helper or equivalent) to
obtain an integer index; if that conversion errors or returns non-integer,
propagate/raise a TypeError with an explanatory message; then use the resulting
integer (handling overflow/negative semantics as before) to perform the in-place
repeat logic. Ensure you reference the W_Deque::__imul__ (W_Deque.imul) function
and use the existing index conversion helper rather than returning
pyre_object::w_not_implemented().
In `@pyre/pyre-interpreter/src/objspace/descroperation.rs`:
- Around line 1015-1019: If binop_dispatch_first(...) is true and
try_dispatch_binary_special(...) returns None (meaning a user override existed
and returned NotImplemented), short-circuit and do not fall through into builtin
fast paths—propagate/return the NotImplemented result immediately instead of
continuing to int/long/float/sequence fast-path logic; update the blocks around
binop_dispatch_first + try_dispatch_binary_special (e.g., the __add__/__radd__
case and the other listed locations) to treat None as the override declining the
operation and return that NotImplemented outcome to the caller.
In `@pyre/pyre-interpreter/src/typedef.rs`:
- Around line 1192-1215: In tuple_descr_new, validate that cls is actually a
user subclass of tuple before copying and tagging the new tuple: after
determining tuple_typeobj and before the "Subclass path" block, call the
existing check_user_subclass (or equivalent routine used elsewhere) with cls and
the tuple type object and if it returns false return a TypeError instead of
proceeding; this prevents arbitrary types (e.g. dict) from being assigned to
w_class when creating the new tuple in tuple_descr_new.
- Around line 4543-4556: The mro builtin reads args[0] unsafely and calls
pyre_object::w_type_get_mro without verifying arity or that the receiver is
actually a type, which can panic or read invalid memory; update the closure
passed to make_builtin_function("mro", ...) to first check args.len() > 0 and
validate the receiver is a type using the existing type-check helper (e.g., the
project's is_type/w_type_check helper or equivalent) before calling
pyre_object::w_type_get_mro, and if the check fails return a TypeError (using
the project's error/exception construction path) instead of doing the unsafe
read; keep the subsequent logic that builds the list with
pyre_object::w_list_new and stores it with dict_storage_store unchanged.
In `@pyre/pyre-jit/src/jit/flatten.rs`:
- Around line 1371-1374: The code currently only emits the post-call Insn::live
when self.lowering_ctx.is_some(); change this so post-call live markers are
emitted whenever insn_needs_trailing_live(&insn) is true regardless of
lowering_ctx. Concretely, update the trailing_live computation in the block that
defines trailing_live (remove the is_some() check) and keep the subsequent
self.emitline(Insn::live(Vec::new())) branch so residual_call_* and
inline_call_* streams handled by flatten_space_operation() still get their
trailing live markers; keep using insn_needs_trailing_live, emitline, and
Insn::live as the referenced symbols.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 655ea195-2a16-4220-846b-5c7ad31b63d9
📒 Files selected for processing (21)
majit/majit-metainterp/src/jitcode/assembler.rsmajit/majit-metainterp/src/optimizeopt/schedule.rspyre/bench/list_reverse.pypyre/pyre-interpreter/src/baseobjspace.rspyre/pyre-interpreter/src/builtins.rspyre/pyre-interpreter/src/call.rspyre/pyre-interpreter/src/display.rspyre/pyre-interpreter/src/eval.rspyre/pyre-interpreter/src/jit_fnaddr.rspyre/pyre-interpreter/src/module/_collections/mod.rspyre/pyre-interpreter/src/module/operator/app_operator.pypyre/pyre-interpreter/src/module/operator/mod.rspyre/pyre-interpreter/src/objspace/descroperation.rspyre/pyre-interpreter/src/objspace/std/mapdict.rspyre/pyre-interpreter/src/pyframe.rspyre/pyre-interpreter/src/typedef.rspyre/pyre-jit/src/jit/assembler.rspyre/pyre-jit/src/jit/codewriter.rspyre/pyre-jit/src/jit/flatten.rspyre/pyre-jit/src/jit/simplify.rspyre/pyre-macros/src/lib.rs
| unsafe fn try_compare_override( | ||
| a: PyObjectRef, | ||
| b: PyObjectRef, | ||
| dunder: &str, | ||
| rdunder: &str, | ||
| ) -> Result<Option<PyObjectRef>, PyError> { | ||
| let a_over = operand_overrides(a, dunder); | ||
| let b_over = operand_overrides(b, rdunder); | ||
| if !a_over && !b_over { | ||
| return Ok(None); | ||
| } | ||
| let reverse_first = b_over && should_try_reverse_first(a, b, rdunder); | ||
| if reverse_first { | ||
| if let Some(method) = lookup_type_special(b, rdunder) { | ||
| if let Some(result) = try_call_special(method, &[b, a])? { | ||
| return Ok(Some(result)); | ||
| } | ||
| } | ||
| } | ||
| if a_over { | ||
| if let Some(method) = lookup_type_special(a, dunder) { | ||
| if let Some(result) = try_call_special(method, &[a, b])? { | ||
| return Ok(Some(result)); | ||
| } | ||
| } | ||
| } | ||
| if b_over && !reverse_first { | ||
| if let Some(method) = lookup_type_special(b, rdunder) { | ||
| if let Some(result) = try_call_special(method, &[b, a])? { | ||
| return Ok(Some(result)); | ||
| } | ||
| } | ||
| } | ||
| Ok(None) |
There was a problem hiding this comment.
Preserve rich-compare fallback semantics after overridden methods return NotImplemented.
try_compare_override() already proved that at least one user-level comparison override exists. If that dispatch returns None, continuing into the numeric fast paths turns an explicit decline into plain value comparison; a builtin-leaf subclass whose __lt__ / __gt__ both return NotImplemented will currently compare like a bare int instead of falling through to the generic comparison path.
Also applies to: 2058-2069, 2070-2100
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyre/pyre-interpreter/src/objspace/descroperation.rs` around lines 2016 -
2049, The compare override flow currently treats a method that returns the
NotImplemented singleton as "no override" and falls through to numeric fast
paths; change the logic so an actual method call that returns NotImplemented is
preserved and propagated as an explicit result. Concretely, in
try_compare_override (and the analogous blocks around lookup_type_special /
try_call_special), ensure that try_call_special does not map a NotImplemented
return into None — either update try_call_special to return Some(NotImplemented)
when the callee returned NotImplemented, or modify try_compare_override to treat
a returned PyObjectRef equal to the NotImplemented singleton as Some(result) and
immediately return Ok(Some(result)) (using operand_overrides,
lookup_type_special, should_try_reverse_first to locate the same call sites).
This preserves the intended fallback semantics where an explicit NotImplemented
from an override prevents falling into numeric fast paths.
| // For plain dict: direct delete. For dict subclass instance: use backing dict. | ||
| unsafe { | ||
| if pyre_object::is_dict(args[0]) { | ||
| crate::baseobjspace::delitem(args[0], args[1])?; | ||
| } else if pyre_object::is_instance(args[0]) { | ||
| // dict subclass — delete from __dict_data__ backing dict | ||
| if let Ok(backing) = crate::baseobjspace::getattr(args[0], "__dict_data__") | ||
| { | ||
| if pyre_object::is_dict(backing) { | ||
| crate::baseobjspace::delitem(backing, args[1])?; | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Don't turn invalid dict deletions into no-ops.
If args[0] is a non-dict instance, or a dict subclass whose __dict_data__ is missing/non-dict, this now returns None without deleting anything or raising. That masks bad receivers and silently drops real deletions.
🛠️ Minimal fix
|args| {
if args.len() < 2 {
return Err(crate::PyError::type_error("__delitem__ requires 2 args"));
}
- // For plain dict: direct delete. For dict subclass instance: use backing dict.
- unsafe {
- if pyre_object::is_dict(args[0]) {
- crate::baseobjspace::delitem(args[0], args[1])?;
- } else if pyre_object::is_instance(args[0]) {
- // dict subclass — delete from __dict_data__ backing dict
- if let Ok(backing) = crate::baseobjspace::getattr(args[0], "__dict_data__")
- {
- if pyre_object::is_dict(backing) {
- crate::baseobjspace::delitem(backing, args[1])?;
- }
- }
- }
+ let recv = args[0];
+ let dict = crate::type_methods::resolve_dict_backing(recv);
+ if dict.is_null() {
+ let tp_name = unsafe { (*(*recv).ob_type).name };
+ return Err(crate::PyError::type_error(format!(
+ "descriptor '__delitem__' for 'dict' objects doesn't apply to a '{tp_name}' object"
+ )));
}
+ crate::baseobjspace::delitem(dict, args[1])?;
Ok(pyre_object::w_none())
},🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyre/pyre-interpreter/src/typedef.rs` around lines 2356 - 2369, The current
deletion code silently no-ops when args[0] is neither a dict nor a dict-subclass
with a valid "__dict_data__" backing, which masks invalid receiver errors;
update the branch handling in the block that uses pyre_object::is_dict and
pyre_object::is_instance so that if args[0] is not a plain dict and either is
not an instance or getattr(args[0], "__dict_data__") is missing/non-dict, you
return a TypeError instead of silently returning None; use the same
error-construction helper/pattern used elsewhere in this module to raise a
TypeError with a clear message (e.g. "descriptor delete for non-dict or missing
__dict_data__") and keep the existing behavior of calling
crate::baseobjspace::delitem when a dict or valid backing dict is present.
| fn expect_reg(op: &Operand, expected: Kind) -> u16 { | ||
| match op { | ||
| Operand::Register(Register { kind, index }) if *kind == expected => *index, | ||
| _ => panic!("expected Register({:?}, _), got {:?}", expected, op), | ||
| _ => panic!( | ||
| "expected Register({:?}, _), got {:?} (op={})", | ||
| expected, | ||
| op, | ||
| current_dispatch_op() | ||
| ), | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
Extend opcode context to other expect_* helpers for uniform diagnostics.
Only expect_reg includes the current opcode in its panic message. Other operand-shape helpers (expect_tlabel, expect_small_u16, expect_descr_vable_array_field, etc.) would benefit equally from this context when diagnosing SSARepr operand mismatches.
Suggested enhancement for consistency
Apply the same pattern to other expect_* helpers:
fn expect_tlabel(op: &Operand) -> &TLabel {
match op {
Operand::TLabel(label) => label,
- _ => panic!("expected TLabel, got {:?}", op),
+ _ => panic!(
+ "expected TLabel, got {:?} (op={})",
+ op,
+ current_dispatch_op()
+ ),
}
}
fn expect_small_u16(op: &Operand) -> u16 {
match op {
Operand::ConstInt(value) => u16::try_from(*value).expect("expected u16-sized ConstInt"),
- _ => panic!("expected ConstInt(u16), got {:?}", op),
+ _ => panic!(
+ "expected ConstInt(u16), got {:?} (op={})",
+ op,
+ current_dispatch_op()
+ ),
}
}And similarly for expect_descr_vable_array_field, expect_descr_vable_array, expect_descr_vable_static_field, etc.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyre/pyre-jit/src/jit/assembler.rs` around lines 1646 - 1655, The panic in
expect_reg already includes current_dispatch_op(), but other helper functions
(e.g., expect_tlabel, expect_small_u16, expect_descr_vable_array_field,
expect_descr_vable_array, expect_descr_vable_static_field, etc.) do not; update
each expect_* helper to include the current opcode context by appending
current_dispatch_op() (or a similar descriptive call used in expect_reg) to
their panic messages so all operand-shape mismatch errors report the dispatch
opcode for consistent diagnostics.
follow-up #112
Summary
This PR delivers two large areas: CPython-parity machinery in the interpreter (builtin-leaf subclass dispatch, dict-subclass/deque/operator support, class-creation and cellvar fixes that together enable the Enum/IntEnum/Flag/IntFlag and namedtuple stdlib paths) and canonical-graph JIT work (the #127
simplify_graphpass port, the #73 canonicalflatten_graphlowering, and vectorizer Codex-parity). The Rust changes are inpyre-interpreter/pyre-jit/majit; the diff carries no enum/namedtuple test fixtures (the only test-ish file touched ischeck.py, forfib_loopheadroom).Interpreter — builtin-leaf subclass dispatch (custom int·float·str·tuple, enabling enum / IntFlag)
int/long/float/bool/strkeepob_typeat the canonical storage type and carry the Python class inw_class; addedbuiltin_subclass_dunder(display.rs) sopy_repr/py_strconsult a__repr__/__str__override resolved aboveobjectin thew_classMRO before the storage-keyed formatting (e.g.repr(IntEnum.X)).tuple.__new__'sdescr_new_wrapper!with a hand-writtentuple_descr_newthat, for a subclass, copies into a fresh tuple and setsw_class = cls(mirroringint_descr_new/float_descr_new), socollections.namedtuplefield access, repr,_replace/_make, and defaults work.descroperationnow gates the binary operators, rich comparisons, and unary operators (pos/neg/invert) on aw_class-resolved override (binop_dispatch_first/try_compare_override/try_unary_override) before theiris_int_like/is_float_pair/is_strstorage fast paths;operand_overrideskeys on the resolved method kind (mutable-codeFUNCTION_TYPE, excluding fixed-code gateway builtins) so alongis not mistaken for an override and does not recurse — covers Python__add__/__or__,__lt__/__eq__, andenum.IntFlag.__invert__.load_method(eval.rs) no longer bindsselffor non-method descriptors (type/property/member/getset such as__class__) or MRO-absent attributes on a builtin-storage instance, fixingself.__class__(value)(compiled asLOAD_METHOD/CALL_METHOD) prepending an extra argument.baseobjspaceresolves__getitem__ontype(cls)'s MRO before the PEP 560__class_getitem__fallback, consults the metaclass__len__for a type receiver, and resolves__contains__on the receiver's dynamic type before the getitem scan — routingColor['RED'],len(Color), andx in Color/x in flagthroughEnumMeta/IntFlag instance methods.Interpreter — class creation, metaclass, cellvars
MAKE_CELLnow wraps a slot only when it does not already hold a cell, sinceinitialize_frame_scopespre-installs an empty cell for every pure cellvar; this stops never-reassigned cellvars like__class__from becoming a cell-of-cell, fixingself.__class__and zero-argsuper()reads (previously infinite recursion in a dict-subclass__repr__callingsuper().__repr__()).build_class(call.rs) strips the compiler-internal class scaffolding (__class__,__classdict__,__classcell__,__classdictcell__) from the class namespace before building the type, captures the__classcell__cell, and binds it to the new class (type_new_classcell); a follow-up keeps__classcell__/__classdictcell__visible in the namespace a custom metaclass receives while still excluding them from the class__dict__, consuming them intype.__new__instead.build_classexecutes the class body directly against a custom__prepare__mapping (a dict subclass such asenum._EnumDict) viasetdictscope_object, so its__setitem__/__getitem__fire mid-body, then mirrors the mapping's contents intoclass_nsand skips the metaclass-path replay; fixes Flag members likeWHITE = RED | GREEN | BLUEreading unresolvedauto()sentinels.type.__new__resolves the dict backing of the namespace before its copy and__set_name__loops, so a dict-subclass namespace (PyDict_Check, notPyDict_CheckExact) is walked instead of producing an empty__dict__.type.mro()method (mro_external), returning the MRO as a fresh list distinct from the__mro__tuple getset; its absence had blockedimport enum.MAKE_CELLshapes: a parameter captured by an inner function, and the implicit__class__cellvar resolving zero-argsuper().Interpreter — dict / dict-subclass / mapdict
dict.__getitem__on a dict-subclass instance now looks up the key in the__dict_data__backing directly and, on a miss, dispatches__missing__against the subclass instance's type (upstreamdictmultiobject.py:166-170) instead of the plain-dict backing — so e.g.defaultdict.__missing__fires;dict_missing_or_key_erroris nowpub(crate).dict.__repr__method (extracted into the shareddisplay::dict_reprhelper, upstreamdictmultiobject.py:130-150 descr_repr) so dict-subclass instances andsuper().__repr__()format their backing rather than falling back to the object repr; unbounddict.__repr__(x)on a non-dict receiver now raisesTypeError(the receiver-rejection lives intypedef.rs, not the.pyref).dict.__delitem__resolves the__dict_data__backing for subclass instances intypedef.rs(mirroring__setitem__), fixing infinite recursion where the instance branch re-looked-up and re-entered the inheriteddict.__delitem__.DevolvedDictTerminatorread/write in mapdict (mapdict.py:383-395): read viagetdict+finditem_str, write viagetdict+setitem_str, gated onattrkind == DICT; added_mapdict_self_refto theMapdictObjecttrait to reach_obj_getdict.switch_to_text_strategypastLIMIT_MAP_ATTRIBUTESremains a documented deferral.(Rust changes for the dict items above live in
typedef.rs/baseobjspace.rs/display.rs/mapdict.rs; thedictmultiobject.pyline citations are upstream PyPy parity attributions, not Rust paths.)Interpreter — deque, operator, builtin-kwargs ABI, misc / bench
_collectionsdeque: bounded the list-backedW_Dequetomaxlen, trimming from the opposite end onappend/appendleft/extend/extendleft, with the bound in the private__maxlen__slot and a read-onlymaxlenproperty; addedextendleft,rotate,count,remove,__contains__,reverse,index,copy,__setitem__,__delitem__,__repr__, routing pop/popleft and the append family through shared snapshot/store helpers.maxlenis validated at construction viagateway_nonnegint_w(TypeError/ValueError),__init__propagates iterable errors,__repr__isReprGuard-protected ([...]), and__getitem__/__setitem__/__delitem__go through adeque_indexhelper mirroringspace.decode_index4. Added rich comparison (__eq__…__ge__, element-wise over both backings,NotImplementedfor non-deque) and repetition (__mul__/__rmul__/__imul__, re-bounded bymaxlen). The port covers these methods specifically; it is not a complete deque.operator: porteditemgetter/attrgetter/methodcalleras app-level callable classes plus the_resolve_attr_chainhelper (verbatimapp_operator.py), installed via theappleveldefsarm, replacing interp-level stubs that returnedargs[0]unchanged (which broke the stdlib namedtuple'sitemgetter(n)accessors); also added app-levelcountOf.length_hintstays interp-level.#[pyre_function]keyword binding (pyre-macros): when a call carries the trailing__pyre_kw__dict, the wrapper rebinds positional+keyword args into a resolved scope viabind_builtin_kwargs(mirroring the gatewayArguments._match_signature) using parameter-name/required tables collected at expansion time; positionals fill left-to-right, keywords match by name, an absent optional becomesPY_NULL, and unknown/duplicate/missing-required raisesTypeError. The positional fast path is unchanged when no kwargs dict is present; varargs fns keep the positional path. Fixesdeque(maxlen=3)previously binding the dict asiterable.pyframe:peekvalues(n)now asserts only the lowerbasebound; the upper-bound assert is skipped for the empty peek (n == 0), which spuriously failed at peak stack depth and crashedcollections/reprlibimports in debug builds. (Its commit also carries thecall.rsbuild_classscaffolding-strip rustfmt rewrap and a stray formatting-only edit to asimplify.rstest — both non-behavioral.)Forwardedenum has noVectorInfovariant (scratch is not clone-stable, lives in the pos-keyedvecinfo_cache), and fix thedefaultdictdoc-comment to state__getitem__invokes/storesdefault_factory(raisingKeyErrorwithout one) rather than short-circuiting tow_none()— the same comment records thatW_DefaultDictremains a stub subclassingobjectnotdict(soisinstance(d, dict)is False) with__missing__/__repr__/copy/__reduce__still absent.bench/list_reverse: raisedREPSfrom 15 to 401 (odd, keeping the reversed result) so the build loop and JIT trace warmup are amortised and the measurement reflectsreverse().check.py: gave craneliftfib_loop3x-vs-cpython headroom (dynasm stays 2x) to absorb slower windows-runner variance on the bignum-add-bound benchmark.JIT — simplify_graph pass port (#127)
eliminate_empty_blockstosimplify.rs(port ofsimplify.py:52-69,not link.target.operations), retargeting each predecessor link through an empty forwarding block, and wired it intoall_passes(); the walker pipeline keeps theblock.dead-predicatecodewriter::eliminate_empty_blocks. Adds a graph-shape test for collapsing a non-dead arg-carrying forwarder.remove_trivial_links' merge bridge to strip the source block's trailing boundarygoto TLabel(target) + Unreachablebefore absorbing the merged target'sper_block_ssarepr, so the target's own terminator is the first terminator and theemit_link_renamings_into_blocksplice lands after the target opcodes.rewrite_dead_forwarder_gotosto run beforeremove_trivial_links/rewrite_trivial_link_mergesso inline byte-stream gotos already name the surviving target when the merge bridge'sstrip_trailing_boundary_gotoreads source terminators (fixes thesource -> dead_forwarder -> targetstrip miss).None,NotImplemented,Ellipsis,True, andFalseprebuilt-singleton addresses injit_static_ref_addrsunder theirmodule::NAMEcatalogue keys, so the front-end same-fileExpr::Pathfold emitsConstRefAddrinstead of a rejected cross-block body-Inputforw_none/w_not_implemented/w_ellipsis/w_bool_from.assembler.rs::patch_labelsfail-loud note) and added a regression test asserting no reachable link targets a dead/dropped block after the subset runs on a graph with a dead switch arm and a trivial forwarder.JIT — canonical flatten_graph lowering (#73)
Constantref (Operand::ConstRef) in the canonicalflatten_graphpath via a newJitCodeBuilder::ref_return_const, mirroringload_const_rby encoding it as a constants-window register index (num_regs_r + pool_idx) patched infinish(); routeref_returnConstRefto it instead ofexpect_reg. Also names the dispatch opname in theexpect_regpanic via aCURRENT_DISPATCH_OPthread-local.setattrtois_pyre_canonical_elidable_hlopalongsidegetattrandtype: all three are paired with an inlineabort_permanentby the walker (StoreAttrarm), so the canonical SSARepr elides the undispatchable HLOp (upstreamrclass.py rtype_setattrrewrites tosetfield_gc).-live-after canonicalresidual_call_*/inline_call_*Insns underlowering_ctx, perjtransform.py:467-482handle_residual_call/handle_regular_call, supplying the post-callguard_no_exception/ inline-boundary resume marker. Adds a unit test pinning the marker after aresidual_call_ir_r. (A follow-up rustfmt-only commit rewraps thetrailing_livebinding/closure; non-behavioral.)calldescr_canraise(effect_info.check_can_raise(false)), reading theCallDescrStubEffectInfooff the Insn, so theEF_CANNOT_RAISEget_current_exceptioncall drops its marker whileinline_call_*stays unconditional; adds a unit test covering both the can-raise and cannot-raise cases.JIT — vectorizer Codex-parity
jitcell_token: Option<&Arc<JitCellToken>>throughoptimize_vector,VectorizingOptimizer::run_optimization, andtry_vectorize, passing it tofinaloplistin place of a hardcodedNone; the standalone caller and theOptimization-traitpropagate_forwardpath passNonewhile the compile path is disconnected, matching the upstreamjitcell_token=Nonedefault (vector.py:123,143,271).forwarded_vecinfoscheduling scratch (schedule.py:20-28) uses apos-keyedvecinfo_cacheinstead ofop._forwarded:Op::cloneresetsforwardedbut preservespos(resoperation.rs:1344,1352), the scheduler reads vecinfo off cloned ops (dependency.rs:221, unroll/schedule clones), andINT_SIGNEXTbytesize is the dynamicarg1value (cast_to_bytesize_staticreturnsNone) recoverable only viaint_signext_vecinfo's setup-time resolver thatvectorization_info_for_op(&Op)cannot reach.Self-review
Prompt & Model
Model:
Prompt:
Answer
Summary by CodeRabbit
Release Notes
dequeimplementation with extend, rotate, count, remove, copy, and rich comparisonsoperatormodule now providescountOf,attrgetter,itemgetter, andmethodcaller__getitem__,__len__, and__contains__operationstupleanddictsubclass handlingmro()method for type objects