objspace, typedef, frame, dict, imp: a user __getattribute__ on any layout, object's remaining text signatures, the collection frame.clear() forced, popitem on the strategy, and create_builtin's pre-filled import metadata - #1094
Conversation
|
Warning Review limit reached
Next review available in: 52 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (12)
WalkthroughThe PR adds CPython parity tests and updates interpreter behavior for attribute access, dictionary ChangesInterpreter parity and runtime behavior
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant PyrexExecution
participant RuntimeFinalization
participant SIGINTHandler
PyrexExecution->>RuntimeFinalization: report KeyboardInterrupt and finalize
RuntimeFinalization->>SIGINTHandler: restore default SIGINT disposition
SIGINTHandler-->>PyrexExecution: terminate with SIGINT status
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5ec5fc46bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ns, | ||
| "__eq__", | ||
| make_builtin_function_with_arity( | ||
| crate::gateway::make_builtin_function_with_arity_and_text_signature( |
There was a problem hiding this comment.
Preserve upstream signature generation
When introspection runs against Pyre as a PyPy port, these new literals expose CPython 3.14 signatures instead of the signatures produced by PyPy's gateway. In pypy/objspace/std/objectobject.py, only __repr__, __init__, and __new__ receive explicit signatures; the remaining methods flow through gateway.py::_generate_text_signature, which derives names from the RPython arguments and returns None for unsupported varargs shapes such as __subclasshook__. Hard-coding all 20 CPython spellings therefore creates observable structural divergence; port the gateway-generation path and reserve literals for the three upstream literals.
AGENTS.md reference: AGENTS.md:L231-L233
Useful? React with 👍 / 👎.
🤖 Codex parity reviewStatic analysis of this diff vs the local RPython/PyPy sources (commit 4af5443). Files in the reviewed diffCodex did not produce a report (exit 1). Last log lines: |
__getattribute__ on any layout, object's remaining text signatures, the collection frame.clear() forced, and popitem on the strategy__getattribute__ on any layout, object's remaining text signatures, the collection frame.clear() forced, popitem on the strategy, and create_builtin's pre-filled import metadata
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e46450f2df
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let (k, v) = pyre_object::dictmultiobject::w_dict_popitem(dict) | ||
| .ok_or_else(|| crate::PyError::key_error("popitem(): dictionary is empty"))?; |
There was a problem hiding this comment.
Root strategy values before allocating popped keys
When popitem() runs on a module globals dict or a mapdict-backed obj.__dict__, this new strategy dispatch reaches implementations that copy/remove the value and then allocate a string key before the value is pinned (celldict.rs:1213-1216 and mapdict.rs:4598-4601). If that allocation triggers a nursery collection, the returned local can still contain the pre-relocation pointer (or, for a removed module value, be reclaimed entirely); pinning k and v after w_dict_popitem returns is too late. Root and reload the value inside each allocating strategy implementation before constructing the key.
Useful? React with 👍 / 👎.
| crate::module_ns_store(w_dict, "__loader__", pyre_object::w_none()); | ||
| crate::module_ns_store(w_dict, "__spec__", pyre_object::w_none()); |
There was a problem hiding this comment.
Seed
__package__ for directly created builtins
When _imp.create_builtin(spec) is called directly, module_from_spec() has not yet initialized import metadata, and load_builtin_module() only seeds __name__; this helper adds __loader__ and __spec__ but leaves __package__ absent, so accessing module.__package__ raises AttributeError. CPython's directly created builtin modules expose all three attributes as None, and the existing module initializer in typedef.rs seeds that same trio, so __package__ should be initialized here as well.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
https://github.com/youknowone/pyre/blob/c09d7ee4f54df9d91fe30655df218c6baeaa166a/pyre-object/src/dictmultiobject.rs#L4169
Keep popitem strategy dispatch traceable
In JIT-compiled code that repeatedly calls dict.popitem(), this annotation forces every operation through a residual call and hides the live-strategy dispatch and concrete popitem implementation from optimization. The corresponding W_DictMultiObject.popitem and strategy methods in pypy/objspace/std/dictmultiobject.py have no dont_look_inside barrier; rpython/rlib/jit.py explicitly defines that decorator as preventing tracing. The translator should learn the required IndexMap/mapdict mutations rather than introducing a permanent JIT boundary around an otherwise line-by-line port.
AGENTS.md reference: AGENTS.md:L231-L233
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6386bd5e47
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| #[cfg(windows)] | ||
| let signaled = libc::raise(libc::SIGINT); | ||
| #[cfg(not(windows))] | ||
| let signaled = libc::kill(libc::getpid(), libc::SIGINT); |
There was a problem hiding this comment.
Use an allowlisted self-signal in sandbox builds
In Linux sandbox builds, real_main installs pyre-sandbox's seccomp filter before executing user code, and that filter allows tkill/tgkill but not kill, with every unlisted syscall trapped. Consequently, an uncaught KeyboardInterrupt in a pyre interact guest reaches this raw kill(2) and exits through the SIGSYS handler with a “blocked syscall” diagnostic rather than terminating by SIGINT as intended. Use an allowlisted self-thread signaling path, or add an appropriately constrained sandbox allowance.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
pyre/pyre-interpreter/src/typedef.rs (1)
19055-19076: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winEnforce the declared positional arity.
make_builtin_function_with_text_signaturedoes not enforce arity, and both closures ignore positional arguments. As a result, invalid calls such asobject.__init_subclass__(int, 1)can return instead of raisingTypeError.Require exactly one positional class argument for
__init_subclass__after splitting the internal keyword carrier. Register__subclasshook__with the arity-aware gateway and require its object argument.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pyre/pyre-interpreter/src/typedef.rs` around lines 19055 - 19076, Update the __init_subclass__ closure to validate the positional arguments returned by split_builtin_kwargs, requiring exactly one class argument before processing keyword arguments. Change the __subclasshook__ registration to use the arity-aware gateway and enforce its required object argument, while preserving existing keyword validation and behavior.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@pyre/extra_tests/parity_tests/dict_popitem_strategy.py`:
- Around line 46-48: Add a bytes-only LIFO test case alongside the existing int,
str, and object cases, using only byte-string keys so it selects
BytesDictStrategy and exercises popitem. Assert reverse insertion order while
covering Vec<u8> key reconstruction and preserving the associated values.
In `@pyre/extra_tests/parity_tests/exception_getattribute_override.py`:
- Around line 54-56: Rename the ambiguous list variable l in the test setup to a
descriptive identifier, and update the subsequent marker assertion to use the
new name while preserving the existing assertions and behavior.
In `@pyre/extra_tests/parity_tests/frame_clear_finalization.py`:
- Around line 74-77: Update the assertions in the generator finalization test so
it requires seen == ["del"] immediately after generator.close() returns, before
del generator and gc.collect(); retain the later collection and final assertion
to verify cleanup remains finalized.
- Around line 127-132: Remove the per-operation CPU-time calculation and the
`per_op_ms < 1.0` assertion from this test. Replace the timing-based check
around `check.assertRaises(Expected, raises)` with a deterministic assertion
that verifies the expected cleanup behavior, using the existing test resources
and observable state.
In `@pyre/extra_tests/parity_tests/keyboard_interrupt_exit_status.py`:
- Around line 56-64: Extend the parity test around the `subprocess.run` `-m`
call by adding a temporary importable module whose execution raises
`KeyboardInterrupt`, while retaining the existing missing-module status-1
assertion. Run that module with `-m` and assert the platform-specific SIGINT
return status plus the expected traceback output.
In `@pyre/pyre-interpreter/src/eval.rs`:
- Around line 3335-3346: Add a parity test covering a context manager’s __exit__
arguments when E is raised inside a with statement. Define __exit__ to assert
its traceback argument is the stored traceback rather than
"overridden-traceback", complementing the existing direct exc.__traceback__
test.
In `@pyre/pyre-interpreter/src/objspace/std/mapdict.rs`:
- Around line 4600-4611: Update the code around plain_direct_read and
maybe_migrate_to_boxed to root w_obj before boxing, then root w_key before
reading and w_value before migration so all GC-managed references survive
allocation or movement. After migration, reload the rooted w_obj, w_key, and
w_value and recompute inst and map before calling self.delitem and returning the
pair.
---
Outside diff comments:
In `@pyre/pyre-interpreter/src/typedef.rs`:
- Around line 19055-19076: Update the __init_subclass__ closure to validate the
positional arguments returned by split_builtin_kwargs, requiring exactly one
class argument before processing keyword arguments. Change the __subclasshook__
registration to use the arity-aware gateway and enforce its required object
argument, while preserving existing keyword validation and behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: fb33cad0-5de0-4292-9428-cc8447ef2707
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (21)
pyre/cpython_tests/baseline.jsonpyre/cpython_tests/run.pypyre/extra_tests/parity_tests/builtin_module_loader_spec.pypyre/extra_tests/parity_tests/dict_popitem_strategy.pypyre/extra_tests/parity_tests/exception_getattribute_override.pypyre/extra_tests/parity_tests/frame_clear_finalization.pypyre/extra_tests/parity_tests/imp_multi_interp_override.pypyre/extra_tests/parity_tests/keyboard_interrupt_exit_status.pypyre/extra_tests/parity_tests/object_init_text_signature.pypyre/pyre-interpreter/src/baseobjspace.rspyre/pyre-interpreter/src/eval.rspyre/pyre-interpreter/src/executioncontext.rspyre/pyre-interpreter/src/importing.rspyre/pyre-interpreter/src/module/imp/interp_imp.rspyre/pyre-interpreter/src/objspace/std/mapdict.rspyre/pyre-interpreter/src/type_methods.rspyre/pyre-interpreter/src/typedef.rspyre/pyre-object/src/celldict.rspyre/pyre-object/src/dictmultiobject.rspyre/pyrex/Cargo.tomlpyre/pyrex/src/lib.rs
| l = L([1, 2, 3]) | ||
| assert l.marker is SENTINEL | ||
| assert L.calls == 1 |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Rename the ambiguous list variable.
Ruff E741 rejects l as an ambiguous identifier. Rename it and update its uses.
Proposed fix
-l = L([1, 2, 3])
-assert l.marker is SENTINEL
+list_instance = L([1, 2, 3])
+assert list_instance.marker is SENTINEL📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| l = L([1, 2, 3]) | |
| assert l.marker is SENTINEL | |
| assert L.calls == 1 | |
| list_instance = L([1, 2, 3]) | |
| assert list_instance.marker is SENTINEL | |
| assert L.calls == 1 |
🧰 Tools
🪛 Ruff (0.16.1)
[error] 54-54: Ambiguous variable name: l
(E741)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyre/extra_tests/parity_tests/exception_getattribute_override.py` around
lines 54 - 56, Rename the ambiguous list variable l in the test setup to a
descriptive identifier, and update the subsequent marker assertion to use the
new name while preserving the existing assertions and behavior.
Source: Linters/SAST tools
| assert seen in ([], ["del"]) | ||
| del generator | ||
| gc.collect() | ||
| assert seen == ["del"] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Require finalization when close() returns.
The compatibility path explicitly collects after an unstarted generator frame is cleared. seen in ([], ["del"]) accepts a regression where generator.close() returns before Watched.__del__ runs.
Require seen == ["del"] before deleting generator.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyre/extra_tests/parity_tests/frame_clear_finalization.py` around lines 74 -
77, Update the assertions in the generator finalization test so it requires seen
== ["del"] immediately after generator.close() returns, before del generator and
gc.collect(); retain the later collection and final assertion to verify cleanup
remains finalized.
| before = resource.getrusage(resource.RUSAGE_SELF) | ||
| for _ in range(count): | ||
| check.assertRaises(Expected, raises) | ||
| after = resource.getrusage(resource.RUSAGE_SELF) | ||
| per_op_ms = (after.ru_utime - before.ru_utime) * 1000.0 / count | ||
| assert per_op_ms < 1.0, per_op_ms |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Remove the CPU-time threshold.
This assertion depends on host speed and ru_utime resolution. It can fail on loaded CI workers. It can also pass when the measurement rounds to zero.
Keep this test deterministic. Assert cleanup behavior instead of elapsed CPU time.
🧰 Tools
🪛 Ruff (0.16.1)
[warning] 129-129: Use pytest.raises instead of unittest-style assertRaises
(PT027)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyre/extra_tests/parity_tests/frame_clear_finalization.py` around lines 127 -
132, Remove the per-operation CPU-time calculation and the `per_op_ms < 1.0`
assertion from this test. Replace the timing-based check around
`check.assertRaises(Expected, raises)` with a deterministic assertion that
verifies the expected cleanup behavior, using the existing test resources and
observable state.
| // Box the key before the read's conversion tail runs: `key` borrows the | ||
| // map node, and the migration below replaces the instance's map. | ||
| let w_key = pyre_object::unicodeobject::box_str_constant(key); | ||
| let w_value = plain_direct_read(curr, &*inst); | ||
| // `plain_direct_read` is only `_prim_direct_read` (mapdict.py:600-601). | ||
| // The read still owes `_direct_read`'s tail (mapdict.py:592-598): an | ||
| // unboxed attribute whose terminator has stopped allowing unboxing | ||
| // converts the whole instance to boxed storage. `getdictvalue` pairs | ||
| // the two the same way. | ||
| maybe_migrate_to_boxed(map, inst, key, DICT); | ||
| self.delitem(w_dict, w_key); | ||
| Some((w_key, w_value)) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Root and reload references across allocation.
w_key crosses plain_direct_read, which boxes unboxed values. w_value then crosses maybe_migrate_to_boxed. Both operations can allocate or move GC-managed objects.
Pin w_obj before allocation. Pin w_key before plain_direct_read. Pin w_value before migration. Reload the rooted references and derive inst and map again before deletion and return. Otherwise a moving collection can leave self.delitem or the returned pair with stale pointers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@pyre/pyre-interpreter/src/objspace/std/mapdict.rs` around lines 4600 - 4611,
Update the code around plain_direct_read and maybe_migrate_to_boxed to root
w_obj before boxing, then root w_key before reading and w_value before migration
so all GC-managed references survive allocation or movement. After migration,
reload the rooted w_obj, w_key, and w_value and recompute inst and map before
calling self.delitem and returning the pair.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 148cbd16d7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| crate::make_builtin_function_with_arity( | ||
| "_override_multi_interp_extensions_check", | ||
| |args| { | ||
| crate::baseobjspace::gateway_int_w(args[0])?; |
There was a problem hiding this comment.
Apply C-int conversion before refusing the override
When the argument is an integer that fits i64 but not a C int (for example 1 << 40), gateway_int_w accepts it and this function raises the main-interpreter RuntimeError; CPython 3.14 instead rejects it with OverflowError before checking the interpreter. The same converter also accepts objects implementing only __int__, while this API requires the integer-index protocol. Use a C-int/index conversion here so invalid arguments retain the API's argument error rather than being misreported as an interpreter-state error.
Useful? React with 👍 / 👎.
`getattr_str_impl` gated the receiver-type `__getattribute__` slot on `is_instance`, a `W_ObjectObject` layout check. An exception subclass is a `W_BaseException` and a `list` subclass a `W_ListObject`, so neither reached the gate: their override was never called and attribute access fell through to `object_getattr_miss`, which runs its own descriptor protocol. Only `__getattr__` still worked, through `instance_getattr_hook_or_err`. objspace.py:664-670 gates on `space.type(w_obj)` with no layout test. `setattr_str` in this file already resolves the type that way, which is why `__setattr__` worked on the same classes and `__getattribute__` did not. The arm runs only for `space.getattr`, not the bare `object.__getattribute__` slot: `object_getattribute` hands its non-instance, non-type receivers back to `getattr_str_impl`, so dispatching there would recurse through every `super().__getattribute__(name)` an override makes. `super`, bound `method`, `types.GenericAlias`, `types.UnionType` and the two weakref proxies register a `__getattribute__` of their own, so `getattribute_if_not_from_object` answers `Some` for them too; a census over 403 types finds no others besides `type` and `module`, which the arm already excludes. Those receivers are served by the shims earlier in the function, so the dispatch is restricted to a heap-type owner and they keep the descriptor protocol they took before. `with_except_start_values` read `__traceback__` through `getattr_str` while building the `__exit__(type, value, traceback)` arguments; pyopcode.py:1358-1362 reads `W_BaseException.w_traceback` directly, so it now reads the slot. Assisted-by: Claude
`init_object_type` stamped a signature on `__new__` and `__init__` only, so the other 19 entries of `object.__dict__` were built with bare `make_builtin_function` / `make_builtin_function_with_arity` and answered `None`. They now use the `_and_text_signature` twins; `__init_subclass__` and `__subclasshook__` are stamped on the inner carrier before `w_classmethod_new` wraps it. The strings are the ones cpython3.14 reports, not the ones `gateway.py:1146 _generate_text_signature` would derive from the RPython parameter names — that generator spells `descr__str__(space, w_obj)` as `($obj, /)` and returns `None` outright for a varargname callable like `descr___subclasshook__`. `extra_tests/parity_tests/run.py` runs every fixture under the system CPython as well as each backend, so only the cpython spelling is assertable. objectobject.py:187-189 hard-codes the same three strings for `__repr__`, `__init__` and `__new__`, so a literal is the upstream shape here rather than a pyre invention. `__sizeof__` is not among them: `object.__sizeof__` no longer exists, so the fixture drops it from the compared set the way `list_python314.py` and `code_python314.py` already do. `__doc__` is left alone. gateway.py:1328-1332 also synthesizes a docstring beside the signature, but pyre's helpers only write `w_text_signature` and these descriptors report `__doc__` as `None` today; changing that is a descriptor-surface change with its own fixtures asserting on it. `object_init_text_signature.py` asserts all 19 against `object.__dict__[name].__text_signature__`, the form this changes; the bound `getattr(object, name)` and `inspect.signature` forms depend on descriptor re-exposure and parsing and are left out. Assisted-by: Claude
`init_frame_type`'s `clear` entry called `finalize_explicitly_cleared_frame_references` after `descr_clear`, so every `frame.clear()` ran a full non-moving major pass plus the finalizer queue. That cost 12.5 ms per call, and `unittest`'s `_AssertRaisesContext.__exit__` reaches it through `traceback.clear_frames(tb)` on every *successful* `assertRaises`: `traceback.clear_frames` measured 13.13 ms/op against 0.00 ms on cpython3.14. `test.test_importlib` timed out past 300s on that; it now runs its 1346 tests in 10.8s. The two `generator.py:243` close paths in `baseobjspace.rs` keep the hook. gh-142766 is about `generator.close()` releasing a suspended frame's locals before it returns, and `extra_tests/parity_tests/generator_python314.py` asserts that; the frame method was given the same call without a caller. `test_frame.py::ClearTest.test_clear_locals` clears traceback frames and then calls `support.gc_collect()` itself before asserting the weakref died, and `jit_inline_traceback_frame_clear.py` asserts only that a completed traceback's frames can be cleared, so neither depends on the clear collecting. Deferring here is the tracing-GC behaviour a refcount-free runtime already has for every other reference drop. `frame_clear_finalization.py` pins both sides: `frame.clear()` plus an explicit `gc.collect()` kills the weakref, `close()` on a suspended and on a try/finally generator finalizes before it returns, `clear()` on an executing frame still raises `RuntimeError`, and an `assertRaises`-shaped loop stays under 1 ms of user CPU per iteration. Assisted-by: Claude
`dict_method_popitem` called `w_dict_items`, which materialises a `Vec` of every pair, took `.last()` and then deleted that key. Draining a dict of N entries was O(N^2), and an Int-strategy dict boxed every key through `w_int_new` on each of the N calls. `dictmultiobject.py:257 descr_popitem` delegates to `W_DictMultiObject.popitem`, which dispatches to the strategy; the new `w_dict_popitem` does that. `DictStrategy::popitem` had no caller outside its own unit tests, so this is the first time the family runs from Python and every impl needed an audit rather than only the one being called: - Object, Unicode, Bytes and Int had no override and fell through to the trait default, which is the `items()` materialisation this commit is removing. They get direct `IndexMap::pop` overrides (`dictmultiobject.py:1119-1121 AbstractTypedStrategy.popitem`). - `ModuleDictStrategy::popitem` bumped no `keys_version` on either storage path. Nothing observed that while it was unreachable; a global read cached against a stale version would now. - `MapDictStrategy::popitem` ended `instance_node_getdictvalue(w_obj, key)?`. That `?` turns a value-lookup miss into the caller's "dictionary is empty", which would skip the delete and raise `KeyError` on a non-empty `__dict__`. `node_search` has already returned the node, so the value is present by construction: it reads that node with `plain_direct_read` instead. Dropping `instance_node_getdictvalue` also drops its `maybe_migrate_to_boxed` side effect, which belongs at the getattr boundary and not on an entry about to be deleted. - Empty and EmptyKwargs return `None`; Kwargs already pops the array tail and already bumps. Bytes and Int allocate the returned key, so the popped value is pinned across that allocation, and `dict_method_popitem` pins both across `w_tuple_new`. The `w_dict_len(dict) == 0` guard stays ahead of the dispatch. It is not redundant: without it an empty mapdict-backed `__dict__` would reach `instance_lock` and `ensure_mapdict_initialized`, and an empty module dict `w_module_dict_object_storage_mut_opt`, before raising. `w_dict_popitem` is `dont_look_inside` (`rlib/jit.py:139`): the concrete strategies pop from `IndexMap` or mapdict storage, neither of which the tracer models as inline heap mutation. `dict_popitem_strategy.py` asserts LIFO across the int, str, object, kwargs, module-dict and instance-`__dict__` strategies plus a `dict` subclass, covers a surrogate-named instance attribute, and pins the empty `KeyError`'s type and message on the empty, drained and re-inserted shapes. Assisted-by: Claude
`_imp.create_builtin` routed through `create_builtin_module` → `startup_builtin_module` → `set_builtin_module_spec`, which fetches a spec from `sys.modules["importlib._bootstrap"].BuiltinImporter` and runs `_init_module_attrs` on it. The module handed back therefore already carried a `__loader__`, and `_bootstrap.py:746` guards `module_from_spec`'s own `_init_module_attrs` with `if override or getattr(module, '__loader__', None) is None`, so the caller's `spec.loader` never landed. `__spec__` is assigned unconditionally, which is why the two disagreed. `set_builtin_module_spec` is right for the native `load_part` path, which never reaches app-level `_init_module_attrs`. `startup_builtin_module_impl` now takes that decision as a parameter: the native wrapper passes `true` and keeps its behaviour, `_imp.create_builtin` passes `false` and seeds `__loader__` and `__spec__` as `None`, which is the state cpython3.14 hands back. On an uncached `errno`, `_imp.create_builtin(spec)` returned a module whose `__loader__` was `_frozen_importlib.BuiltinImporter` and whose `__spec__` was already a `ModuleSpec`; both are now `None`, matching cpython3.14. `test.test_importlib` goes from `FAILED (failures=1, skipped=65)` to `OK (skipped=65)` over 1346 tests, and its baseline entry moves off the stale `IMPORTERROR`. `set_sys_module` stays on this path. cpython3.14 does not insert on a direct `_imp.create_builtin` call — a second `del sys.modules[name]` raises `KeyError` there and succeeds here — but removing it segfaults, so the rooting it provides is still load-bearing and the divergence is left in place. The source-imported `BuiltinImporter.__module__` still reads `_frozen_importlib` where cpython3.14 reads `importlib._bootstrap`. That survives this change, so it is a separate defect rather than another symptom of the stamping. Assisted-by: Claude
ModuleDictStrategy::popitem's object-storage arm bumped the keys version without calling strategy.mutated(), so a reader of a popped global kept the strategy's cached view (celldict.py:166-173). MapDictStrategy::popitem read the value with plain_direct_read, which is only _prim_direct_read (mapdict.py:600-601); the read still owed _direct_read's migrate-to-boxed tail (mapdict.py:592-598). The key is boxed before the migration replaces the instance's map. Assisted-by: Claude
run_module and run_source classified the error as a KeyboardInterrupt before printing it, and after finalize_runtime and maybe_print_jit_stats reset SIGINT to SIG_DFL and raise it at the process instead of calling process::exit(1) (app_main.py:1133-1153). pyrex gains a libc dependency for signal/kill/getpid. The parity fixture asserts the exit statuses: -SIGINT on POSIX, 0xC000013A on win32, 1 for RuntimeError, 3 for SystemExit(3), 0 for a caught KeyboardInterrupt. Assisted-by: Claude
The function converts its argument with gateway_int_w and then raises RuntimeError naming the main interpreter. pyre runs one interpreter, so the refusing arm is the only reachable one and the override keeps no state. importlib.util._incompatible_extension_module_restrictions therefore raises from __enter__. Assisted-by: Claude
…les as PASS test_runpy's file run as __main__ does not run test_runpy — it starts libregrtest over the whole suite (491 modules on CPython 3.14, 492 here), so script mode could only time out on it. It joins DOTTED_IDENTITY_MODULES alongside test_descr and test_enum. baseline.json records test_import, test_modulefinder and test_runpy as PASS on dynasm; test_pkg and test_pkgutil already read PASS on main. Assisted-by: Claude
…wo fixtures `MapDictStrategy::popitem` held `w_key`, `w_value` and the instance carrier in Rust locals across `box_str_constant`, `plain_direct_read` and `maybe_migrate_to_boxed`, each of which can allocate and so run a minor collection that forwards a nursery address. They are pinned with `gc_roots::pin_roots` and read back from the shadow stack after every such call. `map`, `curr` and `key` need no pin: map nodes are not GC-allocated. `w_dict_popitem` no longer carries `dont_look_inside`. `w_dict_delitem` and `w_dict_clear` mutate the same `IndexMap` and mapdict storages untraced, and `dictmultiobject.py` has no such decorator. Measured: `check.py --backend dynasm` and `--backend cranelift` both stay at 404 passed with no jit-stats movement. `keyboard_interrupt_exit_status.py`'s `-m` case ran a module that fails to import, which never reaches `run_module`'s interrupt ending; it now also runs a module that raises KeyboardInterrupt and asserts the signal status there. `dict_popitem_strategy.py` gains a bytes-only dict, which is the only input that reaches that strategy's arm. `exception_getattribute_override.py` renames its one-letter list binding. Assisted-by: Claude
`ParseErrorType::Lexical(LexicalErrorType::UnrecognizedToken)` reached the
SyntaxError carrying the parser's own `Got unexpected token {tok}`. The
normalizing match in the compile-error conversion gains an arm that splits it
the way `pytokenizer.py:130-140` does: a non-printable character reports
`invalid non-printable character U+{:04X}` and a printable one
`invalid character '{tok}' (U+{:04X})`, the format widening rather than
truncating an astral code point. Printability is
`rustpython_unicode::classify::is_printable`, the predicate behind
`str.isprintable`; `char::is_control` is Cc-only and would let U+00A0 through.
A printable ASCII character reports plain `invalid syntax`:
`test_syntax.py:1459` asserts that for `1 $ 2`, and `:2238` keeps
`invalid character` for the non-ASCII case.
Offsets and line numbers are untouched — all ten measured rows keep offset 5.
test.test_code_module reaches PASS and is recorded; test.test_syntax stays at
12 failures, none naming either message, and its `1 $ 2` doctests now pass.
Assisted-by: Claude
…e override `exception_getattribute_override.py` covered `__traceback__` only through attribute access, which the override answers. A `with` body that raises the same exception reaches `__exit__` through the unwinder, which reads the stored slot, so the two disagree by construction and only one of them was pinned. Assisted-by: Claude
There was a problem hiding this comment.
💡 Codex Review
https://github.com/youknowone/pyre/blob/4af544321fe4c26457c12acc5a80abcb773d2577/pyre-interpreter/src/objspace/std/mapdict.rs#L4609
Publish the mapdict wrapper with its carrier
When obj.__dict__.popitem() runs while the wrapper is still in the nursery, this pin_roots call can trigger a collection after publishing only w_obj; tracing the carrier then forwards its __dict__ wrapper, but the local w_dict retains the old address and is later passed to self.delitem. The new result rooting therefore still permits a stale-pointer dereference or heap corruption under minor collection. Publish w_obj and w_dict together and reload the wrapper before deletion.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Five commits on top of
origin/main: two attribute-protocol correctness fixes, the collection everyframe.clear()was forcing, thedict.popitemstrategy routing, and the import metadata_imp.create_builtinwas pre-filling.__getattribute__on a non-objectlayoutobjspace: dispatch a user __getattribute__ on any receiver layoutgetattr_str_implgated the receiver-type__getattribute__slot onis_instance, which is aW_ObjectObjectlayout check. An exception subclass is aW_BaseExceptionand alistsubclass aW_ListObject, so neither reached the gate — the override was simply never called, and attribute access fell through toobject_getattr_missand its own descriptor protocol. Only__getattr__still fired, throughinstance_getattr_hook_or_err.E().argsran the override zero times.objspace.py:664-670gates onspace.type(w_obj)with no layout test.setattr_strin the same file already resolves the type that way, which is exactly why__setattr__worked on these classes while__getattribute__did not.Two things keep the widening from re-entering:
space.getattronly, not for the bareobject.__getattribute__slot.object_getattributehands its non-instance, non-type receivers back togetattr_str_impl, so dispatching there would recurse through everysuper().__getattribute__(name)an override makes.typeandmodule(already excluded), exactlysuper, boundmethod,types.GenericAlias,types.UnionTypeand the two weakref proxy types answerSomefromgetattribute_if_not_from_object. Those are served by the shims earlier in the function — the bound-method one forwards to__func__— so the dispatch is restricted to a heap-type owner and they keep the descriptor protocol they already took. The change is strictly additive.The same commit fixes
with_except_start_values, which read__traceback__back throughgetattr_strwhile building the__exit__(type, value, traceback)arguments.pyopcode.py:1358-1362readsW_BaseException.w_tracebackdirectly, so it now reads the slot — otherwise a__getattribute__override would newly observe the interpreter's own read.object's remaining text signaturestypedef: give object's remaining 20 callables their __text_signature__init_object_typestamped a signature on__new__and__init__only; the other 20 entries ofobject.__dict__were built with the baremake_builtin_function/make_builtin_function_with_arityhelpers and answeredNone. They now use the_and_text_signaturetwins, and__init_subclass__/__subclasshook__are stamped on the inner carrier beforew_classmethod_newwraps it.The strings are the ones cpython3.14 reports, not the ones
gateway.py:1146 _generate_text_signaturewould derive from the RPython parameter names — that generator spellsdescr__str__(space, w_obj)as($obj, /)and returnsNoneoutright for a varargname callable likedescr___subclasshook__.extra_tests/parity_tests/run.pyruns every fixture under the system CPython as well as each backend, so only the cpython spelling is assertable, andobjectobject.py:187-189hard-codes the same three strings for__repr__,__init__and__new__— a literal is the upstream shape here, not a pyre invention.__doc__is deliberately left alone.gateway.py:1328-1332synthesizes a docstring beside the signature, but pyre's helpers only writew_text_signatureand these descriptors report__doc__asNonetoday; changing that is a descriptor-surface change with its own fixtures asserting on it.The collection
frame.clear()forcedframe: stop clear() forcing an old-generation collectioninit_frame_type'sclearentry calledfinalize_explicitly_cleared_frame_referencesafterdescr_clear, so everyframe.clear()ran a full non-moving major pass plus the finalizer queue.That is not a frame-object curiosity.
unittest's_AssertRaisesContext.__exit__callstraceback.clear_frames(tb)on every successfulassertRaises, andclear_framescallstb_frame.clear()per traceback frame — so the whole vendored CPython suite pays it.Per-op user CPU, same script on each:
traceback.clear_frames(tb)tb.tb_frame.clear()loopMeasured with 162 modules loaded — a heap the size of a real test run — the same two operations cost 13.13 ms and 12.50 ms per op, because the forced pass scales with the old generation.
End to end,
test.test_importlibused to time out past 300s. It now runs its 1346 tests in 10.8s (cpython3.14: 4.03s). One test in that module still fails, for an unrelated pre-existing reason — see Follow-up below.The two
generator.py:243close paths inbaseobjspace.rskeep the hook, and the doc comment is narrowed to say so. gh-142766 is aboutgenerator.close()releasing a suspended frame's locals before it returns;extra_tests/parity_tests/generator_python314.pyasserts that, and it still passes. The frame method had been given the same call without a caller that needs it:test_frame.py::ClearTest.test_clear_locals— the obvious test for "frame.clear()released the reference" — callssupport.gc_collect()itself afterclear_traceback_frames(...)and only then asserts the weakref is dead.test.test_frameis recordedSKIP("implementation detail") inpyre/cpython_tests/baseline.json, so the gate does not run it either way.jit_inline_traceback_frame_clear.pyasserts only that a completed traceback's frames can be cleared, not when their contents die.Deferring here is the behaviour a refcount-free runtime already has for every other reference drop.
frame_clear_finalization.pypins both sides:frame.clear()plus an explicitgc.collect()kills the weakref;close()on a suspended generator and on one closed insidetry/finallyfinalizes before it returns;clear()on an executing frame still raisesRuntimeError; and anassertRaises-shaped loop stays under 1 ms of user CPU per iteration.dict.popitemon the strategydict: route popitem through the strategy instead of items().last()dict_method_popitemcalledw_dict_items, which materialises aVecof every pair, took.last(), then deleted that key. Draining a dict of N entries was O(N²), and an Int-strategy dict boxed every key throughw_int_newon each of the N calls.dictmultiobject.py:257 descr_popitemdelegates toW_DictMultiObject.popitem, which dispatches to the strategy.DictStrategy::popitemhad no caller outside its own unit tests, so this is the first time the family runs from Python — every impl needed an audit, not just the one being called:IndexMap::popitems()materialisation being removeddictmultiobject.py:1119-1121 AbstractTypedStrategy.popitem)IndexMap::popkeys_versionon either storage pathnode_search(..., DICT)nodeinstance_node_getdictvalue(w_obj, key)?plain_direct_readNoneTwo of those were latent defects the routing would have exposed:
?turns a value-lookup miss into the caller's "dictionary is empty" — it would skip the delete and raiseKeyErroron a non-empty__dict__.node_searchhas already returned the node, so the value is present by construction. Droppinginstance_node_getdictvaluealso drops itsmaybe_migrate_to_boxedside effect, which belongs at the getattr boundary, not on an entry about to be deleted.keys_versionbump whilepopitemwas unreachable. A global read cached against a stale version would.Bytes and Int allocate the returned key, so the popped value is pinned across that allocation, and
dict_method_popitempins both acrossw_tuple_new.The
w_dict_len(dict) == 0guard stays ahead of the dispatch. It is not redundant: without it an empty mapdict-backed__dict__would reachinstance_lockandensure_mapdict_initialized, and an empty module dictw_module_dict_object_storage_mut_opt, before raising.w_dict_popitemisdont_look_inside(rlib/jit.py:139) — the concrete strategies pop fromIndexMapor mapdict storage, neither of which the tracer models as inline heap mutation.dict_popitem_strategy.pyasserts LIFO across the int, str, object, kwargs, module-dict and instance-__dict__strategies plus adictsubclass, covers a surrogate-named instance attribute, and pins the emptyKeyError's type and message on the empty, drained and re-inserted shapes.The import metadata
_imp.create_builtinpre-filledimp: stop create_builtin pre-filling __loader__ and __spec__Removing the
frame.clear()collection lettest.test_importlibfinish for the first time, and it finished with exactly one failure:Two classes that print identically but are different objects.
test_importlib/util.py:63 import_importlibbuilds aSourcevariant withblocked=('_frozen_importlib', '_frozen_importlib_external'), so the test holds a second, source-importedBuiltinImporterand asserts the loaded module's__loader__is that one._imp.create_builtinrouted throughcreate_builtin_module→startup_builtin_module→set_builtin_module_spec, which fetches a spec fromsys.modules["importlib._bootstrap"].BuiltinImporterand runs_init_module_attrson it. The module handed back therefore already carried a__loader__, and_bootstrap.py:746guardsmodule_from_spec's own_init_module_attrswithif override or getattr(module, '__loader__', None) is None— so the caller'sspec.loadernever landed.__spec__is assigned unconditionally, which is whymodule.__spec__.loaderwas right whilemodule.__loader__was wrong. The two disagreeing is the signature of this bug.On an uncached
errno:_imp.create_builtin(spec).__loader__BuiltinImporterNoneNone_imp.create_builtin(spec).__spec__ModuleSpecNoneNoneset_builtin_module_specis right for the nativeload_partpath, which never reaches app-level_init_module_attrs.startup_builtin_module_implnow takes that decision as a parameter — the native wrapper passestrueand is unchanged,_imp.create_builtinpassesfalse.test.test_importlibgoes fromFAILED (failures=1, skipped=65)toOK (skipped=65)over 1346 tests, and its baseline entry moves off the staleIMPORTERROR, so those tests are gated from now on.Two divergences are deliberately left, both stated in the commit:
set_sys_modulestays on this path. cpython3.14 does not insert on a direct_imp.create_builtincall — a seconddel sys.modules[name]raisesKeyErrorthere and succeeds here — but removing it segfaults, so the rooting it provides is still load-bearing.BuiltinImporter.__module__still reads_frozen_importlibwhere cpython3.14 readsimportlib._bootstrap. It survives this fix, so it is a separate defect rather than another symptom.A main regression that main has already fixed
Earlier CI runs on this PR are red on
test.test_datetime:That is not this branch's, and it is already gone. Measured with the same five commits:
e4f299c1159PASS 46, FAIL 0, no regressions1391a9656dd(#1091)test_datetimeFAIL 3/3aaefe84f9a3(adds9d2fff92649= #1063)test_datetimePASS 3/3The gate had not been green on any main sha between
678319fcf23and9d2fff92649, and neither07e6c4aff3e(#1074) nor1391a9656dd(#1091) ever got a CI run of its own. #1063 closed it — it stopstry_walker_specialize_newtuple_objectdeclining arity 2, so a BUILD_TUPLE pair is now the canonical array-backed tuple rather than amakespecialisedtuple2pair, and it teachestry_walker_inline_resolved_user_callto seed a*argscallee's vararg local.The PR's first CI run merged against main at
1391a9656dd, before #1063 landed. The current tip carries the fix.Verification
All of the below were run at the branch tip, each after a
pyre/scripts/extract-llbc.pyextraction taken after the last Rust edit.check.py --backend dynasmcheck.py --backend craneliftcheck.py --backend wasmextra_tests/parity_tests/run.pycargo test -p pyre-object -p pyre-jit-trace -p pyre-interpreter --features dynasmcargo fmt --all -- --checkcpython_tests/run.py --backend dynasm --baseline … --jobs 3 --timeout 300PASS 46, FAIL 0, no regressionsat the earlier base; at the current base the only failure is thetest.test_datetimemain regression described aboveNew parity fixtures, each
cpython=OK dynasm=OK cranelift=OK:exception_getattribute_override.py,object_init_text_signature.py,frame_clear_finalization.py,dict_popitem_strategy.py,builtin_module_loader_spec.py.Two
check.pydynasm runs failed a wall-clock ratio gate on this shared host and were attributed by re-running each alone on the same binary:synth/dir_full_mropassed 3/3 with the ratio reading 4.3x, 2.3x and 1.4x across identical runs (its whole budget is 0.02s), andsynth/range_ctor_in_looppassed 2/2 with a direct timed run measuring 5.97s user CPU against 10.19s wall. Neither is a correctness or jitstats diff, and the two backends that passed 391/391 and 387/387 saw neither on the same tree.The branch was rebased onto a newer
origin/maintwice during this work, which invalidates locally built binaries and the LLBC extraction each time; the numbers above were taken after re-extraction at their respective bases, and the CI run on this push is the authority for the current one.Summary by CodeRabbit
Bug Fixes
dict.popitem()behavior, including LIFO ordering, module namespaces, subclasses, and unusual attribute names.KeyboardInterruptnow exits with standard interrupt status.Compatibility
Tests