builtins/eval: de-leak the __pyre_kw__ kwargs marker ABI and propagate raising key __eq__ from LOAD_GLOBAL - #821
Conversation
Builtin keyword calls pack a trailing marker dict keyed by "__pyre_kw__" holding the marker sentinel. Three changes to that handling: - Route the fixed-name lookups (print end/sep/file/flush, kwarg_get, has_builtin_kwargs; __build_class__ __pyre_kw__/metaclass; type_methods format; _pickle import-map; math prod/nextafter __pyre_kw__/start/steps) through borrow-based w_dict_getitem_str instead of w_dict_lookup(w_str_new(name)); the latter allocated an immortal W_UnicodeObject + Wtf8Buf wrapper per lookup. Both are the same Option-returning strategy dispatch, and kwargs dicts hold exact-str keys, so the borrowed probe is behavior-preserving. - Cache the constant "__pyre_kw__" key behind a OnceLock (w_kw_marker_key) instead of minting a fresh immortal w_str_new per keyworded call in call_with_kwargs and pack_pyre_kwargs. The marker dict is collectable and drops only the borrowed key pointer on collection. - Gate nextafter's kwargs detection on is_kw_marker_sentinel over the value, matching prod; the presence-only .is_some() check misclassified a positional dict carrying a "__pyre_kw__" string key as keyword arguments. Assisted-by: Claude
…_str load_global_value (interpreter) and jit_load_name_from_namespace (the JIT LOAD_GLOBAL extern) both split the globals lookup by hand: an is_dict fast path through the unchecked w_dict_getitem_str and a dict-subclass path through finditem_str. The unchecked probe returns a plain Option, so a raising key __eq__ during the bucket comparison (a stored non-string key that hash-collides with the looked-up name) was swallowed as a miss instead of propagating — and the outcome differed depending on whether the frame was traced. finditem_str now takes the borrowed-string shortcut for shortcut dicts itself and drains the dict-key error, so route both cases through finditem_str(w_globals, name), matching pyopcode.py:958-960. Assisted-by: Claude
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
WalkthroughThe interpreter replaces temporary string-object dictionary lookups with direct string-key retrieval, centralizes the kwargs-marker key, and routes global lookups through unified mapping dispatch. ChangesDictionary lookup consolidation
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🤖 Codex parity reviewStatic analysis of this diff vs the local RPython/PyPy sources (commit 0001c7c). Files in the reviewed diff1. Regressions to PyPy parity introduced by this patchNone. 2. Other mismatches introduced by this patchNone. 3. Pre-existing mismatches (already present before this patch)None. 4. Structural adaptations
|
Summary
Two related changes to builtin keyword handling and
LOAD_GLOBALname resolution, motivated by a per-call immortal-allocation (RSS) leak and a dict-key__eq__error-propagation gap.builtins: de-leak and harden the__pyre_kw__kwargs marker ABIBuiltin keyword calls pack a trailing marker dict keyed by
"__pyre_kw__"holding the marker sentinel.printend/sep/file/flush,kwarg_get,has_builtin_kwargs;__build_class____pyre_kw__/metaclass;type_methodsformat;_pickleimport-map;mathprod/nextafter__pyre_kw__/start/steps) usedw_dict_lookup(dict, w_str_new(name)), allocating an immortalW_UnicodeObject+Wtf8Bufwrapper per lookup solely to key the dict. Routed through borrow-basedw_dict_getitem_str, which hashes the&strdirectly. Both are the sameOption-returning strategy dispatch, and kwargs dicts hold exact-str keys, so the swap is behavior-preserving.call_with_kwargs/pack_pyre_kwargsminted a fresh immortalw_str_new("__pyre_kw__")per keyworded call. Cached once behind aOnceLock(w_kw_marker_key). The marker dict is collectable and drops only the borrowed key pointer on collection.nextafterdetected kwargs with a presence-only.is_some()check on the"__pyre_kw__"key, misclassifying a positional dict that merely carries such a string key. Gated onis_kw_marker_sentinelover the value, matchingprod.eval/runtime_ops: collapse theLOAD_GLOBALglobals lookup tofinditem_strload_global_value(interpreter) andjit_load_name_from_namespace(the JITLOAD_GLOBALextern) both split the globals lookup by hand — anis_dictfast path through the uncheckedw_dict_getitem_strand a dict-subclass path throughfinditem_str. The unchecked probe returns a plainOption, so a raising key__eq__during the bucket comparison (a stored non-string key that hash-collides with the looked-up name) was swallowed as a miss instead of propagating — and the outcome differed depending on whether the frame was traced.finditem_strnow takes the borrowed-string shortcut for shortcut dicts itself and drains the dict-key error, so both cases route throughfinditem_str(w_globals, name), matchingpyopcode.py:958-960. The hand-rolledw_dict_getitem_strfast path was a pre-shortcut workaround that is now redundant.Verification
python pyre/check.py --backend dynasm(referencepython3.14): 329/329 PASS.len(s)× 2M, JIT off,PYRE_GC_INTERP=1— RSS flat at ~47MB (200k == 2M); the wrapper allocation is gone andfinditem_str's borrowed shortcut keeps the builtins/globals lookup allocation-free.__eq__, injected intobuiltins.__dict__/globals()and hit viaLOAD_GLOBAL, now propagatesValueError(matchingpython3.14) instead of being swallowed as a miss — verified on both the interpreter and JIT paths (JIT on/off × globals/builtins).math.nextafter(1.0, 2.0, {"__pyre_kw__": 1})now raisesTypeError(matchingpython3.14) instead of being misread as keyword arguments.🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Performance