Skip to content

import: run source importlib, module specs, and authoritative sys.path - #731

Merged
youknowone merged 7 commits into
mainfrom
import
Jul 23, 2026
Merged

import: run source importlib, module specs, and authoritative sys.path#731
youknowone merged 7 commits into
mainfrom
import

Conversation

@youknowone

@youknowone youknowone commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Foundational import-compatibility work toward passing CPython's test_import
and test_importlib.

Run the real source importlib

Remove the native importlib / importlib.machinery / importlib.abc stubs
that injected fake classes, so importlib/__init__.py runs, binds
__import__, and machinery/abc re-export the real _bootstrap /
_bootstrap_external classes. Set __spec__ / __loader__ / __package__
on builtin modules (app-level BuiltinImporter.find_spec +
_init_module_attrs) and on source modules
(_bootstrap_external._fix_up_module).

Authoritative sys.path

Imports now read the live Python sys.path list, so user mutations and
PYTHONPATH take effect — the same precedence check_sys_modules already
gives the live sys.modules dict.

  • python_sys_path_dirs returns None only while sys does not exist yet;
    once it does the Python list is authoritative even when empty, so
    del sys.path / sys.path.clear() make imports fail instead of resurrecting
    a native seed.
  • The native SYS_PATH seed is now only a pre-sys staging buffer. It is
    flushed into sys.path when the sys module is created (forcing stdlib
    detection first so the stdlib is present even under -S / -S -P), then
    mutated in place afterward — sys.path keeps a single stable list object.
  • The one-way sync_python_sys_path mirror (which replaced the list object)
    and its -m / -c / REPL call sites are deleted.

implementation.cache_tag

Drop the dot (pyre-3.14pyre-314) so PEP 3147 source_from_cache
dot-count parsing works.

_io.IncrementalNewlineDecoder

Real implementation (was a stub returning None) so
_bootstrap_external.decode_source and universal-newline reads work.

Verification

  • check.py: dynasm + cranelift 289/289, both backends.
  • sys.path behavior checked against CPython/PyPy: startup population,
    list-object identity stability, insert/append, authoritative-empty
    (del / clear), PYTHONPATH, -m, and -S / -S -P stdlib presence.

A long tail remains before the suites fully pass (namespace packages, frozen
machinery, LazyLoader, resources/zipfile).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Improved module import behavior and search-path handling, including support for environment-provided paths.
    • Added universal newline decoding support to the _io module.
    • sys.path is now initialized and updated more reliably during startup and interactive use.
  • Bug Fixes

    • Improved module metadata such as __spec__, __loader__, __package__, and __file__.
    • Fixed builtin module resolution and import behavior.
    • Updated the Python implementation cache tag to pyre-314.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Import initialization now applies module metadata fixups and uses live sys.path entries for seeding and lookup. The _io module adds IncrementalNewlineDecoder with universal-newline translation and state handling.

Changes

Import runtime changes

Layer / File(s) Summary
Module metadata and builtin registration
pyre/pyre-interpreter/src/importing.rs
Frozen importlib registrations are removed; builtin and source modules receive importlib metadata fixups, and builtin loading rereads modules after startup.
Live sys.path seeding and lookup
pyre/pyre-interpreter/src/importing.rs, pyre/pyre-interpreter/src/module/sys/vm.rs, pyre/pyrex/src/lib.rs, pyre/pyrex/src/repl.rs
PYTHONPATH and runtime seeds populate sys.path, post-initialization updates modify the live list, and module lookup reads string entries from it. Execution paths import sys before site setup. The implementation cache tag changes to pyre-314.

_io newline decoder

Layer / File(s) Summary
Incremental newline decoding
pyre/pyre-interpreter/src/module/_io/_io_app.py, pyre/pyre-interpreter/src/module/_io/mod.rs
Adds IncrementalNewlineDecoder with translation, newline tracking, pending-CR handling, state methods, and _io export wiring.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant run_source
  participant importhook
  participant sys
  participant importer
  run_source->>importhook: import sys before site initialization
  importhook->>sys: create seeded sys.path
  sys->>importer: expose live path entries
  importer-->>run_source: resolve modules from sys.path
Loading

Possibly related PRs

Poem

A rabbit hops through sys.path bright,
While loaders dress modules just right.
Newlines softly turn,
Pending CRs learn,
And imports bloom in moonlit light.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main importlib, module spec, and sys.path changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch import

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ee0a9ba27a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pyre/pyre-interpreter/src/importing.rs Outdated
// sandbox interpreter takes its search path from the controller, so it
// does not read the host environment here.
#[cfg(not(feature = "sandbox"))]
if let Ok(pythonpath) = host_os::var("PYTHONPATH") {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor -E/-I when seeding PYTHONPATH

In non-sandbox launches with PYTHONPATH set, this block appends those directories even when the launcher was invoked with -E or -I; -I sets ignore_environment too, and the help text advertises -E as ignoring PYTHON* environment variables. As a result PYTHONPATH=/tmp/shadow pyre -E -c 'import victim' (or -I) can still import from the environment-controlled directory before stdlib/user code runs, defeating the isolation flag. Please gate this on !ignore_environment_flag() (or avoid reading PYTHONPATH when the launcher recorded -E/-I).

Useful? React with 👍 / 👎.

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

🤖 Codex parity review

Static analysis of this diff vs the local RPython/PyPy sources (commit 4690e79).
Updated: 2026-07-23T03:35:00.289Z

Files in the reviewed diff
pyre/pyre-interpreter/src/importing.rs
pyre/pyre-interpreter/src/module/_io/_io_app.py
pyre/pyre-interpreter/src/module/_io/mod.rs
pyre/pyre-interpreter/src/module/sys/vm.rs
pyre/pyrex/src/lib.rs
pyre/pyrex/src/repl.rs

1. Regressions to PyPy parity introduced by this patch

  • pyre/pyre-interpreter/src/importing.rs:2142 ↔ pypy/interpreter/baseobjspace.py:721 and pyre/pyrex/src/lib.rs:741 ↔ pypy/interpreter/baseobjspace.py:721: _bootstrap._install() is now run twice for -m: once automatically after loading importlib._bootstrap, then again by init_importlib_bootstrap. PyPy installs _frozen_importlib once during space setup. The second run appends duplicate BuiltinImporter, FrozenImporter, and PathFinder entries to import machinery lists.

2. Other mismatches introduced by this patch

  • pyre/pyre-interpreter/src/importing.rs:1124 ↔ lib-python/3/importlib/_bootstrap.py:924: builtin startup runs before _init_module_attrs; upstream creates a module with its spec attributes and only then calls loader.exec_module. A builtin initializer that observes __spec__, __loader__, or __package__ therefore sees different state.

  • pyre/pyre-interpreter/src/importing.rs:990 ↔ lib-python/3/importlib/_bootstrap.py:745: errors from BuiltinImporter.find_spec, _init_module_attrs, and their attribute lookups are discarded (let Ok(...) else { return Ok(()) }), whereas upstream propagates errors except for assignment failures explicitly caught inside _init_module_attrs. Monkey-patching either bootstrap helper can therefore make a Pyre builtin import succeed silently where PyPy raises.

  • pyre/pyre-interpreter/src/module/_io/_io_app.py:395 ↔ pypy/module/_io/interp_textio.py:51: IncrementalNewlineDecoder intentionally omits io_check_errors’ development-mode error-handler lookup. With dev mode enabled, PyPy rejects an unknown errors handler during construction; Pyre accepts it until/if an underlying decoder uses it.

3. Pre-existing mismatches (already present before this patch)

None.

4. Structural adaptations

  • pyre/pyre-interpreter/src/importing.rs:524 ↔ pypy/interpreter/baseobjspace.py:721: Pyre loads importlib and bootstrap modules from source files rather than installing PyPy’s compiled _frozen_importlib mixed module. This is a bootstrap/compiler adaptation.

  • pyre/pyre-interpreter/src/importing.rs:1789 ↔ lib-python/3/importlib/_bootstrap.py:1561: the Rust-native resolver reads string filesystem entries from sys.path directly, rather than executing the Python sys.meta_path / sys.path_hooks protocol. This is a fundamental native-importer adaptation; non-filesystem import hooks remain unsupported.

  • pyre/pyre-interpreter/src/module/_io/mod.rs:859 ↔ pypy/module/_io/moduledef.py:31: IncrementalNewlineDecoder is installed from app-level Python source rather than as PyPy’s interplevel W_IncrementalNewlineDecoder type. The exposed API is structurally adapted to the Rust object model.

- Drop the native importlib / importlib.machinery / importlib.abc module
  registrations so the on-disk importlib package loads from source: its
  __init__ binds __import__, and machinery/abc re-export the real finder,
  loader and ModuleSpec classes from the frozen _bootstrap /
  _bootstrap_external, which already carry the full surface. The native stubs
  only injected placeholder object classes that shadowed them.

- Give builtin modules __spec__/__loader__/__package__ from
  BuiltinImporter.find_spec + _bootstrap._init_module_attrs, and source
  modules their __spec__/__loader__/__file__/__cached__ from
  _bootstrap_external._fix_up_module. Both are gated on the importlib
  bootstrap being wired and are best-effort so a module imported while the
  bootstrap itself is still executing falls back to the previous None seeding;
  _bootstrap._setup fixes up the pre-wire builtins in bulk. load_part re-reads
  the builtin from sys.modules after the app-level call, which can relocate it.

- find_in_sys_path reads the live Python sys.path first so sys.path mutations
  and PYTHONPATH are honored, then appends the native SYS_PATH seed
  (deduplicated) to keep the stdlib and defaults searchable and to cover the
  pre-sync bootstrap window. init_sys_path seeds PYTHONPATH entries.

Assisted-by: Claude
'pyre-3.14' put a dot inside the cache tag, so _bootstrap_external's PEP 3147
source_from_cache dot-count parse rejected the resulting __pycache__ names.
Use 'pyre-314', matching the dot-free tag convention.

Assisted-by: Claude
It was a stub returning None, so _bootstrap_external.decode_source did
None.decode and every app-level SourceLoader.get_source raised. Add the real
class (a standalone type, matching the C _io type rather than a
codecs.IncrementalDecoder subclass) to _io_app.py and register it.

Assisted-by: Claude
python_sys_path_dirs now returns Option: None only while the sys module
does not exist yet (the pre-sys bootstrap window falls back to the native
SYS_PATH seed), otherwise Some(dirs) — a missing, non-list, or empty
sys.path searches nothing. Add create_sys_path_list to build the initial
Python list from the native seed; not yet wired to sys-module creation.

Assisted-by: Claude
Complete the authoritative sys.path change.

- create_sys_path_list flushes the native SYS_PATH seed into sys.path when the
  sys module is created (register_module), forcing stdlib detection first so the
  stdlib is on sys.path even under -S / -S -P, before any user code reads it.
- add_sys_path appends to the live list in place once sys exists (GC-pinned,
  deduplicated) and only stages in the seed before that, so sys.path keeps a
  single stable list object and mutations are never lost.
- Delete the one-way sync_python_sys_path mirror and its -m / -c / REPL call
  sites; it replaced the list object and is superseded by the flush plus append.

Assisted-by: Claude
Windows registers the posix builtin (never nt), so os.path is posixpath
and site.removeduppaths() rewrites drive-letter sys.path entries into
<cwd>/D:... garbage at startup, making every stdlib source import fail.
Until nt registration lands, a live-list miss on Windows retries the
native seed so the stdlib stays importable.

Assisted-by: Claude
… validation

- Skip PYTHONPATH under -E / -I (ignore_environment); split it on the
  platform path-list separator and keep empty components, extending the
  seed with the raw split (app_main.setup_and_fix_paths).
- Propagate errors raised by _fix_up_module instead of mapping them to
  the None-seeding fallback; the getattr fallback for the partially
  initialised bootstrap window stays.
- IncrementalNewlineDecoder: accept errors=None as strict, reject
  non-str errors and unencodable handler names, coerce translate via
  __index__, and raise TypeError when a wrapped decoder returns a
  non-str result (interp_textio.py); pass final positionally.

Assisted-by: Claude

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4690e79804

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

use pyre_object::gc_roots::{pin_root, push_roots, shadow_stack_get, shadow_stack_len};

let entry = dir.to_string_lossy();
if get_sys_module("sys").is_none() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Stage stdlib while sys.path is still absent

When create_sys_path_list() forces ensure_stdlib_path() during sys.register_module, load_part has already inserted sys into sys.modules but module_ns_store(..., "path", ...) has not run yet. This condition therefore sends the stdlib add down the live-sys.path branch, where the missing path entry returns without staging anything; ensure_stdlib_path has already set its DONE flag, so normal -c/script/-m startup leaves sys.path without the stdlib and import site/stdlib imports fail. Treat a sys module with no list-valued path as still pre-flush and append to SYS_PATH instead.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pyre/pyre-interpreter/src/importing.rs`:
- Around line 1173-1180: Update the PYTHONPATH handling in the
environment-loading block to skip processing when the retrieved value is empty,
while continuing to split and preserve empty components for non-empty values.
Keep the existing platform-specific separator and path extension behavior
unchanged.

In `@pyre/pyre-interpreter/src/module/_io/_io_app.py`:
- Around line 377-417: Update the IncrementalNewlineDecoder docstring to
identify importlib._bootstrap_external.decode_source as the decoder=None caller
instead of TextIOWrapper. In IncrementalNewlineDecoder.decode, invoke the
wrapped decoder’s decode method with final passed by keyword as final=final,
preserving the existing decoder=None path.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 35f7883d-73e3-4437-85ba-8a1151136434

📥 Commits

Reviewing files that changed from the base of the PR and between a1de59b and 4690e79.

📒 Files selected for processing (6)
  • pyre/pyre-interpreter/src/importing.rs
  • pyre/pyre-interpreter/src/module/_io/_io_app.py
  • pyre/pyre-interpreter/src/module/_io/mod.rs
  • pyre/pyre-interpreter/src/module/sys/vm.rs
  • pyre/pyrex/src/lib.rs
  • pyre/pyrex/src/repl.rs
💤 Files with no reviewable changes (1)
  • pyre/pyrex/src/repl.rs

Comment on lines +1173 to +1180
if !ignore_environment_flag() {
if let Ok(pythonpath) = host_os::var("PYTHONPATH") {
let sep = if cfg!(windows) { ';' } else { ':' };
// Empty components are preserved — an empty `sys.path` entry
// denotes the current directory (app_main.setup_and_fix_paths
// extends with the raw split).
path.extend(pythonpath.split(sep).map(PathBuf::from));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

PyPy app_main setup_and_fix_paths PYTHONPATH empty string handling

💡 Result:

In Python and PyPy environments, the handling of the PYTHONPATH environment variable follows the standard convention where an empty string or the absence of the variable results in different behaviors depending on the implementation version [1]. Historically, and consistent with standard CPython behavior, an empty entry in the PYTHONPATH environment variable (or an explicitly empty PYTHONPATH) is often interpreted as the current directory (".") [1][2]. While there have been community proposals and discussions to change this behavior—specifically to ignore empty entries to enhance security and prevent unintentional inclusion of the current directory—standard Python implementations generally maintain this long-standing interpretation [1][2]. Regarding PyPy specifically, PyPy aims to be highly compatible with CPython [3]. Its path initialization logic, which performs functions analogous to CPython's getpath, follows the standard Python path configuration rules [4][5][6]. Consequently, if you are working with PyPy's initialization or path-fixing routines (such as those mirroring app_main logic), you should expect the standard Python behavior: 1. PYTHONPATH: The variable is parsed using the OS-specific path delimiter (e.g., ':' on POSIX, ';' on Windows) [5]. 2. Empty Entries: An empty string entry within this delimited list typically resolves to the current working directory [1][2]. If you are implementing custom setup or path-fixing logic, it is recommended to explicitly sanitize paths by filtering out empty strings before adding them to sys.path, as this avoids security risks associated with the current directory being implicitly included in the module search path [2].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file =="
fd -a 'importing.rs$' . || true

echo "== target surrounding =="
if [ -f pyre/pyre-interpreter/src/importing.rs ]; then
  wc -l pyre/pyre-interpreter/src/importing.rs
  sed -n '1145,1205p' pyre/pyre-interpreter/src/importing.rs | nl -ba -v1145
fi

echo "== PYTHONPATH references in repo =="
rg -n "PYTHONPATH|setup_and_fix_paths|pythonpath" pyre pyre-interpreter . --glob '!target/**' --glob '!node_modules/**' || true

echo "== Rust split behavior (host compiler only; no repo code executed) =="
rustc --edition=2021 --crate-type=lib - <<'RS'
pub fn rust_split_behaviour() -> usize {
    let sep = ':';
    let mut path = Vec::new();
    let pythonpath = String::from("");
    path.extend(pythonpath.split(sep).map(String::from));
    path.len()
}

pub fn rust_nonempty_empty_component_behaviour() -> Vec<String> {
    let pythonpath = String::from(":a");
    pythonpath.split(':').map(String::from).collect()
}
RS

Repository: youknowone/pyre

Length of output: 339


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== target surrounding =="
sed -n '1145,1205p' pyre/pyre-interpreter/src/importing.rs

echo "== PYTHONPATH references in repo =="
rg -n "PYTHONPATH|setup_and_fix_paths|pythonpath" pyre pyre-interpreter . --glob '!target/**' --glob '!node_modules/**' || true

echo "== Python stdlib behavior probe =="
python3 - <<'PY'
for s, sep in [("", ":"), (":a", ":"), ("a:", ":"), ("a:b", ")"):]]:
    print(repr(s), sep, "=>", repr(s.split(sep)))
PY

Repository: youknowone/pyre

Length of output: 17781


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== PyPy app_main setup_and_fix_paths lines 350-375 =="
sed -n '350,375p' pypy/interpreter/app_main.py

echo "== CPython test_empty_PYTHONPATH_issue16309 =="
sed -n '415,440p' lib-python/3/test/test_cmd_line.py

echo "== CPython test_getpath empty PYTHONPATH cases =="
sed -n '590,630p' lib-python/3/test/test_getpath.py
sed -n '860,945p' lib-python/3/test/test_getpath.py

echo "== Python string splitting behavior probe =="
python3 - <<'PY'
for s, sep in [("", ":"), (":a", ":"), ("a:", ":"), ("a:b", "abc")]:
    print(repr(s), sep, "=>", repr(s.split(sep)))
PY

Repository: youknowone/pyre

Length of output: 6928


Skip an empty PYTHONPATH value before splitting.

host_os::var("PYTHONPATH") returns Ok("") when the variable is set but empty, and "".split(sep) produces [""], which adds cwd as an empty sys.path entry. PyPy’s app_main.setup_and_fix_paths and CPython’s documented behavior skip an empty PYTHONPATH entirely while preserving empty components inside a non-empty value.

🐛 Proposed fix
         #[cfg(not(feature = "sandbox"))]
         if !ignore_environment_flag() {
-            if let Ok(pythonpath) = host_os::var("PYTHONPATH") {
+            if let Ok(pythonpath) = host_os::var("PYTHONPATH") {
+                // An empty value is skipped entirely (app_main guards
+                // `if pythonpath:`); empty components inside a non-empty
+                // value are still preserved.
+                if !pythonpath.is_empty() {
                     let sep = if cfg!(windows) { ';' } else { ':' };
                     // Empty components are preserved — an empty `sys.path` entry
                     // denotes the current directory (app_main.setup_and_fix_paths
                     // extends with the raw split).
                     path.extend(pythonpath.split(sep).map(PathBuf::from));
+                }
             }
         }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if !ignore_environment_flag() {
if let Ok(pythonpath) = host_os::var("PYTHONPATH") {
let sep = if cfg!(windows) { ';' } else { ':' };
// Empty components are preserved — an empty `sys.path` entry
// denotes the current directory (app_main.setup_and_fix_paths
// extends with the raw split).
path.extend(pythonpath.split(sep).map(PathBuf::from));
}
if !ignore_environment_flag() {
if let Ok(pythonpath) = host_os::var("PYTHONPATH") {
// An empty value is skipped entirely (app_main guards
// `if pythonpath:`); empty components inside a non-empty
// value are still preserved.
if !pythonpath.is_empty() {
let sep = if cfg!(windows) { ';' } else { ':' };
// Empty components are preserved — an empty `sys.path` entry
// denotes the current directory (app_main.setup_and_fix_paths
// extends with the raw split).
path.extend(pythonpath.split(sep).map(PathBuf::from));
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pyre/pyre-interpreter/src/importing.rs` around lines 1173 - 1180, Update the
PYTHONPATH handling in the environment-loading block to skip processing when the
retrieved value is empty, while continuing to split and preserve empty
components for non-empty values. Keep the existing platform-specific separator
and path extension behavior unchanged.

Source: Coding guidelines

Comment on lines +377 to +417
`_io.IncrementalNewlineDecoder` is a standalone type, not a
`codecs.IncrementalDecoder` subclass; `decode_source` and `TextIOWrapper`
construct it with `decoder=None` to translate an already-decoded string.
"""

_LF = 1
_CR = 2
_CRLF = 4

def __init__(self, decoder, translate, errors="strict"):
if errors is None:
errors = "strict"
elif not isinstance(errors, str):
raise TypeError(
"TextIOWrapper() argument 'errors' must be str or None, not %s"
% type(errors).__name__
)
else:
# io_check_errors minus the dev-mode handler lookup — a codecs
# import here would recurse through decode_source.
errors.encode("utf-8", "strict")
if not isinstance(translate, int):
try:
translate = translate.__index__()
except AttributeError:
raise TypeError(
"'%s' object cannot be interpreted as an integer"
% type(translate).__name__
) from None
self.errors = errors
self.translate = translate
self.decoder = decoder
self.seennl = 0
self.pendingcr = False

def decode(self, input, final=False):
# decode input (with the eventual \r from a previous pass)
if self.decoder is None:
output = input
else:
output = self.decoder.decode(input, final)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Find construction sites and confirm decoder=None usage + positional final compatibility.
rg -nP -C3 '\bIncrementalNewlineDecoder\s*\(' pyre/pyre-interpreter/src/module/_io/
rg -nP -C3 'decode_source' pyre/pyre-interpreter/src/module/_io/
# Inspect any wrapped decoder .decode signatures referenced by TextIOWrapper
rg -nP -C2 '\.decode\s*\(' pyre/pyre-interpreter/src/module/_io/_io_app.py

Repository: youknowone/pyre

Length of output: 1624


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== files =="
git ls-files pyre/pyre-interpreter/src/module/_io/

echo "== IncrementalNewlineDecoder definitions/usages across repo =="
rg -nP -C3 '\bIncrementalNewlineDecoder\s*\(' .

echo "== decode_source references across repo =="
rg -nP -C3 '\bdecode_source\b' .

echo "== TextIOWrapper definition area =="
fd -a '(_io_app\.py$|textio.*\.py$|textiowrapper.*\.py$)' pyre/pyre-interpreter/src/module/_io/ | sed 's#^\./##'
wc -l pyre/pyre-interpreter/src/module/_io/_io_app.py
sed -n '300,460p' pyre/pyre-interpreter/src/module/_io/_io_app.py

echo "== decoder.decode call patterns in _io_app =="
rg -n 'decoder\.decode|\.decode\(' pyre/pyre-interpreter/src/module/_io/_io_app.py

Repository: youknowone/pyre

Length of output: 18931


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== _io_app TextIOWrapper area =="
sed -n '220,310p' pyre/pyre-interpreter/src/module/_io/_io_app.py

echo "== _pyio decoder creation and TextIOWrapper decode call =="
sed -n '1948,2305p' lib-python/3/_pyio.py

echo "== stdlib incremental decoder signatures =="
python3 - <<'PY'
import codecs, io, inspect
for cls in [codecs.IncrementalDecoder, io.TextIOWrapper]:
    print(f"{cls.__module__}.{cls.__name__}: {inspect.signature(cls.__init__ or cls)}")
    if hasattr(cls, 'decode'):
        print(f"  decode signature: {inspect.signature(cls.decode)}")
PY

echo "== PyPy _io app IncrementalNewlineDecoder construction sites =="
rg -n -C4 'IncrementalNewlineDecoder\(' pyre/pyre-interpreter/src/module/_io pyre/pyre-interpreter

Repository: youknowone/pyre

Length of output: 15248


Fix the stale docstring and keep the wrapped decoder call compatible.

TextIOWrapper does not construct IncrementalNewlineDecoder(None, ...): the normal read path constructs codecs.getincrementaldecoder(encoding)(errors) first and then wraps that decoder. Update this docstring to use importlib._bootstrap_external.decode_source as the decoder=None caller, and pass final through as final=final for the wrapped incremental decoder rather than relying on positional-only semantics.

🧰 Tools
🪛 Ruff (0.15.21)

[warning] 386-386: Missing return type annotation for special method __init__

Add return type annotation: None

(ANN204)


[warning] 391-392: Use format specifiers instead of percent format

Replace with format specifiers

(UP031)


[warning] 403-404: Use format specifiers instead of percent format

Replace with format specifiers

(UP031)


[warning] 412-412: Missing return type annotation for private function decode

(ANN202)


[error] 412-412: Function argument input is shadowing a Python builtin

(A002)


[warning] 412-412: Boolean default positional argument in function definition

(FBT002)


[warning] 414-417: Use ternary operator output = input if self.decoder is None else self.decoder.decode(input, final) instead of if-else-block

Replace if-else-block with output = input if self.decoder is None else self.decoder.decode(input, final)

(SIM108)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pyre/pyre-interpreter/src/module/_io/_io_app.py` around lines 377 - 417,
Update the IncrementalNewlineDecoder docstring to identify
importlib._bootstrap_external.decode_source as the decoder=None caller instead
of TextIOWrapper. In IncrementalNewlineDecoder.decode, invoke the wrapped
decoder’s decode method with final passed by keyword as final=final, preserving
the existing decoder=None path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant