gh-128067: Fix pyrepl overriding printed output without newlines #138732

JanEricNitschke · 2025-09-10T08:58:58Z

I tried a bit to debug this and narrowed it down to these two lines for windows and unix terminals respectively:

Because while properly running the repl in unix doesnt cause the issue, running the test i added also fails there before these changes.

I am not 100% sure if this is the base and 100% correct way, but it causes the test to pass on unix and fixes the issue on windows for me.

I also attempted to try to be able to run these tests for windows (because thats where we actually saw the issue) but didnt manage unfortunately.

With the fix in the unix console:

 python -m coverage run --pylib --branch --source=pyrepl Lib/test/regrtest.py  test_pyrepl -m test_no_newline
Using random seed: 4126788206
0:00:00 load avg: 0.17 Run 1 test sequentially in a single process
0:00:00 load avg: 0.17 [1/1] test_pyrepl
0:00:04 load avg: 0.17 [1/1] test_pyrepl passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 4.7 sec
Total tests: run=1 (filtered)
Total test files: run=1/1 (filtered)
Result: SUCCESS

Without:

 python -m coverage run --pylib --branch --source=pyrepl Lib/test/regrtest.py  test_pyrepl -m test_no_newline
Using random seed: 3832564122
0:00:00 load avg: 0.30 Run 1 test sequentially in a single process
0:00:00 load avg: 0.30 [1/1] test_pyrepl
test test_pyrepl failed -- Traceback (most recent call last):
  File "/mnt/d/Programming/Projects/cpython/Lib/test/support/__init__.py", line 2923, in wrapper
    return func(*args, **kwargs)
  File "/mnt/d/Programming/Projects/cpython/Lib/test/test_pyrepl/test_pyrepl.py", line 1819, in test_no_newline
    self.assertIn(expected_output_sequence, cleaned_output)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'Something pretty long>>> exit()' not found in 'print(\'Something pretty long\', end=\'\')\r\nexit()\r\nPython 3.15.0a0 (heads/main:c919d02ede, Sep  6 2025, 15:55:27) [GCC 13.3.0] on linux\r\nType "help", "copyright", "credits" or "license" for more information.\r\n\x1b[1A\n>>> print(\'Something pretty long\', end=\'\')\x1b[42D\n\rSomething pretty long\x1b[1A\n>>> exit()\x1b[10D\n\r'

0:00:02 load avg: 0.30 [1/1/1] test_pyrepl failed (1 failure)

== Tests result: FAILURE ==

1 test failed:
    test_pyrepl

Total duration: 2.0 sec
Total tests: run=1 (filtered) failures=1
Total test files: run=1/1 (filtered) failed=1
Result: FAILURE

Issue: [Windows] Python 3.13 New REPL prompt swallows characters with print(..., end=" ") #128067

bedevere-app · 2025-09-10T08:59:04Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

bedevere-app · 2025-09-10T11:41:31Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

bedevere-app · 2025-09-10T13:13:49Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

chris-eibl · 2025-09-14T13:07:43Z

As written in the issue, this works fine on

Windows 10 (in a legacy console and the new Windows terminal )
WSL Ubuntu-24.04 (again in a legacy console and the new Windows terminal)
native Ubuntu 24.0.4.3 LTS

Lib/test/test_pyrepl/test_pyrepl.py

chris-eibl · 2025-09-14T13:17:29Z

Lib/_pyrepl/windows_console.py

-            self._move_relative(0, len(self.screen) - 1)
-            self.__write("\n")
+            if self.screen:
+                self._move_relative(0, len(self.screen) - 1)


Spot on, len(self.screen) - 1 should not get negative here.

chris-eibl · 2025-09-14T13:20:48Z

Please add a News entry, see Quick reference #8 in the devguide.

chris-eibl · 2025-09-14T13:22:40Z

Please merge with latest default, because the pyrepl Windows tests are not run otherwise (#138801).

JanEricNitschke · 2025-09-14T13:26:59Z

Please merge with latest default, because the pyrepl Windows tests are not run otherwise (#138801).

So merge with current main and "merge" not "rebase", right? So basically just clicking the "Update branch" button here.

chris-eibl · 2025-09-14T13:42:16Z

I suggest to add tests like

def test_no_newline(self, _os_write):
    code = "1"
    events = code_to_events(code)
    _, con = handle_events_unix_console(events)
    self.assertNotIn(call(ANY, b'\n'), _os_write.mock_calls)
    con.restore()

def test_newline(self, _os_write):
    code = "\n"
    events = code_to_events(code)
    _, con = handle_events_unix_console(events)
    _os_write.assert_any_call(ANY, b"\n")
    con.restore()

in test_unix_console.py and test_windows_console.py. Otherwise, for Windows we won't have any test coverage.

I think they are best placed before test_simple_addition?

Both tests fail before the PR and succeed with the PR.

So these tests might be enough, but of course your current test is more explict. But since Windows doesn't have the pty (Pseudo-terminal utilities), I don't know how to do a similar test there.

The filtering of the many escape sequences might induce some maintenance burden, but I don't think it is a big issue.

chris-eibl · 2025-09-14T13:44:55Z

Please merge with latest default, because the pyrepl Windows tests are not run otherwise (#138801).

So merge with current main and "merge" not "rebase", right? So basically just clicking the "Update branch" button here.

Correct. For reviewers it is easier to follow without squashing.

For the final merge from your fork to the upstream branch, squashing is used, anyway.

JanEricNitschke · 2025-09-14T14:06:49Z

Added the tests. I put the news entry under Windows because the main user visible effect here is on windows, although we did make changes to both unix and windows consoles.

chris-eibl

LGTM. Thank you @JanEricNitschke.

chris-eibl · 2025-09-14T17:30:59Z

I just found out, that in clear we can now change from

cpython/Lib/_pyrepl/windows_console.py

Line 517 in 8b5ce31

self.screen = [""]

to

cpython/Lib/_pyrepl/unix_console.py

Line 562 in 1c984ba

self.screen = []

like in unix_console.py (and like it is initialized in prepare in both implementations).

JanEricNitschke · 2025-09-14T17:58:57Z

Changed this for the windows console to be the same as for the unix one.

JanEricNitschke · 2025-09-16T17:36:36Z

Added myself to ACKS after the review/approval, hope thats alright.

ambv · 2026-01-02T13:01:50Z

Lib/_pyrepl/windows_console.py

        self.__write(CLEAR)
        self.posxy = 0, 0
-        self.screen = [""]
+        self.screen = []


The "" here before was a clear attempt at working around the same issue that you're solving here, but I agree the if statement is cleaner and more explicit.

In unix_console.py clear() is setting self.screen to an empty list, but as you noticed this wasn't triggering any bug because the __gone_tall boolean would prevent this path being taken in refresh(). In unit tests that wasn't the case and so they were failing before your fix.

I'm okay with being defensive here, especially that this makes the behavior symmetrical between Windows and Unix.

ambv · 2026-01-02T13:03:37Z

Sorry this took me so long to get to. I had to understand what's going on here before pushing the green button. Confirmed this is the correct fix and tested it resolves the issue on Windows 11 while maintaining behavior on macOS and Ubuntu 22.04 LTS. Thank you so much for your work, @JanEricNitschke and @chris-eibl for your careful review.

miss-islington-app · 2026-01-02T13:04:41Z

Thanks @JanEricNitschke for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

pythonGH-138732) (cherry picked from commit 8a2deea) Co-authored-by: Jan-Eric Nitschke <47750513+JanEricNitschke@users.noreply.github.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl>

bedevere-app · 2026-01-02T13:04:50Z

GH-143350 is a backport of this pull request to the 3.14 branch.

pythonGH-138732) (cherry picked from commit 8a2deea) Co-authored-by: Jan-Eric Nitschke <47750513+JanEricNitschke@users.noreply.github.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl>

bedevere-app · 2026-01-02T13:04:54Z

GH-143351 is a backport of this pull request to the 3.13 branch.

bedevere-bot · 2026-01-02T13:19:23Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 CentOS9 NoGIL 3.x (tier-1) has failed when building commit 8a2deea.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1609/builds/4528) and take a look at the build logs.
Check if the failure is related to this commit (8a2deea) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1609/builds/4528

Failed tests:

test_pyrepl

Failed subtests:

test_no_newline - test.test_pyrepl.test_pyrepl.TestMain.test_no_newline

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/test/support/__init__.py", line 3022, in wrapper
    return func(*args, **kwargs)
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/test/test_pyrepl/test_pyrepl.py", line 1884, in test_no_newline
    self.assertIn(expected_output_sequence, cleaned_output)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 'Something pretty long>>> exit()' not found in 'print(\'Something pretty long\', end=\'\')\r\nexit()\r\nPython 3.15.0a3+ free-threading build (heads/main:8a2deea1fc7, Jan  2 2026, 13:14:38) [GCC 11.5.0 20240719 (Red Hat 11.5.0-14)] on linux\r\nType "help", "copyright", "credits" or "license" for more information.\r\n>>> \x1b[34hprint(\'Something pretty long\', end=\'\')\x1b[42D\n\rSomething pretty long>>> \x1b[34hexit()\x1b[10D\n\r'

ambv · 2026-01-02T13:21:29Z

The buildbot failure is related. I'm on it.

…es (GH-138732) (GH-143350) (cherry picked from commit 8a2deea) Co-authored-by: Jan-Eric Nitschke <47750513+JanEricNitschke@users.noreply.github.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl>

…es (GH-138732) (GH-143351) (cherry picked from commit 8a2deea) Co-authored-by: Jan-Eric Nitschke <47750513+JanEricNitschke@users.noreply.github.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl>

Fix pyrepl overriding printed output without newlines

074d743

JanEricNitschke requested review from ambv, lysnikolaou and pablogsal as code owners September 10, 2025 08:58

bedevere-app bot added the awaiting review label Sep 10, 2025

bedevere-app bot mentioned this pull request Sep 10, 2025

[Windows] Python 3.13 New REPL prompt swallows characters with print(..., end=" ") #128067

Closed

Define more save escape sequences

771b387

Fix allowed character insert escape

011a7ca

chris-eibl reviewed Sep 14, 2025

View reviewed changes

Lib/test/test_pyrepl/test_pyrepl.py Show resolved Hide resolved

chris-eibl reviewed Sep 14, 2025

View reviewed changes

📜🤖 Added by blurb_it.

713b649

Merge branch 'main' into fix-pyrepl

edf20b7

Add dedicated unittests for unix and windows console

a43a319

chris-eibl approved these changes Sep 14, 2025

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Sep 14, 2025

chris-eibl mentioned this pull request Sep 14, 2025

gh-134746: implement Ctrl+Alt+L clear display keymap in pyREPL #138521

Open

Harmonize clear between windows and unix

8c75345

Add myself to ACKS

3308891

chris-eibl mentioned this pull request Sep 23, 2025

gh-130168: pyrepl can messup and poll is not thread safe #138617

Closed

JanEricNitschke and others added 2 commits October 17, 2025 17:09

Merge remote-tracking branch 'origin/main' into fix-pyrepl

611a1f8

Merge branch 'main' into fix-pyrepl

22e9542

ambv reviewed Jan 2, 2026

View reviewed changes

ambv added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Jan 2, 2026

ambv merged commit 8a2deea into python:main Jan 2, 2026
58 checks passed

bedevere-app bot removed the awaiting core review label Jan 2, 2026

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Jan 2, 2026

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Jan 2, 2026

Uh oh!

gh-128067: Fix pyrepl overriding printed output without newlines #138732

gh-128067: Fix pyrepl overriding printed output without newlines #138732

Uh oh!

Conversation

JanEricNitschke commented Sep 10, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app bot commented Sep 10, 2025

Uh oh!

bedevere-app bot commented Sep 10, 2025

Uh oh!

bedevere-app bot commented Sep 10, 2025

Uh oh!

chris-eibl commented Sep 14, 2025

Uh oh!

Uh oh!

chris-eibl Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

chris-eibl commented Sep 14, 2025

Uh oh!

chris-eibl commented Sep 14, 2025

Uh oh!

JanEricNitschke commented Sep 14, 2025

Uh oh!

chris-eibl commented Sep 14, 2025

Uh oh!

chris-eibl commented Sep 14, 2025

Uh oh!

JanEricNitschke commented Sep 14, 2025

Uh oh!

chris-eibl left a comment

Choose a reason for hiding this comment

Uh oh!

chris-eibl commented Sep 14, 2025

Uh oh!

JanEricNitschke commented Sep 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JanEricNitschke commented Sep 16, 2025

Uh oh!

ambv Jan 2, 2026

Choose a reason for hiding this comment

Uh oh!

ambv commented Jan 2, 2026

Uh oh!

Uh oh!

miss-islington-app bot commented Jan 2, 2026

Uh oh!

bedevere-app bot commented Jan 2, 2026

Uh oh!

bedevere-app bot commented Jan 2, 2026

Uh oh!

bedevere-bot commented Jan 2, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

ambv commented Jan 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JanEricNitschke commented Sep 10, 2025 •

edited by bedevere-app bot

Loading

JanEricNitschke commented Sep 14, 2025 •

edited

Loading