Lib/pty.py major revision

[8vasu]

BPO	41818
Nosy	@gpshead, @pfmoore, @ronaldoussoren, @tjguk, @ned-deily, @encukou, @asvetlov, @zware, @koobs, @zooba, @kulikjak, @aeros, @8vasu
PRs	bpo-41818: Updated tests for the standard pty library #22962 bpo-41818: test_openpty succeed on Gentoo, don't expect to fail on this platform #23514 bpo-41818: Make test_openpty() avoid unexpected success due to number of rows and/or number of columns being == 0. #23526 bpo-41818: test_master_read() in test_pty.py should be expected to fail on every platform that is not Linux; that includes Solaris. #23533 bpo-41818: Fix test_master_read() so that it succeeds on all platforms that either raise OSError or return b"" upon reading from master #23536 bpo-41818: Make additions to the tty module #23546 bpo-41818: Add termios.tcgetwinsize(), termios.tcsetwinsize(). Update docs. #23686 bpo-41818: Add os.login_tty() for *nix. #23740 bpo-41818: Close file descriptors in test_openpty #24119 bpo-41818: ++ termios versionadded markers. #27987 bpo-41818: Add os.login_tty() for *nix. #29658

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-09-19.23:19:34.173>
labels = ['OS-mac', '3.8', '3.9', '3.10', 'library', 'tests', 'OS-windows']
title = 'Lib/pty.py major revision'
updated_at = <Date 2021-11-20.10:06:35.452>
user = 'https://github.com/8vasu'

bugs.python.org fields:

activity = <Date 2021-11-20.10:06:35.452>
actor = 'soumendra'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'macOS', 'Tests', 'Windows', 'FreeBSD']
creation = <Date 2020-09-19.23:19:34.173>
creator = 'soumendra'
dependencies = []
files = []
hgrepos = []
issue_num = 41818
keywords = ['patch']
message_count = 22.0
messages = ['377195', '377198', '377202', '381830', '381839', '381843', '381845', '381847', '381852', '381870', '381922', '381930', '381941', '381965', '381968', '381980', '382016', '384408', '384413', '385257', '400389', '400393']
nosy_count = 13.0
nosy_names = ['gregory.p.smith', 'paul.moore', 'ronaldoussoren', 'tim.golden', 'ned.deily', 'petr.viktorin', 'asvetlov', 'zach.ware', 'koobs', 'steve.dower', 'kulikjak', 'aeros', 'soumendra']
pr_nums = ['22962', '23514', '23526', '23533', '23536', '23546', '23686', '23740', '24119', '27987', '29658']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue41818'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

• gh-85984: Remove legacy Lib/pty.py code. #92365
• gh-85984: Utilize new "winsize" functions from termios in pty tests. #101831
• gh-85984: New additions and improvements to the tty library. #101832
• gh-85984: Major revision of the pty library. #101833
• gh-85984: Add POSIX pseudo-terminal functions. #102413
• [3.11] gh-85984: Utilize new "winsize" functions from termios in pty tests. (GH-101831) #104652
• gh-85984: Document change in return type of tty functions #110028
• [3.12] gh-85984: Document change in return type of tty functions (GH-110028) #110324
• gh-85984: Add _POSIX_VDISABLE from unistd.h to termios module. #114985

[8vasu]

The current pty library has the following issues:

1. Does not set slave termios. Documented in the source.

2. Does not set initial slave window size. Documented in the source. Does not handle SIGWINCH. See bpo-41494, bpo-41541. This is essential in the following practical scenarios: i. creating split windows/panes while using a terminal multiplexer; ii. when resizing GUI terminal emulator window, especially relevant when using tiling window managers; iii. resizing an ansi-term window created inside a GNU Emacs frame.

3. Does not perform signal handling. Signals must be blocked during sensitive portions of code.

4. Hangs on FreeBSD. See bpo-26228.

5. Includes deprecated functions pty.master_open(), pty.slave_open().

6. In pty.fork(), the fallback code should try using TIOCSCTTY first. It is still using the old method of opening a tty to make it the controlling tty. Currently even SysV based systems provide TIOCSCTTY. See https://stackoverflow.com/questions/51593530/code-explanation-for-glibc-login-tty-function-openttyname-immediately-f

The current version of pty.spawn() uses pty.fork() internally. However, pty.fork() closes slave and only returns (pid, master_fd). To update winsize, access to slave is necessary. Further, slave termios must be properly set. The proposed modifications do this by implementing a login_tty(3) based function ( tty.login() ), and using that in pty.spawn() instead of pty.fork(). tty.login() tries TIOCSCTTY before falling back to the old SysV method because Python currently does not provide an interface to the native login_tty(3).

7. tty.setraw() is called right after tty.tcgetattr(). This increases redundancy of code because tty.setraw() itself makes an identical tty.tcgetattr() call.

8. Requires testing/porting to more platforms. Solaris, Illumos, macOS, Cygwin, etc. Windows ConPTY?

9. There should be an option in pty.spawn() to turn off slave's ECHO flag. For example, when it is being used in a pipe. See util-linux/util-linux@1eee1ac and util-linux/util-linux@75ccd75#diff-3834a3d25eeaf20d9d0dcb05a46995f6

10. Tests are incomplete. Tests consider OSes such as Tru64 but not {Free/Net/Open/...}BSD.

Find ongoing work here: https://github.com/8vasu/pypty2

[aeros]

In addition to the above, if a major revision is made to pty, I'd suggest also addressing the issue of "master/slave" terminology, and replace it with something comparable like "parent/child". There's an open devguide issue (python/devguide#605) to more explicitly state terms to avoid, and support for avoiding usage of "slave/master" seems uncontroversial (especially in any new code).

[8vasu]

Makes sense. I will happily make a change of terminology in the pypty2 repository after the most desirable alternative is determined based on the choice of the majority. I think 'mother/son' sounds cute while still retaining the same initials as before; people used to the older
terminology will find this easy to remember. Terminology such as parent/child and server/client might make it a little confusing.

[asvetlov]

New changeset c13d899 by Soumendra Ganguly in branch 'master':
bpo-41818: Updated tests for the standard pty library (GH-22962)
c13d899

[vstinner]

This change broke x86 Gentoo buildbots: bpo-42463.

[asvetlov]

#23514 has the fix, waiting for all buildbots finish before pressing "Merge" button.
Gentoo bots are green.