gh-109191: Fix build with newer editline

[Bo98]

Fixes compile with BSD libedit versions since April 2023.

Requires backport to 3.12 (should cleanly apply).
Requires backport to 3.11 and 3.10 using this manual cherry-pick: Bo98@b0c571f
I believe 3.9 and older don't technically support libedit except Apple's 2012 version, but a similar cherry-pick could be made if desired.

• Issue: Build fails with recent libedit versions #109191

[cpython-cla-bot]

All commit authors signed the Contributor License Agreement.

[bedevere-app]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[erlend-aasland]

Requires backport to 3.12 (should cleanly apply). Requires backport to 3.11 and 3.10 using this manual cherry-pick: Bo98@b0c571f I believe 3.9 and older don't technically support libedit except Apple's 2012 version, but a similar cherry-pick could be made if desired.

FYI, 3.10 and older is in security-fix-mode; I don't think this applies as a security fix. Backports to 3.12 and 3.11 are allowed for bug-fixes; is this a bugfix?

[erlend-aasland]

@corona10: you've recently dived into the readline issues. Would you mind taking a look at this PR?

[bedevere-app]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[Bo98]

Backports to 3.12 and 3.11 are allowed for bug-fixes; is this a bugfix?

I consider it so. It's certainly not a new feature.

Compile error fixes, including those adjusting for new OS and library versions, seem to have been backported before unless I'm mistaken? I consider this the same category as those. Future FreeBSD and NetBSD will ship with this newer editline out of the box too.

Risk should also be low.

[bedevere-app]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[corona10]

@corona10: you've recently dived into the readline issues. Would you mind taking a look at this PR?

I will take a look by this weekend, I need time to investigate it :)

[erlend-aasland]

@corona10: we should have a conditional CI matrix for readline changes, similar to our OpenSSL CI matrix.

[erlend-aasland]

This looks good to me. I'll wait for Donghee's approval before merging.

[erlend-aasland]

Oh, BTW, can you produce a NEWS entry for this, @Bo98?

[corona10]

@corona10: we should have a conditional CI matrix for readline changes, similar to our OpenSSL CI matrix.

Good idea

[corona10]

@Bo98 cc @erlend-aasland

Since the change was introduced quite recent days
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libedit/readline/readline.h.diff?r1=1.53&r2=1.54&sortby=date

I think that we have to support old versions(1.53) too.
WDYT?

[corona10]

@erlend-aasland

Requires backport to 3.11 and 3.10 using this manual cherry-pick: Bo98@b0c571f

About the backport patch, I am not sure if we handle this as the bug or not?
IMO, If the user uses CPython <= 3.12, they should use libedit <= 1.53 not the latest version.
How did we deal with similar issue for OpenSSL case?

[Bo98]

Since the change was introduced quite recent days http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libedit/readline/readline.h.diff?r1=1.53&r2=1.54&sortby=date

I think that we have to support old versions(1.53) too.

This PR still supports older versions as it checks for the existence of rl_compdisp_func_t, which didn't exist prior to 1.54. So < 1.54 will still use the VFunction type cast as before.

This is evidenced by the compile working with Apple's libedit, which is based on 1.33.

About the backport patch, I am not sure if we handle this as the bug or not?
IMO, If the user uses CPython <= 3.12, they should use libedit <= 1.53 not the latest version.
How did we deal with similar issue for OpenSSL case?

Had a dig into this, and found the OpenSSL 3 changes to _hashlib (#84659):

• 3.9 got initial support for the new FIPS API from OpenSSL 3.0 alpha. 3.8 didn't receive a backport for this change as 3.8 didn't contain FIPS API so there was no need to backport. 3.8 did receive a backport to satisfy the compiler and silence deprecation warnings (OpenSSL 3.0.0: define OPENSSL_API_COMPAT 1.1.1 #87965).
• Final release of OpenSSL 3.0 came when 3.11 was the main branch, but the fixes for the final release were backported to 3.10 and 3.9.
• _ssl changes were split into multiple issues, but seemed to be backported accordingly at the time, e.g. OpenSSL 3.0.0: handle empty cadata consistently #88086.

Some of those backports (e.g. _hashopenssl.c in 4ddd5da) were fairly large too, and could have affected runtime behaviour. Tbh, I would normally have expected the bar to be higher to only cover compile-level adjustments or small changes, though it seems that wasn't the case here.

With that said, the fixes for libedit in this PR are only compile-level adjustments - there is no runtime change as it's just a type cast adjustment. Both branches of the #if defined are identical otherwise.

[erlend-aasland]

I think that we have to support old versions(1.53) too. WDYT?

It seems to be there is no problem with old versions; AC_CHECK_TYPE will check for the type def and the correct signature will be used.

About the backport patch, I am not sure if we handle this as the bug or not?

Build failure => build bug. IMO, it is a bug fix.

IMO, If the user uses CPython <= 3.12, they should use libedit <= 1.53 not the latest version.

Why?

If we are to restrict the upper version requirement for a third-party dependency, we must document that, and preferrably also implement compile time and runtime checks for that. I don't think any other dependency has an upper version limit.

How did we deal with similar issue for OpenSSL case?

Which issue were you thinking of?

[corona10]

@Bo98 @erlend-aasland
After rereading this issue and PR, I noticed that there was a misunderstanding from myself.
LGTM and let's add the backport patch,

[miss-islington]

Thanks @Bo98 for the PR, and @corona10 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

[miss-islington]

Sorry, @Bo98 and @corona10, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker f4cb0d27cc08f490c42a22e646eb73cc7072d54a 3.11

[bedevere-app]

GH-110562 is a backport of this pull request to the 3.12 branch.

[erlend-aasland]

Sorry, @Bo98 and @corona10, I could not cleanly backport this to 3.11 due to a conflict.

@Bo98, do you want to do the backport?

[bedevere-app]

GH-110575 is a backport of this pull request to the 3.11 branch.