Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-105323: Update readline module to detect apple editline variant #108665

Merged
merged 19 commits into from
Sep 29, 2023

Conversation

corona10
Copy link
Member

@corona10 corona10 commented Aug 30, 2023

@corona10 corona10 changed the title gh-105223: Introduce --with-readline=apple option gh-105323: Introduce --with-readline=apple option Aug 30, 2023
@corona10 corona10 requested a review from a team as a code owner August 30, 2023 03:07
@corona10 corona10 closed this Aug 30, 2023
@corona10 corona10 reopened this Aug 30, 2023
@ned-deily
Copy link
Member

Without having time to look closely at this now. I feel pretty strongly against adding a new configuration option for what has always worked by default before. There should be a better way.

@corona10
Copy link
Member Author

Without having time to look closely at this now. I feel pretty strongly against adding a new configuration option for what has always worked by default before. There should be a better way.

Yeah let's discuss it after your travel :) Enjoy!

@corona10
Copy link
Member Author

corona10 commented Aug 30, 2023

I feel pretty strongly against adding a new configuration option for what has always worked by default before.

The different signature has existed since macOS 10.5. (2007)
We didn't notice for a long time, and it was never been an issue because the compiler didn't detect it..
But I think that this is the correct way to handle the Apple issue. The default option is still available but it will not be the correct way for Apple BSD editline backend.

@erlend-aasland
Copy link
Contributor

I agree with @ned-deily. Before proposing new PRs, let's instead lay out the three scenarios in detail and then see how we can improve configure and readline.c preprocessor guards :)

@corona10
Copy link
Member Author

corona10 commented Aug 30, 2023

@erlend-aasland @ned-deily

Backend configure header Real verison Macro Version (RL_READLINE_VERSION) Install Source rl_startup_hook rl_pre_input_hook User
readline --with-readline=readline readline/readline.h 8.2.1 0x0802 brew / MacPorts https://ftp.gnu.org/gnu/readline/ typedef int rl_hook_func_t (void); typedef int rl_hook_func_t (void); Linux/BSD/ macOS/...
libedit w/readline wrapper --with-readline=editline editline/readline.h 20221030-3.1 0x0402 brew / MacPorts https://www.thrysoee.dk/editline/ rl_hook_func_t(void); rl_hook_func_t(void); Linux/BSD/ macOS/...
libedit --with-readline=editline editline/editline.h 1.17.1 n/a https://github.com/troglobit/editline https://github.com/troglobit/editline n/a n/a Linux/BSD/ macOS/...
Apple editline --with-readline=apple readline/readline.h editline/readline.h libedit-57 0x0402 XCode SDK https://github.com/apple-oss-distributions/libedit/ typedef int Function(const char *, int); typedef int Function(const char *, int); macOS

First of all, I love seamless approach, but here is the table for why
we should handle Apple editline as an independent backend.
Apple editline only shares the same header name, and it uses fully different signature and implementation.

IIUC, There is no way to distinguish between editline and Apple editline except using grep (I don't prefer this option it's too unstable) because they are using same macro versions.

Suppose Apple editline is viewed as an independent implementation. In that case, I think it is better in terms of consistency to treat it separately as an Apple option as if the edlitline option was added.

@erlend-aasland
Copy link
Contributor

Hm, I wonder if there are really four flavours we need to take into account:

  • GNU readline (readline/readline.h)
  • libedit w/readline wrapper (editline/readline.h)
  • libedit (editline/editline.h)
  • Apple libedit (editline/readline.h and readline/readline.h, both identical)

@corona10
Copy link
Member Author

libedit w/readline wrapper (editline/readline.h)
libedit (editline/editline.h)

When I refer the libedit hompage, editline/readline.h only exists.
I am not sure where I can refer the libedit (editline/editline.h) as official case.

@erlend-aasland
Copy link
Contributor

libedit w/readline wrapper (editline/readline.h)
libedit (editline/editline.h)

When I refer the libedit hompage, editline/readline.h only exists. I am not sure where I can refer the libedit (editline/editline.h) as official case.

Minix Editline (https://troglobit.com/projects/editline/) provides editline/editline.h.

@corona10
Copy link
Member Author

Minix Editline (https://troglobit.com/projects/editline/) provides editline/editline.h.

Thanks @erlend-aasland, I updated the table :)

@ronaldoussoren
Copy link
Contributor

Suppose Apple editline is viewed as an independent implementation. In that case, I think it is better in terms of consistency to treat it separately as an Apple option as if the edlitline option was added.

Apple's lib edit isn't really an independent implementation, but is a very old copy of NetBSD libedit. This old revision in the NetBSD CVS(!) server has similar definitions.

That said, this is just nitpicking on your statement. For handling the differences you might as well treat this as the "apple" implementation because I expect nobody else uses the old code base.

Do you know if the newer libedit is consistently better than Apple's copy? If it is we could consider to just ship a current version with our installers, but that's for a different issue.

@corona10 corona10 changed the title [WIP] gh-105323: Introduce --with-readline=apple option gh-105323: Update readline module to detect apple editline variant Sep 15, 2023
@corona10
Copy link
Member Author

corona10 commented Sep 15, 2023

Okay, using AX_CHECK_TYPEDEF can provide a seamless solution.
@ned-deily @ronaldoussoren @erlend-aasland PTAL

CI for FreeBSD is unrelated to this change.

configure.ac Outdated
AS_VAR_IF([with_readline], [readline], [
AX_CHECK_TYPEDEF(Function, readline/readline.h, [,AC_DEFINE([WITH_APPLE_EDITLINE]),])
Copy link
Member Author

@corona10 corona10 Sep 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure that we have to support for AS_VAR_IF([with_readline], [edit], [...]) case too

@corona10 corona10 changed the title gh-105323: Update readline module to detect apple editline variant [WIP] gh-105323: Update readline module to detect apple editline variant Sep 15, 2023
@corona10
Copy link
Member Author

Oops, I need more testing.

configure.ac Outdated
@@ -5830,6 +5831,12 @@ AC_ARG_WITH(
[with_readline=readline]
)

dnl gh-105323: Need to handle the macOS editline as an alias of readline.
AS_CASE([$ac_sys_system/$ac_sys_release],
[Darwin/*], [AX_CHECK_TYPEDEF(Function, readline/readline.h, AC_DEFINE([WITH_APPLE_EDITLINE]))],
Copy link
Member Author

@corona10 corona10 Sep 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erlend-aasland cc @ronaldoussoren

I am not sure why AC_DEFINE([WITH_APPLE_EDITLINE]) is not generated in configure file.
Checking itself looks okay but the problem is if the condition is true it doesn't do as my expectation.

checking whether right shift extends the sign bit... yes
checking for getc_unlocked() and friends... yes
checking for Function in readline/readline.h... yes

@corona10 corona10 changed the title [WIP] gh-105323: Update readline module to detect apple editline variant gh-105323: Update readline module to detect apple editline variant Sep 22, 2023
@corona10
Copy link
Member Author

corona10 commented Sep 22, 2023

@ronaldoussoren @ned-deily @erlend-aasland

Finally, I updated the PR to be the working solution. I checked both cases in my local environment.
PTAL

FYI, I found some comments about AX_CHECK_TYPEDEF / AC_CHECK_TYPEDEF are broken, so they recommend to use
AC_CHECK_TYPE.

Thank you @ronaldoussoren and @ned-deily to suggest better solution :)

configure.ac Outdated Show resolved Hide resolved
@corona10
Copy link
Member Author

@ronaldoussoren @ned-deily @erlend-aasland

gentle ping!

Modules/readline.c Outdated Show resolved Hide resolved
configure.ac Outdated Show resolved Hide resolved
configure.ac Outdated Show resolved Hide resolved
Copy link
Contributor

@erlend-aasland erlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Dong-hee; this looks good to me! (And thanks for looking into these readline issues!) AFAICS, this looks like a solution that should please all involved core devs. Please wait for a thumbs-up from Ronald and the other Ned before merging 🙂

Doc/using/configure.rst Outdated Show resolved Hide resolved
Co-authored-by: Erlend E. Aasland <[email protected]>
Modules/readline.c Outdated Show resolved Hide resolved
Copy link
Contributor

@ronaldoussoren ronaldoussoren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@corona10 corona10 merged commit 501939c into python:main Sep 29, 2023
18 checks passed
@corona10 corona10 deleted the gh-105223-apple-readline branch September 29, 2023 12:18
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants