Document configure options regarding Python (module, app)

[jimklimov]

Closes: #1787

Also note #1792

[AppVeyorBot]

❌ Build nut 2.8.0.290-master failed (commit dbe7211288 by @jimklimov)

[gdt]

In general this feels too complicated.

• If python is used as a build tool that should be separate from the target.
• It seems nut should find and select a single python (to be used after nut is installed)
• The default should steer to python3.N first, and bare python3 (which by definition is a pointer that might get changed)
• There is code that works with 3 as the standard approach. If there's py27 code, fine, install that when 2 is selected. If that's done optionally also, make that opt in. I can't imagine anyone wanting that unless their system is python3-deficient.
• I see zero need for dispatching at runtime. That just seems like extra complexity that solves a problem people shouldn't have.

[gdt]

You mentioned realpath and I am not sure you meant that literally.
POSIX specifies realpth(3) but there is no requirement for a realpath(1) and NetBSD (and likely others) does not have one.

A thought is that given a --with-python=FOO, one can just run `FOO-config --prefix`` to find the prefix, avoiding a lot of complexity.

[jimklimov]

Well, for systems that have it - literally indeed. For others, fall back to the "FOO" pathname (which may be a symlink so less useful).

Good point about FOO-config, quite worth trying it as a fallback too. Not ubiquitous though, fewer and fewer projects and/or packages ship a FOO-config script/program. Looked now quickly at systems under my fingertips, and it is indeed a mixed bag for python variants available here, in particular.

UPDATE: I also don't quite see options that would tell me the "real path" to interpreter:

• MinGW:

$ python3-config
Usage: /mingw64/bin/python3-config --prefix|--exec-prefix|--includes|--libs|--cflags|--ldflags|--extension-suffix|--help|--abiflags|--configdir|--embed

$ python3-config --exec-prefix
C:/msys64/mingw64

$ python3-config --configdir
C:/msys64/mingw64/lib/python3.10/config-3.10

$ ls -la C:/msys64/mingw64/lib/python3.10/config-3.10
total 198
drwxr-xr-x 1 root root     0 Aug 17 10:01 .
drwxr-xr-x 1 root root     0 Aug 17 10:02 ..
-rw-r--r-- 1 root root 86653 Aug 12 20:00 Makefile
-rw-r--r-- 1 root root 15350 Aug 12 20:00 Setup
-rw-r--r-- 1 root root   524 Aug 12 20:00 Setup.config
-rw-r--r-- 1 root root    41 Aug 12 20:00 Setup.local
-rw-r--r-- 1 root root  3445 Aug 12 20:00 config.c
-rw-r--r-- 1 root root  1623 Aug 12 20:00 config.c.in
-rwxr-xr-x 1 root root 15358 Aug 12 20:00 install-sh
-rwxr-xr-x 1 root root  7855 Aug 12 20:00 makesetup
-rwxr-xr-x 1 root root  2046 Aug 12 20:00 python-config.py
-rw-r--r-- 1 root root  2947 Aug 12 20:00 python.o

• WSL2 Ubuntu:

$ python3.8-config
Usage: /usr/bin/python3.8-config --prefix|--exec-prefix|--includes|--libs|--cflags|--ldflags|--extension-suffix|--help|--abiflags|--configdir|--embed

$ python3.8-config --exec-prefix
/usr

$ ls -la /usr/lib/python3.8/config-3.8-x86_64-linux-gnu
total 18816
drwxr-xr-x  2 root root     4096 Jan  5 12:42 .
drwxr-xr-x 30 root root    20480 Jan  5 12:42 ..
-rw-r--r--  1 root root    89066 Nov 14 13:59 Makefile
-rw-r--r--  1 root root    14679 Nov 14 13:59 Setup
-rw-r--r--  1 root root     1263 Nov 14 13:59 Setup.local
-rw-r--r--  1 root root     5357 Nov 14 13:59 config.c
-rw-r--r--  1 root root     1623 Nov 14 13:59 config.c.in
-rwxr-xr-x  1 root root    15368 Nov 14 13:59 install-sh
-rw-r--r--  1 root root  8880268 Nov 14 13:59 libpython3.8-pic.a
-rw-r--r--  1 root root 10103676 Nov 14 13:59 libpython3.8.a
lrwxrwxrwx  1 root root       40 Nov 14 13:59 libpython3.8.so -> ../../x86_64-linux-gnu/libpython3.8.so.1
-rwxr-xr-x  1 root root     7848 Nov 14 13:59 makesetup
-rwxr-xr-x  1 root root     2128 Nov 14 13:59 python-config.py
-rw-r--r--  1 root root    97552 Nov 14 13:59 python.o

Maybe something can be gleaned from those files, but feels like a not quite portable adventure... but being a fallback to try and skip in case of troubles, may be good enough, e.g.:

$ head -1 C:/msys64/mingw64/lib/python3.10/config-3.10/python-config.py
#!/mingw64/bin/python3.10.exe

$ head -1 /usr/lib/python3.8/config-3.8-x86_64-linux-gnu/python-config.py
#! /usr/bin/python3.8

The python print(sys.executable) effectively returns argv[0] so the symlink name if called like that.

[gdt]

I thought that e.g. python3.10-config was the standard approach in python's install -- that's not the kind of thing pkgsrc makes up. But fair point that if others don't follow that, you can't relay on it.

But, I only really care what happens when one provides the full path to a real program, since that's how my world is.

[gdt]

Thanks for working on this. I tested this and it's mostly ok. However, with --with-python=/usr/pkg/bin/python3.10 I still ended up with files installed to python2.7. I'm rebuilding with --without-python2 which is easy enough, but I don't think that should be necessary after specifying a python version.

[jimklimov]

As a counter-offer, I'll update the docs with the suggestion to use the option :)

[jimklimov]

Heh, pythonX-config is (always?..) a python script itself, so has a useful shebang...

UPDATE: NOT always

[jimklimov]

Neat:

:; ./autogen.sh && ./configure --without-all --with-pynut
...
checking for python... python
configure: WARNING: A python program name without a specific version number was requested (may be a symlink prone to change over time): /usr/bin/python
checking python interpeter to call... /usr/bin/python (sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0))
checking python site-packages location... /usr/local/lib/python3.8/dist-packages
checking for python2... python2
checking python2 interpeter to call... /usr/bin/python2 (sys.version_info(major=2, minor=7, micro=18, releaselevel='final', serial=0))
checking python2 site-packages location... /usr/local/lib/python2.7/dist-packages
configure: WARNING: A python3 program name was not specified during configuration, will default to '/usr/bin/python3.8' (derived from --with-python setting which has a suitable version)
checking python3 interpeter to call... /usr/bin/python3.8 (sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0))
checking python3 site-packages location... /usr/local/lib/python3.8/dist-packages
...

NUT Configuration summary:
==========================
...
* install NUT-Monitor desktop application:      no
* install PyNUT binding module: yes
* Default  Python  interpreter: /usr/bin/python
* Default  Python  site-packages:       /usr/local/lib/python3.8/dist-packages
* Specific Python2 interpreter: /usr/bin/python2
* Specific Python2 site-packages:       /usr/local/lib/python2.7/dist-packages
* Specific Python3 interpreter: /usr/bin/python3.8
* Specific Python3 site-packages:       /usr/local/lib/python3.8/dist-packages
...

So here the python value as defaulted was used almost as such -- just resolved to /usr/bin/python (with warnings about the uncertainty). Similarly for python2.

However for python3 defaulted to try the value of PYTHON first because it has major=3, the complete /usr/bin/python3.8 path got discovered.

Wondering how well MSYS2/mingw would play with that, considering its resolution to windows paths rather than POSIXy shims:

$ which python ; python -c 'import sys; print(sys.executable);'
/mingw64/bin/python
C:/msys64/mingw64/bin/python.exe

And ouch, this does not help build confidence (gotta filter discovered names after all):

$ head -1 /mingw64/bin/python-config
#!/bin/sh

[jimklimov]

Hm, seems --without-python2 now misbehaves

[gdt]

Let me know when you want me to retest (i.e., when you have come to a stopping place and think it's ok).

[jimklimov]

Let me know when you want me to retest (i.e., when you have come to a stopping place and think it's ok).

Hopefully found the responsible typo, fixed.

:; rm -f Makefile configure ; ./autogen.sh && ./configure --without-all --with-pynut --without-python2
...
checking for python... python
configure: WARNING: A python program name without a specific version number was requested (may be a symlink prone to change over time): /usr/bin/python
checking python interpeter to call... /usr/bin/python (sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0))
checking python site-packages location... /usr/local/lib/python3.8/dist-packages

checking python2 interpeter to call... no

configure: WARNING: A python3 program name was not specified during configuration, will default to '/usr/bin/python3.8' (derived from --with-python setting which has a suitable version)
checking python3 interpeter to call... /usr/bin/python3.8 (sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0))
checking python3 site-packages location... /usr/local/lib/python3.8/dist-packages

...

* install NUT-Monitor desktop application:      no
* install PyNUT binding module: yes
* Default  Python  interpreter  :       /usr/bin/python
* Default  Python  site-packages:       /usr/local/lib/python3.8/dist-packages
* Specific Python2 interpreter  :
* Specific Python2 site-packages:
* Specific Python3 interpreter  :       /usr/bin/python3.8
* Specific Python3 site-packages:       /usr/local/lib/python3.8/dist-packages

[gdt]

I made a distfile from this and am able to build it in pkgsrc with only the chosen python. So WORKSFORME and thanks for hacking on this.

[jimklimov]

Super! Thanks for the report!