configure behavior about python is hard for packaging

[gdt]

I am using nut git master as of today, building in pkgsrc.

(From my viewpoint python 2.7 is now history and we might as well gc the support. But that's off point.)

pkgsrc chooses a python version to depend on and ensures it is there, and sets variables in our Makefile that I can pass in the env or to configure, in order to use that version (and only that version). So I don't want nut's configure to look, I want to pass the specific version.

It's unsound to use env or python3 because we are installing in some particular versions site-packages, so we can't have that just change. It has to match.

I am already passing -I/-L CFLAGS/LDFLAGS for python3.10 but I think that's not important.

The python configure code appears not to take a version as input, and it looks for python3 before python3.N. It doesn't look for 3.10. It finds it without a pass, and testclient ends up with "#!/usr/bin/env python3", and this is not allowed (not bound to the dependency precisely). I want "#!/usr/pkg/bin/python3.10" after I pass in --with-python=/usr/pkg/bin/python3.10 or PYTHON= (whatever works is fine in env or configure cmd line).

My suggestions (severable :-) are:

• rip out python2 stuff
• take a --with-python=/path/to/python3.N arg
• if not, for python3, look for python3.10 through python3.7 or maybe lower by default. Skip python3 as dangerous and people who want that can use --with-python.

[jimklimov]

Please take a look at ci_build.sh use of PYTHON envvar which does optionally set --with-python=/some/path (and is directly used by autogen.sh to run helper scripts that provide includes for configure.ac. Indeed, documentation missed that flag, will update. IIRC the envvar (and PYTHON2 and PYTHON3 for targeted activity) also acts as default for configure if a CLI option is not used.

So far there is no plan to "rip out" python2 support, since NUT project aims to do its best to build and run on all systems released this millenium, as much as is reasonably feasible (e.g. where older versions ran okay). Being off upstream support does not mean there are no boxes still running that obsoleted version and depending on it with no (non-disruptive) way forward. To that extent, PyNUT module, NUT-Monitor GUI app, and build helper scripts were either changed syntactically to work with at least python-2.7 and several versions of python-3.x that I could get in various CI-tested distros, or in case of GUI app (and changes of GUI backend involved) - split into two that can be alternately installed.

The NUT CI farm does test several implementations in its build matrix, so the mechanism is known to work :)

Note also that the PYTHON version called for build helper scripts (from autogen.sh and some Makefiles -- after parsing by configure) can be named anything as long as it can run the scripts momentarily, in that it does not have use for the long-term stability of installation paths as you mention above.

The m4/nut_check_python.m4 does search for "site-packages" location of the selected python binary, if it can, to generate make install recipe paths for that. Actual testing for this was limited and manual however (to make install and run the app) so maybe some things do not work/integrate well enough currently.

Note that the scripts/python/app/NUT-Monitor wrapper script which dispatches Py2/Py3 implementations does try to make use of PYTHONPATH to prefer a ../module location -- primarily for in-tree testing of current development. If the app is installed in a filesystem tree like this (co-bundling PyNUT module rather than site-package'ing it), presence of the module in Python locations might not matter for it at all (and might e.g. change along with distro-preferred version, to be used by other consumers if any). Not that I promote doing it this way, but it is possible without much effort.

As for probing short-hand program names (python, python2 and python3, possibly with dash-number spelling) it is somewhat a vague point - many OSes do not deliver those at all nowadays, others use them as symlinks to currently selected default version for the user (e.g. debian alternatives or illumos facets).

CFLAGS/LDFLAGS should not matter here, PyNUT is purely Python and uses the text protocol to talk to NUT data server (probably just a subset of the protocol at the moment, as it was an implementation detail for the GUI app originally).

[jimklimov]

@gdt : does https://github.com/networkupstools/nut/pull/1788/files#diff-52c1fe4e78fac41f4ab264a9efe9022181337befe30aea69a1887c7ac8d095e1 (change of docs/configure.txt in #1788, if some force-push changes the link) address it enough to close this issue and solve your packaging problems?

[gdt]

I think so and will try it, but I find it very verbose and confusing.

My view is that when building nut, it should use either 0 or 1 python implementations, and after finding that implementation should tightly bind to it. That's the subject of a not-yet-opened ticket; the question here is how to achieve that.

I think I tried --with-python=/usr/pkg/bin/python3.10 and had issues with the test script still having /usr/bin/env in it. I'll do that again.

[jimklimov]

Hm, looking at detection code, I suppose there can be a bug: if it failed to find the program or failed to confirm it (run and get version), it reverts to PYTHON=no and later can default via PYTHON2/3 vars if found. Should have aborted if it can not fulfill the explicit requirement.

As for one implementation - some OSes ship(ed) both pythons side by side; NUT has no reason to not deliver both modules then. Shebang de-facto matters only for the programs (app variants, and test script).

[gdt]

With that I get

checking python interpeter to call... /usr/pkg/bin/python3.10 (sys.version_info(major=3, minor=10, micro=9, releaselevel='final', serial=0))
checking python site-packages location... /usr/pkg/lib/python3.10/site-packages
checking for python2... no
checking for python2.7... no
checking for python-2.7... no
checking for python... python
configure: WARNING: Version reported by /usr/bin/env python was not suitable as python2
checking python2 interpeter to call... no
checking for python3... python3
checking python3 interpeter to call... /usr/bin/env python3 (sys.version_info(major=3, minor=10, micro=9, releaselevel='final', serial=0))
checking python3 site-packages location... /usr/pkg/lib/python3.10/site-packages

and in config.status

S["HAVE_PYTHON3_SITE_PACKAGES_TRUE"]=""                                                                                                            
S["PYTHON3_SITE_PACKAGES"]="/usr/pkg/lib/python3.10/site-packages"                                                                                 
S["HAVE_PYTHON3_FALSE"]="#"                                                                                                                        
S["HAVE_PYTHON3_TRUE"]=""                                                                                                                          
S["PYTHON3"]="/usr/bin/env python3"                                                                                                                
S["HAVE_PYTHON2_SITE_PACKAGES_FALSE"]=""                                                                                                           
S["HAVE_PYTHON2_SITE_PACKAGES_TRUE"]="#"                                                                                                           
S["PYTHON2_SITE_PACKAGES"]=""                                                                                                                      
S["HAVE_PYTHON2_FALSE"]=""                                                                                                                         
S["HAVE_PYTHON2_TRUE"]="#"                                                                                                                         
S["PYTHON2"]=""                                                                                                                                    
S["HAVE_PYTHON_SITE_PACKAGES_FALSE"]="#"                                                                                                           
S["HAVE_PYTHON_SITE_PACKAGES_TRUE"]=""                                                                                                             
S["PYTHON_SITE_PACKAGES"]="/usr/pkg/lib/python3.10/site-packages"                                                                                  
S["HAVE_PYTHON_FALSE"]="#"                                                                                                                         
S["HAVE_PYTHON_TRUE"]=""                                                                                                                           
S["PYTHON"]="/usr/pkg/bin/python3.10"                                                                                                              

and I did find the correct substitution in PyNUT.py and the test client.

After building and installing, it was still ok.

Somehow I did not end up succeeding earlier while trying to figure this out from reading the m4 sources.

[gdt]

I think where we're at is:

• configure help needs merging
• I have solved my current problem
• The dispatch/PYTHONPATH stuff, which I would likely find troublesome, is not installed by default, so it's not bothering me.
  So I'm going to hit close and thanks for the discussion.