[BUG] broken RPATH entries on python3 modules

[rprichard]

The Python 3 module DSOs in our Linux prebuilt have a broken RPATH setting (/../lib, instead of the intended $ORIGIN/../lib). The python3 and libpython3.9.so are fine, though.

I suppose it's harmless, maybe? If it were to break something, maybe it would matter on a RHEL/CentOS system where /lib64 has the 64-bit binaries and /lib has the 32-bit ones. I think we want this RPATH on the python binary. For the modules, if we wanted an rpath at all, we'd want $ORIGIN and/or $ORIGIN/../... None of the extension modules depend on any other prebuilt python DSO, though, so we could clear the rpath for everything except python, if the build system let us do that.

RPATH values:

$ cd /x/kokoro/prebuilts/python/linux-x86
$ for p in $(find . -type f | sort); do readelf -d $p 2>/dev/null | sed -e 's#$# -- '$p'#' | grep RPATH; done
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib] -- ./bin/python3.9
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib] -- ./lib/libpython3.9.so.1.0
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib] -- ./lib/libpython3.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/audioop.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/binascii.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/cmath.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_codecs_cn.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_codecs_hk.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_codecs_iso2022.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_codecs_jp.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_codecs_kr.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_codecs_tw.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_contextvars.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_crypt.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_csv.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_ctypes_test.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_curses.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_curses_panel.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_dbm.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_decimal.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_elementtree.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_gdbm.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_json.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_lsprof.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_md5.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/mmap.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_multibytecodec.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_multiprocessing.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/nis.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/ossaudiodev.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/parser.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_posixshmem.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/readline.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/resource.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_sha1.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_sha256.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_sha3.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_sha512.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/spwd.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_sqlite3.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_ssl.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_statistics.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/syslog.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/termios.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_testbuffer.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_testcapi.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_testimportmultiple.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_testinternalcapi.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_testmultiphase.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/unicodedata.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/xxlimited.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_xxsubinterpreters.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_xxtestfuzz.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so
 0x000000000000000f (RPATH)              Library rpath: [/../lib] -- ./lib/python3.9/lib-dynload/_zoneinfo.cpython-39-x86_64-linux-gnu.so

Also, because it's interesting, here are the current DT_NEEDED dependencies:

$ declare -A deps; deps=(); for lib in $(find . -type f | sort); do for need in $(readelf -d $lib 2>/dev/null | sed -ne 's/^.*Shared library: \[\(.*\)\]$/\1/p'); do deps[$need]+=":$(basename $lib)"; done; done
$ for lib in "${!deps[@]}"; do printf '%-20s : %s\n' "$lib" "${deps[$lib]}" ; done
libsqlite3.so.0      : :_sqlite3.cpython-39-x86_64-linux-gnu.so
libpython3.9.so.1.0  : :python3.9:libpython3.so
libreadline.so.6     : :readline.cpython-39-x86_64-linux-gnu.so
libm.so.6            : :libpython3.9.so.1.0:audioop.cpython-39-x86_64-linux-gnu.so:cmath.cpython-39-x86_64-linux-gnu.so:_ctypes_test.cpython-39-x86_64-linux-gnu.so:_datetime.cpython-39-x86_64-linux-gnu.so:_decimal.cpython-39-x86_64-linux-gnu.so:math.cpython-39-x86_64-linux-gnu.so
libtinfo.so.5        : :_curses.cpython-39-x86_64-linux-gnu.so
libcrypt.so.1        : :_crypt.cpython-39-x86_64-linux-gnu.so
libpanel.so.5        : :_curses_panel.cpython-39-x86_64-linux-gnu.so
libc.so.6            : most/all of them
librt.so.1           : :_posixshmem.cpython-39-x86_64-linux-gnu.so
libffi.so.6          : :_ctypes.cpython-39-x86_64-linux-gnu.so
libssl.so.1.0.0      : :_ssl.cpython-39-x86_64-linux-gnu.so
libbz2.so.1.0        : :_bz2.cpython-39-x86_64-linux-gnu.so
libncurses.so.5      : :_curses.cpython-39-x86_64-linux-gnu.so
libdl.so.2           : :libpython3.9.so.1.0:_ctypes.cpython-39-x86_64-linux-gnu.so
libgdbm_compat.so.3  : :_dbm.cpython-39-x86_64-linux-gnu.so
libcrypto.so.1.0.0   : :_hashlib.cpython-39-x86_64-linux-gnu.so:_ssl.cpython-39-x86_64-linux-gnu.so
libnsl.so.1          : :nis.cpython-39-x86_64-linux-gnu.so
libz.so.1            : :binascii.cpython-39-x86_64-linux-gnu.so:zlib.cpython-39-x86_64-linux-gnu.so
libgdbm.so.3         : :_gdbm.cpython-39-x86_64-linux-gnu.so
libutil.so.1         : :libpython3.9.so.1.0
libpthread.so.0      : most/all of them

Edit: There is a small lib/libpython3.so ELF file (not a symlink), soname of libpython3.so, that needs libpython3.9.so.1.0, so it needs its rpath of $ORIGIN/../lib.

[DanAlbert]

We can probably safely disable some of those modules. We only need to include what LLDB and the NDK itself need. sqlite, curses, readline, and bz2 seem likely. The others are harder to be certain of. We probably don't need SSL proper but hashlib depends on libcrypto from the SSL library anyway, and that's maybe used somewhere? iirc we used to have a dependency on hashlib in the build, but idr where or why.

[rprichard]

hashlib can still provide most hash functions even if openssl isn't available. They're all implemented in native code, too, for acceptable performance. (Exceptions seem to be arcane, like hashlib.new("blake2b512"), which uses _hashlib with our Linux+Windows prebuilts and doesn't work with the mac prebuilt.) Most of the commonly-used hash functions are implemented in modules like _sha256 or _md5.

[rprichard]

The rpath values are fixed by https://android-review.googlesource.com/c/platform/external/python/cpython3/+/1788257.

[rprichard]

Probably should leave this open until prebuilts are updated.

[rprichard]

I noticed that our Python 3 REPL on Linux has no line editing on newer systems, probably because prebuilts/python/linux-x86/lib/python3.9/lib-dynload/readline.cpython-39-x86_64-linux-gnu.so depends on libreadline.so.6 which gLinux doesn't have. (Ubuntu 18.04 only has libreadline5 and libreadline7. Ubuntu 20.04 only has libreadline5 and libreadline8.) Line editing does work on Ubuntu 16.04 and CentOS 7, which have libreadline.so.6.

I see a comment in the python3 readline.c module:

/*
 * It is possible to link the readline module to the readline
 * emulation library of editline/libedit.
...
 */

We distribute libncurses.so.6 and libedit.so.0 dynamically with the Android Clang prebuilt, which contains LLDB. Maybe we want to configure Python to use libedit's readline emulation, and maybe we want both libedit and the _curses module to use a bundled copy of libncurses. (But see #1565).

I'm working on switching the Linux python3 build over to use Docker (centos:7 base container). I have it linking libffi statically, and currently, there are no readline or _curses modules. (curses exists but can't be imported because it needs _curses.)

[rprichard]

Prebuilts updated:
https://android-review.googlesource.com/c/platform/prebuilts/python/linux-x86/+/1817162