Statically linked `sqlite3` fails checks due to missing `-lm`

[zanieb]

Bug report

Bug description:

On Linux, if I build a static version of sqlite3 (e.g., ./configure --disable-shared && make from a sqlite3 tarball)

You can see that ./configure for CPython fails the SQLite checks (added back in #30016):

$ ./configure 2>&1 | grep sqlite
checking for sqlite3 >= 3.15.2... yes
checking for sqlite3.h... yes
checking for sqlite3_bind_double in -lsqlite3... no
checking for sqlite3_column_decltype in -lsqlite3... no
checking for sqlite3_column_double in -lsqlite3... no
checking for sqlite3_complete in -lsqlite3... no
checking for sqlite3_progress_handler in -lsqlite3... no
checking for sqlite3_result_double in -lsqlite3... no
checking for sqlite3_set_authorizer in -lsqlite3... no
checking for sqlite3_trace_v2 in -lsqlite3... no
checking for sqlite3_trace in -lsqlite3... no
checking for sqlite3_value_double in -lsqlite3... no
checking for sqlite3_load_extension in -lsqlite3... no
checking for sqlite3_serialize in -lsqlite3... no
checking for --enable-loadable-sqlite-extensions... no

And in the config.log there are a bunch of failures due to undefined references to functions in libm, e.g.:

configure:15469: checking for sqlite3_bind_double in -lsqlite3
configure:15492: gcc -o conftest   -I/usr/local/include  -I$(srcdir)/Modules/_sqlite -L/usr/local/lib -lsqlite3  -L/usr/local/lib conftest.c -lsqlite3  -L/usr/local/lib -lsqlite3  -ldl  >&5
/usr/bin/ld: /usr/local/lib/libsqlite3.a(sqlite3.o): in function `logFunc':
/repo/sqlite-autoconf-3470100/sqlite3.c:131765:(.text+0x26858): undefined reference to `log2'
/usr/bin/ld: /repo/sqlite-autoconf-3470100/sqlite3.c:131768:(.text+0x268d4): undefined reference to `log'
/usr/bin/ld: /repo/sqlite-autoconf-3470100/sqlite3.c:131762:(.text+0x268ec): undefined reference to `log10'
/usr/bin/ld: /repo/sqlite-autoconf-3470100/sqlite3.c:131750:(.text+0x2691c): undefined reference to `log'

CPython is determing SQLite's requirements via pkg-config if it can at:

cpython/configure.ac

Lines 4204 to 4209 in 492b224

	PKG_CHECK_MODULES(
	[LIBSQLITE3], [sqlite3 >= 3.15.2], [], [
	LIBSQLITE3_CFLAGS=${LIBSQLITE3_CFLAGS-""}
	LIBSQLITE3_LIBS=${LIBSQLITE3_LIBS-"-lsqlite3"}
	]
	)

And -lm is included in the pkg-config for the static sqlite3 build, but it's in the private libs and requires the --static flag to be included in the --libs output:

$ pkg-config --static --libs sqlite3 
-L/usr/local/lib -lsqlite3 -lm 
$ pkg-config --libs sqlite3             
-L/usr/local/lib -lsqlite3 

However, PKG_CHECK_MODULES doesn't pass this flag. There's a PKG_CHECK_MODULES_STATIC macro which does, but CPython doesn't seem to use this yet. The alternative is set the flag manually at configure time:

PKG_CONFIG="$PKG_CONFIG --static"

which changes all the PKG_CHECK_MODULES calls to include the --static flag. However, there's a missing export for LIBS which prevents this from resolving the issue:

$ git rev-parse HEAD
f9a5a3a3ef34e63dc197156e9a5f57842859ca04
$ export PKG_CONFIG="pkg-config --static"
$ ./configure 2>&1 | grep sqlite
checking for sqlite3 >= 3.15.2... yes
checking for sqlite3.h... yes
checking for sqlite3_bind_double in -lsqlite3... no
checking for sqlite3_column_decltype in -lsqlite3... no
checking for sqlite3_column_double in -lsqlite3... no
checking for sqlite3_complete in -lsqlite3... no
checking for sqlite3_progress_handler in -lsqlite3... no
checking for sqlite3_result_double in -lsqlite3... no
checking for sqlite3_set_authorizer in -lsqlite3... no
checking for sqlite3_trace_v2 in -lsqlite3... no
checking for sqlite3_trace in -lsqlite3... no
checking for sqlite3_value_double in -lsqlite3... no
checking for sqlite3_load_extension in -lsqlite3... no
checking for sqlite3_serialize in -lsqlite3... no
checking for --enable-loadable-sqlite-extensions... no
checking for stdlib extension module _sqlite3... missing
^C

This is "solved" with the patch:

diff --git a/configure.ac b/configure.ac
index badb19d5589..5ed147f5482 100644
--- a/configure.ac
+++ b/configure.ac
@@ -4221,6 +4221,7 @@ dnl bpo-45774/GH-29507: The CPP check in AC_CHECK_HEADER can fail on FreeBSD,
 dnl hence CPPFLAGS instead of CFLAGS.
   CPPFLAGS="$CPPFLAGS $LIBSQLITE3_CFLAGS"
   LDFLAGS="$LIBSQLITE3_LIBS $LDFLAGS"
+  LIBS="$LIBSQLITE3_LIBS $LIBS"
 
   AC_CHECK_HEADER([sqlite3.h], [
     have_sqlite3=yes

As demonstrated by:

$ export PKG_CONFIG="pkg-config --static"
$ ./configure 2>&1 | grep sqlite
checking for sqlite3 >= 3.15.2... yes
checking for sqlite3.h... yes
checking for sqlite3_bind_double in -lsqlite3... yes
checking for sqlite3_column_decltype in -lsqlite3... yes
checking for sqlite3_column_double in -lsqlite3... yes
checking for sqlite3_complete in -lsqlite3... yes
checking for sqlite3_progress_handler in -lsqlite3... yes
checking for sqlite3_result_double in -lsqlite3... yes
checking for sqlite3_set_authorizer in -lsqlite3... yes
checking for sqlite3_trace_v2 in -lsqlite3... yes
checking for sqlite3_value_double in -lsqlite3... yes
checking for sqlite3_load_extension in -lsqlite3... yes
checking for sqlite3_serialize in -lsqlite3... yes
checking for --enable-loadable-sqlite-extensions... no

For context, this was reported in astral-sh/python-build-standalone#449 and some of the findings here are repeated there.

I've posted the patch in #128322

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-128321: Set LIBS instead of LDFLAGS during sqlite3 library build checks #128322
• [3.12] gh-128321: Set LIBS instead of LDFLAGS when checking sqlite3 requirements (GH-128322) #128355
• [3.13] gh-128321: Set LIBS instead of LDFLAGS when checking sqlite3 requirements (GH-128322) #128356

[zanieb]

cc @erlend-aasland — I understand you're the expert in this area.

[zanieb]

In addition to #128322, we may also want to include -lm at

cpython/configure.ac

Line 4207 in 492b224

LIBSQLITE3_LIBS=${LIBSQLITE3_LIBS-"-lsqlite3"}

If the pkg-config check fails, we fallback to including these flags. However, without -lm all of our subsequent checks will fail! Happy to post a pull request for that too.

[eli-schwartz]

In addition to #128322, we may also want to include -lm at

cpython/configure.ac

Line 4207 in 492b224

LIBSQLITE3_LIBS=${LIBSQLITE3_LIBS-"-lsqlite3"}

If the pkg-config check fails, we fallback to including these flags. However, without -lm all of our subsequent checks will fail! Happy to post a pull request for that too.

This will result in over-linking when sqlite3 is used as a shared library (no pkg-config --static).

[zanieb]

This will result in over-linking when sqlite3 is used as a shared library

Ah true — we could just include it during the checks but it seems like a bit of a pain.