Stack overflow test errors in Alpine after GH-130398

[bitdancer]

Bug report

Bug description:

After GH-130398 (0142236) was applied, test.test_dynamic.RebindBuiltinsTests.test_load_global_specialization_failure_keeps_oparg and test.test_functools.TestLRUC.test_lru_recursion fail with RecursionError: Stack overflow (used 96 kB) while calling a Python object on Alpine linux with python compiled with musl.

I don't know the significance of this; the tests were already skipped on wasi and/or emscripten before that commit, which also use musl. However, the fact that a stack overflow happens where one did not previously happen is worrisome for the stability of python on Alpine.

Let me know if there is any debugging assistance I can provide.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Linked PRs

• gh-131338: Disable new stack size code for non-glibc. #134336

[bitdancer]

@markshannon

[markshannon]

If it is anything like the emscrpiten issue, then it might be that musl libc claims to support pthread_getattr_np, but does not actually support it.

Is HAVE_PTHREAD_GETATTR_NP defined and if it is, does pthread_getattr_np give sensible stack values?

[bitdancer]

Alpine musl does indeed have pthread_getattr_np, and claims to support it as of 0.9.10 ;) (http://git.musl-libc.org/cgit/musl/tree/src/thread/pthread_getattr_np.c).

Putting

    printf("guard_size %lu\n", guard_size);
    printf("stack_size %lu\n", stack_size);

in ceval.c gives me:

guard_size 0
stack_size 126976

but I have no context to know if that is "sensible" or not ;) If it means anything, the stack_size between runs of python -m unittest test.test_functools.TestLRUC.test_lru_recursion varies between three values, the above and 135168 and 131072.

In 3.13 alpine-musl python can handle fib calls up to what would at the time of call exceed sys.getrecursionlimit (1000), but on master it caps out at less than 100 (which is what triggers the lru test failure). So I'm guessing that means the above number is "not sensible" ;)

I don't know enough about this subject to know if/what to report to musl about this, but I definitely would not like to see 3.14 going out the door with a recursion-hobbled python on Alpine. Note that I'm planning to volunteer to be tier3 support for alpine (x86_64-linux-musl) once I can get the tests passing on the buildbot one way or another (and I can set up my own buildbot if needed).

[bitdancer]

Others have encountered this problem with Alpine/MUSL reporting a small stack size via pthread_getattr_np.

As things stand, if we change the test so we can see the recursion error message, we get:

RecursionError: Stack overflow (used 96 kB) while calling a Python object

If this code is disabled for Alpine/MUSL, we instead get:

RecursionError: Stack overflow (used 3907 kB) while calling a Python object

[bitdancer]

I've proposed a PR to only enable this for GLIBC, where we know it works. It seems like it would be better to only enable it for those platforms where we know it works, if there are others.

[bitdancer]

Note; although someone applied the unsupported platform tag, I have learned that this does also affect at least emscripten, which is supported. I think emscripten crafted a local hack to get around it, but it seems to me to be better to make this opt-in to reduce the chance of future problems.

@hugovk I think this should be addressed one way or another before the release candidates.