Components: Python, C++
Environment
- pyarrow 25.0.0 (PyPI wheel, bundled mimalloc), pandas + pyarrow dataset/parquet path
- Python 3.14.4, CPython, Linux x86_64 (Ubuntu-family, kernel 7.0.0), uvicorn/FastAPI service
- Not OOM (VmRSS 161 MB at crash, process peak 130 MB), not disk.
Summary
If the first import of pyarrow and Arrow's first allocation happen on a short-lived worker thread that subsequently exits, a later allocation on a new thread crashes with SIGSEGV inside the bundled mimalloc's per-thread heap initialisation. When the import + first allocation happen on the main thread (or any thread that stays alive), the same workload is stable under heavy concurrency.
This bites any FastAPI/uvicorn-style service that lazily imports pandas/pyarrow inside a request handler: anyio's worker threads have a 10-second idle lifetime (WorkerThread.MAX_IDLE_TIME), so the import lands on a thread that is pruned seconds later, and the next burst of traffic on fresh threads segfaults the process.
Crash
From a core dump (gdb, thread apply all bt; crashing thread was the newest of 23):
#5 mi_thread_init () libarrow.so.2500
#6 _mi_malloc_generic ()
#8 arrow::BaseMemoryPoolImpl<MimallocAllocator>::Allocate(...)
#12 arrow::AllocateEmptyBitmap(...)
#14 parquet::SerializedFile::PreBuffer(...)
#45 pyarrow._dataset.Scanner.to_table
#49 _PyEval_EvalFrameDefault
Disassembly at the fault: mov (%rdx),%rax then => mov 0x18(%rax),%rax with rax == 0 — a NULL dereference inside mi_thread_init.
Isolation matrix (3 runs per configuration, deterministic)
| pyarrow import |
first Arrow allocation |
outcome |
| throwaway thread (exits) |
same throwaway thread |
SEGV 3/3 |
| main thread |
throwaway thread |
survived 3/3 |
| throwaway thread |
main thread |
survived 3/3 |
| main thread |
main thread |
survived 3/3 |
The trigger is thread identity at load/first-allocation time, not concurrency: with the import on the main thread, 8 concurrent fresh threads performed 6,000 parquet reads with no fault.
Reproduction shape
import threading
def load_and_read():
import pyarrow.dataset as ds # first pyarrow import in the process
ds.dataset("some.parquet").to_table() # first Arrow allocation
t = threading.Thread(target=load_and_read)
t.start(); t.join() # thread exits — heap/TLS torn down
# later: allocations from NEW short-lived threads (e.g. anyio worker threads
# serving an ASGI app) crash in mi_thread_init.End-to-end reproduction (our service shape): uvicorn app whose handler lazily imports pandas and reads parquet; hammer with 21 concurrent GETs per burst, 12 s idle between bursts (idle > anyio's 10 s worker lifetime so each burst gets fresh threads). Pre-workaround the process died with signal 11 on the third burst, reproducibly. Post-workaround: 10 bursts, 210 requests, all 200.
Hypothesis
mimalloc's process-wide initialisation appears to bind state to the thread that performed it; when that thread exits, its TLS/heap teardown leaves a structure NULL that mi_thread_init later dereferences on a new thread. (We can share the full backtrace and the isolation script on request.)
Workarounds (verified)
- Import pyarrow and perform one real allocation on the main thread before any worker-thread use (we ship this as a startup warm-up).
ARROW_DEFAULT_MEMORY_POOL=system (set before import) also sidesteps it, at the cost of changing the allocator globally.
Reporter context: hit in production behind FastAPI/uvicorn; root-caused from an apport core dump; happy to provide the isolation script, full thread apply all bt, and library build info.
Components: Python, C++
Environment
Summary
If the first import of pyarrow and Arrow's first allocation happen on a short-lived worker thread that subsequently exits, a later allocation on a new thread crashes with SIGSEGV inside the bundled mimalloc's per-thread heap initialisation. When the import + first allocation happen on the main thread (or any thread that stays alive), the same workload is stable under heavy concurrency.
This bites any FastAPI/uvicorn-style service that lazily imports pandas/pyarrow inside a request handler: anyio's worker threads have a 10-second idle lifetime (
WorkerThread.MAX_IDLE_TIME), so the import lands on a thread that is pruned seconds later, and the next burst of traffic on fresh threads segfaults the process.Crash
From a core dump (gdb,
thread apply all bt; crashing thread was the newest of 23):Disassembly at the fault:
mov (%rdx),%raxthen=> mov 0x18(%rax),%raxwith rax == 0 — a NULL dereference insidemi_thread_init.Isolation matrix (3 runs per configuration, deterministic)
The trigger is thread identity at load/first-allocation time, not concurrency: with the import on the main thread, 8 concurrent fresh threads performed 6,000 parquet reads with no fault.
Reproduction shape
End-to-end reproduction (our service shape): uvicorn app whose handler lazily imports pandas and reads parquet; hammer with 21 concurrent GETs per burst, 12 s idle between bursts (idle > anyio's 10 s worker lifetime so each burst gets fresh threads). Pre-workaround the process died with signal 11 on the third burst, reproducibly. Post-workaround: 10 bursts, 210 requests, all 200.
Hypothesis
mimalloc's process-wide initialisation appears to bind state to the thread that performed it; when that thread exits, its TLS/heap teardown leaves a structure NULL that
mi_thread_initlater dereferences on a new thread. (We can share the full backtrace and the isolation script on request.)Workarounds (verified)
ARROW_DEFAULT_MEMORY_POOL=system(set before import) also sidesteps it, at the cost of changing the allocator globally.Reporter context: hit in production behind FastAPI/uvicorn; root-caused from an apport core dump; happy to provide the isolation script, full
thread apply all bt, and library build info.