Split out of #5757 (the body-corruption half is fixed by #5778). This is a separate, pre-existing concurrency bug.
Symptom
When one Perry process is both an HTTP server and a fetch client — an in-process server that calls fetch() (against itself or any host) while serving (SSR, webhooks, API gateway, tests) — a fetch request can hang forever: the promise never settles, the process never exits.
import { createServer } from "node:http";
const s = createServer((req, res) => {
let c: Buffer[] = [];
req.on("data", (b: Buffer) => c.push(b));
req.on("end", () => res.end("len=" + Buffer.concat(c).length));
});
s.listen(0, async () => {
const port = (s.address() as any).port;
for (let i = 1; i <= 40; i++) {
const r = await fetch(`http://localhost:${port}/`, { method: "POST", body: "x".repeat(i) });
await r.text(); // eventually hangs here
}
console.log("done"); process.exit(0);
});
Single requests usually succeed; it becomes reliable under repeated/concurrent requests and under machine load.
Diagnosis
lldb on the hung process: main thread parked on the event-loop condvar, all tokio workers idle, the I/O reactor parked on kevent — i.e. the reqwest request future is parked and never re-woken. The fetch resolution is never queued, so the main loop's idle-cap re-checks find nothing to drain → permanent.
- Body-agnostic (string and Buffer both hang) and reproduces on
main.
- The node
http/https client (different pump) and cross-process fetch (separate runtimes) are not affected — only the in-process loopback where the reqwest client and the hyper server share the tokio runtime (async_bridge::RUNTIME, reached by the server via perry_ffi).
- It's a reqwest pooled-keep-alive ↔ in-process-hyper-server lost-wakeup on the shared runtime, exacerbated by load.
What was tried (and why it's not enough)
pool_max_idle_per_host(0) (disable idle keep-alive): 100% reliable under low load, but still hangs under load, and costs cross-request connection reuse.
- A dedicated tokio runtime for fetch (separate from the server's): cuts the rate
100%→25% but a permanent residual remains (a deeper same-process lost-wakeup in the await/Promise.all machinery).
A complete fix needs to address the underlying lost-wakeup (reqwest/hyper on the shared runtime, and/or the main-thread event-loop ↔ tokio handoff), not just timing.
Environment
Split out of #5757 (the body-corruption half is fixed by #5778). This is a separate, pre-existing concurrency bug.
Symptom
When one Perry process is both an HTTP server and a fetch client — an in-process server that calls
fetch()(against itself or any host) while serving (SSR, webhooks, API gateway, tests) — a fetch request can hang forever: the promise never settles, the process never exits.Single requests usually succeed; it becomes reliable under repeated/concurrent requests and under machine load.
Diagnosis
lldbon the hung process: main thread parked on the event-loop condvar, all tokio workers idle, the I/O reactor parked onkevent— i.e. the reqwest request future is parked and never re-woken. The fetch resolution is never queued, so the main loop's idle-cap re-checks find nothing to drain → permanent.main.http/httpsclient (different pump) and cross-process fetch (separate runtimes) are not affected — only the in-process loopback where the reqwest client and the hyper server share the tokio runtime (async_bridge::RUNTIME, reached by the server viaperry_ffi).What was tried (and why it's not enough)
pool_max_idle_per_host(0)(disable idle keep-alive): 100% reliable under low load, but still hangs under load, and costs cross-request connection reuse.100%→25% but a permanent residual remains (a deeper same-process lost-wakeup in the await/Promise.allmachinery).A complete fix needs to address the underlying lost-wakeup (reqwest/hyper on the shared runtime, and/or the main-thread event-loop ↔ tokio handoff), not just timing.
Environment
main; macOS arm64.