Repository navigation
램 누수가 있어요 #314
Description
Client or integration
Codex CLI
Area
CLI
Summary
살려주세요
Reproduction
조금 쓰다보면 저렇게 누수가 발생하는 것 같아요
Version
2.7.31
Operating system
windows 11
Provider and model
No response
Logs or error output
Screenshots and supporting files
No response
Redacted configuration
Checks
- I searched existing issues and documentation.
- I removed secrets, tokens, account details, request credentials, and personal data.
Activity
lidge-jun commentedon Jul 23, 2026
빨리 고치겠습니다 ㅠㅠ
lidge-jun commentedon Jul 23, 2026
Thanks for the report and the screenshot — a single Bun process at ~23 GB RSS is clear evidence, and I dug into it.
TL;DR: this looks like a Bun 1.3.14 runtime issue on Windows, not an opencodex application leak. We audited every long-lived in-process store on current
dev(request-log ring buffer: capped at 200 entries; responses continuation state: 1-hour TTL with pruning on every access; usage log: windowed file reads; SSE relays: pull-based passthrough with abort-on-cancel). None of them can grow anywhere near 23 GB.What does match your profile is the pinned Bun runtime (
1.3.14) on Windows:fetch().bodypiped throughTransformStreamdoes not propagate backpressure oven-sh/bun#28035 —fetch().bodystreaming fails to propagate backpressure; a proxy repro reached 13.5 GB RSS. This is exactly the opencodex workload (long-running localhost streaming proxy).What we'll do: evaluate bumping the pinned Bun runtime from 1.3.14 to 1.4.x in an upcoming release (it touches CI/release automation, so it goes through our review lane first).
What you can try meanwhile:
ocx stop && ocx start) when memory climbs — the leak is in the runtime, so a restart fully reclaims it.bun upgrade) and run opencodex under it — per #32585 this resolved the same class of problem.I'm closing this as tracked-upstream with the Bun pin bump on our roadmap. If it still happens after the next release (or under Bun 1.4.x), please comment here and we'll reopen and dig deeper — a
process.memoryUsage()sample (rss vs heapUsed) at high memory would be especially helpful.lidge-jun commentedon Jul 23, 2026
One correction to the workaround above: plain
bun upgradewill report 1.3.14 as the latest — that is expected. 1.4.0 is currently only on Bun's canary channel, not stable. To actually try it:(and
bun upgrade --stableto roll back to 1.3.14.) The reporter in oven-sh/bun#32585 was running a canary 1.4.0 binary. Canary builds carry their own risk, so treat this as an experiment — the reliable workaround remains restarting the proxy when memory climbs.On our side, the plan is to evaluate the runtime pin bump once Bun 1.4 reaches a stable release.
devlog: 260723 issue triage r2 — new-pile sweep (#314 memleak, Sol la…
devlog: correct #314 Bun 1.4 guidance — canary-only, bump waits for s…
devlog(win-mem-safestream): docs-first roadmap — 000 plan, 001 resear…
feat(stream): runtime stream-capability gate + persisted streamMode s…
feat(stream): eager bounded single-reader SSE relay behind runtime ga…
feat(server): RSS memory watchdog + authed /api/system/memory endpoint (
feat(doctor): service memory/runtime section via /api/system/memory (#…
docs: Windows memory troubleshooting page + structure SoT sync (#314 …
fix(codex): refuse responses input beyond the advertised context window
fix(codex): refuse responses input beyond the advertised context window
9 remaining items
devlog: 260723 issue triage r2 — new-pile sweep (lidge-jun#314 memlea…
devlog: correct lidge-jun#314 Bun 1.4 guidance — canary-only, bump wa…
devlog(win-mem-safestream): docs-first roadmap — 000 plan, 001 resear…
feat(stream): runtime stream-capability gate + persisted streamMode s…
feat(stream): eager bounded single-reader SSE relay behind runtime ga…
feat(server): RSS memory watchdog + authed /api/system/memory endpoint (
feat(doctor): service memory/runtime section via /api/system/memory (l…
docs: Windows memory troubleshooting page + structure SoT sync (lidge…
kirin-bot139 commentedon Oct 4, 2026
Follow-up with a current Bun 1.4.x field sample, since the maintainer note above asked for a report if this still reproduces under 1.4.x.
Environment (sanitized):
Observed-memory timeline from /api/system/memory (scalar-only):
At the ~4.65 GiB sample:
This suggests the multi-GiB growth is not explained by the configured app-owned retained stores or responseState. It also does not currently correlate with slower model TTFT in this workload: over 17 consecutive 15-minute samples, first-output P50 median was ~5.0 s and memory-vs-first-output correlation was negative/noisy, while network loss stayed 0% on the measured route.
I have not overridden the bundled runtime or patched OpenCodex. The current mitigation is a conservative idle-only restart guard: it only invokes the official restart path after memory is high and there are no active turns plus a sustained no-request gap.
Clean restart follow-up: the guard later found a natural idle window and used the supported drain/restart path. Observed memory dropped from ~4.35 GiB immediately before restart to ~264 MiB immediately after replacement, with a new service PID and health/readiness both returning 200. A four-minute usage window spanning the restart contained 6/6 HTTP 200 rows and no 429/499/502. The configured app-owned stores remained well below budget before restart, so the restart result reinforces that the multi-GiB retained/committed memory was outside the bounded app-owned state measured above.
For a separate runtime sanity check, I compared the bundled Bun 1.4.0 against an isolated official Bun 1.4.2 binary without changing the live service or PATH. A small AsyncLocalStorage stress case and a local streaming fetch proxy case (including client-side early cancellation) both returned external memory to ~0.1 MiB after GC on both versions; 1.4.2 did not show a clear retention or throughput advantage in those isolated cases. I therefore did not override the production runtime.
If useful, I can provide additional scalar-only long-uptime samples. No request/response content, credentials, account ids, local paths, or private endpoints are included.
Ingwannu commentedon Oct 4, 2026
Reopening for investigation based on the new Windows 11 / OpenCodex 2.77.0 / bundled Bun 1.4.0 observation, as requested in the earlier maintainer follow-up. The previous attribution to the older Bun runtime is not a sufficient explanation for this new sample. This is not yet a confirmed leak or a confirmed Bun defect.
The distinction in these measurements matters:
/api/system/memoryselectsobservedBytes = max(rss, external, arrayBuffers)for thresholding. WhenobservedMetricisexternal, that value is not another measurement of RSS. The 256 MiBappOwnedBytesbudget covers registered retained stores, not the whole process or native runtime. Keeping those stores near 69 MiB is useful evidence, but cannot by itself identify where the remaining growth comes from. Spill bytes are a separate disk-retention counter and should not be added to resident memory. The TTFT samples do not establish the cause of the memory growth either way.Please attach the scalar-only samples you already offered, ideally the existing 15-minute series rather than a new stress run:
uptimeSeconds, with byte units kept consistent;bunVersion,bunRevision,bunRuntimeSource,streamModeandeagerRelay;rss,heapUsed,heapTotal,external,arrayBuffers,observedBytes/observedMetric, andjscHeapcounters;appOwnedBytesretained/evictable/pinned/over-budget totals, plusresponseState, response-spill totals andactiveTurnCount;Any samples from a naturally quiet interval would help distinguish continued growth from a retained plateau. Before/after data from your next independently planned restart would also be useful, but please do not restart, force GC, change the bundled runtime or increase load just for this request. The endpoint's eager-relay value is the no-client-rewrite baseline, not proof that every request used that path.
No request/response contents, credentials, account IDs, paths, private endpoints or heap dumps are needed. I have only checked the current reporting/accounting contracts; I have not reproduced this Windows workload or changed your guard/service. Please keep the issue open while we assess the new evidence.
Ingwannu commentedon Oct 4, 2026
The edited restart follow-up is useful: in your reported workload, replacing the service process reduced the observed counter from about 4.35 GiB to 264 MiB. That supports process-lifetime retention/accounting as an investigation target, but does not identify the allocation owner. Low registered-store totals do not rule out unregistered application buffers, transport buffers, allocator retention or a runtime accounting issue. We should not relabel all of the unexplained counter as a confirmed native/Bun leak.
The two isolated AsyncLocalStorage/streaming checks returning near baseline after GC are narrow negative controls. They do not reproduce the long-lived concurrent workload, and do not justify a production runtime override. Likewise, six HTTP-200 rows around replacement are useful limited availability evidence, not proof that every in-flight session or stream survived.
The existing request for the raw scalar time series remains the next evidence step; no additional stress run, forced GC, runtime change or restart is requested. I have not reproduced or changed your service/guard, and am keeping this open without a cause or fix claim.
lidge-jun commentedon Oct 7, 2026
Thank you for the additional Bun 1.4.0 observations. The growth and restart reduction are useful measurements, but neither identifies the allocation owner; the earlier Bun attribution does not explain the newer sample. Please reopen with current versions, sanitized
ocx doctoroutput, and a minimal reproduction with separate RSS/heap/external counters and evidence narrowing the responsible component.Closing as not reproducible with the current evidence; this is not a fix claim.