Skip to content

A module that spawns a closure capturing a caller's closure fails with Stack underflow inside a coroutine #783

Description

@Masterplanner25

Summary

A module function that wraps a caller-supplied closure in a closure of its own and spawns the wrapper works from main and fails with Stack underflow when the same call is made from inside a coroutine.

This is the #691 / #696 family in a third direction. Found while building std:loop for #182 — run_after(ms, body) is the natural shape for it, and it was unusable.

Reproduction

wrapmod.nd:

fn wrap_and_spawn(body) {
    return spawn(fn() {
        return body()
    })
}
import "./wrapmod.nd" as m
fn main() {
    spawn(coroutine(fn() {
        m.wrap_and_spawn(fn(){ print("m2") })
        return 1i
    }))
    run_loop()
}
Runtime error at probe.nd:2:99: Stack underflow
  at __anon_3 (probe.nd)
  called from __anon_1 (wrapmod.nd:3:16)

The same program with m.wrap_and_spawn(...) called directly from main prints m2 and exits 0.

The matrix

where the wrap lives called from main called inside a coroutine
the same file works works
a module works Stack underflow

std:async's parallel is unaffected because it spawns the caller's closure directly rather than wrapping it — verified in both positions.

Diagnosis

Instrumented _foreign_closure_origin and load_coroutine_context across both cases. The failing call reports:

('foreign_origin', '__anon_3', 'is_foreign=True', 'origin=None', 'caller_vm=False')

The caller's closure is correctly detected as foreign and resolution returns None. Walking the four sources in _foreign_closure_origin:

  1. origin_ctx from a _ClosureProxy — nothing was wrapped; the same-VM _try_enter_module_call fast path (BUG-105: async builtins via std: module wrappers fall back to sync (ASYNC-MOD-001) #105) does not wrap arguments.
  2. The frame walk for cross_module_ctx — by the time the spawned coroutine runs body(), the wrap_and_spawn frame has been popped. This is BUG: a closure RETURNED from a module runs against the caller's chunk — silent truncation at top level #696's "every source records a context a call is still inside of".
  3. self._caller_vm — None on the same-VM path. This is the branch that saves the from-main case, where invoke_function creates a detached child VM and caller_vm=True.
  4. _module_owning(fn_info) — searches the loaded modules. The closure was compiled in main, which is not a module, so nothing owns it.

So the gap is that ownership resolution covers every chunk except the root program's. #696 resolved closures a module returned by finding the owning module; a closure the root program passed in, invoked after the module frame has popped, has no such owner to find.

Fix direction

Give _foreign_closure_origin a final fallback to the VM's own base context when no module claims the closure — a closure that is foreign to the currently-loaded chunk and owned by no loaded module belongs to the root program by elimination.

Worth checking with the same care the rest of this family got: the resolution order matters, and the root fallback must come after _module_owning, or a closure genuinely owned by a module would resolve to the root.

Why it is filed rather than fixed

Found while implementing #182, and it is not #182's subject. std:loop's run_after uses the new spawn_after(ms, fn) builtin, which spawns the caller's closure directly and so never poses the question — the same shape async.parallel has always used. That is a route around this defect, not a fix for it: the wrapping shape is an ordinary thing to write and still fails.

tests/test_nodus_native_timers.py::test_run_after_fires_from_inside_a_coroutine pins the behaviour that would regress if run_after were ever rewritten to wrap.

Affected versions

Current main (9cec213). The _module_owning resolution this falls through arrived with #696 in 5.9.0; before that the case failed for the earlier reason and was equally broken.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions