max_memory_mb: bound how much a run grows the process (#160)

CHANGELOG.md

@@ -146,6 +146,54 @@
   stores and no record of which. The report counts **waiting** runs separately,
   because that is the population a backend switch strands.
 
+- **#160: `max_memory_mb` — an embedder can bound how much a run grows the process.**
+
+  ```python
+  NodusRuntime(max_memory_mb=256)
+  ```
+
+  `max_steps` bounds instructions and `timeout_ms` bounds wall clock; neither
+  stopped a script growing a list until the host got a `MemoryError`. Off by
+  default like the other two, because how much memory a run may have is host
+  policy.
+
+  **Three of the issue's four proposed approaches were measured and rejected**,
+  and the numbers are the reason this reads RSS from the OS instead:
+
+  | approach | measured |
+  |---|---|
+  | `tracemalloc` | **64x slowdown** (0.026s → 1.681s on a 300k loop), 11.9 µs/read |
+  | `sys.getallocatedblocks()` | 0.8 µs, but counts **blocks not bytes**, and absent on PyPy |
+  | RSS via the OS | **4.4 µs**, real megabytes, Windows and POSIX |
+
+  Polled every 256 instructions — about 0.6% overhead against Nodus's ~2.75 µs per
+  instruction (#173). It started at 2000 and was lowered because a string doubling
+  in a loop reached a `MemoryError` *between* two checks.
+
+  **Growth, not absolute RSS.** An embedded runtime shares a process with its
+  host, and an absolute bound would make a 100 MB limit fire immediately inside a
+  2 GB host. The baseline is read at the start of each run, and the VM carries a
+  single absolute *ceiling* rather than a limit plus a baseline plus a counter --
+  because `tests/test_vm_attribute_budget.py` caught the first version at **81
+  instance attributes against PyPy'''s 80-attribute map limit**, which cost ~9x
+  throughput the last time it was crossed (#702). Four attributes became one.
+
+  **Refused at construction where the platform cannot be metered**, rather than
+  accepted and silently unenforced — a security control an operator believes they
+  have is worse than none, which is what #473 and #478 were both filed for.
+
+  **It bounds a footprint, not an allocation count.** RSS is a process measure:
+  a 25 MB allocation can be satisfied from pages the process already holds and
+  move the reading not at all -- or down, if something else frees concurrently.
+  A test asserting otherwise passed locally and failed on CI with `179421184 not
+  greater than 180469760`, which is the meter behaving correctly and the
+  assumption being wrong.
+
+  Stated plainly in `SECURITY_POSTURE.md §5` rather than left implied: polling
+  bounds growth *over time*, not a **single** allocation. One enormous list is
+  served before the next check. OS-level limits (`ulimit -v`, cgroups, a container
+  cap) remain the answer for hard memory safety, not something this replaces.
+
 ### Fixes
 
 - **#725: a persisted deadline now means the same thing in every process.**

docs/governance/SECURITY_POSTURE.md

@@ -112,6 +112,7 @@ The security controls available are:
 | Runtime-state floor | — | always on | Guest writes into the runtime's state — `.nodus/`, **and wherever `NODUS_RUN_STATE_ROOT` points** (#585) — are refused; no policy can override it |
 | Call stack cap | `max_frames` | `None` → `MAX_STACK_DEPTH` (10,000) | Deep recursion raises `Call stack overflow`; tighten to 200–1000 for untrusted code |
 | Instruction limit | `max_steps` | `MAX_STEPS` (large) | Prevents infinite loops from running indefinitely |
+| Memory limit | `max_memory_mb` | `None` (unbounded) | Aborts when the run has grown the process past the bound (#160). Bounds growth over time, not a single allocation — see §5 |
 | Wall-clock limit | `timeout_ms` | `None` (no deadline) | Prevents long-running scripts from blocking the host |
 
 > **`max_frames` had no working default through v4.1.1**
@@ -150,8 +151,21 @@ The Nodus sandbox is not a full security sandbox. It does not protect against:
 - **CPU exhaustion via tight computation** — `max_steps` limits instructions but not
   CPU time; a tight loop can consume significant CPU before the step limit fires.
   Use `timeout_ms` in addition to `max_steps`.
-- **Memory exhaustion** — No limit on heap allocation. A script that builds a large
-  list or map can exhaust host memory. No equivalent of `max_memory`.
+- **Memory exhaustion, in one direction only.** `max_memory_mb` (#160) bounds how
+  much a run may grow the process, polled every 256 instructions against the OS's
+  own RSS reading. It stops the case it was filed for — a script that grows a list
+  in a loop — and it is **off by default**, like `max_steps` and `timeout_ms`.
+
+  What it cannot do: polling bounds growth *over time*, not a **single**
+  allocation. A program that asks for one enormous list gets it, and the check
+  fires afterwards if the process survives. Only an OS-level limit (`ulimit -v`, a
+  cgroup, a container memory cap) prevents that, and for untrusted code it remains
+  the answer rather than something this replaces.
+
+  Two more caveats before relying on it. It measures **process** RSS, so a host
+  allocating concurrently on another thread is counted against the guest's budget.
+  And it is refused at construction where the platform cannot be metered, rather
+  than accepted and silently unenforced.
 - **Subprocess execution** — `std:subprocess` (v4.0+) allows arbitrary process execution.
   Disable via `allow_subprocess=False` on `NodusRuntime`. When enabled, the subprocess
   binary and its arguments are unrestricted — only `stdout`/`stderr` redirect paths and

src/nodus/runtime/embedding.py

@@ -13,6 +13,7 @@
 from nodus.runtime.errors import coerce_error, legacy_error_dict
 from nodus.runtime.diagnostics import LangRuntimeError, LangSyntaxError, HostFunctionError
 from nodus.support.config import MAX_STDOUT_CHARS, MAX_STEPS
+from nodus.runtime.memory import memory_metering_available, rss_bytes
 from nodus.runtime.module_loader import ModuleLoader
 from nodus.tooling.sandbox import capture_output, configure_vm_limits
 from nodus.runtime.schema_contract import (
@@ -275,6 +276,7 @@ def __init__(
         allowed_commands: list[str] | None = None,
         allowed_hosts: list[str] | None = None,
         max_frames: int | None = None,
+        max_memory_mb: float | None = None,
         on_error: Callable | None = None,
         coroutine_timeout_ms: int | None = None,
         event_sinks: list | None = None,
@@ -501,6 +503,21 @@ def __init__(
         self.allowed_commands = allowed_commands
         self.allowed_hosts = allowed_hosts
         self.max_frames = max_frames
+        # #160. Refused at construction when the platform cannot meter, rather
+        # than accepted and silently unenforced -- a security control an operator
+        # believes they have is worse than none, which is what #473 and #478 were
+        # both filed for.
+        if max_memory_mb is not None:
+            if not isinstance(max_memory_mb, (int, float)) or max_memory_mb <= 0:
+                raise ValueError("max_memory_mb must be a positive number of megabytes")
+            if not memory_metering_available():
+                raise RuntimeError(
+                    "max_memory_mb cannot be enforced on this platform: this "
+                    "runtime could not read the process's resident memory. Use an "
+                    "OS-level limit (ulimit -v, a cgroup, or a container memory "
+                    "cap) instead of a limit that would not fire."
+                )
+        self.max_memory_mb = max_memory_mb
         self.on_error = on_error
         self.coroutine_timeout_ms = coroutine_timeout_ms
         self._event_sinks: list = list(event_sinks) if event_sinks else []
@@ -1231,6 +1248,17 @@ def _run_source_locked(
         resolved_frames = self.max_frames if max_frames is None else max_frames
         if resolved_frames is not None:
             vm.max_frames = resolved_frames
+        if self.max_memory_mb is not None:
+            # The baseline is read **here**, at the start of the run, so the VM
+            # carries a single absolute ceiling instead of a limit plus a
+            # baseline plus a counter -- four instance attributes where one will
+            # do, and PyPy's 80-attribute map limit made that difference matter
+            # (#702). Reading it now rather than at construction also means the
+            # budget covers what *this call* grows, not whatever the host
+            # allocated in between.
+            baseline = rss_bytes()
+            if baseline is not None:
+                vm.max_memory_bytes = baseline + int(self.max_memory_mb * 1048576)
 
         with capture_output(max_stdout_chars=resolved_stdout) as (stdout, stderr):
             try:

src/nodus/runtime/memory.py

@@ -0,0 +1,147 @@
+"""Reading this process's resident memory, cheaply enough to poll (#160).
+
+`max_steps` bounds instructions and `timeout_ms` bounds wall clock, but neither
+bounds the heap: a script that grows a list in a loop runs until the host gets a
+`MemoryError` or the OS kills it. `max_memory_mb` is the missing third bound.
+
+**Three of the four approaches the issue proposed were measured and rejected**,
+which is why this module reads RSS from the OS rather than counting allocations:
+
+- `tracemalloc` — **64x slowdown** on allocation-heavy code (0.026s → 1.681s on a
+  300k-iteration loop), and 11.9 µs per `get_traced_memory()`. Unusable even as an
+  opt-in, and #173 already names throughput as the bootstrapping blocker.
+- `sys.getallocatedblocks()` — cheap (0.8 µs) but counts pymalloc **blocks, not
+  bytes**, so it cannot back a limit expressed in megabytes without inventing a
+  conversion. It is also absent on PyPy.
+- Counting allocations in the VM's value constructors — deterministic, but it
+  measures *elements*, and an embedder asked for megabytes. Left on the table
+  rather than dismissed: it is the portable answer if a deterministic bound is
+  ever wanted alongside this one.
+
+Reading RSS costs **6.8 µs** on Windows, which at the polling interval the VM uses
+is a few nanoseconds per instruction. That is the trade this module exists to make.
+
+**What this cannot do**, stated because a memory limit implies more than it
+delivers: polling bounds *growth over time*, not a single allocation. A program
+that asks for one enormous list gets it, and the check fires afterwards — by which
+point the process may already be dead. Only an OS-level limit (ulimit, cgroups, a
+container memory cap) prevents that, and `SECURITY_POSTURE.md §5` says so.
+"""
+
+from __future__ import annotations
+
+import sys
+
+_PAGE_SIZE = 4096
+
+
+def _rss_windows():
+    import ctypes
+    import ctypes.wintypes as wintypes
+
+    class _ProcessMemoryCounters(ctypes.Structure):
+        _fields_ = [
+            ("cb", wintypes.DWORD),
+            ("PageFaultCount", wintypes.DWORD),
+            ("PeakWorkingSetSize", ctypes.c_size_t),
+            ("WorkingSetSize", ctypes.c_size_t),
+            ("QuotaPeakPagedPoolUsage", ctypes.c_size_t),
+            ("QuotaPagedPoolUsage", ctypes.c_size_t),
+            ("QuotaPeakNonPagedPoolUsage", ctypes.c_size_t),
+            ("QuotaNonPagedPoolUsage", ctypes.c_size_t),
+            ("PagefileUsage", ctypes.c_size_t),
+            ("PeakPagefileUsage", ctypes.c_size_t),
+        ]
+
+    kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
+    psapi = ctypes.WinDLL("psapi", use_last_error=True)
+    kernel32.GetCurrentProcess.restype = wintypes.HANDLE
+    # Declared explicitly: without argtypes the call silently returns a zeroed
+    # struct on 64-bit, which reads as "0 MB resident" and would make the limit
+    # unenforceable while looking like it worked.
+    psapi.GetProcessMemoryInfo.argtypes = [
+        wintypes.HANDLE, ctypes.POINTER(_ProcessMemoryCounters), wintypes.DWORD
+    ]
+    psapi.GetProcessMemoryInfo.restype = wintypes.BOOL
+    handle = kernel32.GetCurrentProcess()
+    # One struct, reused. Allocating a fresh one per read doubled the cost
+    # (15 us against 7), and this is on a polling path.
+    counters = _ProcessMemoryCounters()
+    counters.cb = ctypes.sizeof(_ProcessMemoryCounters)
+    size = ctypes.sizeof(_ProcessMemoryCounters)
+    reference = ctypes.byref(counters)
+
+    def read() -> int | None:
+        if not psapi.GetProcessMemoryInfo(handle, reference, size):
+            return None
+        return int(counters.WorkingSetSize)
+
+    read()  # prove it works now, not at the first limit check
+    return read
+
+
+def _rss_proc_statm():
+    """Linux: field 2 of /proc/self/statm is resident pages."""
+    def read() -> int | None:
+        try:
+            with open("/proc/self/statm", "rb") as handle:
+                return int(handle.read().split()[1]) * _PAGE_SIZE
+        except (OSError, IndexError, ValueError):
+            return None
+
+    if read() is None:
+        raise OSError("/proc/self/statm unreadable")
+    return read
+
+
+def _rss_getrusage():
+    """macOS and other POSIX: `ru_maxrss`, in bytes on Darwin and KiB on Linux.
+
+    A *peak*, not a current reading, so it never falls. For a limit that is the
+    conservative direction -- it can fire late but not early on freed memory.
+    """
+    import resource
+
+    # Reached through `getattr` rather than attribute access: `resource` is
+    # POSIX-only, so mypy on Windows reports "Module has no attribute
+    # getrusage" -- while a `type: ignore` would be flagged as unused on the
+    # Linux runner that CI type-checks on. This is correct on both.
+    getrusage = getattr(resource, "getrusage")
+    rusage_self = getattr(resource, "RUSAGE_SELF")
+    scale = 1 if sys.platform == "darwin" else 1024
+
+    def read() -> int | None:
+        return int(getrusage(rusage_self).ru_maxrss) * scale
+
+    read()
+    return read
+
+
+def _select_reader():
+    for candidate in (_rss_windows, _rss_proc_statm, _rss_getrusage):
+        try:
+            return candidate()
+        except Exception:  # noqa: BLE001 - any failure means "try the next one"
+            continue
+    return None
+
+
+_READER = _select_reader()
+
+
+def rss_bytes() -> int | None:
+    """This process's resident set size, or None where it cannot be read."""
+    if _READER is None:
+        return None
+    return _READER()
+
+
+def memory_metering_available() -> bool:
+    """Can `max_memory_mb` be enforced here?
+
+    Asked at construction so an unenforceable limit is **refused** rather than
+    accepted and quietly ignored. A limit that is declared and not enforced is
+    the pattern #473 and #478 were both filed for, and it is worse than no limit:
+    it is a security control an operator believes they have.
+    """
+    return rss_bytes() is not None

src/nodus/vm/vm.py

@@ -56,6 +56,7 @@ def __len__(self) -> int:
     _effective_timeout_ms,  # #596: read the step budget before suspending
 )
 from nodus.services.memory_runtime import GLOBAL_MEMORY_STORE, MemoryStore, delete_value, get_value, has_value, list_keys, put_value
+from nodus.runtime.memory import rss_bytes
 from nodus.runtime.runtime_stats import runtime_time_ms, scheduler_stats, task_snapshot
 from nodus.runtime.runtime_events import RuntimeEventBus
 from nodus.runtime.capability import ALLOW, ASK, BUILTIN_CAPABILITIES, DEFAULT_FLOOR, ApprovalChannel, CapabilityPolicy, CapabilityRequest, emit_denied, inherit_authority
@@ -331,6 +332,20 @@ def __init__(
         self.max_frames: int | None = MAX_STACK_DEPTH
         self.max_steps: int | None = None
         self.deadline: float | None = None
+        # #160: the third bound. `max_steps` limits instructions and `deadline`
+        # limits wall clock; neither stops a script growing a list until the host
+        # gets a MemoryError. Off by default like the other two, and for the same
+        # reason -- how much memory a run may have is host policy.
+        #
+        # An **absolute RSS ceiling**, not a limit plus a baseline. The caller
+        # sets it to `rss_at_start + budget`, so it still bounds *growth* -- and
+        # it costs the VM ONE instance attribute rather than four. That is not
+        # tidiness: `tests/test_vm_attribute_budget.py` caught the first version
+        # of this change at 81 attributes against PyPy's 80-attribute map limit,
+        # which cost ~9x throughput the last time it was crossed (#702, #488).
+        # The interval is a class constant below, and the modulo in
+        # `record_instruction` replaces a "last checked" counter.
+        self.max_memory_bytes: int | None = None
         self.trace_scheduler = trace_scheduler
         self.scheduler_output = scheduler_output
         self._task_counter = 0
@@ -2389,6 +2404,14 @@ def builtin_workflow_resume_payload(self):
     #: Options `workflow_wait`'s map form accepts. Closed and checked, like the
     #: `budget` and step-option vocabularies -- an unknown key is a mistake the
     #: author can fix now, not a silently discarded declaration (#490's rule).
+    #: Instructions between RSS samples for `max_memory_bytes` (#160). 256, not
+    #: the deadline's 100 and not the 2000 this started at: one RSS read costs
+    #: ~4.4 us against ~2.75 us per Nodus instruction (#173), so this is ~0.6%
+    #: overhead -- while 2000 was coarse enough that a program doubling a string
+    #: reached a MemoryError between two checks. A class attribute, so it costs
+    #: no per-instance slot.
+    MEMORY_CHECK_INTERVAL = 256
+
     WAIT_OPTION_KEYS = ("correlation_key", "payload", "deadline_ms", "schema", "on_timeout")
 
     def builtin_workflow_wait(self, event_type, correlation_key=None, payload=None, deadline_ms=None):
@@ -3193,6 +3216,39 @@ def run_closure(self, closure, args: list, workflow_context: dict | None = None,
             if saved_module_ctx is not None:
                 self._restore_module_ctx(saved_module_ctx)
 
+    def _check_memory_limit(self) -> None:
+        """Abort if this run has grown the process past its bound (#160).
+
+        Measures **growth since the VM started**, not absolute RSS. An embedded
+        runtime shares a process with its host, and a host with a 2 GB footprint
+        would trip a 100 MB absolute limit before running an instruction. Growth
+        is the quantity an embedder can reason about without knowing their own
+        baseline -- with the caveat, stated in `SECURITY_POSTURE.md`, that a host
+        allocating concurrently on another thread is counted too.
+
+        Polling bounds growth *over time*, not a single allocation: one enormous
+        list is served before the next check runs. That is a real limit of this
+        approach and the reason OS-level limits remain the answer for hard memory
+        safety.
+        """
+        # Bound locally. The `is not None` guard is at the call site in
+        # `record_instruction`, and extracting the body into a helper drops
+        # mypy's narrowing across that boundary -- the same trap CLAUDE.md
+        # records from the bytecode-cache refactor.
+        ceiling = self.max_memory_bytes
+        if ceiling is None:
+            return
+        current = rss_bytes()
+        if current is None:
+            return  # metering vanished mid-run; refused at construction, not here
+        if current > ceiling:
+            err = RuntimeLimitExceeded(
+                f"Memory limit exceeded: this run grew the process to "
+                f"{current // 1048576} MB, past its {ceiling // 1048576} MB ceiling"
+            )
+            self.emit_runtime_error(err)
+            raise err
+
     def record_instruction(self) -> None:
         self.instructions_executed += 1
         if self.task_step_budget is not None:
@@ -3210,6 +3266,9 @@ def record_instruction(self) -> None:
             err = RuntimeLimitExceeded("Execution step limit exceeded")
             self.emit_runtime_error(err)
             raise err
+        if (self.max_memory_bytes is not None
+                and self.instructions_executed % self.MEMORY_CHECK_INTERVAL == 0):
+            self._check_memory_limit()
         if self.instructions_executed - self._last_batch_emit >= self._instruction_batch_size:
             count = self.instructions_executed - self._last_batch_emit
             # Advanced whether or not the event is emitted: leaving it behind