Step memoization with idempotency keys for workflow replay #755

Part of #752 (Durable Execution v0.1 epic).

Problem

The replay model for durable workflows requires that on crash recovery (or any replay), completed steps return their cached output instead of re-executing. Without this, the workflow re-invokes every step — including external calls — every time it resumes.

Proposed shape

A workflow-aware checkpoint store with a schema specialized for the workflow runtime's access pattern:

Integration with existing primitives

• Commits ride inside the existing `transaction(fn)` from `resources/transaction.ts` — this is what gives `ctx.atomic` its differentiator: the workflow checkpoint and the user's table writes commit as one LMDB/RocksDB transaction.
• For `ctx.step` (non-atomic): the checkpoint is its own small transaction, separate from anything the step body did in external systems.
• For `ctx.atomic`: the checkpoint write is spliced into the user-supplied `transaction(fn)` so both commit together.

Retention

The global audit log retention default is 24 hours (`auditStore.ts:109`). Workflow checkpoints need their own retention policy — independent, configurable, defaulting to something workflow-lifecycle-appropriate (workflow completes → checkpoints retained N days, then GC'd; running workflows → retained indefinitely).

Indexes

• Primary: `(workflowId, stepId)` for replay lookup.
• Secondary: `(workflowId, completedAt)` for transcript reconstruction.
• Possibly tertiary: `(idempotencyKey)` for cross-workflow dedup if we expose it as an API.

Idempotency-key generation

The runtime needs a stable key per logical step invocation that is identical across replays. Proposed derivation: `hash(workflowId || stepId || invocationCount)` for in-loop steps, with the engine providing the value to step bodies as `memo.idempotencyKey` for forwarding to external services (the doc's email/LLM examples).

Open questions

• Hard size cap per checkpoint output? Large LLM responses or vector blobs are common — paginate, or store large outputs in a separate blob ref?
• Compression of output payloads on commit?
• Cross-workflow idempotency-key namespace (engine-internal vs. user-supplied for at-most-once external calls)?

Out of scope

Step memoization for steps that span databases — that requires cross-DB atomicity which is the v0.2 follow-up in #752.

🤖 Filed by Claude on behalf of @kriszyp