checkpoint: consume engram-parser v0.3 checkpoint/tensor APIs and retire in-tree GGUF ownership #47

2026-09 direction update

This issue remains the checkpoint-boundary migration tracker, but the target has changed.

cortex-tensor is no longer being developed as an independent checkpoint/tensor framework. Its role is becoming ANN backend interoperability and reference execution for ANN/SNN hybrid systems.

The old plan to consume engram-parser 0.2.0 only to wrap parse_checkpoint_layout is superseded.

Goal

Consume the published engram-parser 0.3.x checkpoint/tensor substrate and retire long-term ownership of GGUF parsing/layout/mmap policy from cortex-tensor.

Version ownership: this issue ships in cortex-tensor v0.2 — Boundary reset & reference backend. The v0.3 dependency requirement refers to engram-parser, not a cortex-tensor release. These versions are independent.

Dense Transformer tensor-role discovery and ANN/SNN/hybrid topology are owned by engram-parser v0.4 (rmems/engram-parser#88 and its milestone). Do not make that later topology work an acceptance gate for this checkpoint-boundary migration.

Cortex should receive normalized checkpoint/tensor information and bind those weights into ANN execution stages. It should not know whether the source checkpoint was GGUF or Safetensors in its core execution APIs.

Target boundary

GGUF / Safetensors
       |
       v
engram-parser
  format-independent checkpoint / tensor metadata / raw payload access
       |
       v
cortex-tensor
  ANN stage weight binding + execution

Scope

• depend on a published engram-parser 0.3.x release when its required checkpoint/tensor surface is ready;
• replace the in-tree GGUF header/KV/tensor-directory walk with engram-owned APIs;
• migrate checkpoint-backed MoE/Transformer weight lookup toward the v0.3 normalized checkpoint/tensor API; defer automatic topology/role discovery to the engram-parser v0.4 track;
• remove new format-specific growth from src/moe/checkpoint.rs / src/moe/gguf.rs;
• preserve compatibility wrappers only as migration shims;
• keep ANN numerical execution and routing math where still useful.

Acceptance criteria

• GGUF and Safetensors consumer paths use engram-owned checkpoint/tensor metadata and payload access.
• Equivalent fixtures preserve tensor inventory and existing reference-backend weight-binding behavior.
• Malformed/unsupported input fails with explicit errors rather than falling back to the retired parser.
• Remove the in-tree GGUF parsing/layout implementation; document any temporary compatibility wrapper and its removal condition.
• Record the exact engram-parser version and required features used for validation; release dependency resolves from the published v0.3.x package.
• No ModelTopology/TensorRole or other v0.4-only API is required to close this issue.

No longer cortex-tensor responsibilities

• owning GGUF mmap policy;
• expanding GGUF dtype/parser coverage;
• Safetensors format parsing;
• GPU/SAAQ synapse-source policy;
• general ML-framework replacement work.

Blocked by / related

• architecture: evolve engram-parser into an ANN/SNN checkpoint substrate engram-parser#85 / engram-parser#85
• safetensors: adopt upstream crate and add mmap-backed raw tensor access engram-parser#86 / engram-parser#86
• checkpoint: add a format-independent tensor/checkpoint abstraction for GGUF and Safetensors engram-parser#87 / engram-parser#87
• release: publish engram-parser v0.3.0 checkpoint-substrate release engram-parser#89 / engram-parser#89

GitHub canonical public issue: #47