Skip to content

Architecture

Jump to bottom
Raul Cardenas Montoya edited this page Sep 17, 2026 · 3 revisions

Architecture

Wiki hero

Role

.gguf file
    │ load_gguf / parse_bytes  (or load_gguf_mmap under --features mmap)
    ▼
GgufLayout { metadata, tensors, bytes }
    │ list_experts
    ▼
(block, expert)*
    │ extract_expert
    ▼
MoeExpertWeights { gate?, up?, down? as RawTensor }
    │ dequantize_packed (Q8_0 / Q5_K / Q6_K / IQ3_M)
    ▼  (downstream: routing, SNN — not this crate)
cortex-tensor / hybrid-fusion / neuromod

.safetensors / shard index / directory   (--features safetensors)
    │ inspect_safetensors_checkpoint
    ▼
SafetensorsManifest { checkpoint, tensors, index }
    │ discover_candidates
    ▼
SafetensorsCandidateSummary (routers + expert groups)

Owns

Area Detail
GGUF v3 deserialize Magic, version, KV, tensor directory
Expert enumeration list_experts
Per-expert raw extract gate/up/down with shape + dtype
Layout-aware dtypes F32/F16/BF16 + quant wire layouts
Packed dequant Q8_0 / Q5_K / Q6_K / IQ3_M block → f32
Optional mmap reader load_gguf_mmap (memmap2)
Safetensors metadata Header, deterministic manifests, shard-index / directory layouts, MoE candidate discovery — behind the safetensors feature

Does not own

Area Owner / note
Matmul / routing / softmax cortex-tensor etc.
Model-family adapters cortex-tensor
Safetensors payload mmap / tensor extraction out of scope for the metadata feature
Hugging Face config.json policy not ported from Corinth
GPU / SIMD never
Corinth experiment orchestration / SAAQ corinth-canal (never a dependency of this crate)

Reversed decision (#57): safetensors support ships in this crate behind the off-by-default safetensors cargo feature — not as a separate safetensors-parser crate. The feature is zero-dep: the upstream safetensors crate, serde_json, and corinth-canal are never used. The extraction is a one-way promotion from rmems/corinth-canal inspiration with no dependency in either direction; hard invariant: engram-parser never depends on corinth-canal.

Design principles

  • Zero dependencies by default — no thiserror, serde, memmap2 on the default path
  • Full file read into Vec<u8> by default (portable); optional mmap maps page-aligned tensor byte ranges for a later GPU crate to host-register
  • Malformed-input guards: max tensor/KV counts, max dims, EOF checks; JSON nesting capped
  • Shard paths resolve relative to the checkpoint root — absolute paths and .. escapes rejected
  • Extraction returns owned Vec<u8> so results outlive parsing choices

Related

Last updated: September 16, 2026 Updated by: KAI Package tip reference: 3dea2c3 (main)

Clone this wiki locally