Repository navigation
MoE Extraction
src/moe/{mod,expert,extract}.rs
blk.{B}.ffn_gate_exps.weight
blk.{B}.ffn_up_exps.weight
blk.{B}.ffn_down_exps.weight
Shape: expert axis is the last dim (GGML innermost-first). Per-expert chunk is a contiguous byte stride: byte_len / n_experts.
blk.{B}.ffn_gate.{E}.weight (or ffn_gate_{E}.weight)
blk.{B}.ffn_up.{E}.weight
blk.{B}.ffn_down.{E}.weight
Whole tensor payload cloned; stacked_slice = false.
Reports a pair if any of gate/up/down is discoverable under either naming scheme. Sorted via BTreeSet.
| Field | Type |
|---|---|
block, expert
|
indices |
gate / up / down
|
Option<RawTensor> |
is_complete() |
all three present |
Prefers stacked names first, then per-expert candidates. If none found → MissingTensor. Expert ≥ available → ExpertOutOfRange.
Self-owning: source_name, dims (expert dim stripped for stacked), dtype, ggml_type, bytes, stacked_slice.
Last updated: September 16, 2026
Updated by: KAI
Package tip reference: 3dea2c3 (main)