Skip to content

[feat] support deepseek_v41: switch blockwise mxfp8 to OCP mxfp8. - #2426

Open
lkk12014402 wants to merge 2 commits into
intel:mainfrom
lkk12014402:feat/deepseek-v41-model-free
Open

lkk12014402 wants to merge 2 commits into
intel:mainfrom
lkk12014402:feat/deepseek-v41-model-free

Conversation

@lkk12014402

Copy link
Copy Markdown
Contributor

Enable DeepSeek-V4.1 llm-compressor export support:

  1. Extend model-type matching to include deepseek_v41, which shares the same native quantized layout as deepseek_v4.
  2. Fix a gap where V4.1 checkpoints skipped the llm-compressor renaming pipeline due to strict model-type matching.
  3. Preserve Engram conditional-memory tensors (layers.{1,14}.engram.*) during conversion, avoiding false matches with Linear weight/scale patterns.
  4. Validate on DeepSeek-V4.1-Flash (552B): 40/40 layers exported successfully, MXFP4 experts and MXFP8 linears converted correctly, with Engram tensors remaining byte-identical to source checkpoints and successful vLLM nightly evaluation.

Signed-off-by: lkk12014402 <kaokao.lv@intel.com>
Signed-off-by: lkk12014402 <kaokao.lv@intel.com>
@AutoRoundBot

Copy link
Copy Markdown
Collaborator

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants