Skip to content

Support additional BitNet models (Falcon-E, BitNet 0.7B)#14

Merged
m96-chan merged 2 commits into
mainfrom
feat/multi-model-support
Feb 22, 2026
Merged

Support additional BitNet models (Falcon-E, BitNet 0.7B)#14
m96-chan merged 2 commits into
mainfrom
feat/multi-model-support

Conversation

@m96-chan

@m96-chan m96-chan commented Feb 22, 2026

Copy link
Copy Markdown
Owner

Summary

  • Fix activation detection: Only arch=bitnet-25 (BitNet 2B-4T) uses relu²; all other architectures default to silu. Removes broken isOfficial heuristic that misclassified Falcon-E (large vocab) and BitNet 0.7B (arch=bitnet).
  • Handle F16 untied LM head: Falcon-E models have output.weight as F16, not I2_S ternary. Keep as raw F16 and route through the f32_matmul shader path instead of BitLinear.
  • Add ChatML template: Auto-detect <|im_start|>/<|im_end|> tokens in vocab and apply ChatML format. Also adds <|im_end|> as a stop token.
  • Fix BITNET_0_7B_CONFIG: Correct hiddenSize (2048→1536), intermediateSize (5632→4096), numAttentionHeads (32→16), numKeyValueHeads (32→16).
  • Add model presets: Falcon-E 1B (~400 MB), Falcon-E 3B (~700 MB), BitNet 0.7B (~200 MB) in web-chat demo.

Closes #1

Files Changed

File Changes
packages/core/src/types.ts Add lmHeadF16?: boolean to ModelConfig
packages/core/src/model/loader.ts Fix activation detection; F16 lm_head handling; skip dummy scale for F16 head
packages/core/src/model/config.ts Fix BITNET_0_7B_CONFIG dimensions
packages/core/src/nn/model.ts F16 untied lm_head path; parameterize dispatchLMHead weight buffer
packages/core/src/tokenizer/tokenizer.ts Add ChatML template; imEndTokenId getter
packages/core/src/index.ts Stop on `<
examples/web-chat/src/main.ts Add 3 model presets

Test plan

  • npm run build passes
  • npm run lint passes
  • Load BitNet 2B-4T — verify no regression (relu², LLaMA 3 template, tied embeddings)
  • Load BitNet 0.7B — verify silu activation, correct config dimensions
  • Load Falcon-E 1B — verify F16 lm_head, ChatML template, <|im_end|> stop token
  • Load Falcon-E 3B — same as 1B

🤖 Generated with Claude Code

m96-chan and others added 2 commits February 22, 2026 10:07
Fix 4 bugs preventing non-2B-4T models from loading:
- Activation detection: only arch=bitnet-25 uses relu², others use silu
- F16 untied LM head: Falcon-E has F16 output.weight, not I2_S ternary
- ChatML template: auto-detect and apply ChatML for Falcon-E models
- BITNET_0_7B_CONFIG: fix wrong hiddenSize/intermediateSize/head counts

Add model presets for Falcon-E 1B, Falcon-E 3B, and BitNet 0.7B.

Closes #1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Falcon-E GGUF repos are under Instruct-GGUF, not plain GGUF.
BitNet 0.7B has no GGUF file available (only safetensors).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@m96-chan m96-chan merged commit f472dbb into main Feb 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support additional BitNet models

1 participant