Skip to content

Support additional BitNet models #1

Description

@m96-chan

Currently only BitNet b1.58 2B-4T (ggml-model-i2_s.gguf) is tested and listed as a preset.

Candidates

  • bitnet_b1_58-large (0.7B) — 1bitLLM/bitnet_b1_58-large
    • Previously had BITNET_0_7B_CONFIG; needs GGUF + end-to-end verification
  • Falcon3-Edge 1B / 3B — BitNet-quantized Falcon models
  • Aramis-2B — Community BitNet model

What "supported" means

  1. GGUF file loads without errors
  2. Generates coherent text
  3. Added as a preset in example apps (if GGUF is publicly hosted)
  4. Listed in docs and README

Notes

  • Auto-detection from GGUF metadata should handle most models already
  • Main blocker is finding reliable public GGUF URLs and verifying output quality
  • BITNET_0_7B_CONFIG and BITNET_3B_CONFIG still exist in code but are untested

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions