Skip to content

install.md

Latest commit

 

History

History
36 lines (26 loc) · 887 Bytes

install.md

File metadata and controls

36 lines (26 loc) · 887 Bytes
Raw

Install

Requirements

  • Linux x86_64, NVIDIA GPU, driver r580+ (CUDA 13)
  • Python >= 3.10, with uv recommended (plain pip + venv works too)

Method 1: Install from PyPI

uv venv && source .venv/bin/activate
uv pip install "freetoken[accel]"

CUDA kernels are JIT-compiled on first use, need a CUDA 13 toolkit with nvcc on PATH.

Method 2: Install from source

git clone https://github.com/FlashML-org/FreeToken.git && cd FreeToken
uv venv && source .venv/bin/activate
uv pip install -e ".[accel]"

Verify

source .venv/bin/activate
ft --version
ft serve --model ~/path/to/Qwen3.6-35B-A3B
curl http://127.0.0.1:1919/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"Qwen3.6-35B-A3B","messages":[{"role":"user","content":"hi"}]}'

Then head to quickstart.md.