NOTE

The tensorrt-llm-moe will be take over by sglang-project team(https://github.com/sgl-project), and the code will be moved to sgl-project/tensorrt-llm-moe.

And We will open source the repo when feature is stable and ready when integrate into SGLang.

Finally, I'm also a member of sglang devlopment team, welcome to join us.

tensorrt-llm-moe

Use TensorRT LLM MoE as a library

Build

apt-get install tensorrt
python3 setup.py develop

Test

end2end MoE test

python3 tests/tensorrt_llm_moe_test.py