expert-sharding

Here is 1 public repository matching this topic...

Distributed inference infrastructure for Mixture-of-Experts models. Run large MoE models on consumer GPUs.

infrastructure rust distributed-systems ai inference moe consumer-hardware llm expert-sharding

Add a description, image, and links to the expert-sharding topic page so that developers can more easily learn about it.

To associate your repository with the expert-sharding topic, visit your repo's landing page and select "manage topics."