I created a WeDLM OpenAI-Compatible Server Setup #5
Description
🚀 Just got Tencent's WeDLM running with an OpenAI-compatible API server! 🎉
Super smooth setup using FastAPI + UV package manager. Now I can easily integrate WeDLM-8B-Instruct into my projects with standard OpenAI client libraries.
📦 Quick setup:
1️⃣ Install dependencies
2️⃣ Download the model
3️⃣ Run the server
4️⃣ Start making API calls!
Already seeing great performance with flash-attention optimization. If you're working with LLMs, definitely worth checking out.
Check it out here: https://gist.github.com/cyysky/bedad2eeb7e440f5838e7cb8d2f11ca6