🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
-
Updated
Sep 7, 2024 - Python
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
💬 Chatbot web app + HTTP and Websocket endpoints for LLM inference with the Petals client
An easy to use and understand method for the average user to test various aspects of intelligence of your LLM in only one run.
Add a description, image, and links to the guanaco topic page so that developers can more easily learn about it.
To associate your repository with the guanaco topic, visit your repo's landing page and select "manage topics."