🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.

A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

An open-source visual programming environment for battle-testing prompts to LLMs.

Turn expensive prompts into cheap fine-tuned models

Laminar - open-source all-in-one platform for engineering AI products. Create data flywheel for your AI app. Traces, Evals, Datasets, Labels. YC S24.

The open LLM Ops platform - Traces, Analytics, Evaluations, Datasets and Prompt Optimization ✨

LLM Prompt Injection Detector

📊 llm.report is an open-source logging and analytics platform for OpenAI: Log your ChatGPT API requests, analyze costs, and improve your prompts.

Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊

The only fully local production-grade Super SDK that provides a simple, unified, and powerful interface for calling more than 200+ LLMs.

Smoothly Manage Multiple LLMs (OpenAI, Anthropic, Azure) and Image Models (Dall-E, SDXL), Speed Up Responses, and Ensure Non-Stop Reliability.

Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry

Integrating AI into every workflow with our open-source, no-code platform, powered by the actor model for dynamic, graph-based solutions.

Visual node-edge graph GUI editor for LangGraph and run with local LLM or online API

Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application

Multi-Agent Conversation Framework in TypeScript