Laminar

Laminar is the open-source platform for tracing and evaluating AI applications.

• Tracing
  • OpenTelemetry-based automatic tracing of common AI frameworks and SDKs (LangChain, OpenAI, Anthropic ...) with just 2 lines of code. (powered by OpenLLMetry).
  • Trace input/output, latency, cost, token count.
  • Function tracing with observe decorator/wrapper.
  • Image tracing.
• Evals
  • Run evals in parallel with a simple SDK
• Datasets
  • Export production trace data to datasets.
  • Run evals on hosted datasets.
• Built for scale
  • Written in Rust 🦀
  • Traces are sent via gRPC, ensuring the best performance and lowest overhead.
• Modern Open-Source stack
  • RabbitMQ for message queue, Postgres for data, Clickhouse for analytics.
• Dashboards for statistics / traces / evaluations / labels.

Documentation

Check out full documentation here docs.lmnr.ai.

Getting started

The fastest and easiest way to get started is with our managed platform -> lmnr.ai

Self-hosting with Docker compose

For a quick start, clone the repo and start the services with docker compose:

git clone https://github.com/lmnr-ai/lmnr
cd lmnr
docker compose up -d

This will spin up a lightweight version of the stack with Postgres, clickhouse, app-server, and frontend. This is good for a quickstart or for lightweight usage. You can access the UI at http://localhost:5667 in your browser.

You will also need to properly configure the SDK, with baseUrl and correct ports. See https://docs.lmnr.ai/self-hosting/setup

For production environment, we recommend using our managed platform or docker compose -f docker-compose-full.yml up -d.

docker-compose-full.yml is heavy but it will enable all the features.

• app-server – core Rust backend
• rabbitmq – message queue for reliable trace processing
• qdrant – vector database
• semantic-search-service – gRPC service for embedding text and storing/retrieving it from qdrant
• frontend – Next.js frontend and backend
• python-executor – gRPC service with lightweight Python sandbox that can run arbitrary code.
• postgres – Postgres database for all the application data
• clickhouse – columnar OLAP database for more efficient trace and label analytics

Index - SOTA browser agent

Index is a state-of-the-art open-source browser agent. It can autonomously perform complex tasks on the web.

You can use Index:

• As an open-source package
• As a cloud API at lmnr.ai. Read the API reference to get started.
• It powers the Chat UI behind lmnr.ai/chat
  • Chat UI is also self-hostable. Read the self-hosting guide below and access the UI at <YOUR-SELF-HOST-URL>:5667/chat

Spinning up Index UI locally

Currently local version of Laminar Index connects to browsers hosted on Scrapybara, and its reasoning is powered by Anthropic's Claude models. You will need API keys for both of these services.

1. Get an Anthropic API key
2. Get a Scrapybara API key
3. Update both ANTHROPIC_API_KEY and SCRAPYBARA_API_KEY values in .env in the root of lmnr
4. Laminar index agent manager will spin up as part of the default docker-compose up.

Contributing

For running and building Laminar locally, or to learn more about docker compose files, follow the guide in Contributing.

TS quickstart

First, create a project and generate a project API key. Then,

npm add @lmnr-ai/lmnr

It will install Laminar TS SDK and all instrumentation packages (OpenAI, Anthropic, LangChain ...)

To start tracing LLM calls just add

import { Laminar } from '@lmnr-ai/lmnr';
Laminar.initialize({ projectApiKey: process.env.LMNR_PROJECT_API_KEY });

To trace inputs / outputs of functions use observe wrapper.

import { OpenAI } from 'openai';
import { observe } from '@lmnr-ai/lmnr';

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const poemWriter = observe({name: 'poemWriter'}, async (topic) => {
  const response = await client.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{ role: "user", content: `write a poem about ${topic}` }],
  });
  return response.choices[0].message.content;
});

await poemWriter();

Python quickstart

First, create a project and generate a project API key. Then,

pip install --upgrade 'lmnr[all]'

It will install Laminar Python SDK and all instrumentation packages. See list of all instruments here

To start tracing LLM calls just add

from lmnr import Laminar
Laminar.initialize(project_api_key="<LMNR_PROJECT_API_KEY>")

To trace inputs / outputs of functions use @observe() decorator.

import os
from openai import OpenAI

from lmnr import observe, Laminar
Laminar.initialize(project_api_key="<LMNR_PROJECT_API_KEY>")

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

@observe()  # annotate all functions you want to trace
def poem_writer(topic):
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "user", "content": f"write a poem about {topic}"},
        ],
    )
    poem = response.choices[0].message.content
    return poem

if __name__ == "__main__":
    print(poem_writer(topic="laminar flow"))

Running the code above will result in the following trace.

Client libraries

To learn more about instrumenting your code, check out our client libraries: