-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Open
Labels
Description
Summary
Add PostHog LLM analytics integration for the OpenAI Agents SDK, enabling automatic tracking of agent execution, tool calls, handoffs, and LLM generations.
Implementation
The integration uses the SDK's official TracingProcessor interface to capture:
- Agent spans (
$ai_spantype=agent) - LLM generations (
$ai_generation) with token counts and costs - Tool/function calls (
$ai_spantype=tool) - Agent handoffs (
$ai_spantype=handoff) - Guardrail checks (
$ai_spantype=guardrail) - Custom spans (
$ai_spantype=custom) - Full trace hierarchy (
$ai_trace) - Conversation linking via
$ai_group_id(maps to SDK'sgroup_idinRunConfig)
Properties
$ai_provider: "openai" (the underlying LLM provider)$ai_framework: "openai-agents" (identifies the framework)$ai_total_tokens: Sum of input + output tokens$ai_error_type: Error categorization (model_behavior_error, user_error, input_guardrail_triggered, output_guardrail_triggered, max_turns_exceeded, unknown)
Usage
from posthog.ai.openai_agents import instrument
instrument(
client=posthog,
distinct_id="user_123",
privacy_mode=False,
)
# Run agents as normal - traces automatically sent to PostHogPull Requests
- posthog-python: SDK integration - feat(ai): add OpenAI Agents SDK integration posthog-python#408
- posthog: Onboarding content - chore(docs): add OpenAI Agents SDK LLM analytics installation #45071
- posthog.com: Docs wrapper - feat(llm-analytics): add OpenAI Agents SDK installation docs posthog.com#14393
- llm-analytics-apps: Example app - feat: add OpenAI Agents SDK example with PostHog instrumentation llm-analytics-apps#31
Branches
- posthog-python
feat/llma-add-openai-agents-sdk: https://github.com/PostHog/posthog-python/tree/feat/llma-add-openai-agents-sdk - llm-analytics-apps
feat/openai-agents-sdk-tracing: https://github.com/PostHog/llm-analytics-apps/tree/feat/openai-agents-sdk-tracing
Real-World Example
- github-standup-agent: A multi-agent system that generates daily standup summaries from GitHub activity. Uses the PostHog OpenAI Agents SDK integration for tracing the Coordinator → Data Gatherer → Summarizer agent workflow.
Follow-up
- Once published, submit PR to openai-agents-python to add PostHog to the official tracing integrations list
- Add UI support for grouping/filtering traces by
$ai_group_id