Skip to content

docs: clarify that trace queries should use observations endpoint - #94

Closed
Lotte-Verheyden wants to merge 1 commit into
mainfrom
clarify-observations-endpoint
Closed

docs: clarify that trace queries should use observations endpoint#94
Lotte-Verheyden wants to merge 1 commit into
mainfrom
clarify-observations-endpoint

Conversation

@Lotte-Verheyden

@Lotte-Verheyden Lotte-Verheyden commented Aug 12, 2026

Copy link
Copy Markdown
Member

Users almost always describe their data as "traces," but the modern query endpoint in the CLI and API is observations, not traces. The traces endpoints are outdated. This updates the CLI reference tip so agents default to observations even when a request is phrased in terms of traces.

Bumps both plugin manifests 1.5.21.5.3 (patch, in lockstep) for the skill-content clarification.


Note

Low Risk
Docs and manifest version bumps only; no runtime or API behavior changes.

Overview
Documentation-only update so agents using the Langfuse CLI skills default to the modern observations API when users ask for “traces,” instead of outdated traces endpoints.

The CLI reference Tips section replaces the narrow “traces list can time out on Cloud” note with explicit guidance: treat user language as traces, but always query via observations (use --trace-id when scoping to a known trace), with a link to the Observations API v1→v2 mapping.

.claude-plugin/plugin.json and .cursor-plugin/plugin.json versions are bumped 1.5.2 → 1.5.3 in lockstep for this skill-content patch.

Reviewed by Cursor Bugbot for commit dc15b8b. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

Copy link
Copy Markdown

@claude review

@Lotte-Verheyden
Lotte-Verheyden deleted the clarify-observations-endpoint branch August 12, 2026 21:02
- Prefer `metrics` over `legacy-metrics-v1s` for the same reason
- Prefer `scores` over `legacy-score-v1s` for list/get operations
- For broad trace queries, `traces list` can time out on Langfuse Cloud — use `observations list` (with `--trace-id` if you're traversing from a known trace) instead. See the [Observations API docs](https://langfuse.com/docs/api-and-data-platform/features/observations-api) for the v1 → v2 mapping.
- Users almost always say "traces," but the modern query endpoint is `observations`, not `traces` — the `traces` endpoints are outdated. Always query via `observations` (add `--trace-id` to scope to a known trace), even when the request is phrased in terms of traces. See the [Observations API docs](https://langfuse.com/docs/api-and-data-platform/features/observations-api) for the v1 → v2 mapping.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new tip claims the traces endpoints are outdated and that agents should always use observations instead, even for trace-phrased requests — but this is factually wrong: per the live CLI schema, traces is a current, non-legacy resource (unlike legacy-observations-v1s/legacy-metrics-v1s/legacy-score-v1s), and traces list exposes trace-level aggregates (e.g. scores_avg, errorCount/warningCount, whole-trace totalCost/latency/totalTokens) that observations cannot produce, while observations has no get/delete action at all. An agent following this tip literally could not answer trace-aggregate queries ("traces with high total cost") or get/delete a specific trace. The fix is to restore the prior scoping — prefer observations list for broad/timeout-prone trace listing, not as an absolute replacement for traces.

Extended reasoning...

The bug: The updated tip at skills/langfuse/references/cli.md:59 states: "the modern query endpoint is observations, not traces — the traces endpoints are outdated. Always query via observations ... even when the request is phrased in terms of traces." Both of the tip's core claims are factually incorrect when checked against the live CLI schema (npx langfuse-cli api __schema, api traces --help, api traces list --help, api observations --help).

Claim 1 — "the traces endpoints are outdated": false. The schema explicitly marks deprecated resources with a legacy- prefix — legacy-observations-v1s, legacy-metrics-v1s, legacy-score-v1s — which is exactly the convention the file's own preceding three bullets (lines 56-58) rely on to justify preferring observations/metrics/scores. traces carries no such prefix; it is a first-class resource with 4 actions (list, get, delete-public, delete-delete-multiple).

Claim 2 — "always query via observations, even for trace-phrased requests": this doesn't hold either. traces list exposes trace-level aggregate field groups that observations list simply does not have: scores_avg/score_categories, errorCount/warningCount/defaultCount/debugCount, and whole-trace totalCost/latency/inputTokens/outputTokens/totalTokens aggregated across all of a trace's observations. observations list only returns per-observation fields plus a trace_context group (tags, release, traceName) — there is no aggregation across a trace's observations and no scores group. observations also has only a list action, with no get/delete, so a request to fetch or delete a specific trace cannot be fulfilled via observations at all.

Concrete failure walkthrough: Suppose a user asks "which traces had the highest total cost last week?" An agent following this tip literally would call observations list (per-observation records) instead of traces list --fields ...totalCost.... Observations don't carry a whole-trace totalCost field, so the agent would have to manually fetch every observation, group by trace_id, and sum costs client-side — slow, error-prone, and exactly the kind of task traces list's aggregate columns exist to avoid. Similarly, "delete trace X" cannot be done via observations (no delete action) — only traces delete-public"/"delete-delete-multiple supports it. In both cases the agent following the tip as written would either fail the task or improvise an incorrect workaround.

Why this happened: the PR's stated intent was to steer agents away from a real problem — traces list timing out on broad queries against Langfuse Cloud, per the pre-PR wording ("For broad trace queries, traces list can time out ... use observations list ... instead"). That scoped guidance was correct. The rewrite over-generalized it into an absolute "traces is outdated, always use observations" claim, which is not supported by the schema and actively blocks trace-aggregate and trace-get/delete workflows.

Fix: restore the original scoping — recommend observations list specifically for broad/timeout-prone trace listing (optionally scoped with --trace-id), while keeping traces list/get/delete as the correct tool for trace-aggregate queries and single-trace get/delete. All three independent verifiers confirmed this against the live schema and none refuted it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant