docs: clarify that trace queries should use observations endpoint - #94
docs: clarify that trace queries should use observations endpoint#94Lotte-Verheyden wants to merge 1 commit into
Conversation
|
@claude review |
| - Prefer `metrics` over `legacy-metrics-v1s` for the same reason | ||
| - Prefer `scores` over `legacy-score-v1s` for list/get operations | ||
| - For broad trace queries, `traces list` can time out on Langfuse Cloud — use `observations list` (with `--trace-id` if you're traversing from a known trace) instead. See the [Observations API docs](https://langfuse.com/docs/api-and-data-platform/features/observations-api) for the v1 → v2 mapping. | ||
| - Users almost always say "traces," but the modern query endpoint is `observations`, not `traces` — the `traces` endpoints are outdated. Always query via `observations` (add `--trace-id` to scope to a known trace), even when the request is phrased in terms of traces. See the [Observations API docs](https://langfuse.com/docs/api-and-data-platform/features/observations-api) for the v1 → v2 mapping. |
There was a problem hiding this comment.
🔴 The new tip claims the traces endpoints are outdated and that agents should always use observations instead, even for trace-phrased requests — but this is factually wrong: per the live CLI schema, traces is a current, non-legacy resource (unlike legacy-observations-v1s/legacy-metrics-v1s/legacy-score-v1s), and traces list exposes trace-level aggregates (e.g. scores_avg, errorCount/warningCount, whole-trace totalCost/latency/totalTokens) that observations cannot produce, while observations has no get/delete action at all. An agent following this tip literally could not answer trace-aggregate queries ("traces with high total cost") or get/delete a specific trace. The fix is to restore the prior scoping — prefer observations list for broad/timeout-prone trace listing, not as an absolute replacement for traces.
Extended reasoning...
The bug: The updated tip at skills/langfuse/references/cli.md:59 states: "the modern query endpoint is observations, not traces — the traces endpoints are outdated. Always query via observations ... even when the request is phrased in terms of traces." Both of the tip's core claims are factually incorrect when checked against the live CLI schema (npx langfuse-cli api __schema, api traces --help, api traces list --help, api observations --help).
Claim 1 — "the traces endpoints are outdated": false. The schema explicitly marks deprecated resources with a legacy- prefix — legacy-observations-v1s, legacy-metrics-v1s, legacy-score-v1s — which is exactly the convention the file's own preceding three bullets (lines 56-58) rely on to justify preferring observations/metrics/scores. traces carries no such prefix; it is a first-class resource with 4 actions (list, get, delete-public, delete-delete-multiple).
Claim 2 — "always query via observations, even for trace-phrased requests": this doesn't hold either. traces list exposes trace-level aggregate field groups that observations list simply does not have: scores_avg/score_categories, errorCount/warningCount/defaultCount/debugCount, and whole-trace totalCost/latency/inputTokens/outputTokens/totalTokens aggregated across all of a trace's observations. observations list only returns per-observation fields plus a trace_context group (tags, release, traceName) — there is no aggregation across a trace's observations and no scores group. observations also has only a list action, with no get/delete, so a request to fetch or delete a specific trace cannot be fulfilled via observations at all.
Concrete failure walkthrough: Suppose a user asks "which traces had the highest total cost last week?" An agent following this tip literally would call observations list (per-observation records) instead of traces list --fields ...totalCost.... Observations don't carry a whole-trace totalCost field, so the agent would have to manually fetch every observation, group by trace_id, and sum costs client-side — slow, error-prone, and exactly the kind of task traces list's aggregate columns exist to avoid. Similarly, "delete trace X" cannot be done via observations (no delete action) — only traces delete-public"/"delete-delete-multiple supports it. In both cases the agent following the tip as written would either fail the task or improvise an incorrect workaround.
Why this happened: the PR's stated intent was to steer agents away from a real problem — traces list timing out on broad queries against Langfuse Cloud, per the pre-PR wording ("For broad trace queries, traces list can time out ... use observations list ... instead"). That scoped guidance was correct. The rewrite over-generalized it into an absolute "traces is outdated, always use observations" claim, which is not supported by the schema and actively blocks trace-aggregate and trace-get/delete workflows.
Fix: restore the original scoping — recommend observations list specifically for broad/timeout-prone trace listing (optionally scoped with --trace-id), while keeping traces list/get/delete as the correct tool for trace-aggregate queries and single-trace get/delete. All three independent verifiers confirmed this against the live schema and none refuted it.
Users almost always describe their data as "traces," but the modern query endpoint in the CLI and API is
observations, nottraces. Thetracesendpoints are outdated. This updates the CLI reference tip so agents default toobservationseven when a request is phrased in terms of traces.Bumps both plugin manifests
1.5.2→1.5.3(patch, in lockstep) for the skill-content clarification.Note
Low Risk
Docs and manifest version bumps only; no runtime or API behavior changes.
Overview
Documentation-only update so agents using the Langfuse CLI skills default to the modern
observationsAPI when users ask for “traces,” instead of outdatedtracesendpoints.The CLI reference Tips section replaces the narrow “
traces listcan time out on Cloud” note with explicit guidance: treat user language as traces, but always query viaobservations(use--trace-idwhen scoping to a known trace), with a link to the Observations API v1→v2 mapping..claude-plugin/plugin.jsonand.cursor-plugin/plugin.jsonversions are bumped 1.5.2 → 1.5.3 in lockstep for this skill-content patch.Reviewed by Cursor Bugbot for commit dc15b8b. Bugbot is set up for automated code reviews on this repo. Configure here.