Skip to content

Remove stale and volatile Langfuse skill guidance - #92

Merged
Lotte-Verheyden merged 2 commits into
mainfrom
review-posthog-skill-advice
Aug 7, 2026
Merged

Remove stale and volatile Langfuse skill guidance#92
Lotte-Verheyden merged 2 commits into
mainfrom
review-posthog-skill-advice

Conversation

@Lotte-Verheyden

@Lotte-Verheyden Lotte-Verheyden commented Aug 7, 2026

Copy link
Copy Markdown
Member

Summary

  • remove stale user-feedback SDK examples and defer implementation details to current Langfuse docs
  • retain durable frontend/server feedback workflow and security guidance
  • replace the universal judge-calibration cutoff with use-case-specific acceptance criteria
  • discover CLI score-creation capabilities from current schema and help instead of a pinned package version
  • bump both plugin manifests to 1.5.2

Validation

  • git diff --check origin/main...HEAD
  • validated both plugin manifests as JSON
  • confirmed manifest versions remain in lockstep
  • confirmed SKILL.md is unchanged

Note

Low Risk
Documentation and plugin version metadata only; no runtime code or security-sensitive logic changes.

Overview
Updates Langfuse skill references so agents rely on live docs instead of copy-pasted SDK snippets, and bumps the Claude and Cursor plugin manifests from 1.5.0 to 1.5.2.

In user-feedback, the inlined LangfuseWeb / Vercel AI SDK examples are removed in favor of pointers to the current user-feedback and score-ingestion guides, while keeping the workflow (implicit vs explicit feedback, public-key-only in the browser, naming rules, and common mistakes).

In judge-calibration, the fixed TPR/TNR > 0.90 ship gate is replaced with use-case-specific acceptance criteria (false positive/negative costs and monitoring vs automation). CLI score-creation advice no longer pins langfuse-cli@0.0.10; it tells readers to inspect current schema and --help instead of assuming capabilities from a version.

Reviewed by Cursor Bugbot for commit 55f9442. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — this is a clean documentation-only change that removes stale/volatile guidance in favor of live docs.

What was reviewed: the user-feedback.md diff (removed inlined LangfuseWeb/Vercel AI SDK code samples in favor of doc links, consistent with the repo's "no committed code" rule) and judge-calibration.md diff (replaced the fixed TPR/TNR>0.90 gate and pinned langfuse-cli@0.0.10 note with use-case-specific acceptance criteria and "inspect current schema/--help" guidance). Confirmed both plugin manifests bump together (1.5.0 → 1.5.2) and SKILL.md/routing/required_access are untouched, matching the diff's own validation notes.

Extended reasoning...

Overview

This PR touches only two Langfuse skill reference files (judge-calibration.md, user-feedback.md) and the two plugin manifests. No SKILL.md changes, no new files, no code executed by the harness — purely prose edits to agent-facing reference docs plus a version bump.

Security risks

None. The change actually improves the security posture of the guidance slightly by generalizing 'use the current browser SDK with a public key only, never expose secret key in browser code' instead of hardcoding an SDK class name that could go stale, while preserving the substantive security warning.

Level of scrutiny

Low. This is exactly the kind of change AGENTS.md encourages: removing committed code samples that could go stale in favor of docs links, and replacing a universal numeric threshold with judgment-based acceptance criteria. Both edits reduce staleness risk rather than introduce it. The version bump (1.5.0 → 1.5.2) is a reasonable patch-level bump for content clarifications, and both manifests stay in lockstep as required.

Other factors

The bug-hunting system found no bugs. Two candidate issues (missing LANGFUSE_PROJECT_INTERFACE in required_access, and a checklist ordering nit in the quality-gate section) were raised and refuted by verifiers, and I independently read both modified files end-to-end and agree neither rises to a real defect — the required_access list is pre-existing/unchanged by this diff, and the 'acceptance criteria' checklist item is a reminder about when to set thresholds, not a claim of strict temporal ordering. No outstanding reviewer comments remain unaddressed.

@Lotte-Verheyden
Lotte-Verheyden merged commit b9958d6 into main Aug 7, 2026
5 checks passed
@Lotte-Verheyden
Lotte-Verheyden deleted the review-posthog-skill-advice branch August 7, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant