Repository navigation
Conversation
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (15)
📝 WalkthroughWalkthroughAdds role-based getting-started pages, evaluation and human-review documentation, five cookbook guides, and a new Cookbook index. Updates navigation with onboarding, evaluation, and cookbook entries. The cookbook content covers agent evaluation, latency diagnosis, prompt version rollouts, and LLM cost tracking with dashboards and alerts. Possibly related PRs
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
docs/get-started/choose-your-path.md (1)
3-3: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExpand reader-unfamiliar acronyms at their first visible use.
docs/get-started/choose-your-path.md#L3-L3: Replacebackend/SREwith “backend/site reliability engineer” or “backend/site reliability engineer (SRE)”.docs/get-started/for-backend-and-sre.md#L2-L2: ExpandSREsin the title to “site reliability engineers (SREs)”.docs/get-started/for-platform-and-security.md#L20-L20: ReplaceKeycloak OIDC providerwith “Keycloak OpenID Connect (OIDC) provider”.As per coding guidelines, acronyms must be spelled out at first use. Based on learnings, only the approved common acronyms may remain unexpanded.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/get-started/choose-your-path.md` at line 3, Expand reader-unfamiliar acronyms at their first use across all affected documentation sites: in docs/get-started/choose-your-path.md lines 3-3, replace “backend/SRE” with “backend/site reliability engineer” or the parenthetical “backend/site reliability engineer (SRE)”; in docs/get-started/for-backend-and-sre.md lines 2-2, expand “SREs” to “site reliability engineers (SREs)”; and in docs/get-started/for-platform-and-security.md lines 20-20, replace “Keycloak OIDC provider” with “Keycloak OpenID Connect (OIDC) provider”.Sources: Coding guidelines, Learnings
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/cookbook/evaluate-an-agent.md`:
- Around line 230-247: Update the regression-results section around the ✗✔ rows
and 66.7% pass rate to explicitly label the displayed output as illustrative and
potentially variable between runs; do not present these values as fixed results
unless replacing the evaluation with a deterministic stub.
- Around line 155-160: Update MentionsFact.evaluate so it does not treat an
empty expected_output as a valid fact; ensure the refund_policy example supplies
a meaningful expected fact or routes that case through a judge-only evaluation
path. Preserve the existing case-insensitive containment check for non-empty
expected outputs.
- Around line 136-145: Remove the redundant Logfire setup calls from the
evaluation script: delete logfire.configure() and
logfire.instrument_pydantic_ai() near the imports, relying on agent.py to
perform telemetry initialization on import. Remove the now-unused logfire import
as well.
In `@docs/cookbook/track-llm-cost.md`:
- Around line 21-22: Update the prerequisites in track-llm-cost.md to explicitly
instruct readers to set the OPENAI_API_KEY environment variable alongside
LOGFIRE_TOKEN, ensuring the sample’s OpenAI authentication setup is complete.
---
Nitpick comments:
In `@docs/get-started/choose-your-path.md`:
- Line 3: Expand reader-unfamiliar acronyms at their first use across all
affected documentation sites: in docs/get-started/choose-your-path.md lines 3-3,
replace “backend/SRE” with “backend/site reliability engineer” or the
parenthetical “backend/site reliability engineer (SRE)”; in
docs/get-started/for-backend-and-sre.md lines 2-2, expand “SREs” to “site
reliability engineers (SREs)”; and in
docs/get-started/for-platform-and-security.md lines 20-20, replace “Keycloak
OIDC provider” with “Keycloak OpenID Connect (OIDC) provider”.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: ad531c9c-adf2-43ca-b979-b01e7338eca8
⛔ Files ignored due to path filters (2)
docs/images/guide/live-view-demo-shop.pngis excluded by!**/*.pngdocs/images/guide/llms-view.pngis excluded by!**/*.png
📒 Files selected for processing (14)
docs/cookbook/debug-a-slow-endpoint.mddocs/cookbook/evaluate-an-agent.mddocs/cookbook/index.mddocs/cookbook/roll-out-a-prompt-safely.mddocs/cookbook/track-llm-cost.mddocs/evaluate/human-review.mddocs/evaluate/index.mddocs/get-started/choose-your-path.mddocs/get-started/for-ai-engineers.mddocs/get-started/for-backend-and-sre.mddocs/get-started/for-platform-and-security.mddocs/get-started/for-product-and-growth.mddocs/get-started/solo-builder.mddocs/nav.json
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
pydantic/logfire(manual)pydantic/pydantic-ai(manual)pydantic/platform(auto-detected)pydantic/pydantic(auto-detected)
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 16 files
Confidence score: 5/5
- Safe to merge after the addressed issues were fixed.
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/cookbook/debug-a-slow-tool-call.md`:
- Line 75: Update the first public-documentation use of DSN in the example to
spell out “database connection string (DSN)” or replace the acronym with the
expanded term, while preserving the connection-string value and subsequent
usage.
- Around line 28-30: Add an explicit LOGFIRE_TOKEN setup step near the
walkthrough prerequisites before the tool-call example, using the copied Logfire
write token; alternatively, pass that token through logfire.configure(). Ensure
the walkthrough configures authentication so traces are sent to the intended
project.
- Around line 83-93: Update the top_products tool so its database operations
match its async signature: use Psycopg’s asynchronous connection and cursor APIs
and await the connection, execute, and fetch operations, preserving the existing
query and result formatting.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 24520d8a-e669-489f-a471-6c5a36c932eb
📒 Files selected for processing (4)
docs/cookbook/debug-a-slow-tool-call.mddocs/cookbook/index.mddocs/get-started/for-ai-engineers.mddocs/nav.json
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
pydantic/logfire(manual)pydantic/pydantic-ai(manual)pydantic/platform(auto-detected)pydantic/pydantic(auto-detected)
🚧 Files skipped from review as they are similar to previous changes (2)
- docs/cookbook/index.md
- docs/get-started/for-ai-engineers.md
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/evaluate/overview.md`:
- Line 52: Update the “Pass/fail (boolean)” documentation example to spell out
“personally identifiable information (PII)” at its first use, while preserving
the existing yes/no guidance and examples.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 2650b0a9-0f90-440b-a59a-808ce05c4166
📒 Files selected for processing (6)
docs/cookbook/evaluate-an-agent.mddocs/cookbook/roll-out-a-prompt-safely.mddocs/evaluate/human-review.mddocs/evaluate/overview.mddocs/get-started/for-ai-engineers.mddocs/nav.json
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
pydantic/logfire(manual)pydantic/pydantic-ai(manual)pydantic/platform(auto-detected)pydantic/pydantic(auto-detected)
🚧 Files skipped from review as they are similar to previous changes (4)
- docs/evaluate/human-review.md
- docs/get-started/for-ai-engineers.md
- docs/nav.json
- docs/cookbook/roll-out-a-prompt-safely.md
|
Macroscope skipped reviewing this pull request. Per-review cost limit exceeded (workspace setting). This review would cost an estimated $5.08, which exceeds your per-review limit of $5.00. The top 3 files driving up this estimate:
Tip To get this pull request reviewed, you can:
|
strawgate
force-pushed
the
docs/ia-entry-points
branch
from
July 21, 2026 15:35
f65ed24 to
2310781
Compare
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 14 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
strawgate
force-pushed
the
docs/ia-entry-points
branch
from
July 22, 2026 00:43
7020c6c to
a0f2bbd
Compare
strawgate
force-pushed
the
docs/ia-entry-points
branch
5 times, most recently
from
July 22, 2026 05:38
74f6b85 to
0d6ce76
Compare
Six runnable recipes plus a landing page: instrument and evaluate an agent, debug a slow endpoint, debug a slow tool call, roll out a prompt safely, and track and alert on LLM cost. Every code example is verified to run end to end, and each recipe closes with a short troubleshooting section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A "choose your path" hub plus five role pages (AI engineer, backend/SRE, platform/security, product/growth, solo builder) that route each audience to the pages that matter for them, each link with a one-line reason. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An Evaluate section introducing the datasets/scorers/scores/experiments model and how human review and user feedback land as the same kind of score. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
strawgate
force-pushed
the
docs/ia-entry-points
branch
from
July 22, 2026 05:42
0d6ce76 to
d15902a
Compare
Contributor
Author
|
Closing in favor of a fresh PR from the same branch (identical commits, rebased onto main). Continued below. |
Contributor
Author
|
Continued in #2102 (same commits, rebased onto main). |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/cookbook/track-llm-cost.md`:
- Around line 34-39: Update the credential placeholders in the setup commands
near the Logfire and OpenAI environment variables to be shell-safe, replacing
angle-bracket placeholders with unquoted names such as YOUR_LOGFIRE_WRITE_TOKEN
and YOUR_OPENAI_API_KEY while preserving the surrounding instructions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 0ea0a1d9-b4f3-47a8-8029-7649ea5083fb
⛔ Files ignored due to path filters (2)
docs/images/guide/live-view-demo-shop.pngis excluded by!**/*.pngdocs/images/guide/llms-view.pngis excluded by!**/*.png
📒 Files selected for processing (15)
docs/cookbook/debug-a-slow-endpoint.mddocs/cookbook/debug-a-slow-tool-call.mddocs/cookbook/evaluate-an-agent.mddocs/cookbook/index.mddocs/cookbook/roll-out-a-prompt-safely.mddocs/cookbook/track-llm-cost.mddocs/evaluate/human-review.mddocs/evaluate/overview.mddocs/get-started/choose-your-path.mddocs/get-started/for-ai-engineers.mddocs/get-started/for-backend-and-sre.mddocs/get-started/for-platform-and-security.mddocs/get-started/for-product-and-growth.mddocs/get-started/solo-builder.mddocs/nav.json
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
pydantic/logfire(manual)pydantic/pydantic-ai(manual)pydantic/platform(auto-detected)pydantic/pydantic(auto-detected)
🚧 Files skipped from review as they are similar to previous changes (7)
- docs/get-started/solo-builder.md
- docs/get-started/for-platform-and-security.md
- docs/evaluate/overview.md
- docs/cookbook/index.md
- docs/cookbook/debug-a-slow-tool-call.md
- docs/cookbook/roll-out-a-prompt-safely.md
- docs/nav.json
Comment on lines
+34
to
+39
| Set your write token and your OpenAI key so the app can send data and make model calls (in local development you can run `logfire projects use <your-project>` instead of the token): | ||
|
|
||
| ```bash | ||
| export LOGFIRE_TOKEN=<your write token from Project → Settings → Write tokens> | ||
| export OPENAI_API_KEY=<your key from platform.openai.com/api-keys> | ||
| ``` |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Use shell-safe credential placeholders.
The commands cannot be copied as written: <...> is parsed as shell redirection, and the placeholders contain unquoted spaces. Replace them with values such as YOUR_LOGFIRE_WRITE_TOKEN and YOUR_OPENAI_API_KEY, or quote the placeholders.
Proposed fix
-export LOGFIRE_TOKEN=<your write token from Project → Settings → Write tokens>
-export OPENAI_API_KEY=<your key from platform.openai.com/api-keys>
+export LOGFIRE_TOKEN='YOUR_LOGFIRE_WRITE_TOKEN'
+export OPENAI_API_KEY='YOUR_OPENAI_API_KEY'📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| Set your write token and your OpenAI key so the app can send data and make model calls (in local development you can run `logfire projects use <your-project>` instead of the token): | |
| ```bash | |
| export LOGFIRE_TOKEN=<your write token from Project → Settings → Write tokens> | |
| export OPENAI_API_KEY=<your key from platform.openai.com/api-keys> | |
| ``` | |
| Set your write token and your OpenAI key so the app can send data and make model calls (in local development you can run `logfire projects use <your-project>` instead of the token): | |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/cookbook/track-llm-cost.md` around lines 34 - 39, Update the credential
placeholders in the setup commands near the Logfire and OpenAI environment
variables to be shell-safe, replacing angle-bracket placeholders with unquoted
names such as YOUR_LOGFIRE_WRITE_TOKEN and YOUR_OPENAI_API_KEY while preserving
the surrounding instructions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds feature-oriented entry points onto the existing docs — all additive (new pages, no moves, no redirects), so this is independent of the larger IA reorg and safe to land on its own.
What's new
Verification
Every recipe was run for real against the SDK (Pydantic AI / Evals 2.x) and a live Logfire instance, which caught and fixed genuine bugs the pages had (they predated the 2.x migration):
Dataset(...)needed the now-requiredname=kwarg — the recipe errored as written; the printed table's columns and numbers were also wrong for Evals 2.x.debug-a-slow-endpoint's install was missingfastapiand thepsycopgdriver;@app.on_event→ lifespan; confirmed the query span nests in the request span anddb.statementis the right attribute.track-llm-cost'sgen_ai.*/operation.costattributes were confirmed present on real spans.skip-runtags so the examples pass the docs format/lint check.Where a recipe shows output, it uses a Gemini Flash model so the numbers are real and reproducible.
Test plan
pytest tests/test_docs.py -k test_formatting— green.