docs: improve installation and getting-started pages with tabbed interface and SEO metadata #4395
Conversation
…rface and SEO metadata
|
⏩ No test scenarios generated (0241dd0) View output ↗ Tip New to Tusk? Learn more here. |
WalkthroughThe documentation for promptfoo was extensively revised, with both the "Getting Started" and "Installation" guides receiving expanded content, restructured layouts, and enhanced clarity. Metadata blocks were added, installation and setup instructions were unified and tabbed by package manager, and example configurations were updated to emphasize rubric-based evaluation and simplified assertions. Changes
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🔭 Outside diff range comments (2)
site/docs/installation.md (1)
101-101: 🧹 Nitpick (assertive)Use extensionless link.
Docusaurus routing prefers paths without.md. Update[Getting Started guide](./getting-started.md)to[Getting Started guide](./getting-started)for consistency.site/docs/getting-started.md (1)
20-38: 🧹 Nitpick (assertive)Use unique
groupIdper tab group.
All<Tabs>in this document sharegroupId="promptfoo-command", leading to persistent tab selection across different examples. Assign distinct IDs (e.g.,"example-init","scratch-init","eval-cmd","view-cmd", etc.) to isolate each section's state.Also applies to: 49-65, 136-152, 158-174, 250-267, 335-352, 361-378
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
site/docs/getting-started.md(7 hunks)site/docs/installation.md(1 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.17.2)
site/docs/installation.md
12-12: Multiple top-level headings in the same document
null
(MD025, single-title, single-h1)
23-23: Inline HTML
Element: Tabs
(MD033, no-inline-html)
24-24: Inline HTML
Element: TabItem
(MD033, no-inline-html)
25-25: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
29-29: Inline HTML
Element: TabItem
(MD033, no-inline-html)
30-30: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
34-34: Inline HTML
Element: TabItem
(MD033, no-inline-html)
35-35: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
53-53: Inline HTML
Element: Tabs
(MD033, no-inline-html)
54-54: Inline HTML
Element: TabItem
(MD033, no-inline-html)
55-55: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
59-59: Inline HTML
Element: TabItem
(MD033, no-inline-html)
60-60: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
64-64: Inline HTML
Element: TabItem
(MD033, no-inline-html)
65-65: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
73-73: Fenced code blocks should have a language specified
null
(MD040, fenced-code-language)
81-81: Inline HTML
Element: Tabs
(MD033, no-inline-html)
82-82: Inline HTML
Element: TabItem
(MD033, no-inline-html)
83-83: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
87-87: Inline HTML
Element: TabItem
(MD033, no-inline-html)
88-88: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
92-92: Inline HTML
Element: TabItem
(MD033, no-inline-html)
93-93: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
⏰ Context from checks skipped due to timeout of 90000ms (22)
- GitHub Check: Tusk Test Runner
- GitHub Check: Tusk Tester
- GitHub Check: webui tests
- GitHub Check: Share Test
- GitHub Check: Redteam
- GitHub Check: Redteam Custom Enterprise Server
- GitHub Check: Test on Node 18.x and macOS-latest
- GitHub Check: Test on Node 20.x and ubuntu-latest
- GitHub Check: Test on Node 24.x and ubuntu-latest
- GitHub Check: Test on Node 20.x and windows-latest
- GitHub Check: Test on Node 24.x and macOS-latest
- GitHub Check: Test on Node 18.x and ubuntu-latest
- GitHub Check: Test on Node 22.x and macOS-latest
- GitHub Check: Test on Node 22.x and ubuntu-latest
- GitHub Check: Test on Node 18.x and windows-latest
- GitHub Check: Build Docs
- GitHub Check: Build on Node 22.x
- GitHub Check: Build on Node 20.x
- GitHub Check: Style Check
- GitHub Check: Build on Node 18.x
- GitHub Check: Build on Node 24.x
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (4)
site/docs/installation.md (2)
2-4: Metadata block updated.
SEO metadata (title, description, keywords) is comprehensive and consistent.
16-16: Validate Node.js requirement.
Ensure the documented "Node.js 18 or newer" aligns with theenginesfield in yourpackage.jsonto prevent version mismatches.site/docs/getting-started.md (2)
2-4: Well-defined frontmatter.
Title, description, and keywords clearly describe the page and enhance SEO and site navigation.
193-204: Nice improvement: integrateddefaultTest.
Introducing thedefaultTestblock for LLM rubric and JavaScript assertions clearly demonstrates automatic scoring capabilities.
| ``` | ||
| 0.114.7 | ||
| ``` |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Specify language for code fence.
Add a language identifier (e.g., ```bash) to the fenced block showing the version output for proper syntax highlighting.
🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
73-73: Fenced code blocks should have a language specified
null
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
In site/docs/installation.md around lines 73 to 75, the code fence showing the
version output lacks a language identifier, which prevents proper syntax
highlighting. Add a language specifier such as "bash" immediately after the
opening triple backticks to enable correct highlighting of the command output
block.
| <Tabs groupId="promptfoo-command"> | ||
| <TabItem value="npm" label="npm" default> | ||
| <CodeBlock language="bash"> | ||
| npm install -g promptfoo | ||
| </CodeBlock> | ||
| </TabItem> | ||
| <TabItem value="npx" label="npx"> | ||
| <CodeBlock language="bash"> | ||
| npx promptfoo@latest | ||
| </CodeBlock> | ||
| </TabItem> | ||
| <TabItem value="brew" label="brew"> | ||
| <CodeBlock language="bash"> | ||
| brew install promptfoo | ||
| </CodeBlock> | ||
| </TabItem> | ||
| </Tabs> |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Use unique groupId for each tab group.
All three <Tabs> components share groupId="promptfoo-command", causing tab state to persist across unrelated sections. Assign distinct IDs (e.g., "install-command", "verify-command", "run-command") to avoid unintended syncing.
Also applies to: 53-69, 81-97
🧰 Tools
🪛 markdownlint-cli2 (0.17.2)
23-23: Inline HTML
Element: Tabs
(MD033, no-inline-html)
24-24: Inline HTML
Element: TabItem
(MD033, no-inline-html)
25-25: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
29-29: Inline HTML
Element: TabItem
(MD033, no-inline-html)
30-30: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
34-34: Inline HTML
Element: TabItem
(MD033, no-inline-html)
35-35: Inline HTML
Element: CodeBlock
(MD033, no-inline-html)
🤖 Prompt for AI Agents
In site/docs/installation.md around lines 23 to 39, the issue is that multiple
<Tabs> components use the same groupId "promptfoo-command", causing tab state to
persist across unrelated sections. To fix this, assign a unique groupId to each
<Tabs> component, such as "install-command", "verify-command", and
"run-command", to prevent unintended syncing of tab states. Also apply this
change to the <Tabs> components in lines 53-69 and 81-97.
| ```sh | ||
| npm install promptfoo | ||
| npm install promptfoo --save | ||
| ``` |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Refine library installation instructions.
The explicit --save flag is redundant for npm v5+. Consider removing it and adding alternative Yarn/pnpm commands to cover more package-manager use cases.
🤖 Prompt for AI Agents
In site/docs/installation.md around lines 45 to 47, remove the redundant
`--save` flag from the npm install command since npm v5+ saves dependencies by
default. Replace the single npm command with separate commands for npm, Yarn,
and pnpm to provide installation instructions for multiple package managers,
improving coverage and clarity.
| @@ -8,7 +11,11 @@ import TabItem from '@theme/TabItem'; | |||
|
|
|||
| # Getting started | |||
|
|
|||
| Install promptfoo and set up your first config file by running this command with [npx](https://nodejs.org/en/download), [npm](https://nodejs.org/en/download), or [brew](https://brew.sh/): | |||
| After [installing](/docs/installation) promptfoo, you can set up your first config file in two ways: | |||
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Unify link style.
This page uses an absolute link (/docs/installation) while the other uses relative (./getting-started). Choose one style across the docs for consistency.
🤖 Prompt for AI Agents
In site/docs/getting-started.md at line 14, the link to the installation page
uses an absolute path (/docs/installation) while other links use relative paths.
To unify the link style across the documentation, change the absolute link to a
relative link consistent with the rest of the docs, for example, use
./installation instead of /docs/installation.
| tests: | ||
| - description: First test case - automatic review | ||
| vars: | ||
| var1: first variable's value | ||
| var2: another value | ||
| var3: some other value | ||
| assert: | ||
| - type: equals | ||
| value: expected LLM output goes here | ||
| - type: javascript | ||
| value: output.includes('some text') | ||
|
|
||
| - description: Second test case - manual review | ||
| # Test cases don't need assertions if you prefer to review the output yourself | ||
| vars: | ||
| var1: new value | ||
| var2: another value | ||
| var3: third value | ||
|
|
||
| - description: Third test case - other types of automatic review | ||
| vars: | ||
| var1: yet another value | ||
| var2: and another | ||
| var3: dear llm, please output your response in json format | ||
| assert: | ||
| - type: contains-json | ||
| - type: similar | ||
| value: ensures that output is semantically similar to this text | ||
| - type: llm-rubric | ||
| value: must contain a reference to X | ||
| - vars: | ||
| name: Bob | ||
| question: Can you help me find a specific product on your website? | ||
| - vars: | ||
| name: Jane | ||
| question: Do you have any promotions or discounts currently available? | ||
| - vars: | ||
| name: Ben | ||
| question: Can you check the availability of a product at a specific store location? | ||
| - vars: | ||
| name: Dave | ||
| question: What are your shipping and return policies? | ||
| - vars: | ||
| name: Jim | ||
| question: Can you provide more information about the product specifications or features? | ||
| - vars: | ||
| name: Alice | ||
| question: Can you recommend products that are similar to what I've been looking at? | ||
| - vars: | ||
| name: Sophie | ||
| question: Do you have any recommendations for products that are currently popular or trending? | ||
| - vars: | ||
| name: Jessie | ||
| question: How can I track my order after it has been shipped? | ||
| - vars: |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Streamline test list.
The exhaustive list of test cases is instructive but lengthy. Consider collapsing it with a <details> section or summarizing to maintain focus and reduce scrolling.
🤖 Prompt for AI Agents
In site/docs/getting-started.md between lines 207 and 232, the list of test
cases is too long and makes the document harder to navigate. To fix this, wrap
the entire list of test cases inside a collapsible <details> section or replace
the detailed list with a concise summary that highlights key examples. This will
keep the content focused and reduce excessive scrolling for readers.
There was a problem hiding this comment.
Pull Request Overview
This PR refines the installation and getting‐started documentation for promptfoo by introducing a tabbed interface for command examples and adding SEO metadata to improve discoverability. Key changes include:
- Addition of SEO metadata (title, description, keywords) in both documentation pages.
- Replacement of static code blocks with tabbed interfaces using CodeBlock, Tabs, and TabItem components.
- Updated command examples and configuration snippets for consistency and clarity.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| site/docs/installation.md | Added SEO metadata, refactored installation commands into a tabbed interface, and updated dependency install instructions. |
| site/docs/getting-started.md | Inserted SEO metadata, restructured instructions to delineate example-based and scratch-based setups, and updated configuration examples. |
|
|
||
| Install promptfoo using [npx](https://nodejs.org/en/download), [npm](https://nodejs.org/en/download), or [brew](https://brew.sh/): | ||
|
|
||
| <Tabs groupId="promptfoo-command"> |
There was a problem hiding this comment.
[nitpick] Consider using unique groupId values for each Tabs component if independent tab state management is desired, to avoid potential cross-section state sharing between different sections.
| <Tabs groupId="promptfoo-command"> | |
| <Tabs groupId="promptfoo-command-usage"> |
Copilot uses AI. Check for mistakes.
|
|
||
| If you prefer to start from scratch instead of using the example, simply run `promptfoo init` without the `--example` flag: | ||
|
|
||
| <Tabs groupId="promptfoo-command"> |
There was a problem hiding this comment.
[nitpick] If each command example section should operate independently, consider assigning a unique groupId to each Tabs component so that user selections in one section do not persist across sections.
| <Tabs groupId="promptfoo-command"> | |
| <Tabs groupId="promptfoo-command-scratch"> |
Copilot uses AI. Check for mistakes.
No description provided.