-
Notifications
You must be signed in to change notification settings - Fork 106
Review Agent #298
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Review Agent #298
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
d37daee
push review agent implementation
msukkari 17fce5c
feedback
msukkari 12002f2
wip integrating review agent into monorepo
msukkari a81cd24
move review agent to web
msukkari 79dd288
feedback
msukkari ef978fd
feedback
msukkari 944aeb5
add rate limit throttling to octokit
msukkari 2d0977e
configure agent ui in app
msukkari 871d410
docs
msukkari a242a69
add review command logic and add logging for review agent to data cac…
msukkari 5323cfa
fix bug with llm returning multiple reviews in single invocation
msukkari c586558
fix doc link bug
msukkari 2307106
feedback and improved docs for review agent
msukkari 0ccd4a8
review agent doc nits
msukkari fa40daf
merge main
msukkari ec9526f
mcp doc nit
msukkari File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
--- | ||
title: "Agents Overview" | ||
sidebarTitle: "Overview" | ||
--- | ||
|
||
<Note> | ||
Have an idea for an agent that we haven't built? Submit a [feature request](https://github.com/sourcebot-dev/sourcebot/discussions/categories/feature-requests) on our GitHub | ||
</Note> | ||
|
||
Agents are automations that leverage the code indexed on Sourcebot to perform a specific task. Once you've setup Sourcebot, check out the | ||
guides below to configure additional agents. | ||
|
||
<CardGroup cols={2}> | ||
<Card horizontal title="Review Agent" icon="gear" href="/docs/agents/review-agent"> | ||
An AI agent that reviews your PRs to identify issues | ||
</Card> | ||
</CardGroup> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
--- | ||
title: AI Code Review Agent | ||
sidebarTitle: AI Code Review Agent | ||
--- | ||
|
||
<Note> | ||
This agent sends data to OpenAI (through an API key you supply) to perform code reviews. This data includes code from the PR being reviewed, as well as additional relevant context from your | ||
codebase that the agent may fetch to perform the review. | ||
</Note> | ||
|
||
This agent provides codebase-aware reviews for your PRs. For each diff, this agent fetches relevant context from Sourcebot and feeds it into an LLM for a detailed review of your changes. | ||
|
||
The AI Code Review Agent is open source and packaged in [Sourcebot](https://github.com/sourcebot-dev/sourcebot). To get started using this agent, [deploy Sourcebot](/self-hosting/overview) | ||
and then follow the configuration instructions below. | ||
msukkari marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
 | ||
|
||
# Configure | ||
|
||
This agent currently only supports reviewing GitHub PRs. You configure the agent by creating a GitHub app, installing it into your GitHub organization, and then giving your app info to Sourcebot. | ||
|
||
Before you get started, make sure you have an OpenAPI account that you can create an OpenAPI key with. | ||
|
||
<Steps> | ||
<Step title="Register a GitHub app"> | ||
Follow the official GitHub guide for [registering a GitHub app](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app) | ||
|
||
- GitHub App name: You can make this whatever you want (ex. Sourcebot Review Agent) | ||
- Homepage URL: You can make this whatever you want (ex. https://www.sourcebot.dev/) | ||
- Webhook URL (**IMPORTANT**): You must set this to point to your Sourcebot deployment at /api/webhook (ex. https://sourcebot.aperture.com/api/webhook). Your Sourcebot deployment must be able to accept requests from GitHub | ||
(either github.com or your self-hosted enterprise server) for this to work. If you're running Sourcebot locally, you can [use smee](https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/quickstart#step-2-get-a-webhook-proxy-url) to [forward webhooks](https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/quickstart#step-6-start-your-server) to your local deployment. | ||
- Permissions | ||
- Pull requests: Read & Write | ||
- Issues: Read & Write | ||
- Contents: Read | ||
- Events: | ||
- Pull request | ||
- Issue comment | ||
</Step> | ||
<Step title="Install the GitHub app in your organization"> | ||
Navigate to your new [GitHub app's page](https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/quickstart#navigate-to-your-app-settings) and press `Install` | ||
</Step> | ||
<Step title="Configure the environment variables in Sourcebot"> | ||
Sourcebot requires the following environment variables to begin reviewing PRs through your new GitHub app: | ||
|
||
- `GITHUB_APP_ID`: The client ID of your GitHub app. Can be found in your [app settings](https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/quickstart#navigate-to-your-app-settings) | ||
- `GITHUB_APP_WEBHOOK_SECRET`: A random webhook secret that you've set in your [app settings](https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/quickstart#navigate-to-your-app-settings). This can be anything (ex. `python -c "import secrets; print(secrets.token_hex(10))"` to generate a random secret) | ||
- `GITHUB_APP_PRIVATE_KEY_PATH`: The path to your app's private key. If you're running Sourcebot from a container, this is the path to this file from within your container | ||
(ex `/data/review-agent-key.pem`). You must copy the private key file into the directory you mount to Sourcebot (similar to the config file). | ||
|
||
You can generate a private key file for your app in the [app settings](https://docs.github.com/en/apps/creating-github-apps/writing-code-for-a-github-app/quickstart#navigate-to-your-app-settings). You must copy this private key file into the | ||
directory that you mount to Sourcebot | ||
 | ||
- `OPENAI_API_KEY`: Your OpenAI API key | ||
- `REVIEW_AGENT_AUTO_REVIEW_ENABLED` (default: `false`): If enabled, the review agent will automatically review any new or updated PR. If disabled, you must invoke it using the command defined by `REVIEW_AGENT_REVIEW_COMMAND` | ||
- `REVIEW_AGENT_REVIEW_COMMAND` (default: `review`): The command that invokes the review agent (ex. `/review`) when a user comments on the PR. Don't include the slash character in this value. | ||
|
||
You can find an example docker compose file below. | ||
- This docker compose file is placed in `~/sourcebot_review_agent_workspace`, and I'm mounting that directory to Sourcebot | ||
- The config and the app private key files are placed in this directory | ||
- The paths to these files are given to Sourcebot relative to `/data` since that's the directory in Sourcebot that I'm mounting to | ||
|
||
```yaml | ||
services: | ||
sourcebot: | ||
image: ghcr.io/sourcebot-dev/sourcebot:latest | ||
pull_policy: always | ||
container_name: sourcebot | ||
ports: | ||
- "3000:3000" | ||
volumes: | ||
- "/Users/michael/sourcebot_review_agent_workspace:/data" | ||
environment: | ||
CONFIG_PATH: "/data/config.json" | ||
GITHUB_APP_ID: "my-github-app-id" | ||
GITHUB_APP_WEBHOOK_SECRET: "my-github-app-webhook-secret" | ||
GITHUB_APP_PRIVATE_KEY_PATH: "/data/review-agent-key.pem" | ||
OPENAI_API_KEY: "sk-proj-my-open-api-key" | ||
``` | ||
</Step> | ||
<Step title="Verify configuration"> | ||
Navigate to the agents page by pressing `Agents` in the Sourcebot nav menu. If you've configured your environment variables correctly you'll see the following: | ||
|
||
 | ||
</Step> | ||
</Steps> | ||
|
||
# Using the agent | ||
|
||
The review agent will not automatically review your PRs by default. To enable this feature, set the `REVIEW_AGENT_AUTO_REVIEW_ENABLED` environment variable to true. | ||
|
||
You can invoke the review agent manually by commenting `/review` on the PR you'd like it to review. You can configure the command that triggers the agent by changing | ||
the `REVIEW_AGENT_REVIEW_COMMAND` environment variable. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -38,4 +38,4 @@ | |
"sourcebot", | ||
"code-intelligence" | ||
] | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
import Link from "next/link"; | ||
import { NavigationMenu } from "../components/navigationMenu"; | ||
import { FaCogs } from "react-icons/fa"; | ||
import { env } from "@/env.mjs"; | ||
|
||
const agents = [ | ||
{ | ||
id: "review-agent", | ||
name: "Review Agent", | ||
description: "An AI code review agent that reviews your PRs. Uses the code indexed on Sourcebot to provide codebase-wide context.", | ||
requiredEnvVars: ["GITHUB_APP_ID", "GITHUB_APP_WEBHOOK_SECRET", "GITHUB_APP_PRIVATE_KEY_PATH", "OPENAI_API_KEY"], | ||
configureUrl: "https://docs.sourcebot.dev/docs/agents/review-agent" | ||
}, | ||
]; | ||
|
||
export default function AgentsPage({ params: { domain } }: { params: { domain: string } }) { | ||
return ( | ||
<div className="flex flex-col items-center overflow-hidden min-h-screen"> | ||
<NavigationMenu domain={domain} /> | ||
<div className="w-full max-w-6xl px-4 mt-12 mb-24"> | ||
<div | ||
className={ | ||
agents.length === 1 | ||
? "flex justify-center items-center min-h-[60vh]" | ||
: "grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-10" | ||
} | ||
> | ||
{agents.map((agent) => ( | ||
<div | ||
key={agent.id} | ||
className={ | ||
agents.length === 1 | ||
? "relative flex flex-col items-center border border-border rounded-2xl p-8 bg-card shadow-xl w-full max-w-xl" | ||
: "relative flex flex-col items-center border border-border rounded-2xl p-8 bg-card shadow-xl" | ||
} | ||
> | ||
{/* Name and description */} | ||
<div className="flex flex-col items-center w-full"> | ||
<h2 className="font-bold text-2xl mb-4 mt-2 text-center text-foreground drop-shadow-sm"> | ||
{agent.name} | ||
</h2> | ||
<p className="text-base text-muted-foreground text-center mb-4 min-h-[56px]"> | ||
{agent.description} | ||
</p> | ||
</div> | ||
{/* Actions */} | ||
<div className="flex flex-col items-center w-full mt-2"> | ||
{agent.requiredEnvVars.every(envVar => envVar in env && env[envVar as keyof typeof env] !== undefined) ? ( | ||
<div className="text-green-500 font-semibold"> | ||
Agent is configured and accepting requests on /api/webhook | ||
</div> | ||
) : ( | ||
<Link | ||
href={agent.configureUrl} | ||
target="_blank" | ||
rel="noopener noreferrer" | ||
className="flex items-center justify-center gap-2 px-5 py-2.5 rounded-md bg-primary text-primary-foreground font-mono font-semibold text-base border border-primary shadow-sm hover:bg-primary/80 focus:outline-none focus:ring-2 focus:ring-primary/60 transition w-1/2" | ||
> | ||
<FaCogs className="text-lg" /> Configure | ||
</Link> | ||
)} | ||
</div> | ||
</div> | ||
))} | ||
</div> | ||
</div> | ||
</div> | ||
); | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.