Skip to content

Commit fdbd724

Browse files
sjarmakclaude
sjarmak
and
claude
committed
feat: add instruction_mcp.md files + pin all SG repos to sg-benchmarks mirrors
- Generate 190 instruction_mcp.md files (V5 preamble + instruction.md) via scripts/generate_instruction_mcp.py, making MCP agent instructions transparent - Pin all 110 Dockerfile.sg_only SOURCEGRAPH env vars to sg-benchmarks mirrors (no more unpinned github.com/ refs that search HEAD) - Create 79 sg-benchmarks mirrors on GitHub for reproducible Sourcegraph indexing - Add scripts: audit_unpinned_repos.py, create_sg_mirrors.py, update_sg_only_mirrors.py, generate_instruction_mcp.py - Mirror creation manifest: configs/mirror_creation_manifest.json (79 mirrors) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 16ce2ce commit fdbd724

File tree

301 files changed

+24776
-124
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

301 files changed

+24776
-124
lines changed

‎benchmarks/ccb_build/bustub-hyperloglog-impl-001/instruction_mcp.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# IMPORTANT: Source Code Access
2+
3+
**Local source files are not present.** Your workspace does not contain source code. You **MUST** use Sourcegraph MCP tools to discover, read, and understand code before making any changes.
4+
5+
**Target Repository:** `github.com/sg-benchmarks/bustub--d5f79431`
6+
- Use `repo:^github.com/sg-benchmarks/bustub--d5f79431$` filter in keyword_search
7+
- Use `github.com/sg-benchmarks/bustub--d5f79431` as the `repo` parameter for go_to_definition/find_references/read_file
8+
9+
10+
## Required Workflow
11+
12+
1. **Search first** — Use MCP tools to find relevant files and understand existing patterns
13+
2. **Read remotely** — Use `sg_read_file` to read full file contents from Sourcegraph
14+
3. **Edit locally** — Use Edit, Write, and Bash to create or modify files in your working directory
15+
4. **Verify locally** — Run tests with Bash to check your changes
16+
17+
## Tool Selection
18+
19+
| Goal | Tool |
20+
|------|------|
21+
| Exact symbol/string | `sg_keyword_search` |
22+
| Concepts/semantic search | `sg_nls_search` |
23+
| Trace usage/callers | `sg_find_references` |
24+
| See implementation | `sg_go_to_definition` |
25+
| Read full file | `sg_read_file` |
26+
| Browse structure | `sg_list_files` |
27+
| Find repos | `sg_list_repos` |
28+
| Search commits | `sg_commit_search` |
29+
| Track changes | `sg_diff_search` |
30+
| Compare versions | `sg_compare_revisions` |
31+
32+
**Decision logic:**
33+
1. Know the exact symbol? → `sg_keyword_search`
34+
2. Know the concept, not the name? → `sg_nls_search`
35+
3. Need definition of a symbol? → `sg_go_to_definition`
36+
4. Need all callers/references? → `sg_find_references`
37+
5. Need full file content? → `sg_read_file`
38+
39+
## Scoping (Always Do This)
40+
41+
```
42+
repo:^github.com/ORG/REPO$ # Exact repo (preferred)
43+
repo:github.com/ORG/ # All repos in org
44+
file:.*\.ts$ # TypeScript only
45+
file:src/api/ # Specific directory
46+
```
47+
48+
Start narrow. Expand only if results are empty.
49+
50+
## Efficiency Rules
51+
52+
- Chain searches logically: search → read → references → definition
53+
- Don't re-search for the same pattern; use results from prior calls
54+
- Prefer `sg_keyword_search` over `sg_nls_search` when you have exact terms
55+
- Read 2-3 related files before synthesising, rather than one at a time
56+
- Don't read 20+ remote files without writing code — once you understand the pattern, start implementing
57+
58+
## If Stuck
59+
60+
If MCP search returns no results:
61+
1. Broaden the search query (synonyms, partial identifiers)
62+
2. Try `sg_nls_search` for semantic matching
63+
3. Use `sg_list_files` to browse the directory structure
64+
4. Use `sg_list_repos` to verify the repository name
65+
66+
---
67+
68+
# Implement HyperLogLog Algorithm
69+
70+
**Repository:** github.com/sg-benchmarks/bustub--d5f79431 (mirror of bustub) (TheAgentCompany GitLab)
71+
**Difficulty:** HARD
72+
**Category:** ccb_tac
73+
**Task Type:** Algorithm Implementation
74+
75+
## Description
76+
77+
Implement the HyperLogLog probabilistic data structure for cardinality estimation in the
78+
bustub database system. This requires understanding the existing codebase structure,
79+
following coding conventions, and implementing the algorithm across 4 files.
80+
81+
## Task
82+
83+
Clone http://the-agent-company.com:8929/root/bustub to /workspace folder and complete
84+
http://the-agent-company.com:8929/root/bustub/-/issues/759 locally.
85+
86+
Specifically, complete 4 files:
87+
- `bustub/src/include/primer/hyperloglog.h`
88+
- `bustub/src/include/primer/hyperloglog_presto.h`
89+
- `bustub/src/primer/hyperloglog.cpp`
90+
- `bustub/src/primer/hyperloglog_presto.cpp`
91+
92+
To ensure compatibility of testing across different operating systems, please change
93+
the line `hash = ((hash << 5) ^ (hash >> 27)) ^ bytes[i];` in your local file
94+
`src/include/common/util/hash_util.h` to:
95+
`hash = ((hash << 5) ^ (hash >> 27)) ^ static_cast<signed char>(bytes[i]);`
96+
97+
## Success Criteria
98+
99+
[x] Repository cloned successfully
100+
[x] All 4 implementation files completed
101+
[x] Unit tests pass (10 tests, graded by pass count)
102+
103+
## Notes
104+
105+
- This task uses TheAgentCompany's pre-built environment
106+
- Deterministic grading via unit tests
107+
- Score is based on number of passing tests (each test worth 0.5 points)

‎benchmarks/ccb_build/camel-fix-protocol-feat-001/environment/Dockerfile.sg_only‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
FROM ccb-repo-camel-1006f047
66

7-
ENV SOURCEGRAPH_REPO_NAME=github.com/apache/camel
7+
ENV SOURCEGRAPH_REPO_NAME=sg-benchmarks/camel--1006f047
88

99
# --- sg_only_env: back up full repo, then truncate source ---
1010
RUN cp -a /workspace /repo_full

‎benchmarks/ccb_build/camel-fix-protocol-feat-001/instruction_mcp.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# IMPORTANT: Source Code Access
2+
3+
**Local source files are not present.** Your workspace does not contain source code. You **MUST** use Sourcegraph MCP tools to discover, read, and understand code before making any changes.
4+
5+
**Target Repository:** `github.com/sg-benchmarks/camel--1006f047`
6+
- Use `repo:^github.com/sg-benchmarks/camel--1006f047$` filter in keyword_search
7+
- Use `github.com/sg-benchmarks/camel--1006f047` as the `repo` parameter for go_to_definition/find_references/read_file
8+
9+
10+
## Required Workflow
11+
12+
1. **Search first** — Use MCP tools to find relevant files and understand existing patterns
13+
2. **Read remotely** — Use `sg_read_file` to read full file contents from Sourcegraph
14+
3. **Edit locally** — Use Edit, Write, and Bash to create or modify files in your working directory
15+
4. **Verify locally** — Run tests with Bash to check your changes
16+
17+
## Tool Selection
18+
19+
| Goal | Tool |
20+
|------|------|
21+
| Exact symbol/string | `sg_keyword_search` |
22+
| Concepts/semantic search | `sg_nls_search` |
23+
| Trace usage/callers | `sg_find_references` |
24+
| See implementation | `sg_go_to_definition` |
25+
| Read full file | `sg_read_file` |
26+
| Browse structure | `sg_list_files` |
27+
| Find repos | `sg_list_repos` |
28+
| Search commits | `sg_commit_search` |
29+
| Track changes | `sg_diff_search` |
30+
| Compare versions | `sg_compare_revisions` |
31+
32+
**Decision logic:**
33+
1. Know the exact symbol? → `sg_keyword_search`
34+
2. Know the concept, not the name? → `sg_nls_search`
35+
3. Need definition of a symbol? → `sg_go_to_definition`
36+
4. Need all callers/references? → `sg_find_references`
37+
5. Need full file content? → `sg_read_file`
38+
39+
## Scoping (Always Do This)
40+
41+
```
42+
repo:^github.com/ORG/REPO$ # Exact repo (preferred)
43+
repo:github.com/ORG/ # All repos in org
44+
file:.*\.ts$ # TypeScript only
45+
file:src/api/ # Specific directory
46+
```
47+
48+
Start narrow. Expand only if results are empty.
49+
50+
## Efficiency Rules
51+
52+
- Chain searches logically: search → read → references → definition
53+
- Don't re-search for the same pattern; use results from prior calls
54+
- Prefer `sg_keyword_search` over `sg_nls_search` when you have exact terms
55+
- Read 2-3 related files before synthesising, rather than one at a time
56+
- Don't read 20+ remote files without writing code — once you understand the pattern, start implementing
57+
58+
## If Stuck
59+
60+
If MCP search returns no results:
61+
1. Broaden the search query (synonyms, partial identifiers)
62+
2. Try `sg_nls_search` for semantic matching
63+
3. Use `sg_list_files` to browse the directory structure
64+
4. Use `sg_list_repos` to verify the repository name
65+
66+
---
67+
68+
# big-code-camel-feat-001: Implement camel-fix Component for FIX Protocol
69+
70+
## Task
71+
72+
Implement a new `camel-fix` component in Apache Camel that enables routing FIX (Financial Information eXchange) protocol messages through Camel routes. The FIX protocol is the standard electronic messaging protocol for securities trading, used by exchanges, brokers, and asset managers worldwide.
73+
74+
The component must follow Apache Camel's standard component architecture:
75+
76+
1. **FixComponent** (`components/camel-fix/src/main/java/org/apache/camel/component/fix/FixComponent.java`):
77+
- Extends `DefaultComponent`
78+
- Annotated `@Component("fix")`
79+
- Creates `FixEndpoint` instances via `createEndpoint()`
80+
- Manages shared FIX engine lifecycle
81+
82+
2. **FixEndpoint** (`components/camel-fix/src/main/java/org/apache/camel/component/fix/FixEndpoint.java`):
83+
- Extends `DefaultEndpoint`
84+
- Annotated `@UriEndpoint(scheme = "fix", syntax = "fix:sessionID", ...)`
85+
- Creates Consumer and Producer instances
86+
- URI format: `fix:sessionID?options`
87+
88+
3. **FixConsumer** (`components/camel-fix/src/main/java/org/apache/camel/component/fix/FixConsumer.java`):
89+
- Extends `DefaultConsumer`
90+
- Receives inbound FIX messages and feeds them into Camel routes
91+
- Lifecycle management: starts/stops FIX acceptor sessions
92+
93+
4. **FixProducer** (`components/camel-fix/src/main/java/org/apache/camel/component/fix/FixProducer.java`):
94+
- Extends `DefaultAsyncProducer`
95+
- Sends outbound FIX messages from Camel exchanges
96+
- Implements `process(Exchange, AsyncCallback)`
97+
98+
5. **FixConfiguration** (`components/camel-fix/src/main/java/org/apache/camel/component/fix/FixConfiguration.java`):
99+
- POJO with `@UriParams` and `@UriParam` annotations
100+
- Fields: configFile, senderCompID, targetCompID, fixVersion, heartBeatInterval, socketConnectHost, socketConnectPort
101+
102+
6. **FixConstants** (`components/camel-fix/src/main/java/org/apache/camel/component/fix/FixConstants.java`):
103+
- Header constants for FIX message type, session ID, sender/target comp IDs
104+
105+
7. **Build and registration**:
106+
- `components/camel-fix/pom.xml` — Maven POM inheriting from `components` parent
107+
- `components/pom.xml` — Add `<module>camel-fix</module>` to modules list
108+
- Component descriptor files for Camel's service loader
109+
110+
Study existing components like `camel-kafka`, `camel-netty`, or `camel-amqp` for the complete pattern.
111+
112+
## Context
113+
114+
- **Repository**: github.com/sg-benchmarks/camel--1006f047 (mirror of apache/camel) (Java, ~2M LOC)
115+
- **Category**: Feature Implementation
116+
- **Difficulty**: hard
117+
- **Subsystem Focus**: components/camel-fix/ (new module), components/pom.xml (registration)
118+
119+
## Requirements
120+
121+
1. Identify all files that need creation or modification
122+
2. Follow existing Camel component patterns (`DefaultComponent`, `DefaultEndpoint`, `@UriEndpoint`)
123+
3. Implement the component with actual code changes
124+
4. Ensure the module integrates correctly with the parent build
125+
126+
## Expected Output
127+
128+
Write your analysis to `/logs/agent/solution.md` with the following structure:
129+
130+
```
131+
## Files Examined
132+
- path/to/file1.ext — examined to understand [pattern/API/convention]
133+
- path/to/file2.ext — modified to add [feature component]
134+
...
135+
136+
## Dependency Chain
137+
1. Define types/interfaces: path/to/types.ext
138+
2. Implement core logic: path/to/impl.ext
139+
3. Wire up integration: path/to/integration.ext
140+
4. Add tests: path/to/tests.ext
141+
...
142+
143+
## Code Changes
144+
### path/to/file1.ext
145+
```diff
146+
- old code
147+
+ new code
148+
```
149+
150+
### path/to/file2.ext
151+
```diff
152+
- old code
153+
+ new code
154+
```
155+
156+
## Analysis
157+
[Explanation of implementation strategy, design decisions, and how the feature
158+
integrates with existing architecture]
159+
```
160+
161+
## Evaluation Criteria
162+
163+
- Compilation: Does the code compile after changes?
164+
- File coverage: Did you modify all necessary files?
165+
- Pattern adherence: Do changes follow existing codebase conventions?
166+
- Feature completeness: Is the feature fully implemented?

‎benchmarks/ccb_build/cgen-deps-install-001/instruction_mcp.md‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# IMPORTANT: Source Code Access
2+
3+
**Local source files are not present.** Your workspace does not contain source code. You **MUST** use Sourcegraph MCP tools to discover, read, and understand code before making any changes.
4+
5+
**Target Repository:** `github.com/sg-benchmarks/cgen--dibench`
6+
- Use `repo:^github.com/sg-benchmarks/cgen--dibench$` filter in keyword_search
7+
- Use `github.com/sg-benchmarks/cgen--dibench` as the `repo` parameter for go_to_definition/find_references/read_file
8+
9+
10+
## Required Workflow
11+
12+
1. **Search first** — Use MCP tools to find relevant files and understand existing patterns
13+
2. **Read remotely** — Use `sg_read_file` to read full file contents from Sourcegraph
14+
3. **Edit locally** — Use Edit, Write, and Bash to create or modify files in your working directory
15+
4. **Verify locally** — Run tests with Bash to check your changes
16+
17+
## Tool Selection
18+
19+
| Goal | Tool |
20+
|------|------|
21+
| Exact symbol/string | `sg_keyword_search` |
22+
| Concepts/semantic search | `sg_nls_search` |
23+
| Trace usage/callers | `sg_find_references` |
24+
| See implementation | `sg_go_to_definition` |
25+
| Read full file | `sg_read_file` |
26+
| Browse structure | `sg_list_files` |
27+
| Find repos | `sg_list_repos` |
28+
| Search commits | `sg_commit_search` |
29+
| Track changes | `sg_diff_search` |
30+
| Compare versions | `sg_compare_revisions` |
31+
32+
**Decision logic:**
33+
1. Know the exact symbol? → `sg_keyword_search`
34+
2. Know the concept, not the name? → `sg_nls_search`
35+
3. Need definition of a symbol? → `sg_go_to_definition`
36+
4. Need all callers/references? → `sg_find_references`
37+
5. Need full file content? → `sg_read_file`
38+
39+
## Scoping (Always Do This)
40+
41+
```
42+
repo:^github.com/ORG/REPO$ # Exact repo (preferred)
43+
repo:github.com/ORG/ # All repos in org
44+
file:.*\.ts$ # TypeScript only
45+
file:src/api/ # Specific directory
46+
```
47+
48+
Start narrow. Expand only if results are empty.
49+
50+
## Efficiency Rules
51+
52+
- Chain searches logically: search → read → references → definition
53+
- Don't re-search for the same pattern; use results from prior calls
54+
- Prefer `sg_keyword_search` over `sg_nls_search` when you have exact terms
55+
- Read 2-3 related files before synthesising, rather than one at a time
56+
- Don't read 20+ remote files without writing code — once you understand the pattern, start implementing
57+
58+
## If Stuck
59+
60+
If MCP search returns no results:
61+
1. Broaden the search query (synonyms, partial identifiers)
62+
2. Try `sg_nls_search` for semantic matching
63+
3. Use `sg_list_files` to browse the directory structure
64+
4. Use `sg_list_repos` to verify the repository name
65+
66+
---
67+
68+
**Sourcegraph Repository:** `github.com/sg-benchmarks/cgen--dibench`
69+
70+
# Dependency Inference Task
71+
72+
## Objective
73+
74+
Edit the build file(s) to include all necessary dependency configurations so the project builds and tests pass.
75+
76+
## Task Information
77+
78+
- **Language**: python
79+
- **Project**: inducer/cgen
80+
- **Build Files to Edit**: `pyproject.toml`
81+
82+
## Environment
83+
84+
```json
85+
{
86+
"SDK": "Python 3.8",
87+
"OS": "ubuntu-22.04"
88+
}
89+
```
90+
91+
## Instructions
92+
93+
1. Analyze the project structure in the repository to understand what external libraries are used
94+
2. Review source code imports and usage to identify all required dependencies
95+
3. Edit the build file(s) listed above to add correct dependency declarations
96+
4. Include ALL dependencies needed for the project to build and run its tests
97+
98+
### Constraints
99+
100+
- Only edit the files listed in "Build Files to Edit"
101+
- Limit edits to dependency-related configurations
102+
- Do not modify source code files

0 commit comments

Comments
 (0)