Skip to content

Commit 03dfbe9

Browse files
sjarmakclaude
sjarmak
and
claude
committed
feat: add 3 new ccb_fix tasks replacing removed protonmail tasks
Adds envoy-udp-proxy-cds-fix-001 (medium, C++), envoy-dfp-host-leak-fix-001 (hard, C++), and terraform-plan-null-unknown-fix-001 (medium, Go) to restore SDLC task count from 167 to 170. All use diff_similarity verification, avoiding the git apply --allow-empty failure that killed the protonmail tasks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 71ad211 commit 03dfbe9

31 files changed

+3322
-3
lines changed

‎benchmarks/ccb_fix/envoy-dfp-host-leak-fix-001/environment/Dockerfile‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
FROM ubuntu:22.04
2+
3+
WORKDIR /workspace
4+
5+
# Install dependencies
6+
RUN apt-get update && apt-get install -y --no-install-recommends \
7+
git curl ca-certificates python3 && \
8+
rm -rf /var/lib/apt/lists/*
9+
10+
# Install Node.js for Claude Code CLI
11+
RUN curl -fsSL https://deb.nodesource.com/setup_18.x | bash - && \
12+
apt-get install -y --no-install-recommends nodejs && \
13+
rm -rf /var/lib/apt/lists/*
14+
15+
# Create claude user before cloning (clone-as-claude pattern)
16+
RUN adduser --disabled-password --gecos '' claude 2>/dev/null || true
17+
18+
# Clone envoy at pre-fix commit as claude user
19+
USER claude
20+
RUN git clone --depth 1 https://github.com/envoyproxy/envoy.git /workspace && \
21+
cd /workspace && \
22+
git fetch --depth 1 origin 5160151e14837c6f352b9d0b3a9f55119a9317e8 && \
23+
git checkout FETCH_HEAD
24+
USER root
25+
26+
ENTRYPOINT []

‎benchmarks/ccb_fix/envoy-dfp-host-leak-fix-001/instruction.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Fix Memory Leak in Dynamic Forward Proxy Cluster
2+
3+
**Repository:** envoyproxy/envoy
4+
**Difficulty:** HARD
5+
**Category:** cross_module_bug_fix
6+
7+
## Description
8+
9+
Envoy's Dynamic Forward Proxy (DFP) cluster has a memory leak that occurs when a host's DNS resolution changes to a new IP address. When a DFP host initially resolves to IP address A, a `LogicalHost` is created and added to both the `host_map_` and the cross-priority host map (tracked by `MainPrioritySetImpl`). When the same host later re-resolves to IP address B, the current code updates the `LogicalHost`'s address in-place via `setNewAddresses()` without removing the old address from the cross-priority host map.
10+
11+
This means IP A remains in the cross-priority host map permanently. When the host's TTL expires and it is removed, only IP B is cleaned up from the map — IP A leaks. Over time, with repeated DNS re-resolution, the cross-priority host map grows without bound.
12+
13+
The fix refactors `addOrUpdateHost()` to remove the old host and create a new `LogicalHost` with the new address, properly emitting both `hosts_added` and `hosts_removed` events to the priority set. The method signature is simplified to remove the accumulated `hosts_added` vector pattern, and `updatePriorityState()` is called directly within `addOrUpdateHost()`. Debug logging is also added to `upstream_impl.cc` for cross-priority host map operations.
14+
15+
## Task
16+
17+
Changes:
18+
- 4 files modified (upstream_impl.cc, cluster.cc, cluster.h, cluster_test.cc)
19+
- 29 additions, 39 deletions
20+
21+
Tasks:
22+
1. Add debug logging to `source/common/upstream/upstream_impl.cc` for cross-priority host map mutations
23+
2. Refactor `addOrUpdateHost()` in `source/extensions/clusters/dynamic_forward_proxy/cluster.cc` to replace in-place address updates with host removal and re-creation
24+
3. Simplify the `addOrUpdateHost()` signature in `cluster.h` (remove the `hosts_added` out-parameter)
25+
4. Update test expectations in `cluster_test.cc` for the new per-host update notification pattern
26+
27+
## Success Criteria
28+
29+
Code changes match the expected ground-truth fix.
30+
Code follows repository conventions.
31+
No regressions in existing functionality.
32+
All 4 modified files updated correctly.
33+
34+
## Testing
35+
36+
Your implementation will be automatically verified:
37+
38+
```
39+
The verifier will compare your code changes against the expected ground-truth diff.
40+
Score = 0.35 * file_recall + 0.45 * line_recall + 0.20 * line_precision
41+
```
42+
43+
**Time Limit:** 15 minutes
44+
**Estimated Context:** 10000 tokens

‎benchmarks/ccb_fix/envoy-dfp-host-leak-fix-001/instruction_mcp.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# IMPORTANT: Source Code Access
2+
3+
**Local source files are not present.** Your workspace does not contain source code. You **MUST** use Sourcegraph MCP tools to discover, read, and understand code before making any changes.
4+
5+
**Target Repository:** `github.com/sg-evals/envoy--5160151e`
6+
- Use `repo:^github.com/sg-evals/envoy--5160151e$` filter in keyword_search
7+
- Use `github.com/sg-evals/envoy--5160151e` as the `repo` parameter for go_to_definition/find_references/read_file
8+
9+
10+
## Required Workflow
11+
12+
1. **Search first** — Use MCP tools to find relevant files and understand existing patterns
13+
2. **Read remotely** — Use `sg_read_file` to read full file contents from Sourcegraph
14+
3. **Edit locally** — Use Edit, Write, and Bash to create or modify files in your working directory
15+
4. **Verify locally** — Run tests with Bash to check your changes
16+
17+
## Tool Selection
18+
19+
| Goal | Tool |
20+
|------|------|
21+
| Exact symbol/string | `sg_keyword_search` |
22+
| Concepts/semantic search | `sg_nls_search` |
23+
| Trace usage/callers | `sg_find_references` |
24+
| See implementation | `sg_go_to_definition` |
25+
| Read full file | `sg_read_file` |
26+
| Browse structure | `sg_list_files` |
27+
| Find repos | `sg_list_repos` |
28+
| Search commits | `sg_commit_search` |
29+
| Track changes | `sg_diff_search` |
30+
| Compare versions | `sg_compare_revisions` |
31+
32+
**Decision logic:**
33+
1. Know the exact symbol? -> `sg_keyword_search`
34+
2. Know the concept, not the name? -> `sg_nls_search`
35+
3. Need definition of a symbol? -> `sg_go_to_definition`
36+
4. Need all callers/references? -> `sg_find_references`
37+
5. Need full file content? -> `sg_read_file`
38+
39+
## Scoping (Always Do This)
40+
41+
```
42+
repo:^github.com/ORG/REPO$ # Exact repo (preferred)
43+
repo:github.com/ORG/ # All repos in org
44+
file:.*\.ts$ # TypeScript only
45+
file:src/api/ # Specific directory
46+
```
47+
48+
Start narrow. Expand only if results are empty.
49+
50+
## Efficiency Rules
51+
52+
- Chain searches logically: search -> read -> references -> definition
53+
- Don't re-search for the same pattern; use results from prior calls
54+
- Prefer `sg_keyword_search` over `sg_nls_search` when you have exact terms
55+
- Read 2-3 related files before synthesising, rather than one at a time
56+
- Don't read 20+ remote files without writing code -- once you understand the pattern, start implementing
57+
58+
## If Stuck
59+
60+
If MCP search returns no results:
61+
1. Broaden the search query (synonyms, partial identifiers)
62+
2. Try `sg_nls_search` for semantic matching
63+
3. Use `sg_list_files` to browse the directory structure
64+
4. Use `sg_list_repos` to verify the repository name
65+
66+
---
67+
68+
# Fix Memory Leak in Dynamic Forward Proxy Cluster
69+
70+
**Repository:** github.com/sg-evals/envoy--5160151e (mirror of envoyproxy/envoy)
71+
**Difficulty:** HARD
72+
**Category:** cross_module_bug_fix
73+
74+
## Description
75+
76+
Envoy's Dynamic Forward Proxy (DFP) cluster has a memory leak that occurs when a host's DNS resolution changes to a new IP address. When a DFP host initially resolves to IP address A, a `LogicalHost` is created and added to both the `host_map_` and the cross-priority host map (tracked by `MainPrioritySetImpl`). When the same host later re-resolves to IP address B, the current code updates the `LogicalHost`'s address in-place via `setNewAddresses()` without removing the old address from the cross-priority host map.
77+
78+
This means IP A remains in the cross-priority host map permanently. When the host's TTL expires and it is removed, only IP B is cleaned up from the map — IP A leaks. Over time, with repeated DNS re-resolution, the cross-priority host map grows without bound.
79+
80+
The fix refactors `addOrUpdateHost()` to remove the old host and create a new `LogicalHost` with the new address, properly emitting both `hosts_added` and `hosts_removed` events to the priority set. The method signature is simplified to remove the accumulated `hosts_added` vector pattern, and `updatePriorityState()` is called directly within `addOrUpdateHost()`. Debug logging is also added to `upstream_impl.cc` for cross-priority host map operations.
81+
82+
## Task
83+
84+
Changes:
85+
- 4 files modified (upstream_impl.cc, cluster.cc, cluster.h, cluster_test.cc)
86+
- 29 additions, 39 deletions
87+
88+
Tasks:
89+
1. Add debug logging to `source/common/upstream/upstream_impl.cc` for cross-priority host map mutations
90+
2. Refactor `addOrUpdateHost()` in `source/extensions/clusters/dynamic_forward_proxy/cluster.cc` to replace in-place address updates with host removal and re-creation
91+
3. Simplify the `addOrUpdateHost()` signature in `cluster.h` (remove the `hosts_added` out-parameter)
92+
4. Update test expectations in `cluster_test.cc` for the new per-host update notification pattern
93+
94+
## Success Criteria
95+
96+
Code changes match the expected ground-truth fix.
97+
Code follows repository conventions.
98+
No regressions in existing functionality.
99+
All 4 modified files updated correctly.
100+
101+
## Testing
102+
103+
Your implementation will be automatically verified:
104+
105+
```
106+
The verifier will compare your code changes against the expected ground-truth diff.
107+
Score = 0.35 * file_recall + 0.45 * line_recall + 0.20 * line_precision
108+
```
109+
110+
**Time Limit:** 15 minutes
111+
**Estimated Context:** 10000 tokens

‎benchmarks/ccb_fix/envoy-dfp-host-leak-fix-001/task.toml‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
version = "1.0"
2+
[metadata]
3+
name = "envoy-dfp-host-leak-fix-001"
4+
description = "Fixes memory leak in dynamic forward proxy cluster when DNS re-resolves to new IP (#31433)"
5+
license = "Apache-2.0"
6+
7+
[task]
8+
id = "envoy-dfp-host-leak-fix-001"
9+
repo = "envoyproxy/envoy"
10+
category = "cross_module_bug_fix"
11+
language = "cpp"
12+
pre_fix_rev = "5160151e14837c6f352b9d0b3a9f55119a9317e8"
13+
ground_truth_rev = "f20b6ca6b3cd1b3f2577d495e939831019d9a86d"
14+
difficulty = "hard"
15+
time_limit_sec = 900
16+
17+
[verification]
18+
type = "test"
19+
command = "bash /tests/test.sh"
20+
21+
reward_type = "diff_similarity"
22+
description = "Similarity between agent diff and expected ground-truth diff"
23+
[environment]
24+
build_timeout_sec = 1800.0
25+
26+
[environment.setup_scripts]
27+
mcp_config = """#!/bin/bash
28+
# Setup Sourcegraph MCP if credentials provided
29+
if [ -n "$SOURCEGRAPH_ACCESS_TOKEN" ] && [ -n "$SOURCEGRAPH_URL" ]; then
30+
echo "Setting up Sourcegraph MCP configuration..."
31+
mkdir -p /root/.config/claude
32+
33+
cat > /root/.config/claude/mcp.json << 'EOF'
34+
{
35+
"mcpServers": {
36+
"sourcegraph": {
37+
"command": "npx",
38+
"args": ["-y", "@sourcegraph/mcp-server"],
39+
"env": {
40+
"SRC_ACCESS_TOKEN": "$SOURCEGRAPH_ACCESS_TOKEN",
41+
"SOURCEGRAPH_URL": "$SOURCEGRAPH_URL"
42+
}
43+
}
44+
}
45+
}
46+
EOF
47+
48+
echo "PASS MCP configuration created"
49+
else
50+
echo "No Sourcegraph credentials provided, MCP disabled"
51+
fi
52+
exit 0
53+
"""

0 commit comments

Comments
 (0)