feat: add GEPA code-state CLI mutator#13
Open
twaldin wants to merge 1 commit into
Open
Conversation
📝 WalkthroughWalkthroughThis PR integrates a custom candidate proposer mechanism into GEPA optimization, enabling HarnessMutatorAdapter to generate new code candidates by extracting context from GEPA's reflective dataset. The feature is wired through optimize_code, exposed via a new CLI command, and tested end-to-end. ChangesCustom Candidate Proposer Integration for GEPA Optimization
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/hone/cli.py`:
- Around line 336-346: Wrap the call to optimize_code in a try/except that
catches Exception; on exception, use typer.secho to print a red error message
including the exception message (and optionally repr) and then raise
typer.Exit(code=2) to terminate cleanly; update the block where
optimize_code(...) is called (referencing the optimize_code call and its
parameters like repo, test_command, seed_instructions, base_commit,
max_metric_calls, objective, background, config, mutator_adapter) so that
failures produce the red message + typer.Exit rather than letting the traceback
bubble.
In `@src/hone/gepa_faithful.py`:
- Around line 295-316: The proposer currently builds new instructions via
_instructions_from_reflection and calls mutate but then returns a GEPA candidate
that still contains the old code_candidate.instructions and drops
components_to_update; update the returned candidate to include the new/returned
instructions and preserve any components_to_update from the mutated candidate.
Concretely, after calling mutate(HoneCodeCandidate(...)), construct the value
passed to candidate_to_gepa using the commit_sha from
mutation.candidate.commit_sha and instructions from
mutation.candidate.instructions (or fall back to the local instructions
variable), and also copy mutation.candidate.components_to_update (or
code_candidate.components_to_update if appropriate) into the returned
HoneCodeCandidate so the proposed text change is actually applied.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b8771ee6-e125-4990-994c-b37f1d3bc085
📒 Files selected for processing (3)
src/hone/cli.pysrc/hone/gepa_faithful.pytests/test_gepa_faithful.py
Comment on lines
+336
to
+346
| result = optimize_code( | ||
| repo_path=repo, | ||
| test_command=test_command, | ||
| seed_instructions=seed_instructions, | ||
| base_commit=base_commit, | ||
| max_metric_calls=max_metric_calls, | ||
| objective=objective, | ||
| background=background, | ||
| config=config, | ||
| mutator_adapter=adapter, | ||
| ) |
There was a problem hiding this comment.
Handle optimize-code failures with CLI-friendly exits.
This path currently lets exceptions bubble as tracebacks. Converting to a red message + typer.Exit(code=2) keeps behavior consistent with other commands and improves automation UX.
💡 Suggested fix
- result = optimize_code(
- repo_path=repo,
- test_command=test_command,
- seed_instructions=seed_instructions,
- base_commit=base_commit,
- max_metric_calls=max_metric_calls,
- objective=objective,
- background=background,
- config=config,
- mutator_adapter=adapter,
- )
+ try:
+ result = optimize_code(
+ repo_path=repo,
+ test_command=test_command,
+ seed_instructions=seed_instructions,
+ base_commit=base_commit,
+ max_metric_calls=max_metric_calls,
+ objective=objective,
+ background=background,
+ config=config,
+ mutator_adapter=adapter,
+ )
+ except Exception as exc:
+ console.print(f"[red]optimize-code failed: {exc}[/red]")
+ raise typer.Exit(code=2) from exc📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| result = optimize_code( | |
| repo_path=repo, | |
| test_command=test_command, | |
| seed_instructions=seed_instructions, | |
| base_commit=base_commit, | |
| max_metric_calls=max_metric_calls, | |
| objective=objective, | |
| background=background, | |
| config=config, | |
| mutator_adapter=adapter, | |
| ) | |
| try: | |
| result = optimize_code( | |
| repo_path=repo, | |
| test_command=test_command, | |
| seed_instructions=seed_instructions, | |
| base_commit=base_commit, | |
| max_metric_calls=max_metric_calls, | |
| objective=objective, | |
| background=background, | |
| config=config, | |
| mutator_adapter=adapter, | |
| ) | |
| except Exception as exc: | |
| console.print(f"[red]optimize-code failed: {exc}[/red]") | |
| raise typer.Exit(code=2) from exc |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/hone/cli.py` around lines 336 - 346, Wrap the call to optimize_code in a
try/except that catches Exception; on exception, use typer.secho to print a red
error message including the exception message (and optionally repr) and then
raise typer.Exit(code=2) to terminate cleanly; update the block where
optimize_code(...) is called (referencing the optimize_code call and its
parameters like repo, test_command, seed_instructions, base_commit,
max_metric_calls, objective, background, config, mutator_adapter) so that
failures produce the red message + typer.Exit rather than letting the traceback
bubble.
Comment on lines
+295
to
+316
| del components_to_update | ||
| code_candidate = ( | ||
| candidate_from_gepa(candidate) if isinstance(candidate, Mapping) else candidate | ||
| ) | ||
| instructions = _instructions_from_reflection( | ||
| fallback=code_candidate.instructions, | ||
| reflective_dataset=reflective_dataset, | ||
| ) | ||
| mutation = self.mutate( | ||
| HoneCodeCandidate( | ||
| commit_sha=code_candidate.commit_sha, | ||
| instructions=instructions, | ||
| ), | ||
| example, | ||
| iteration=iteration, | ||
| ) | ||
| return candidate_to_gepa( | ||
| HoneCodeCandidate( | ||
| commit_sha=mutation.candidate.commit_sha, | ||
| instructions=code_candidate.instructions, | ||
| ) | ||
| ) |
There was a problem hiding this comment.
Return updated instruction text from proposer output.
propose_new_texts builds reflection-derived instructions but returns the old candidate.instructions and ignores components_to_update. That can turn text proposals into a no-op even though a new code commit is produced.
💡 Suggested fix
- del components_to_update
code_candidate = (
candidate_from_gepa(candidate) if isinstance(candidate, Mapping) else candidate
)
instructions = _instructions_from_reflection(
fallback=code_candidate.instructions,
reflective_dataset=reflective_dataset,
)
@@
- return candidate_to_gepa(
- HoneCodeCandidate(
- commit_sha=mutation.candidate.commit_sha,
- instructions=code_candidate.instructions,
- )
- )
+ updated_instructions = (
+ instructions
+ if "instructions" in set(components_to_update)
+ else code_candidate.instructions
+ )
+ return candidate_to_gepa(
+ HoneCodeCandidate(
+ commit_sha=mutation.candidate.commit_sha,
+ instructions=updated_instructions,
+ )
+ )📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| del components_to_update | |
| code_candidate = ( | |
| candidate_from_gepa(candidate) if isinstance(candidate, Mapping) else candidate | |
| ) | |
| instructions = _instructions_from_reflection( | |
| fallback=code_candidate.instructions, | |
| reflective_dataset=reflective_dataset, | |
| ) | |
| mutation = self.mutate( | |
| HoneCodeCandidate( | |
| commit_sha=code_candidate.commit_sha, | |
| instructions=instructions, | |
| ), | |
| example, | |
| iteration=iteration, | |
| ) | |
| return candidate_to_gepa( | |
| HoneCodeCandidate( | |
| commit_sha=mutation.candidate.commit_sha, | |
| instructions=code_candidate.instructions, | |
| ) | |
| ) | |
| code_candidate = ( | |
| candidate_from_gepa(candidate) if isinstance(candidate, Mapping) else candidate | |
| ) | |
| instructions = _instructions_from_reflection( | |
| fallback=code_candidate.instructions, | |
| reflective_dataset=reflective_dataset, | |
| ) | |
| mutation = self.mutate( | |
| HoneCodeCandidate( | |
| commit_sha=code_candidate.commit_sha, | |
| instructions=instructions, | |
| ), | |
| example, | |
| iteration=iteration, | |
| ) | |
| updated_instructions = ( | |
| instructions | |
| if "instructions" in set(components_to_update) | |
| else code_candidate.instructions | |
| ) | |
| return candidate_to_gepa( | |
| HoneCodeCandidate( | |
| commit_sha=mutation.candidate.commit_sha, | |
| instructions=updated_instructions, | |
| ) | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/hone/gepa_faithful.py` around lines 295 - 316, The proposer currently
builds new instructions via _instructions_from_reflection and calls mutate but
then returns a GEPA candidate that still contains the old
code_candidate.instructions and drops components_to_update; update the returned
candidate to include the new/returned instructions and preserve any
components_to_update from the mutated candidate. Concretely, after calling
mutate(HoneCodeCandidate(...)), construct the value passed to candidate_to_gepa
using the commit_sha from mutation.candidate.commit_sha and instructions from
mutation.candidate.instructions (or fall back to the local instructions
variable), and also copy mutation.candidate.components_to_update (or
code_candidate.components_to_update if appropriate) into the returned
HoneCodeCandidate so the proposed text change is actually applied.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test Plan
Note: plain python3 -m pytest -q currently fails before collection because a globally auto-loaded pytest_httpbin plugin imports missing flask; project tests pass with plugin autoload disabled.
Summary by CodeRabbit
Release Notes
optimize-codeCLI command for automated code optimization across git repositories using GEPA framework.