Skip to content

Add CodeQualityRefine plugin #4

@macabeus

Description

@macabeus

Currently, the pipeline stops execution as soon as a binary match is achieved.

Proposed improvement

We can extend the pipeline to continue iterating after a match in order to improve code quality besides just matching it.

To achieve this, we need metrics to evaluate whether the generated code is improving over time. Possible approaches include:

  • Static analysis metrics such as cyclomatic complexity
  • Detecting known code smells (e.g. use of goto)
  • Asking an LLM to evaluate the code and provide a quality score based on the target codebase's standards

Once a match is found, the pipeline would enter a refinement phase where it attempts to improve code quality. The process stops when either:

  • A target quality score is reached, or
  • A maximum number of refinement attempts is exceeded
flowchart TD
    A[Prompt Loader] --> |Load prompts from directory| B

    B[Claude Runner]
    C[Compiler]
    D[Objdiff]
    B --> |Generate C| C
    C --> |Compilation Error → Retry| B
    C --> |Compile to object file| D
    D --> |Mismatch → Retry| B

    D --> |Match found| E[Code Quality Refiner]
    D --> |Max retries exceeded| F[Fail]

    E --> J[Compiler]
    J --> K[Objdiff]

    J --> |Compilation Error → Retry| E
    K --> |Mismatch → Retry| E

    K --> |Max retries exceeded| G[Stop]
    K --> |Target score achieved| H[Success]
Loading

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions