Skip to content

feat(scaffold): initialize project + schema v0 + data policy (GH #1/#2/#3) - #8

Merged
rmems merged 17 commits into
mainfrom
feat/initialize-scaffold
Jun 21, 2026
Merged

feat(scaffold): initialize project + schema v0 + data policy (GH #1/#2/#3)#8
rmems merged 17 commits into
mainfrom
feat/initialize-scaffold

Conversation

@rmems

@rmems rmems commented Jun 17, 2026

Copy link
Copy Markdown
Owner

User description

feat(scaffold): initialize project + schema v0 + data policy (GH #1/#2/#3)

Summary

Initial project scaffolding and foundational artifacts for Operation Prometheus.

  • Completes GitHub #1: Project scaffold (directories + .gitkeep, root .gitignore, datasets/README.md, README layout links). No models/ directory created per instruction.
  • Implements GitHub #2: Define trajectory JSON schema v0
    • schemas/pr_trajectory.schema.json (draft v0, not final)
    • Required fields + enums for task_type, outcome, training_use
    • Tiny committed example: datasets/examples/trajectory-v0-example.json
    • Basic validation passes
  • Implements GitHub #3: Data policy and dataset hygiene rules
    • docs/data-policy.md
    • Allowed public sources, excluded sources, manual inspection requirement
    • Distinguishes public engineering history vs. raw chat log scraping
    • Aligns with AGENTS.md "Do Not" section

Also adds support for using the global personal Beads DB (raulmc- prefix) from this repo:

  • .beads/README.md documents BEADS_DIR=/home/raulmc/.beads usage
  • .gitignore updated to ignore .beads/* except the README (no local DB fragmentation)
  • Tracked beads issues: raulmc-vge and raulmc-9cq (created, claimed, implemented, closed)

Changes

.gitignore
README.md
.beads/README.md
docs/data-policy.md
schemas/pr_trajectory.schema.json
datasets/examples/.gitkeep
datasets/examples/trajectory-v0-example.json
+ .gitkeep files in docs/, schemas/, scripts/, evals/, datasets/{cards,jsonl,raw}/

Type of Change

  • Documentation / scaffold / initial setup
  • New schema (v0 draft)
  • Policy / hygiene rules

Pre-flight Checklist

  • git status clean before commit
  • AGENTS.md validation run (ruff check, pytest -q, validate script placeholders)
  • Beads used for tracking (bd create, --claim, close)
  • No secrets / raw data / model weights committed
  • Pushed to both github and gitlab remotes for testing

Testing

  • Schema example validates against the v0 JSON schema (lightweight Python check)
  • All files staged/committed cleanly on feat/initialize-scaffold
  • Branch pushed to both remotes

Related

This is the foundation before any GitHub extraction scripts or larger datasets.


CodeAnt-AI Description

Set up the project structure and document the first trajectory data rules

What Changed

  • Added the main repo folders for docs, schemas, scripts, evals, and dataset outputs so the project has a clear layout from the start
  • Documented what can and cannot be committed in datasets, including small examples, cards, and local-only raw data
  • Added the first trajectory schema and a sample record to show the expected data shape
  • Published the data policy covering allowed public sources, excluded material, and the manual review step before training use
  • Updated the README with the new layout and links to the schema and policy

Impact

✅ Clearer dataset submission rules
✅ Easier navigation of the repository
✅ Safer public-data collection

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

rmems added 2 commits June 17, 2026 00:04
- Add docs/, schemas/, scripts/, evals/, datasets/{cards,jsonl,raw}/ with .gitkeep
- Add root .gitignore protecting secrets, caches, raw datasets, model weights
- Add datasets/README.md explaining no large raw data or weights committed
- Update README.md with Repository Layout section linking key dirs
- No models/ directory created (user instruction)
- Work on feat/initialize-scaffold branch

Refs: #1
…ds support

Implements:
- GitHub #2: Define trajectory JSON schema v0
  - Add schemas/pr_trajectory.schema.json with required fields, enums for task_type/outcome/training_use
  - Add tiny safe example under datasets/examples/trajectory-v0-example.json
  - Basic validation passes; schema documented as v0 (not final)
- GitHub #3: Create data policy and dataset hygiene rules
  - Add docs/data-policy.md (allowed public sources, excluded material, manual inspection, public history vs chat logs)
- Global beads DB usability (raulmc- prefix)
  - Add .beads/README.md explaining BEADS_DIR=/home/raulmc/.beads usage and workflow
  - Update .gitignore to ignore .beads/* except the README (project uses shared global DB, no local embeddeddolt)
- Update root README.md with Repository Layout links + direct references to the new schema/policy and GitHub issues
- Track beads issues: raulmc-vge (in_progress), raulmc-9cq (in_progress)

All changes on feat/initialize-scaffold.
Refs:
- #2
- #3
- Original scaffold: #1

Validation per AGENTS.md performed (ruff/pytest/validate_jsonl placeholders clean).
@codeant-ai

codeant-ai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR bootstraps the Operation Prometheus repository by adding a .gitignore, placeholder .gitkeep files for all top-level directories, a Draft-07 JSON Schema for PR trajectory records, a conforming example record, and documentation covering repository layout, dataset commit rules, and a data hygiene policy.

Changes

Operation Prometheus Initial Repository Setup

Layer / File(s) Summary
Gitignore and directory scaffolding
.gitignore, docs/.gitkeep, schemas/.gitkeep, scripts/.gitkeep, evals/.gitkeep, datasets/cards/.gitkeep, datasets/examples/.gitkeep, datasets/jsonl/.gitkeep, datasets/raw/.gitkeep
.gitignore defines patterns for secrets, Python artifacts, large datasets, model weights, and editor/OS files. All top-level directories gain .gitkeep placeholders to remain tracked in git.
PR Trajectory schema and example record
schemas/pr_trajectory.schema.json, datasets/examples/trajectory-v0-example.json
Draft-07 JSON Schema defines required fields, enum constraints, URI-formatted source_urls, a structured validation array, optional bounded quality_score, and additionalProperties: false. A conforming example record documents a CI workflow fix trajectory.
README updates and data policy
README.md, datasets/README.md, docs/data-policy.md
README.md gains repository layout and schema/data-policy sections. datasets/README.md establishes critical commit rules. docs/data-policy.md defines allowed sources, excluded material, raw export handling, and mandatory manual inspection requirements.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related issues

Poem

🐇 Hippity-hop, a repo takes shape,
With schemas and .gitkeep files in a cape!
No secrets committed, no weights in the tree,
Just trajectories tracked for posterity.
The rabbit approves — it's tidy and free! 🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main changes: project scaffolding, schema v0, and data policy for Operation Prometheus, with GitHub issue references.
Description check ✅ Passed The description comprehensively details all changes across scaffolding, schema, data policy, and Beads DB support, directly corresponding to the changeset and PR objectives.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/initialize-scaffold

Comment @coderabbitai help to get the list of available commands and usage tips.

@rmems rmems self-assigned this Jun 17, 2026
@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Jun 17, 2026

@amazon-q-developer amazon-q-developer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

This PR establishes a solid foundation for Operation Prometheus with comprehensive scaffolding, schema definition, and data policy documentation. The structure is well-organized and aligns with the project's goals.

Critical Issue Found

  • .gitignore contains conflicting rules for .beads/ that will prevent the README from being tracked (blocking correct repository setup)

Overall Assessment

Once the gitignore conflict is resolved, this PR provides a good baseline for future development. The JSON schema is properly structured, the data policy is thorough and responsible, and the example validates correctly against the schema.

Action Required: Fix the gitignore conflict before merge.


You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.

Comment thread .gitignore Outdated
Comment on lines +77 to +78
# Beads (local working state if not using git-tracked db)
.beads/

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛑 Logic Error: Conflicting gitignore rules for .beads/ will prevent the README from being tracked. Line 78 ignores the entire .beads/ directory, then line 82 tries to ignore .beads/* with an exception on line 83. When a directory itself is ignored (line 78), Git won't descend into it to apply the exceptions. Remove the duplicate on line 78 to ensure .beads/README.md can be tracked as intended.

Suggested change
# Beads (local working state if not using git-tracked db)
.beads/
# Beads local state (this project uses the GLOBAL personal DB at ~/.beads with raulmc- prefix)
# Only the documentation README is tracked here.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kilo agent: xAI/ Grok Build 0.1

Fixed in commit 768b31a.

  • Removed the blanket .beads/ rule (it was conflicting with the selective rules below).
  • Kept only:
    .beads/*
    !.beads/README.md
    
    This allows the global DB documentation README to be tracked while ignoring other local state, exactly as intended for the raulmc- prefix setup.

The suggestion has been applied. Thank you.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request establishes the repository layout, data policy, and initial schema (v0) for the Operation Prometheus trajectory datasets. It introduces a comprehensive .gitignore and datasets/README.md to prevent committing large raw datasets or model weights, a data-policy.md document detailing data hygiene rules, and a JSON schema pr_trajectory.schema.json with a corresponding example. The review feedback suggests enhancing the JSON schema by enforcing a regex pattern on the repo field and marking critical fields (like comment in review_signals and type/result in validation) as required.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +13 to +16
"repo": {
"type": "string",
"description": "GitHub repository in owner/repo form."
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure data consistency and prevent malformed repository names, consider adding a regular expression pattern to enforce the owner/repo format.

    "repo": {
      "type": "string",
      "pattern": "^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$",
      "description": "GitHub repository in owner/repo form."
    },

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kilo agent: xAI/ Grok Build 0.1

Accepted. Added in the latest push:

"repo": {
  "type": "string",
  "pattern": "^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$",
  ...
}

This enforces the owner/repo form at schema validation time. Good catch.

Comment on lines +45 to +52
"items": {
"type": "object",
"properties": {
"author": { "type": "string" },
"comment": { "type": "string" },
"suggestion": { "type": "string" }
}
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

An item in review_signals should contain at least a comment to be meaningful. Consider marking the comment field as required for each review signal item.

Suggested change
"items": {
"type": "object",
"properties": {
"author": { "type": "string" },
"comment": { "type": "string" },
"suggestion": { "type": "string" }
}
},
"items": {
"type": "object",
"properties": {
"author": { "type": "string" },
"comment": { "type": "string" },
"suggestion": { "type": "string" }
},
"required": ["comment"]
},

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kilo agent: xAI/ Grok Build 0.1

Done. The review_signals item now declares:

"required": ["comment"],
"additionalProperties": false

A review signal without a comment body will no longer validate. Author and suggestion remain optional (they can be useful for attribution but are not always present in public data).

Comment on lines +65 to +72
"items": {
"type": "object",
"properties": {
"type": { "type": "string", "enum": ["ci", "test", "manual", "review", "other"] },
"result": { "type": "string", "enum": ["pass", "fail", "flaky"] },
"detail": { "type": "string" }
}
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure that validation events are informative, consider requiring both the type and result fields for each item in the validation array.

Suggested change
"items": {
"type": "object",
"properties": {
"type": { "type": "string", "enum": ["ci", "test", "manual", "review", "other"] },
"result": { "type": "string", "enum": ["pass", "fail", "flaky"] },
"detail": { "type": "string" }
}
},
"items": {
"type": "object",
"properties": {
"type": { "type": "string", "enum": ["ci", "test", "manual", "review", "other"] },
"result": { "type": "string", "enum": ["pass", "fail", "flaky"] },
"detail": { "type": "string" }
},
"required": ["type", "result"]
},

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kilo agent: xAI/ Grok Build 0.1

Applied:

"validation": {
  "items": {
    ...
    "required": ["type", "result"],
    "additionalProperties": false
  }
}

Empty or partial validation entries are now rejected. This strengthens the training signal for validation outcomes. Thank you.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.gitignore:
- Around line 77-83: The .gitignore file has conflicting rules where the pattern
`.beads/` on line 78 ignores the entire directory, which prevents git from
traversing into it to process the negation pattern `!.beads/README.md` that
follows later. Remove the line containing `.beads/` (the unconditional ignore
rule) and keep the remaining patterns `.beads/*` and `!.beads/README.md` which
will correctly ignore all contents of the .beads/ directory while allowing the
README.md file to be tracked.

In `@docs/data-policy.md`:
- Around line 51-53: The fenced code block in data-policy.md that displays
"issue/review signal → before state → patch/fix → validation → outcome" is
missing a language identifier, which violates the markdownlint MD040 rule. Add a
language identifier such as `text` immediately after the opening triple
backticks (```). This ensures the code block is properly formatted and the
markdown linter passes without warnings.

In `@README.md`:
- Line 52: The README.md file at the schemas/ directory description incorrectly
lists both Pydantic models and JSON Schema as contents. Update the description
of the schemas/ line to reflect that it contains only JSON Schema definitions,
removing the reference to Pydantic models to accurately represent the current
repository structure and avoid confusing contributors about what is actually
included in the schemas directory.

In `@schemas/pr_trajectory.schema.json`:
- Around line 45-72: The nested object schemas for items in both the
review_signals and validation arrays lack required property constraints and
allow additional properties. To strengthen the data contract, add a required
array specifying which properties must be present in each item object (for
review_signals items: author, comment, suggestion; for validation items: type,
result, detail) and set additionalProperties to false in both nested object
definitions to prevent unexpected fields from being included in the records.
🪄 Autofix (Beta)

✅ Autofix completed


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 9b73c172-917c-4193-a662-a5b7c819cf29

📥 Commits

Reviewing files that changed from the base of the PR and between a04ca23 and 878c753.

📒 Files selected for processing (14)
  • .gitignore
  • README.md
  • datasets/README.md
  • datasets/cards/.gitkeep
  • datasets/examples/.gitkeep
  • datasets/examples/trajectory-v0-example.json
  • datasets/jsonl/.gitkeep
  • datasets/raw/.gitkeep
  • docs/.gitkeep
  • docs/data-policy.md
  • evals/.gitkeep
  • schemas/.gitkeep
  • schemas/pr_trajectory.schema.json
  • scripts/.gitkeep
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Kilo Code Review
🧰 Additional context used
🪛 markdownlint-cli2 (0.22.1)
docs/data-policy.md

[warning] 51-51: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🔇 Additional comments (11)
.gitignore (1)

41-46: ⚡ Quick win

Verify datasets/raw/.gitkeep exists in the PR.

Line 44 of .gitignore references !datasets/raw/.gitkeep, but this file was not provided in this review. The PR objectives mention creating .gitkeep files in "datasets/ subdirectories (cards, jsonl, raw)", and the pattern on lines 43–44 explicitly excludes the .gitkeep file from the ignore rule.

Please confirm that datasets/raw/.gitkeep has been created as part of this PR.

docs/.gitkeep (1)

1-2: LGTM!

schemas/.gitkeep (1)

1-2: LGTM!

scripts/.gitkeep (1)

1-2: LGTM!

evals/.gitkeep (1)

1-2: LGTM!

datasets/cards/.gitkeep (1)

1-2: LGTM!

datasets/examples/.gitkeep (1)

1-2: LGTM!

datasets/jsonl/.gitkeep (1)

1-2: LGTM!

datasets/raw/.gitkeep (1)

1-1: LGTM!

datasets/examples/trajectory-v0-example.json (1)

1-29: LGTM!

datasets/README.md (1)

1-19: LGTM!

Comment thread .gitignore Outdated
Comment thread docs/data-policy.md Outdated
Comment thread README.md Outdated
Comment thread schemas/pr_trajectory.schema.json

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 878c753984

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"description": "Pull request number within the repo."
},
"source_urls": {
"type": "array",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require at least one source URL

With source_urls only typed as an array, a record like "source_urls": [] still satisfies this required field. That leaves accepted trajectory records without any PR/issue provenance, which conflicts with the public-source/manual-inspection policy and makes later dataset audits impossible; add minItems: 1 here.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kilo agent: xAI/ Grok Build 0.1

All addressed together with the other P2 items in the hardening patch (see replies above).

Summary of what landed:

  • id: minLength 1
  • source_urls: minItems 1
  • patch: minLength 1
  • review_signals + validation items now have required + additionalProperties:false

The empty-array and blank-ID cases are now rejected at the schema level. This directly supports the public-source + manual-inspection policy.

Comment thread schemas/pr_trajectory.schema.json
Comment thread schemas/pr_trajectory.schema.json
Comment thread schemas/pr_trajectory.schema.json Outdated
Comment thread schemas/pr_trajectory.schema.json
@rmems rmems added documentation Improvements or additions to documentation enhancement New feature or request labels Jun 17, 2026
…ld 0.1)

- .gitignore: removed conflicting blanket `.beads/` (amazon-q-developer, CodeRabbit)
- README.md: schemas/ description corrected to "JSON Schema v0 draft" only (no Pydantic claim) (CodeRabbit)
- docs/data-policy.md: added language tag ` ```text ` to the trajectory diagram (MD040) (CodeRabbit)
- schemas/pr_trajectory.schema.json:
  * `id`: added `minLength: 1`
  * `repo`: added `pattern` for owner/repo (gemini-code-assist)
  * `source_urls`: added `minItems: 1`
  * `patch`: added `minLength: 1`
  * `review_signals` items: `required: ["comment"]`, `additionalProperties: false`
  * `validation` items: `required: ["type", "result"]`, `additionalProperties: false` (gemini, CodeRabbit, chatgpt-codex-connector)

All changes keep the v0 draft minimal and aligned with AGENTS.md + data policy.

Cites: Kilo agent: xAI/ Grok Build 0.1

Addresses threads in #8

Refs beads: raulmc-s6w
@kilo-code-bot

kilo-code-bot Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Code Review Roast 🔥

Verdict: No Issues Found | Recommendation: Merge

Oh wait, this PR is actually clean. I need to sit down. I had my flamethrower warmed up and everything.

📊 Overall: Like finding a unicorn in production — I didn't think clean PRs existed anymore, but here we are. This bootstrap PR nails the basics: .gitignore that doesn't blow up in your face, a JSON Schema that validates, and example data that actually conforms to it. Someone call the Smithsonian, we've got a museum piece on our hands.

Files Reviewed (12 files)
  • .gitignore — clean
  • README.md — clean
  • docs/data-policy.md — clean
  • datasets/README.md — clean
  • providers/README.md — clean
  • schemas/pr_trajectory.schema.json — clean (all P2 issues addressed)
  • datasets/examples/trajectory-v0-example.json — clean, validates
  • Various .gitkeep files — placeholders, no issues
Previous Review Summaries (7 snapshots, latest commit e32e1f0)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit e32e1f0)

Verdict: No Issues Found | Recommendation: Merge

Oh wait, this PR is actually clean. I need to sit down. I had my flamethrower warmed up and everything.

📊 Overall: Like the second pancake — the first had weird edges, but this one cooked right.

Files Reviewed (1 file)
  • schemas/pr_trajectory.schema.json — no new issues (incremental formatting and guardrail improvements)

Previous review (commit fc68ed9)

Verdict: No Issues Found | Recommendation: Merge

Oh wait, this PR is actually clean. I need to sit down. I had my flamethrower warmed up and everything.

📊 Overall: Like the second pancake — the first had weird edges, but this one cooked right.

Files Reviewed (1 file)
  • schemas/pr_trajectory.schema.json — no new issues (incremental formatting and guardrail improvements)

Previous review (commit b7ff5df)

Verdict: Issues Resolved | Recommendation: Merge

Overview

Severity Count
🚨 critical 0
⚠️ warning 0
💡 suggestion 0
🤏 nitpick 0
Issue Details (click to expand)

No current issues found in the incremental diff. The previous schema emptiness loopholes remain fixed.


🏆 Best part: The schema actually learned guardrails this time — minLength, minItems, required nested fields, and a sane PR-number floor. Shocking. It almost looks maintainable.

💀 Worst part: None. The incremental diff is boring in the best possible way, like a deployment with no pager duty.

📊 Overall: Like the second pancake — the first had weird edges, but this one cooked right.

Files Reviewed (14 files)
  • schemas/pr_trajectory.schema.json - no new issues; previous schema guardrails resolved
  • .gitignore - no new issues; previous beads conflict resolved, JSONL policy intentionally loosened
  • datasets/README.md - clean, carried forward
  • datasets/cards/.gitkeep - clean, carried forward
  • datasets/examples/.gitkeep - clean, carried forward
  • datasets/examples/trajectory-v0-example.json - clean, carried forward
  • datasets/jsonl/.gitkeep - clean, carried forward
  • datasets/raw/.gitkeep - clean, carried forward
  • docs/.gitkeep - clean, carried forward
  • docs/data-policy.md - clean, carried forward
  • evals/.gitkeep - clean, carried forward
  • scripts/.gitkeep - clean, carried forward
  • README.md - clean, carried forward
  • schemas/.gitkeep - clean, carried forward

Previous review (commit 520bb23)

Verdict: Issues Resolved | Recommendation: Merge

Overview

Severity Count
🚨 critical 0
⚠️ warning 0
💡 suggestion 0
🤏 nitpick 0
Issue Details (click to expand)
File Line Roast
schemas/pr_trajectory.schema.json 53,76 Previously flagged: missing minLength on newly-required fields. Now fixed with minLength: 1 guardrails.

🏆 Best part: The fix commit landed cleanly — minLength: 1 added to suggestion and detail, and required arrays properly set. Like finding a unicorn in production — I didn't think clean PRs existed anymore, but here we are.

💀 Worst part: None. The previous emptiness-loophole issues are now properly guarded.

📊 Overall: Like a second pancake — the first one had some shape issues, but this one cooked right.

Files Reviewed (14 files)
  • schemas/pr_trajectory.schema.json - issues resolved
  • .gitignore - clean
  • datasets/README.md - clean
  • datasets/cards/.gitkeep - clean
  • datasets/examples/.gitkeep - clean
  • datasets/examples/trajectory-v0-example.json - clean
  • datasets/jsonl/.gitkeep - clean
  • datasets/raw/.gitkeep - clean
  • docs/.gitkeep - clean
  • docs/data-policy.md - clean
  • evals/.gitkeep - clean
  • scripts/.gitkeep - clean
  • README.md - clean

Previous review (commit add4a9f)

Verdict: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
🤏 nitpick 2
Issue Details (click to expand)
File Line Roast
schemas/pr_trajectory.schema.json 55 Added suggestion to required but forgot minLength: 1 on it and sibling fields — empty strings will sneak through validation like a ninja in an empty hallway.
schemas/pr_trajectory.schema.json 78 detail is now required in validation events but without minLength — an empty detail string passes validation with the enthusiasm of a placeholder commit message.
---

🏆 Best part: The previous P2 hardening feedback was actually taken seriously — minLength on id/patch, minItems on source_urls, required on nested objects. That's the kind of responsiveness that makes a reviewer put the flamethrower down.

💀 Worst part: The new changes required fields but forgot to guard against empty strings. It's like installing a security door but leaving the key under the mat. minLength: 1 exists for a reason — use it.

📊 Overall: Like a house that got a new roof but left the windows open — major progress, but the schema still leaks emptiness. Two quick minLength additions away from "shut up and merge" territory.

Files Reviewed (1 changed file in incremental diff + 7 from prior review)
  • schemas/pr_trajectory.schema.json - 2 new nitpicks (missing minLength on newly-required fields)
  • .gitignore - resolved (previous P2 on conflicting rules addressed)
  • docs/data-policy.md - resolved (markdownlint fix landed)
  • README.md - resolved (Pydantic misrepresentation corrected)
  • .gitkeep files - clean
  • datasets/README.md - clean
  • datasets/examples/trajectory-v0-example.json - clean

Fix these issues in Kilo Cloud

Previous review (commit 4b39fa6)

Verdict: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
🤏 nitpick 2
Issue Details (click to expand)
File Line Roast
schemas/pr_trajectory.schema.json 55 Added suggestion to required but forgot minLength: 1 on it and sibling fields — empty strings will sneak through validation like a ninja in an empty hallway.
schemas/pr_trajectory.schema.json 78 detail is now required in validation events but without minLength — an empty detail string passes validation with the enthusiasm of a placeholder commit message.
---

🏆 Best part: The previous P2 hardening feedback was actually taken seriously — minLength on id/patch, minItems on source_urls, required on nested objects. That's the kind of responsiveness that makes a reviewer put the flamethrower down.

💀 Worst part: The new changes required fields but forgot to guard against empty strings. It's like installing a security door but leaving the key under the mat. minLength: 1 exists for a reason — use it.

📊 Overall: Like a house that got a new roof but left the windows open — major progress, but the schema still leaks emptiness. Two quick minLength additions away from "shut up and merge" territory.

Files Reviewed (1 changed file in incremental diff + 7 from prior review)
  • schemas/pr_trajectory.schema.json - 2 new nitpicks (missing minLength on newly-required fields)
  • .gitignore - resolved (previous P2 on conflicting rules addressed)
  • docs/data-policy.md - resolved (markdownlint fix landed)
  • README.md - resolved (Pydantic misrepresentation corrected)
  • .gitkeep files - clean
  • datasets/README.md - clean
  • datasets/examples/trajectory-v0-example.json - clean

Fix these issues in Kilo Cloud

Previous review (commit 768b31a)

Verdict: No Issues Found | Recommendation: Merge

Oh wait, this PR is actually clean. I need to sit down. I had my flamethrower warmed up and everything.

📊 Overall: Like finding a unicorn in production — I didn't think clean PRs existed anymore, but here we are.

Files Reviewed (7 files)
  • .gitignore - clean, properly ignores secrets/datasets/model-weights
  • schemas/pr_trajectory.schema.json - hardened schema after feedback (minLength, minItems, required fields)
  • datasets/examples/trajectory-v0-example.json - valid example conforming to schema
  • docs/data-policy.md - clear policy document
  • README.md - accurate repository layout documentation
  • datasets/README.md - clear commit rules
  • .gitkeep files - standard placeholders

Reviewed by laguna-m.1-20260312:free · 1,582,647 tokens

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 768b31aa92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +69 to +70
"validation": {
"type": "array",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require at least one validation event

After the current schema requires type and result for validation items, "validation": [] still satisfies this required field because the array has no minItems. That lets extracted trajectory records pass without any CI/test/manual validation signal, which undermines the stated patch/fix → validation → outcome dataset step; add minItems: 1 here if every accepted record must include validation evidence.

Useful? React with 👍 / 👎.

"description": "GitHub repository in owner/repo form."
},
"pr_number": {
"type": "integer",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject non-positive PR numbers

For GitHub PR trajectories, pr_number must identify an actual pull request number, but this schema accepts 0 and negative integers. If an extractor emits one of those values, the record can pass validation while later provenance links or GitHub API lookups point to an impossible PR; add a minimum: 1 constraint here.

Useful? React with 👍 / 👎.

Comment on lines +60 to +61
"before_context": {
"type": "string",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject blank before-state context

Because before_context is top-level required but only constrained to be a string, "before_context": "" still validates. That accepts trajectory records with no pre-change code/state snippet or file reference, breaking the repository's code state → patch/fix signal and making repair examples much harder to audit; add a non-empty constraint here.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
schemas/pr_trajectory.schema.json (2)

19-22: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Constrain pr_number to valid PR IDs.

Line 19 currently accepts 0 and negative values, which are invalid PR numbers and can pollute downstream datasets.

Proposed schema fix
     "pr_number": {
       "type": "integer",
+      "minimum": 1,
       "description": "Pull request number within the repo."
     },
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@schemas/pr_trajectory.schema.json` around lines 19 - 22, The pr_number field
in the schema currently has no minimum value constraint, allowing invalid PR
numbers like 0 and negative values to pass validation. Add a minimum value
constraint to the pr_number field definition to ensure only positive integers
are accepted, specifically by adding a minimum property set to 1 to restrict
pr_number to valid pull request identifiers.

69-81: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Require at least one validation event.

Line 69 makes validation required, but [] still passes. That weakens the trajectory contract for consumers expecting post-change verification evidence.

Proposed schema fix
     "validation": {
       "type": "array",
+      "minItems": 1,
       "items": {
         "type": "object",
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@schemas/pr_trajectory.schema.json` around lines 69 - 81, The validation array
property is marked as required but currently allows an empty array, which
defeats the purpose of enforcing post-change verification evidence. Add a
minItems constraint set to 1 on the validation array definition to ensure at
least one validation event object is always present when the validation property
is included in the schema.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@schemas/pr_trajectory.schema.json`:
- Around line 19-22: The pr_number field in the schema currently has no minimum
value constraint, allowing invalid PR numbers like 0 and negative values to pass
validation. Add a minimum value constraint to the pr_number field definition to
ensure only positive integers are accepted, specifically by adding a minimum
property set to 1 to restrict pr_number to valid pull request identifiers.
- Around line 69-81: The validation array property is marked as required but
currently allows an empty array, which defeats the purpose of enforcing
post-change verification evidence. Add a minItems constraint set to 1 on the
validation array definition to ensure at least one validation event object is
always present when the validation property is included in the schema.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0d53956b-f252-472a-ab32-73c00ccb6263

📥 Commits

Reviewing files that changed from the base of the PR and between 878c753 and 768b31a.

📒 Files selected for processing (4)
  • .gitignore
  • README.md
  • docs/data-policy.md
  • schemas/pr_trajectory.schema.json
💤 Files with no reviewable changes (1)
  • .gitignore
📜 Review details
🔇 Additional comments (2)
README.md (1)

52-52: LGTM!

docs/data-policy.md (1)

51-53: LGTM!

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Note

Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it.

Fixes Applied Successfully

Fixed 1 file(s) based on 1 unresolved review comment.

Files modified:

  • schemas/pr_trajectory.schema.json

Commit: 4b39fa6e85d485f143a8ac6237f1de37e4b9c445

The changes have been pushed to the feat/initialize-scaffold branch.

Time taken: 2m 21s

Fixed 1 file(s) based on 1 unresolved review comment.

Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
Comment thread schemas/pr_trajectory.schema.json Outdated
Comment thread schemas/pr_trajectory.schema.json Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@schemas/pr_trajectory.schema.json`:
- Line 78: The required array in the validation items schema section currently
includes "detail" alongside "type" and "result", but based on your stated
intent, only "type" and "result" should be mandatory fields. Remove "detail"
from the required array so that the field becomes optional rather than enforced,
allowing validation events that don't need explanatory text (such as simple
automated results like "CI passed") to omit the detail field without forcing
placeholder values.
- Line 55: The required constraint in the review_signals schema definition
currently enforces all three fields (author, comment, and suggestion) as
mandatory. Locate the required array in the review_signals schema and remove
"suggestion" from the list to make it optional, keeping only "author" and
"comment" as required fields. This allows review signals to be captured even
when reviewers identify issues without providing specific actionable
suggestions.
🪄 Autofix (Beta)

❌ Autofix failed (check again to retry)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a2d9ad0f-e410-45aa-a368-c7dfe4a5cb6a

📥 Commits

Reviewing files that changed from the base of the PR and between 768b31a and 4b39fa6.

📒 Files selected for processing (1)
  • schemas/pr_trajectory.schema.json
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Kilo Code Review
🔇 Additional comments (2)
schemas/pr_trajectory.schema.json (2)

1-45: LGTM!


60-101: LGTM!

Comment thread schemas/pr_trajectory.schema.json Outdated
Comment thread schemas/pr_trajectory.schema.json Outdated
@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Note

Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it.

⚠️ Branch updated during autofix.

The branch was updated while autofix was in progress. Please try again.

rmems and others added 2 commits June 17, 2026 17:43
Co-authored-by: kilo-code-bot[bot] <240665456+kilo-code-bot[bot]@users.noreply.github.com>
Co-authored-by: kilo-code-bot[bot] <240665456+kilo-code-bot[bot]@users.noreply.github.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@schemas/pr_trajectory.schema.json`:
- Around line 55-58: The field definitions for author, comment, and suggestion
are not nested inside a properties object, causing invalid JSON schema syntax.
Wrap these three field definitions (author, comment, suggestion) inside a
properties object block, ensuring they are properly nested as key-value pairs
within the properties object. Then close the properties object with a closing
brace before the additionalProperties field. Apply the same fix to the detail
field at line 80 where the same nesting issue occurs.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: cb823aa7-8c9c-4439-a99c-ee89d266bac8

📥 Commits

Reviewing files that changed from the base of the PR and between 4b39fa6 and add4a9f.

📒 Files selected for processing (1)
  • schemas/pr_trajectory.schema.json
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Kilo Code Review

Comment thread schemas/pr_trajectory.schema.json Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: add4a9ff50

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread schemas/pr_trajectory.schema.json Outdated
Comment on lines +55 to +58
"author": { "type": "string", "minLength": 1 },
"comment": { "type": "string", "minLength": 1 },
"suggestion": { "type": "string", "minLength": 1 }
"additionalProperties": false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Fix invalid JSON in the trajectory schema

As committed, this schema is not valid JSON: the author/comment/suggestion constraints are inserted after the nested properties object without forming a valid key/value sequence. Any validator or loader fails while parsing the schema before it can validate trajectory records, so the new schema cannot be used until these constraints are moved inside properties or the object syntax is repaired.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kilo agent: xAI/ Grok Build 0.1

All the new P2 points have been addressed:

  • pr_number: now has "minimum": 1
  • before_context: now has "minLength": 1
  • validation: now has "minItems": 1 at the array level + required ["type", "result"] on items (detail optional but has minLength when present)
  • review_signals items: required only ["comment"] (suggestion is optional per CodeRabbit's own later suggestion to relax it; all three fields have minLength:1 to prevent empty strings)
  • .gitignore: removed the datasets/jsonl/*.jsonl rule so small curated examples can be committed (as documented in datasets/README.md)

The schema was also fully rewritten to be valid JSON (previous autofix merges had introduced duplicate keys and fields outside the properties object — that was the critical parse error CodeRabbit flagged).

Schema on the branch is now clean and valid.

Pushed in the latest commits on this PR.

Happy to adjust further if needed.

Comment thread schemas/pr_trajectory.schema.json Outdated
Comment on lines +50 to +53
"properties": {
"author": { "type": "string" },
"comment": { "type": "string" },
"suggestion": { "type": "string" }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require content for each review signal

After the schema syntax is repaired, a review_signals item still only declares optional fields and has no required/anyOf constraint, so an extractor could emit review_signals: [{}] or an item with only author and still satisfy the item schema. That leaves records claiming review signal provenance without any actual review comment or suggestion for review-to-patch training; require at least comment or suggestion when an item is present.

Useful? React with 👍 / 👎.

Comment thread .gitignore Outdated
# Keep the .gitkeep files tracked, ignore actual data
datasets/raw/*
!datasets/raw/.gitkeep
datasets/jsonl/*.jsonl

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow curated JSONL records to be tracked

When someone tries to add a small curated JSONL trajectory under the documented datasets/jsonl/ location, this pattern hides it from normal git add/git status; I confirmed git check-ignore -v datasets/jsonl/sample.jsonl points here. That contradicts datasets/README.md's guidance that datasets/jsonl/ is for cleaned linked JSONL records with only small examples/schema samples/cards committed, and can cause validated records to remain local unless force-added or moved elsewhere.

Useful? React with 👍 / 👎.

…ls and validation (Kilo: xAI/ Grok Build 0.1)

- review_signals items: author, comment, suggestion now all have minLength:1 (addresses kilo-code-bot nitpick)
- validation items: detail now has minLength:1 (addresses kilo-code-bot nitpick)

Also fixed broken JSON syntax from previous merges (duplicate keys, missing commas) that was causing Biome parse failures in CodeRabbit.

Cites: Kilo agent: xAI/ Grok Build 0.1

This should make the schema fully tight against empty strings in required fields.

Refs PR #8 new comments from kilo-code-bot and CodeRabbit Biome warning.
…t P2s (Kilo: xAI/ Grok Build 0.1)

- Create providers/ for external model/inference provider adapters and config (no weights)
- Add providers/README.md documenting purpose and what does not belong
- Update README layout to list providers/
- Fix schema if/then: add "required": ["training_use"] so conditional only applies when field present (kilo-code-bot)
- Tighten repo pattern to reject impossible GitHub owner names (chatgpt-codex P2)
- Make example self-consistent: repo/ci-demo#17, only matching source_url (chatgpt-codex P2)
- Fix sample patch hunk ranges + minimal context (chatgpt-codex P2)

Refs PR #8, closes remaining open review threads on this batch.
@rmems
rmems merged commit 74ffd7d into main Jun 21, 2026
2 checks passed
@rmems
rmems deleted the feat/initialize-scaffold branch June 21, 2026 00:28
@rmems rmems added this to the v0.1 Foundation milestone Jul 31, 2026
rmems pushed a commit that referenced this pull request Aug 2, 2026
Five findings from the Codex review of 5a98c3c.

Local agent/tracker state in curated patches (P1). PR #26's record
embedded full diffs for .beads/ (config, git hooks, tracker exports) and
.claude/settings.json — 12 of its 25 files. The .beads/issues.jsonl
export carried a personal email in an `owner` field, so a curated
artifact shipped contributor PII alongside machine-local config that
docs/data-policy.md excludes. Extended `_is_noise_patch_path`, which
already exists for exactly this, with a directory-prefix rule.

Expected-failure cue still lacked intent (P1). The
`failure behavior|mode|case|path|handling` alternative enclosed the only
fail token in "CI showed failure behavior on Linux", inverting it to
pass. Replaced with a deliberately-invalid-fixture cue, which is what
PR #24 actually reads on; a bare failure noun is no longer a cue.

Self-references from the raw record (P2). The guard only covered
synthesized stubs, so a pre-collected same-repo issue matching the PR
number survived into `seen` and out again. Now filtered on the way in.

Copilot reviewer counted as CI (P2). `copilot-pull-request-reviewer`
missed `_REVIEW_APP_CHECK_MARKERS`, so reviewer automation bolstered a
passing CI result. Added the marker and moved it to review_apps.

Stale indexes (P2). README called grok-ozempic "shortlist only" and
STATUS listed the extraction as an open gap with the obsolete six-PR
roadmap including dropped #8.

PR #26 regenerated: patch_chars 67711 -> 50149, before_context file
count 25 -> 13, manifest sha256/bytes updated. No other record changes.
rmems added a commit that referenced this pull request Aug 2, 2026
* feat: grok-ozempic-v0 trajectories (revised 7-PR shortlist)

Extracts the grok-ozempic v0 dataset for GH #11, with the shortlist
re-scored against what the pipeline actually keeps rather than raw
comment volume.

Shortlist changes:
- add #43 (quantize-goz1 CLI) and #33 (test refactor) — both fill the
  previously empty review-to-patch bucket; #33 also supplies the only
  refactor task_type in the set
- drop #8: yields 2 review signals, one being "@copilot Make changes to
  the pull request"
- defer #42: 51 kept signals collapse to 8 identical ack bodies, and it
  is Python against a Rust card with no language_by_pr override

Measured by replaying every PR through lib/bots.is_bot_user and
lib.normalize.extract_review_signals (cap 8). rmems is the repo's only
human account, so most signal comes via the gemini/codex allowlist.

Also adds a task_type override for #26, whose non-conventional title
("Verify grok-ozempic aligns...") fell through to "other" despite adding
grok1_inventory.rs and alignment.rs.

7 records, all 8 review signals, quality 0.90-0.95, strict schema +
policy validation clean. Raw records stay local per .gitignore.

Refs #11

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(normalize): repair linked-issue provenance and expected-failure validation

Addresses review findings on #17.

Linked-issue provenance:
- Drop self-referential linked issues. Issues and PRs share one number
  space, so a synthesized /issues/<pr> URL resolves back to the PR itself.
  This is what put issues/26 on the PR #26 record instead of real issue
  provenance.
- Add LINKED_ISSUE_OVERRIDE for PR bodies that name their originating
  issue without a GitHub close keyword ("Implements GitHub #22",
  "for issues #16, #22, and #27", "addressing issues #28 and #20").
  parse_linked_issue_numbers only recognises close keywords by design, so
  PRs #29/#33/#43 shipped with the PR URL as their only source.

Expected-failure validation:
- A negative-test line ("Manual invalid schema fixture confirms ingest
  failure behavior") matched the fail-word heuristic and stored PR #24's
  test result as `fail` despite every listed command and CI passing.
  Fail words are now checked per occurrence, so a deliberate failure no
  longer inverts the result while a real one alongside it still does.

Dataset: source_urls corrected on records 26/29/33/43, PR #24 test result
set to pass, manifest source_url_count/sha256/bytes regenerated.

* fix(normalize): require explicit expected-failure markers; case-fold override keys

Both findings from the Codex review of 3401a79.

Expected-failure cue was too loose (P1). The reporting-verb alternative
matched any of confirms/verified/checks/ensures followed by a fail word
within 80 characters, so ordinary failure reports were inverted to
`pass` — reproduced with "CI checks failed on Linux" and "Verified that
cargo test failed". Every alternative now carries an explicit
intentionality marker (expected/intended/intentional/deliberate, "fails
as expected", failure behaviour|mode|case|path|handling, negative test).
PR #24 still classifies as pass via "ingest failure behavior".

Override keys are now case-folded (P2). GitHub repo slugs are
case-insensitive but `parse_repo` preserves whatever the CLI was given,
so collecting with `--repo RMEMS/GROK-OZEMPIC` missed every lookup in
TASK_TYPE_OVERRIDE, DOMAIN_OVERRIDE, TRAINING_USE_OVERRIDE and the new
LINKED_ISSUE_OVERRIDE — silently relabelling PR #26 and dropping the
issue provenance just restored. All four tables now resolve through
`_override_key`.

Datasets are unchanged; both fixes are guarded by new regression tests.

* fix: drop local agent config from patches; tighten expected-failure cue

Five findings from the Codex review of 5a98c3c.

Local agent/tracker state in curated patches (P1). PR #26's record
embedded full diffs for .beads/ (config, git hooks, tracker exports) and
.claude/settings.json — 12 of its 25 files. The .beads/issues.jsonl
export carried a personal email in an `owner` field, so a curated
artifact shipped contributor PII alongside machine-local config that
docs/data-policy.md excludes. Extended `_is_noise_patch_path`, which
already exists for exactly this, with a directory-prefix rule.

Expected-failure cue still lacked intent (P1). The
`failure behavior|mode|case|path|handling` alternative enclosed the only
fail token in "CI showed failure behavior on Linux", inverting it to
pass. Replaced with a deliberately-invalid-fixture cue, which is what
PR #24 actually reads on; a bare failure noun is no longer a cue.

Self-references from the raw record (P2). The guard only covered
synthesized stubs, so a pre-collected same-repo issue matching the PR
number survived into `seen` and out again. Now filtered on the way in.

Copilot reviewer counted as CI (P2). `copilot-pull-request-reviewer`
missed `_REVIEW_APP_CHECK_MARKERS`, so reviewer automation bolstered a
passing CI result. Added the marker and moved it to review_apps.

Stale indexes (P2). README called grok-ozempic "shortlist only" and
STATUS listed the extraction as an open gap with the obsolete six-PR
roadmap including dropped #8.

PR #26 regenerated: patch_chars 67711 -> 50149, before_context file
count 25 -> 13, manifest sha256/bytes updated. No other record changes.

* fix(normalize): filter noise dirs in diff prefilter; require asserted intent

Two P1s from the Codex review of 79191d7, both on code from that commit.

The diff prefilter was basename-only. `_filter_noise_from_diff` returns
early unless a `_NOISE_PATCH_BASENAMES` token appears in the text, so the
new `_NOISE_PATCH_DIRS` rule never ran on the inline/sidecar path — the
one the collector normally supplies. A diff touching only .beads/ or
.claude/ files would have kept its hunks intact. PR #26 was filtered only
because its .beads/.gitignore happens to contain the substring ".env".
The prefilter now scans for directories as well as basenames.

The fixture cue accepted a bad adjective as intent. "Invalid fixture was
repaired, but cargo test failed on Linux" matched, because the gap
between "fixture" and the fail word allowed any non-period characters and
swallowed the clause boundary. The fixture must now be the subject of a
confirming verb whose object is the failure (confirms/verifies/triggers/
demonstrates/shows/proves), with \w+ gaps that cannot cross a comma.

PR #24 still classifies as pass on "invalid schema fixture confirms
ingest failure behavior". Datasets are unchanged; both fixes affect
future normalization only, and both are pinned by regression tests.

* fix(normalize): nested agent state, exact copilot check, unmet expected failures

Four findings from the Codex review of b6dd6ae.

Nested agent state (P1). `_NOISE_PATCH_DIRS` was matched as a root
prefix, so a monorepo's pkg/.claude/settings.json or
workspace/.beads/config.yaml kept its hunk. Now matched as a path
component, with `beads/` and `.beadsfoo/` still excluded.

Copilot marker was a generic product term (P2). Substring "copilot"
would also classify real CI such as "Copilot integration tests" as a
review app, dropping it from ci_checks — and if it were the only
successful check the record would report no validation evidence at all.
Narrowed to the exact `copilot-pull-request-reviewer` check name.

Expected failure that never happened (P2). "The invalid fixture was
expected to fail, but it passed unexpectedly" describes a negative test
that accepted input it should have rejected, but classified as pass. The
cue proves intent, not that the failure occurred. A contradiction now
forces `fail`. This one predates the expected-failure work — verified
that disabling `_fail_words_all_expected` gives the same result — but it
is a real gap in the same area.

Also: `_fail_words_all_expected` returned True when a section had cue
spans and no fail words at all, vacuously. Now False.

Truncation inventory (P2). docs/source-repos.md claimed #11, #24 and #26
all truncate at the 96 KiB budget. Only #11 does and carries the footer;
#24 (77,970) and #26 (50,149) are complete. Corrected, and noted that
#26 is smaller than its raw diff because agent state is filtered.

* fix(normalize): recognize absence-of-failure wording as a failed negative test

Codex P2 on b4de520. The contradiction matcher added in that commit only
covered `passed`/`succeeded` and the exact phrase `did not fail`, so "The
expected failure did not occur" and "was not observed" still suppressed
to pass — reproduced both.

Generalized to absence wording: did not / does not / was not / were not /
never, followed within two words by fail / occur / observed / seen /
triggered / raised / reported / happen.

The contradiction only applies when an expected-failure cue is also
present, so ordinary resolved-failure prose ("previously failed tests are
now passing") is unaffected — covered by the test alongside the new
cases. PR #24 still classifies as pass. Datasets unchanged.

* fix(normalize): scope expected-failure contradiction to a single sentence

Codex P2 on a562309. The contradiction check tested the cue and the
absence wording independently against the whole section, so an unrelated
later clause matched: "Invalid fixture fails as expected. The warning did
not occur" forced a passing validation to `fail`. Reproduced, along with
the list-item form.

`_expected_failure_was_contradicted` now splits on sentence boundaries
and newlines and requires both to appear in the same part, so the
contradiction must actually negate the failure it sits with. Same-sentence
cases are unaffected.

This one produced a false `fail` rather than a false `pass`, so no shipped
record could have been inflated by it. Datasets unchanged; PR #24 still
classifies as pass.

* docs: align shortlist wording with closed_by_pr semantics; refresh STATUS date

Two of four findings from the CodeRabbit review of a562309.

The shortlist table said "Closes #22" for #26 and "Closes #16, #22, #27"
for #29, but those numbers come from LINKED_ISSUE_OVERRIDE, whose stubs
carry closed_by_pr: False precisely because the PR bodies use reference
wording ("Implements GitHub #22"), not close keywords. The doc therefore
contradicted the dataset it describes. Changed to "References", matching
the "Advances" wording already used for the other override-backed rows.

STATUS.md still read "Last updated: 2026-07-24" while recording the
2026-08-02 extraction.

Also added the missing docstring on `_override_key`.

The other two findings are not changes: the `.env.` prefilter token is
already covered (".env" is a substring of ".env.production.local", so the
prefilter fires and the file is filtered — verified), and review-signal
dedupe is issue #18, deliberately deferred. Both answered on their threads.

* fix(normalize): treat soft-wrapped lines as one statement

Codex P2 on 6a59e9d, a regression from the sentence-scoping fix in
76ba370. Splitting on every newline broke Markdown soft wraps, so "The
expected failure\nwas not observed" landed the cue and its contradiction
in separate parts and the contradiction never fired — a false `pass`.
Reproduced both wrapped forms.

_SENTENCE_SPLIT now breaks only on real statement boundaries: sentence
punctuation, a blank line, or the start of a list item. A bare newline is
no longer a boundary.

Both the wrapped forms and the genuine boundaries (sentence, list item,
paragraph) are pinned by tests. Datasets unchanged; PR #24 still passes.

* chore(ci): drop .gitlab-ci.yml

The project builds on GitHub Actions; the GitLab pipeline duplicated the
same lint/test/validate stages and had drifted — its validate stage ran
`validate_jsonl.py` without `--strict-policy`, so it was a weaker gate
than the GitHub one it mirrored.

* docs: align #29 domain with the card; correct #33 review-density wording

Two findings from the CodeRabbit review of 5f4d741.

The shortlist table gave #29 a domain of "validation, CI" while both
datasets/cards/grok-ozempic-v0.json (domain_by_pr) and the record itself
say "validation" — the only row of the seven that disagreed. Dropped the
", CI"; the CI aspect is already carried by the Title ("Docker CI") and
the Signal column ("Docker + cargo audit"), so nothing is lost.

#33 was described as "highest human review density in the repo", but the
measured table counts signals *after* the bot filter, and this section
states rmems is the repo's only human account — most of those 70 kept
signals are gemini/codex. Reworded to "highest filtered review-signal
density", which is what was actually measured.

* fix: strip linked Macroscope notes; allow modifiers in expected-failure cue

Two findings from the Codex review of 695e068.

Macroscope summaries were never stripped. `_MACROSCOPE_NOTE` required the
literal phrase "Macroscope summarized", but the attribution renders as
`<a href="...">Macroscope</a> summarized` — name and verb are not
adjacent, so the pattern never matched. PR #24's validation event
therefore carried the bot's entire change summary after its six real
commands, truncated mid-footer at the 1500-char limit. The regex now
tolerates markup between name and verb; record #24's test detail drops
from 1504 chars to 358 and ends cleanly at its Closes list.

Expected-failure cue required `expected` to sit immediately before the
fail word, so "the expected test failure was observed" was recorded as a
failed validation. Now allows up to two modifiers. Bounded deliberately:
"We expected a clean run but saw 3 test failures" and "Not expected: the
suite failed on main" both still classify as fail, and both are pinned.

Manifest sha256/bytes regenerated.

* fix(bots): scope Macroscope stripping to one note; revert cue widening

Both findings from the Codex review of 2075610 were regressions in that
same commit.

The Macroscope pattern was destroying content. Its leading `.*?` reached
forward for the attribution, so a PR body with a legitimate `[!NOTE]`
before a later bot summary lost the human note *and* every validation
line between them. Reproduced: human note and `cargo test` commands both
vanished. Now matches one `> [!NOTE]` blockquote run at a time and drops
it only when that block carries the attribution, mirroring how
_GEMINI_IMPORTANT_BLOCK already works.

The expected-failure cue widening is reverted. Allowing modifiers between
the intent marker and the fail word let "The CI was expected to be
failing on Linux until the runner is repaired" — a known-broken run —
suppress to `pass`. That is the fabricating direction, so the round-9
improvement is not worth its cost; its test is removed with it.

Known limitation, deliberately left: "intentionally failing on Windows"
still suppresses to `pass`. Restricting the cue to negative-test subjects
fixes it but then reads "expected to fail, but it passed unexpectedly" as
`pass` instead. Measured both — every configuration retains exactly one
false pass; only removing the heuristic has none.

* fix(normalize): filter .kilo/ agent state; recognize asserted negative tests

Two findings from the Codex review of 39a7a1a.

.kilo/ was missing from _NOISE_PATCH_DIRS (P1), so a collected PR
carrying local Kilo agent state would leak it into a curated patch — the
same class as the .beads/ and .claude/ leak fixed earlier, which shipped
a contributor email. This repo's own .codacy.yml already excludes
.kilo/**, so the classification was settled, just not applied here.
Near-misses `kilo/`, `.kilofoo/` and `src/kilo_backend.rs` stay unfiltered.

The `negative test` cue only ever spanned those two words, so
"Negative test confirms failure behavior" left the fail token outside any
cue span and recorded a successful negative test as `fail`. Extended
through an explicit observing/confirming verb — the same shape as the
fixture alternative — rather than by loosening the intent marker, which
is what produced last round's false pass. "Negative test suite reported 3
failures", "Negative tests failed on Windows" and "Negative test added.
CI checks failed on Linux" all still classify as fail, and all three are
pinned.

Full 13-case matrix green. 75 tests, ruff clean, --strict-policy clean.

* fix(bots): keep a separator when dropping a Macroscope note

CodeRabbit finding on 2b7b340, and a real defect in the note-scoping fix
from 39a7a1a. `_NOTE_BLOCK` consumes the newline preceding the block, so
replacing an attributed block with "" spliced the line before the note
onto the line after it. Reproduced: two distinct validation commands
merged into

    - cargo test --features cli- invalid fixture confirms failure behavior

Returns "\n" instead. Non-Macroscope blocks are still returned verbatim,
so unattributed notes are untouched either way.

The regression test asserts the surrounding lines survive as two separate
entries, not just that the bot text is gone — the previous test passed
while the splice was happening because it only checked for absence.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Local & Cloud Agents Aug 2, 2026
rmems added a commit that referenced this pull request Aug 18, 2026
Live gh scan dated 2026-08-16 across the extraction fleet:

- corinth-canal: add Candidate next wave (#125-#128, #138, #142 with
  measured raw review counts; note the GH#147 safetensors series and
  the #126->#138 recovery pair)
- grok-ozempic: add Candidate next wave (#69-#79 GOZ1 v2/v3 + expert
  remedies; flag #83 [112 reviews] and #86 beyond the wave)
- limen-neural: refresh Later waves pointers with live numbers, note
  the Limen-Neural -> rmems transfers, add the remaining issue-#28
  wave B repos (plasticity-lab, brainstem-daemon, synaptic-mesh, nir-rs)
- new worktrees-hives.md shortlist (9 PRs; #61 80 rev / #65 60 / #63 53,
  plus the #88-#105 lab-CLI wave) - shortlist drafted
- new theseus-quarry.md shortlist (7 of 9 merged PRs; #13 35 rev,
  #8 28, #9 17) - shortlist drafted
- new wave-c.md rmems org pointer table for issue #29 repos
- _index.md: Parked (Tier C) section; regenerate Index for the three
  new docs (build_status.py)

Full wave B/C shortlists land when #28/#29 are decomposed.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
cursor Bot pushed a commit that referenced this pull request Aug 20, 2026
Temporarily insert conflicting shared-dict rows for #8/#9/#12, assert the
card still wins, then restore the table so the test stays isolated.

Co-authored-by: Raul Montoya Cardenas  <montoyaraul34@gmail.com>
rmems added a commit that referenced this pull request Aug 20, 2026
* data: extract theseus-quarry-v0 telemetry trajectories

Shortlist five merged PRs (#13, #9, #12, #11, #8). Defer #16 and #18.
Per-PR domain and task_type labels live on the dataset card only.

Co-authored-by: Raul Montoya Cardenas  <montoyaraul34@gmail.com>

* test: prove Theseus-Quarry domain_by_pr beats a planted DOMAIN_OVERRIDE

Temporarily insert conflicting shared-dict rows for #8/#9/#12, assert the
card still wins, then restore the table so the test stays isolated.

Co-authored-by: Raul Montoya Cardenas  <montoyaraul34@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
rmems pushed a commit that referenced this pull request Aug 22, 2026
Fourth confirmed instance of the same pattern (neuromod #9,
kinetic-signals #1): both retained review findings still describe
defects present in the merged patch, with no corrective follow-up
captured -- new_cortical() mixes absolute and relative voltage
conventions the runtime kinetics don't handle, and reset() hard-codes
resting state by temperature instead of deriving it from the neuron's
own parameters. Recategorized repair -> bug-prediction via the card's
training_use_buckets and synced both doc mirrors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request size:L This PR changes 100-499 lines, ignoring generated files

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

Create data policy and dataset hygiene rules Define trajectory JSON schema v0 Initialize project scaffold for Operation Prometheus

1 participant