Skip to content

Validate safe current-action continuation - #45

Merged
dlxeva merged 7 commits into
masterfrom
agent/continuation-eval-v2
Aug 12, 2026
Merged

Validate safe current-action continuation#45
dlxeva merged 7 commits into
masterfrom
agent/continuation-eval-v2

Conversation

@dlxeva

@dlxeva dlxeva commented Aug 12, 2026

Copy link
Copy Markdown
Owner

What changed

  • add a machine-readable real-case registry with explicit evidence boundaries
  • evaluate no state, raw history, Resume, and Manifest across three current real projects
  • compile one canonical Current Action across Resume, Manifest, and Handoff
  • refuse stale or contradictory actions and expose source, timestamp, and reconciliation status
  • validate the repair with deterministic replay and a sealed dual-blind Luna Max rerun
  • add bilingual AML leaderboard recognition and explain the boundary between the competition retriever and FlowGrid Core

Why

The continuation study showed that having project state is not enough when different views expose different next actions. A stale FLG view could be worse than no state because it encouraged a fresh Agent to continue obsolete work. This iteration makes current-action selection deterministic and tests the repair against the same held projects.

Result

  • 211 tests passed
  • repository-source Smoke passed
  • repaired Resume ranked first in all three held projects
  • repaired Resume and Manifest received 0/12 critical-failure scorer-view votes
  • current-state freshness and JSON evidence checks passed

Evidence boundary

The three-project result validates this targeted current-action repair on owner-operated projects using one model family. The AML #8 result belongs to the independent FlowGrid AML Retriever under the leaderboard Add/Search contract. Neither result establishes FlowGrid Core adoption, retention, product-market fit, or universal superiority.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aee2c798a8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

snapshot_content,
state,
pending_patches_count=len(pending_patches),
framing_goal_defined=bool(framing_goal),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Treat the initial framing placeholder as undefined

Immediately after flg init, FRAMING.md#Goals contains (to be defined), but _first_section_line does not recognize that value as a placeholder, so bool(framing_goal) is true. Resume and Manifest consequently classify the valid initialization action (Run 'flg frame'...) as needs_reconciliation, while Handoff's separate placeholder-aware parser classifies it as current; this blocks the documented first-run continuation flow and violates the new canonical-action contract.

Useful? React with 👍 / 👎.

@@ -0,0 +1,23 @@
"""Unit checks for the current-state freshness classifier."""

from scripts.check_current_state import product_changes_after_state_update

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Make top-level scripts importable during pytest collection

The CI workflow invokes the installed pytest console script, whose import path does not include the repository root; tests/conftest.py adds only src/, and the editable package also installs only packages under src. Consequently this import (and the analogous one in test_smoke_command_resolution.py) raises ModuleNotFoundError: No module named 'scripts' during collection, so every CI matrix job fails before running tests. Add the repository root to the test path or expose these helpers through an installed package.

Useful? React with 👍 / 👎.

current_action = resolve_current_action(
snapshot_content,
state,
pending_patches_count=len(pending_patches),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the same pending-patch set for Handoff actions

After flg review accepts all decisions but before the patch is merged, Context's _pending_patch_summaries excludes that patch using decision_review_status == "accepted", whereas Handoff still counts it because its header remains pending_review. Passing this different count makes a Snapshot action such as Review pending patches resolve to needs_reconciliation in Resume/Manifest but current in Handoff, reintroducing the exact cross-view action disagreement this change is intended to eliminate.

Useful? React with 👍 / 👎.

print("current-state: refreshed in working tree")
return 0

last_update = _git("log", "-1", "--format=%H", "--", str(STATE_DOC))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fetch history before checking state-document freshness

In the inspected .github/workflows/ci.yml, actions/checkout@v4 is used without a fetch-depth override, so the job has a one-commit shallow history. In that checkout, git log -- docs/product/current-state.md treats the shallow boundary commit as the file's latest introduction even when the file was not changed there, making last_update equal HEAD and the following diff empty; therefore CI reports the document current for exactly the later product-only commits this check is meant to reject. Fetch sufficient history before relying on this path log.

Useful? React with 👍 / 👎.

@dlxeva
dlxeva merged commit 1b4fb2b into master Aug 12, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant