Skip to content

Mixed-version writes erase Round.CoBots, dropping co-reviewer trigger claims #37

Description

@kristofferR

Split out of the review of #35, where it was raised three times. The mechanism is real; it is documented there as a rollout requirement rather than solved in code, and this issue is the place to solve it properly.

The mechanism

Round.CoBots is additive on schema v3, so an old binary reads the state fine. But Go's encoding/json drops unknown fields, so an old binary that writes state marshals the round without cobots — erasing every Bugbot/Macroscope command id and CAS claim the new binary recorded.

Blast radius

Narrower than it looks, and worth stating precisely so this is not over-fixed:

  • CodeRabbit's no-double-fire invariant is untouched. It rests on FireSlot plus CommandID, both legacy fields old binaries preserve. Codex is dual-written into CodexCommandID/CodexCommandedAt/CodexClaimedAt for exactly this reason.
  • What an erasure can cost is a duplicate co-reviewer trigger comment, which spends no CodeRabbit quota.
  • observe narrows it further: the next pass sees the already-posted bugbot run on the PR and suppresses a repost. The exposed window is between claim and post — seconds — not the claim's lifetime.

Why not just bump the schema

Considered and rejected: an old binary meeting a version it does not know auto-reinitialises the shared state, losing every round in the fleet rather than one map — and it would do so during exactly the rolling deployment this is about. That trades a bounded loss for an unbounded one.

Current mitigation

Documented in #35 under "Rollout requirement": upgrade the whole fleet before enabling Bugbot or Macroscope.

Possible approaches

  • Dual-write the co-bot map into a field old binaries already preserve (as Codex is), at the cost of a fixed set of bots.
  • A version-tolerant round representation that round-trips unknown fields (decode into json.RawMessage and re-emit), which fixes this class of problem generally rather than for CoBots alone.
  • A staged rollout marker that makes new-binary features conditional on the whole fleet having upgraded.

Activity

  1. kristofferR commented on Jul 27, 2026

    @kristofferR
    OwnerAuthor

    Closed in effect by the version tolerance (#43) plus schema v4 (#55), now released as crq 2.0.0.

    Round and State round-trip JSON members they do not recognise (internal/state/tolerant.go), so a binary that writes state no longer erases fields it has never heard of — CoBots included. That removed the mechanism this issue describes for every additive field from v3 onward.

    What it could not do is protect against a binary older than the tolerance itself, and that happened here on 2026-07-27: a crq from before tolerant.go shared the ref and wiped dispatch, drain, writers and cobots from every round. crq doctor now reports other crq installs on the host, compared by content, because the version string cannot tell two builds apart.

    Schema v4 closes the remaining gap the hard way: an older binary now refuses the payload instead of writing a lossy one (#49). The cost is that every host must be upgraded together, which is stated in the README's environment table.

    Leaving this open only if you want the Codex dual-write removed — foldLegacyCodex is still legacy-authoritative in both directions, which is the last thing here that exists for mixed versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions