Skip to content

fix(#435): ProjectionProgressRow.AgentStatus is nullable — no store persists it - #519

Merged
jeremydmiller merged 1 commit into
mainfrom
fix-435-agent-status-nullable
Jul 17, 2026
Merged

jeremydmiller merged 1 commit into
mainfrom
fix-435-agent-status-nullable

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Answers the open question raised on polecat#324: "I could not verify that AgentStatus and LastHeartbeat are actually available from pc_event_progression... please say so on #435 rather than stubbing quietly."

Saying so. 🙂

Finding: AgentStatus has no data source in either store

ProjectionProgressRow.AgentStatus is typed non-nullable string, but no event store can populate it today. Both stores model the column and read it; neither writes it.

Marten ships mt_mark_event_progression_extended(), and its SQL genuinely does write the columns:

INSERT INTO {databaseSchema}.mt_event_progression (name, last_seq_id, last_updated, heartbeat, agent_status, pause_reason, running_on_node)
VALUES (name, last_encountered, transaction_timestamp(), p_heartbeat, p_agent_status, p_pause_reason, p_running_on_node)
...

But the only reference to that function in the codebase is EventGraph.FeatureSchema.cs:101, which yields a SystemFunction so the DDL gets created. Nothing ever invokes it. Meanwhile ProjectionProgressStatement selects agent_status and ShardStateSelector reads it into ShardState.AgentStatus.

Polecat is the same shape: MarkExtendedProjectionProgress writes the columns and is constructed only from tests (polecat#323); PolecatDatabase.AllProjectionProgress reads them back.

Net: agent_status is always NULL in both stores. An implementer of ReadProjectionProgressAsync would have to invent a value — "Unknown", "" — purely to satisfy the type. That's the quiet stub polecat#324 asked us to avoid.

(LastHeartbeat is fine. Polecat does write heartbeat in production. It's already nullable, and both stores only model the column under extended progression tracking, so null is the honest answer when it's off.)

Change

string AgentStatus → string? AgentStatus, plus docs explaining when it's null and a test pinning the semantics. Symmetric with LastHeartbeat, already documented as "null when the store does not track a heartbeat for it".

Why nullable rather than trimming the field

polecat#324 floated trimming. Nullable seems better:

  • Reversible. If agent state is ever wired up (either store, or a daemon-level feature), a nullable field starts carrying real data with no API change. Re-adding a trimmed field would be a breaking change.
  • Symmetric. LastHeartbeat already established "nullable = the store doesn't track this" for this exact record.
  • Honest today. null reads as "no agent state persisted for this cell," which is precisely true.

Happy to trim instead if you'd rather the record carry only fields with a live data source — it's a small change either way, and the reason to decide now is that this hasn't shipped yet: #518/#517 are merged but the latest released JasperFx.Events is 2.27.0, which has no ProjectionProgressRow. Cheap now, breaking later.

Worth a separate look

That neither store writes agent state at all may itself be the bug — the columns, Marten's _extended function, and Polecat's MarkExtendedProjectionProgress all exist and are all unreachable, which reads like a feature that was built and never connected. This PR only makes the contract honest about today's reality; wiring agent state up (if wanted) is a daemon-level design question for both stores. Tracked on the Polecat side at polecat#323.

Tests

Full EventTests green on net9 + net10: 512 passed, 0 failed each.

🤖 Generated with Claude Code

…ersists it

AgentStatus was typed non-nullable string, but no event store can populate it
today. Both Marten and Polecat create an agent_status column on the progression
table and read it back, yet neither has a code path that writes one:

- Marten ships mt_mark_event_progression_extended(), whose SQL does write
  agent_status/pause_reason/running_on_node — but the only reference to it is
  EventGraph.FeatureSchema.cs, which registers it as a SystemFunction so the DDL
  is created. Nothing invokes it.
- Polecat has MarkExtendedProjectionProgress, which likewise writes the columns
  and is constructed only from tests (polecat#323).

So agent_status reads NULL in both stores, and an implementer of
ReadProjectionProgressAsync would have to invent a value to satisfy the type.
Make it string? so a store with nothing to report can say so — symmetric with
LastHeartbeat, already nullable and documented as "null when the store does not
track a heartbeat for it".

Caught before release: #518/#517 are merged but ProjectionProgressRow has not
shipped (latest JasperFx.Events is 2.27.0), so this costs nothing now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jeremydmiller
jeremydmiller merged commit 2707a0e into main Jul 17, 2026
1 of 2 checks passed
jeremydmiller added a commit that referenced this pull request Jul 20, 2026
…aemon (#538)

* feat(#537): wire the extended progression write path into the async daemon

Extended progression tracking (ExtendedProgressionEnabled) created the schema
columns (heartbeat, agent_status, pause_reason, running_on_node) and a faithful
read surface, but no daemon path ever persisted them — the write half was built
and never connected (#519). This adds the daemon-level wiring so both Marten and
Polecat can implement the store-side write:

- IEventDatabase.WriteExtendedProgressionAsync(ShardState, CancellationToken):
  the store-side persistence hook, default-implemented as a graceful no-op so
  existing stores compile and degrade silently until they implement it. The
  contract forbids touching progression (last_seq_id) from this path so a
  telemetry write can never race a batch commit into losing progress.
- IEventStore.ExtendedProgressionEnabled (default false): store-level gate,
  read live per publication so runtime opt-in (e.g. CritterWatch's forced
  enable) is honored.
- ExtendedProgressionWriter: a ShardStateTracker observer the daemon subscribes
  at construction. Status transitions (Started/Paused/Stopped + pause_reason)
  write immediately; heartbeat ticks and batch-commit publications are
  throttled to one write per 5s per shard. Writes are serialized on a
  background block and are strictly best-effort: a failing store write is
  logged at debug and can never fail or stall the shard, and a slow database
  can never back up the tracker's publication loop. Carries the assigned node
  number through to running_on_node when a distribution layer stamped it.

The heartbeat write path is deliberately NOT coupled to progress advance — a
paused or stalled shard is exactly when the persisted heartbeat/status matters
most (the CritterWatch node-down / HWM-frozen alerting case, CritterWatch#750).

Closes #537

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: re-trigger stuck queued build

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant