Skip to content

Add GET /telemetry/events to read archived telemetry from S2#285

Draft
archandatta wants to merge 10 commits into
mainfrom
archand/kernel-1338/add-s2-events-get
Draft

Add GET /telemetry/events to read archived telemetry from S2#285
archandatta wants to merge 10 commits into
mainfrom
archand/kernel-1338/add-s2-events-get

Conversation

@archandatta

@archandatta archandatta commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a paginated read path for browser telemetry archived in S2, so events stay available after they age out of the live SSE ring buffer (GET /telemetry/stream).

  • GET /telemetry/events — reads archived telemetry envelopes for the browser in ascending sequence order. Returns a top-level JSON array of envelopes plus X-Has-More / X-Next-Offset pagination headers (matching the offset_pagination convention used by the list endpoints). Empty result serializes as [], never null.
  • Cursor pagination over S2offset is an S2 sequence cursor (pass the previous response's X-Next-Offset); since/until bound the time window; category filters; limit is the page size (default 100, max 1000). has_more is derived from the S2 read's next-position vs the stream tail, so it is correct whether a page stops on the count, byte, or time bound.
  • events.S2Reader — one-shot, bounded S2 reader injected into ApiService; always terminates (caps at the tail when unbounded).
  • Configurable batcherS2_BATCHER_LINGER / S2_BATCHER_MAX_RECORDS env vars wire into the existing S2Config (defaults unchanged: 100ms / 50).
  • TelemetryEventCategory promoted to a named schema component so the category query param can reference it (behavior-preserving; generated type byte-identical).
  • Fix: telemetry read endpoints no longer self-emit api_call events. The per-request api_call middleware was firing for ReadTelemetryEvents / StreamTelemetryEvents, appending to the very stream being read — a feedback loop that, at page size 1, prevented a paginated read from ever catching the tail. Reads are now side-effect-free.

Envelope/event schemas are unchanged. The response is encoded directly from events.Envelope (matching the SSE stream and publish endpoints), so all telemetry endpoints emit an identical envelope shape.

Consumers paginate via X-Next-Offset (an S2 stream position) — not the envelope's seq field; the param/header docs call this out.

This endpoint is harmless until a consumer calls it, so it can ship independently. A follow-up on the control-plane API will mirror it and wire Stainless offset_pagination so the SDK auto-paginates.

Test plan

  • Unit: handler (S2-disabled empty path), buildReadOptions (offset-over-since precedence, Count mapping), pageSize clamping, filterByCategory.
  • e2e (server/e2e, real S2 via KI_E2E_BACKEND=docker): round-trip, category filter, empty-window [], multi-page cursor walk (ascending, no gaps/dupes), and a read-is-side-effect-free assertion. Auto-skips without S2_BASIN/S2_ACCESS_TOKEN/S2_STREAM.
  • go build, go vet, package tests green; make oapi-generate output committed.

Live runs

Ran against a real S2 stream (freshly built headless image, KI_E2E_BACKEND=docker):

--- PASS: TestReadTelemetryEvents
--- PASS: TestReadTelemetryEventsPagination
ok  github.com/kernel/kernel-images/server/e2e

Also exercised by hand over HTTP at scale — published ~1000 events and paged the full stream at limit=50:

pages walked: 19
total_records=950  min_seq=1  max_seq=950  duplicates=0  gaps=0
RESULT: PASS — full stream 1..950 returned exactly once, contiguous, in order

And confirmed limit=1 terminates cleanly (the feedback-loop fix), which it would not have before.

🤖 Generated with Claude Code

archandatta and others added 6 commits June 11, 2026 12:52
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… limit

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@archandatta archandatta force-pushed the archand/kernel-1338/add-s2-events-get branch from bb9b965 to 0f44c61 Compare June 11, 2026 12:55
archandatta and others added 4 commits June 11, 2026 14:13
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reading telemetry emitted its own api_call event back into the stream,
a feedback loop that prevents a paginated read from catching the tail.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant