Skip to content

feat(log): add component supervision logging#16499

Draft
bartsmykla wants to merge 2 commits into
masterfrom
feat/log-component-supervision
Draft

feat(log): add component supervision logging#16499
bartsmykla wants to merge 2 commits into
masterfrom
feat/log-component-supervision

Conversation

@bartsmykla
Copy link
Copy Markdown
Contributor

@bartsmykla bartsmykla commented May 6, 2026

Motivation

When a component exits with an error or panics, the logs say nothing useful: no component name, no stack trace, no indication of whether the process is still running. On-call has to read the code to figure out which goroutine died.

Implementation information

Added runComponent, a supervision wrapper called from all three goroutine launch sites (non-leader start, runtime add, leader start). It:

  • Logs component starting and component stopped as bookend info lines
  • Recovers panics, logs the component name and stack trace, forwards the error to errCh
  • Rate-limits panic logs per component name (10s window) to avoid log storms; the allowed log line includes a suppressed count so the operator can tell one crash from a cascade
  • Uses non-blocking sends on errCh to avoid goroutine leaks after manager.Start returns; errCh is now buffered by 1 so the first error is never dropped even if a component panics before Start reaches its select

Added NamedComponent interface so components can provide a stable log name; falls back to %T otherwise.

The panic log level is Error (not double-logged at the manager level since the error is returned from Start). Normal exit errors are logged at Info for the same reason - the error is also returned from Start.

Signed-off-by: Bart Smykla <bartek@smykla.com>
@bartsmykla bartsmykla added ci/skip-e2e-test PR: Don't run e2e tests ci/run-full-matrix PR: Runs all possible e2e test combination (expensive use carefully) and removed ci/skip-e2e-test PR: Don't run e2e tests labels May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/run-full-matrix PR: Runs all possible e2e test combination (expensive use carefully)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant