Skip to content

Async ingest: document the eventual-consistency contract + optional sync ack / ingest-lag signal #230

Description

@EricAndrechek

Area: ingest — gap + docs · found via WaveHouse-Stats dogfooding

Expected: a way to know when an accepted event is queryable, or to opt into waiting for it.

Actual: POST /v1/ingest returns 200 after the NATS publish, not after the ClickHouse write (internal/api/ingest.go:153-187). A just-accepted event isn't immediately visible. There's no sync-ack mode and no exposed ingest-lag / queue-depth signal, so consumers can't distinguish "not yet flushed" from "genuinely missing."

Impact (Stats): this is the reason the reconciler needs a settle window before declaring drift — and we're sizing that window blind.

Scope: at minimum, document the eventual-consistency contract before launch (cheap, important for anyone building on ingest). Larger follow-on: an optional synchronous ack mode and/or an exposed ingest-lag metric.


From WAVEHOUSE-FEEDBACK.md (WaveHouse-Stats dogfooding); validated by code-read against 0f8826c on 2026-06-04.

Activity

  1. added
    documentationImprovements or additions to documentation
    enhancementNew feature or request
    area/ingestIngest pipeline (Bento, batching, DLQ)
    area/docsDocumentation, site/, README
    on Jun 4, 2026
  2. coderabbitai commented on Jun 4, 2026

    @coderabbitai
    🔗 Related PRs

    #182 - refactor: full api --> ingest --> clickhouse --> dlq refactor [merged]


    📝 Issue Planner

    Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.

    • Create Plan

    🧪 Issue enrichment is currently in open beta.

    You can configure auto-planning by selecting labels in the issue_enrichment configuration.

    To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

    issue_enrichment:
      auto_enrich:
        enabled: false

    💬 Have feedback or questions? Drop into our discord!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/docsDocumentation, site/, READMEarea/ingestIngest pipeline (Bento, batching, DLQ)area/observabilityMetrics, logs, traces, health, profilingdocumentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions