Skip to content

KV API and misc #100

@thedodd

Description

@thedodd
  • KV API for targeted updates of event data fields. Events identified by their partition & offset (which is absolute per partition).
  • Hadron Client & CLI will expose interfaces for this.
    • Should support querying for an event by offset & partition.
    • Should support updating an event by offset and partition.
  • Events should be updated when written to a stream to include the Hadron partition & offset as extension fields.

  • (Move to new ticket) Delivery count tracking. Will just have Stream subscription controller & pipeline controller emit metrics for these when it happens. Different metric types, one for Streams the other for Pipelines.

  • (Move to new ticket) We will be blocked on this by the upstream metrics project, as they do not yet have an API for removing registered metrics. Forking and running this should be quite easy.

  • (Move to new ticket) Delivery count tracking should be able to cause alerts to be generated. Add a new panel with a table of Stream events which have high delivery counts, and one for Pipelines as well.

  • (Move to different new ticket) Probably support nack with delay. This would only be useful for Streams when we support out of order processing. And only useful for Pipelines when we do not limit max concurrency for stale jobs due to not having active stage consumers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions