Skip to content

Commit 225eab3

Browse files
committed
docs: add usage guidance for downstream_sink_latency metric
1 parent c2b813c commit 225eab3

File tree

1 file changed

+32
-0
lines changed

1 file changed

+32
-0
lines changed
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
title: Monitor downstream_sink_latency in Changefeeds
3+
summary: How to interpret and monitor the downstream sink latency metric to detect bottlenecks in CockroachDB changefeeds.
4+
---
5+
6+
#### Downstream sink acknowledgment latency
7+
8+
**Metric**: `changefeed.stage.downstream_sink_latency`
9+
10+
Measures the time it takes for a downstream sink (e.g., Kafka, cloud storage, webhook) to acknowledge messages. High values may indicate bottlenecks or backpressure at the end of the pipeline.
11+
12+
**When it's high**:
13+
- Sink is slow to ACK; may cause backpressure or stall the changefeed.
14+
15+
**Possible causes**:
16+
- Network latency between CockroachDB and the sink
17+
- Throttling or limits on the sink
18+
- Large payloads or inefficient encoding
19+
- Sink I/O capacity constraints (e.g., disk, Kafka throughput)
20+
21+
**Where to monitor**:
22+
- CockroachDB Admin UI → *Metrics → Changefeeds*
23+
- Prometheus/Grafana if metrics are scraped
24+
25+
**Suggested alert**:
26+
- For example: `changefeed.stage.downstream_sink_latency: p95 > 1s over 1m`, sustained
27+
28+
**Mitigation strategies**:
29+
- Scale the sink or increase throughput
30+
- Enable batching or compression
31+
- Tune changefeed sink settings (partitioning, parallelism)
32+
- Diagnose with logs and sink system metrics

0 commit comments

Comments
 (0)