Skip to content

Commit b3ab2d8

Browse files
authored
feat(exemplar): disk budget arithmetic inside 8 GiB
Implements the frozen #201 contract. Q1 - 8 GiB budget table documented in CLAUDE.md (4.5 GiB main relational tier, 1.5 GiB aggregate.db, 0.5 GiB DLQ, 0.5 GiB WAL/temp, 1 GiB mandatory headroom) with per-component measured high-water gauges. Q2 - EXEMPLAR_BYTES_GLOBAL_WINDOW default 8 MiB -> 3 MiB. New EXEMPLAR_RETENTION_DAYS=2 drives a separate transactional purge of exemplar traces, spans, logs, their FTS rows and expired weak references, running ahead of the HOT_RETENTION_DAYS purge on the same hourly tick. Aggregate retention stays 7 days. Q3 - synthesized logs are metered: every one reserves len(body)+len(attributesJSON)+logRowFixedBytes against the selected trace's per-trace budget and the shared per-service/global window budgets, under EXEMPLAR_SYNTH_LOGS_PER_SPAN=8 and EXEMPLAR_SYNTH_LOGS_PER_TRACE=64. Refusals drop the log, count synth_per_span|synth_per_trace|budget_bytes, and stamp the trace truncated. Q4 - reservation lifecycle replaces immediate charging: reserve before row construction, commit when the primary queue or DLQ accepts the batch, release only when the row never reached a destination. Reserved bytes bind the cap. Bytes accepted downstream are never refunded on selection eviction; count slots still are. Q5 - disk watchdog with staged shedding and hysteresis. statfs on DATA_DISK_PATH is the enforcement source, ceiling = min(DATA_DISK_BUDGET_MB, usable volume capacity). >=90% admits only error exemplars; >=95% disables all raw admission and the exemplar DLQ fallback, purges the expired exemplar tier, checkpoints the WAL and fails readiness. Recovery needs <90% and <85% respectively. Raw shedding never fails a successful Export; an authoritative aggregate commit hitting ENOSPC/SQLITE_FULL does, with RESOURCE_EXHAUSTED.
1 parent 8f0f524 commit b3ab2d8

26 files changed

Lines changed: 3049 additions & 75 deletions

CLAUDE.md

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -369,6 +369,10 @@ Key settings in `internal/config/config.go`:
369369
- `OTEL_EXPORTER_OTLP_ENDPOINT` — enables self-instrumentation (empty = off)
370370
- `DEFAULT_TENANT` (`default`) — assigned to rows ingested without explicit tenant
371371
- `HOT_RETENTION_DAYS` (7) — drives `RetentionScheduler`; range 1..36500
372+
- `EXEMPLAR_RETENTION_DAYS` (2) — separate, shorter retention for the raw exemplar tier in aggregate mode; validated 1..`HOT_RETENTION_DAYS`. See the Data Disk Budget section
373+
- `EXEMPLAR_BYTES_GLOBAL_WINDOW` (**3 MiB**, was 8 MiB) / `EXEMPLAR_BYTES_PER_SERVICE_WINDOW` (512 KiB) — instance-wide and per-service byte budget per 5-minute window
374+
- `EXEMPLAR_SYNTH_LOGS_PER_SPAN` (8), `EXEMPLAR_SYNTH_LOGS_PER_TRACE` (64) — count caps on logs synthesized from span events and span status
375+
- `DATA_DISK_BUDGET_MB` (8192), `DATA_DISK_PATH` (`./data`) — disk watchdog ceiling and the volume it `statfs`-es
372376
- `SAMPLING_RATE` (1.0), `SAMPLING_ALWAYS_ON_ERRORS` (true), `SAMPLING_LATENCY_THRESHOLD_MS` (500)
373377
- `METRIC_MAX_CARDINALITY` (10000), `METRIC_MAX_CARDINALITY_PER_TENANT` (0 = unlimited), `API_RATE_LIMIT_RPS` (100). The per-tenant cap is checked first; when set, a noisy tenant cannot exhaust the global pool. Overflow is labeled by tenant via `otelcontext_tsdb_cardinality_overflow_by_tenant_total{tenant_id}` (`__global__` sentinel when the global cap was the trigger).
374378
- `MCP_ENABLED` (true), `MCP_PATH` (/mcp)
@@ -449,6 +453,97 @@ Failure-mode gauges (prefix `OtelContext_`):
449453
- `retention_last_success_timestamp` — Unix seconds; alert when stale relative to the hourly tick
450454
- `retention_rows_purged_total`, `retention_purge_duration_seconds`, `retention_vacuum_duration_seconds` — throughput and latency
451455

456+
### Data Disk Budget — 8 GiB (#201)
457+
458+
The platform targets a single 8 GiB data volume. The budget is a promise about
459+
that volume, not about a table, so **enforcement reads `statfs` on
460+
`DATA_DISK_PATH`** — the only figure that also counts WAL frames, SQLite temp
461+
files, free pages the file has not handed back, and anything else sharing the
462+
volume. Per-component file sizes are attribution, never enforcement: a budget
463+
enforced against summed file sizes reports 60% while `write()` returns ENOSPC.
464+
465+
| Tier | Allocation | Covers |
466+
|---|---|---|
467+
| Main relational tier | **4.5 GiB** | Raw trace/span/log exemplars, synthesized logs, investigations and other main-DB metadata, indexes, FTS5, database free pages |
468+
| `aggregate.db` | **1.5 GiB** | Aggregate buckets, delta log, baselines, identity tables, and their indexes |
469+
| DLQ | **0.5 GiB** | Existing `DLQ_MAX_DISK_MB` cap |
470+
| WAL/SHM + temp | **0.5 GiB** | `-wal`/`-shm` sidecars of both databases, SQLite temp files, TLS material, transient maintenance overhead |
471+
| Headroom | **1 GiB** | Mandatory and unused |
472+
473+
**Unused allocation in one tier does not authorize another tier to consume the
474+
final 1 GiB.** The seven-day gate (#202) validates these numbers against
475+
measured high-water marks; it does not quietly reallocate them after a failure.
476+
477+
Gauges: `otelcontext_disk_budget_bytes` (the effective ceiling — min of
478+
`DATA_DISK_BUDGET_MB` and the usable volume capacity),
479+
`otelcontext_disk_used_bytes`, `otelcontext_disk_used_ratio`,
480+
`otelcontext_disk_component_bytes{component}` and
481+
`otelcontext_disk_component_high_water_bytes{component}` for
482+
`main_db|aggregate_db|dlq|wal`, `otelcontext_disk_shedding_state`,
483+
`otelcontext_disk_shedding_transitions_total{from,to}`.
484+
485+
#### Exemplar-tier retention
486+
487+
`EXEMPLAR_RETENTION_DAYS=2` runs a **separate transactional purge** of exemplar
488+
traces, spans, logs, their FTS rows and expired weak references (spans whose
489+
trace row is gone), ahead of the `HOT_RETENTION_DAYS` purge on the same hourly
490+
tick. Trace and span deletes share one transaction per batch, so a reader never
491+
sees spans whose trace row has already gone. `logs_fts` is content-linked and
492+
trigger-synced, so index entries die with their rows. Aggregate retention stays
493+
seven days. Wired only in `AGGREGATE_MODE=aggregate`: in legacy and shadow the
494+
raw rows ARE the dataset and a two-day purge would be data loss.
495+
496+
Arithmetic behind the 3 MiB default: two days = 576 five-minute windows;
497+
576 × 3 MiB = 1.69 GiB of charged payload; at the provisional 2× DB/index/FTS
498+
amplification ≈ 3.38 GiB, leaving ≈ 1.12 GiB of margin inside the 4.5 GiB main
499+
tier. 4 MiB/window consumes the whole tier under the same optimistic assumption
500+
— it stays configurable, it is not the default until #202 proves it fits.
501+
Throughput: `otelcontext_exemplar_rows_purged_total{table}`,
502+
`otelcontext_exemplar_purge_duration_seconds`.
503+
504+
#### Synthesized-log metering
505+
506+
Every log synthesized from a span event or span status reserves
507+
`len(body) + len(attributesJSON) + logRowFixedBytes` against its trace's
508+
per-trace budget AND the shared per-service/global window budgets, under
509+
`EXEMPLAR_SYNTH_LOGS_PER_SPAN` and `EXEMPLAR_SYNTH_LOGS_PER_TRACE`. They do
510+
**not** consume the ordinary log-exemplar quota — that budget is for logs a
511+
client actually sent — but they are not weightless either: a span carrying two
512+
hundred exception events used to write two hundred rows no budget had ever
513+
seen. Refusals drop the log, count
514+
`otelcontext_exemplar_dropped_total{signal="logs",reason}` with
515+
`synth_per_span|synth_per_trace|budget_bytes`, and stamp the trace `truncated`.
516+
517+
#### Reservation lifecycle (no unconditional refunds)
518+
519+
Bytes are **reserved** before a row is constructed, **committed** when the
520+
primary queue or the DLQ accepts the batch, and **released** only when the row
521+
is dropped before submission or permanently lost because both destinations
522+
refused it. Reserved bytes bind the cap exactly like committed ones. Once a
523+
submission is accepted the charge is **monotonic for that window**: displacing
524+
the trace later releases the count slot (a slot is a seat, not a byte) and
525+
never the bytes — refunding bytes already on disk is how a window writes past
526+
its cap. `Batch.Reservation` carries the charge to the submit boundary.
527+
528+
#### Staged shedding, hysteresis, and the one ENOSPC exception
529+
530+
| Volume usage | State | Behaviour |
531+
|---|---|---|
532+
| ≥ 90% | `errors_only` | Only error trace/log exemplars are admitted; healthy/slow/WARN raw retention off |
533+
| ≥ 95% | `raw_off` | ALL new raw exemplar admission off, exemplar DLQ fallback closed, immediate expired-exemplar purge + `wal_checkpoint(TRUNCATE)`, `/ready` → 503 |
534+
535+
Hysteresis: recover from `raw_off` only below 90%, from `errors_only` only
536+
below 85%. A failed `statfs` HOLDS the current state — shedding because a
537+
syscall failed would be an outage caused by the safety mechanism.
538+
539+
Raw shedding **never** turns a successful aggregate Export into a retryable
540+
failure. One exception: if the **authoritative** aggregate commit fails with
541+
`ENOSPC` or `SQLITE_FULL` (`aggregate.IsDiskFull`), the Export MUST fail with
542+
`RESOURCE_EXHAUSTED`/429 — under the durable-ACK contract a success response
543+
asserts the deltas are committed, and acknowledging data that was not stored is
544+
data loss with better branding. Shadow mode is unaffected: there the legacy raw
545+
path is still the source of truth.
546+
452547
## Security & Supply Chain
453548

454549
OtelContext targets the OpenSSF Best Practices `passing` badge (project [12646](https://www.bestpractices.dev/en/projects/12646)) and ships a six-job OSS-CLI security stack, supplemented by **SonarCloud SAST as a required gate** (board reversal 2026-04-28). No CodeQL, no NVD-direct tooling. Cost: $0 for the OSS-CLI tier; SonarCloud is free for public repos.

internal/aggregate/diskfull.go

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
package aggregate
2+
3+
import (
4+
"errors"
5+
"io/fs"
6+
"strings"
7+
"syscall"
8+
)
9+
10+
// Disk-exhaustion classification for the authoritative commit path (#201 Q5).
11+
//
12+
// Raw exemplar shedding is explicitly forbidden from turning a successful
13+
// aggregate Export into a retryable failure — the aggregate numbers are the
14+
// dataset, the exemplars are diagnostics, and refusing telemetry because a
15+
// diagnostic could not be stored is the wrong trade in every direction.
16+
//
17+
// There is exactly one exception, and it is not really about shedding: if the
18+
// AUTHORITATIVE aggregate commit itself fails because the device is out of
19+
// space, the Export must fail. Under the durable-ACK contract (#160) a success
20+
// response asserts the deltas are in a committed transaction. Answering OK for
21+
// data that hit ENOSPC is data loss with better branding, and the client's
22+
// retry is the only thing that can still save it.
23+
//
24+
// Detection is by error inspection rather than by asking the filesystem,
25+
// because the only reading that matters is the one the write actually got.
26+
27+
// sqliteFullMessages are the message fragments SQLITE_FULL (13) and its ENOSPC
28+
// cousins surface through the pure-Go driver, which does not export a typed
29+
// error the way mattn/go-sqlite3 does. Matched case-insensitively.
30+
var sqliteFullMessages = []string{
31+
"database or disk is full", // SQLITE_FULL
32+
"no space left on device", // ENOSPC surfaced as text
33+
"disk is full",
34+
}
35+
36+
// IsDiskFull reports whether err is a device-out-of-space failure.
37+
//
38+
// It checks the typed errno first (errors.Is unwraps *fs.PathError and any
39+
// wrapping the storage layer added) and falls back to the driver's message
40+
// text, which is the only channel the pure-Go SQLite driver offers for
41+
// SQLITE_FULL.
42+
func IsDiskFull(err error) bool {
43+
if err == nil {
44+
return false
45+
}
46+
if errors.Is(err, syscall.ENOSPC) || errors.Is(err, syscall.EDQUOT) {
47+
return true
48+
}
49+
var pathErr *fs.PathError
50+
if errors.As(err, &pathErr) && errors.Is(pathErr.Err, syscall.ENOSPC) {
51+
return true
52+
}
53+
msg := strings.ToLower(err.Error())
54+
for _, frag := range sqliteFullMessages {
55+
if strings.Contains(msg, frag) {
56+
return true
57+
}
58+
}
59+
return false
60+
}
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
package aggregate
2+
3+
import (
4+
"errors"
5+
"fmt"
6+
"io/fs"
7+
"syscall"
8+
"testing"
9+
)
10+
11+
// The classifier decides whether a failed authoritative commit is a disk-full
12+
// failure, which is the one condition that MUST fail an Export (#201 Q5). A
13+
// false negative acknowledges data that was never stored.
14+
func TestIsDiskFull(t *testing.T) {
15+
cases := []struct {
16+
name string
17+
err error
18+
want bool
19+
}{
20+
{"nil", nil, false},
21+
{"unrelated", errors.New("constraint violation"), false},
22+
{"bare ENOSPC", syscall.ENOSPC, true},
23+
{"wrapped ENOSPC", fmt.Errorf("commit deltas: %w", syscall.ENOSPC), true},
24+
{"path error", &fs.PathError{Op: "write", Path: "/data/aggregate.db", Err: syscall.ENOSPC}, true},
25+
{"wrapped path error", fmt.Errorf("group commit: %w", &fs.PathError{Op: "write", Path: "/data/aggregate.db", Err: syscall.ENOSPC}), true},
26+
{"quota exceeded", fmt.Errorf("write wal: %w", syscall.EDQUOT), true},
27+
{"SQLITE_FULL text", errors.New("database or disk is full (13)"), true},
28+
{"ENOSPC text", errors.New("write /data/aggregate.db-wal: no space left on device"), true},
29+
{"mixed case", errors.New("SQL logic error: Database Or Disk Is Full"), true},
30+
// Adjacent failures that are NOT disk-full: misclassifying them would
31+
// turn an ordinary commit error into a different gRPC code.
32+
{"disk io error", errors.New("disk I/O error"), false},
33+
{"readonly", errors.New("attempt to write a readonly database"), false},
34+
}
35+
for _, tc := range cases {
36+
t.Run(tc.name, func(t *testing.T) {
37+
if got := IsDiskFull(tc.err); got != tc.want {
38+
t.Fatalf("IsDiskFull(%v) = %v, want %v", tc.err, got, tc.want)
39+
}
40+
})
41+
}
42+
}

internal/api/health_disk_test.go

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
package api
2+
3+
import (
4+
"net/http"
5+
"testing"
6+
)
7+
8+
// Readiness reflects disk pressure (#201 Q5). At raw-off the process still
9+
// serves reads and still accounts aggregates, but it can no longer retain the
10+
// diagnostics anyone comes here for; an orchestrator should stop aiming fresh
11+
// ingest at it. Errors-only is degraded coverage, not an unready process.
12+
//
13+
// readyChecks is shared with health_aggregate_test.go.
14+
func TestReadyReflectsDiskPressure(t *testing.T) {
15+
s := newTestServer(t)
16+
17+
code, checks := readyChecks(t, s)
18+
if checks["disk"] != "skipped" {
19+
t.Fatalf("disk check = %q without a watchdog, want skipped", checks["disk"])
20+
}
21+
if code != http.StatusOK {
22+
t.Fatalf("/ready = %d with no watchdog, want 200", code)
23+
}
24+
25+
state, healthy := "none", true
26+
s.SetDiskPressureProbe(func() (string, bool) { return state, healthy })
27+
28+
for _, tc := range []struct {
29+
state string
30+
healthy bool
31+
want int
32+
}{
33+
{"none", true, http.StatusOK},
34+
{"errors_only", true, http.StatusOK},
35+
{"raw_off", false, http.StatusServiceUnavailable},
36+
} {
37+
state, healthy = tc.state, tc.healthy
38+
code, checks = readyChecks(t, s)
39+
if code != tc.want {
40+
t.Errorf("/ready = %d at disk state %q, want %d", code, tc.state, tc.want)
41+
}
42+
if checks["disk"] != tc.state {
43+
t.Errorf("disk check = %q, want %q", checks["disk"], tc.state)
44+
}
45+
}
46+
}

internal/api/health_handlers.go

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,19 @@ func (s *Server) handleReady(w http.ResponseWriter, r *http.Request) {
100100
checks["aggregate_store"] = "ok"
101101
}
102102

103+
// Disk pressure. At >=95% of the enforcement ceiling the raw exemplar
104+
// path is off entirely; readiness says so rather than letting an
105+
// orchestrator keep aiming ingest at a nearly full volume.
106+
if s.diskPressure == nil {
107+
checks["disk"] = "skipped"
108+
} else {
109+
state, ok := s.diskPressure()
110+
checks["disk"] = state
111+
if !ok {
112+
ready = false
113+
}
114+
}
115+
103116
status := http.StatusOK
104117
if !ready {
105118
status = http.StatusServiceUnavailable

internal/api/server.go

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,14 @@ type Server struct {
4343
// no orchestrator routes traffic to a process whose shards are only
4444
// half-replayed (#173). nil means "no store configured" and is skipped.
4545
aggregateRecovered func() bool
46+
47+
// diskPressure reports the disk watchdog's state (#201 Q5): a label for
48+
// the readiness breakdown and whether the process should still be
49+
// considered ready. At raw-off it is not — the platform still serves
50+
// reads and still accounts aggregates, but it can no longer retain the
51+
// diagnostics anyone comes here for, and an orchestrator should stop
52+
// routing fresh ingest at it. nil means "no watchdog" and is skipped.
53+
diskPressure func() (string, bool)
4654
}
4755

4856
// NewServer creates a new API server.
@@ -102,6 +110,13 @@ func (s *Server) SetAggregateRecoveryProbe(fn func() bool) {
102110
s.aggregateRecovered = fn
103111
}
104112

113+
// SetDiskPressureProbe registers a callback returning the disk watchdog's
114+
// state label and whether readiness should pass. Pass nil (the default) when
115+
// no watchdog is configured.
116+
func (s *Server) SetDiskPressureProbe(fn func() (string, bool)) {
117+
s.diskPressure = fn
118+
}
119+
105120
// RegisterRoutes registers API endpoints on the provided mux.
106121
func (s *Server) RegisterRoutes(mux *http.ServeMux) {
107122
// Metadata & Discovery

internal/config/config.go

Lines changed: 53 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -432,6 +432,30 @@ type Config struct {
432432
ExemplarLogsWarnPerServiceWindow int // Default 20
433433
ExemplarMaxSpansPerTrace int // Default 500
434434
ExemplarMaxBytesPerTrace int // Default 262144 (256 KiB)
435+
436+
// Exemplar-tier retention and synthesized-log metering (#201 Q2/Q3).
437+
//
438+
// ExemplarRetentionDays is SHORTER than HotRetentionDays on purpose: in
439+
// aggregate mode the raw rows are exemplars attached to a seven-day
440+
// aggregate dataset, and 576 five-minute windows (two days) at the 3 MiB
441+
// global window budget is 1.69 GiB of charged payload — 3.38 GiB at the
442+
// provisional 2x DB/index/FTS amplification, inside the 4.5 GiB main tier
443+
// with ~1.12 GiB of margin. Seven days of the same rate does not fit.
444+
ExemplarRetentionDays int // Default 2, validated 1..HotRetentionDays
445+
ExemplarSynthLogsPerSpan int // Default 8
446+
// ExemplarSynthLogsPerTrace bounds the synthesized logs one retained trace
447+
// may carry across all its spans.
448+
ExemplarSynthLogsPerTrace int // Default 64
449+
450+
// Data-volume budget and disk watchdog (#201 Q1/Q5).
451+
//
452+
// DataDiskBudgetMB is the configured ceiling for everything this process
453+
// writes: main relational tier, aggregate.db, DLQ, WAL/temp, and the
454+
// mandatory unused headroom. Enforcement uses the LOWER of this and the
455+
// usable volume capacity — a 4 GiB PVC does not become 8 GiB because the
456+
// config says so.
457+
DataDiskBudgetMB int // Default 8192 (8 GiB)
458+
DataDiskPath string // Default ./data — any path on the data volume
435459
}
436460

437461
func Load(customPath string) (*Config, error) {
@@ -608,17 +632,27 @@ func Load(customPath string) (*Config, error) {
608632
AggregateMaxDimTuplesPerTenant: getEnvInt("AGGREGATE_MAX_DIM_TUPLES_PER_TENANT", 5000),
609633
AggregateMaxDimTuples: getEnvInt("AGGREGATE_MAX_DIM_TUPLES", 50000),
610634
// Bounded exemplar retention (aggregate mode only)
611-
ExemplarTracesPerServiceWindow: getEnvInt("EXEMPLAR_TRACES_PER_SERVICE_WINDOW", 25),
612-
ExemplarTracesGlobalWindow: getEnvInt("EXEMPLAR_TRACES_GLOBAL_WINDOW", 1500),
613-
ExemplarBytesPerServiceWindow: getEnvInt("EXEMPLAR_BYTES_PER_SERVICE_WINDOW", 512*1024),
614-
ExemplarBytesGlobalWindow: getEnvInt("EXEMPLAR_BYTES_GLOBAL_WINDOW", 8*1024*1024),
635+
ExemplarTracesPerServiceWindow: getEnvInt("EXEMPLAR_TRACES_PER_SERVICE_WINDOW", 25),
636+
ExemplarTracesGlobalWindow: getEnvInt("EXEMPLAR_TRACES_GLOBAL_WINDOW", 1500),
637+
ExemplarBytesPerServiceWindow: getEnvInt("EXEMPLAR_BYTES_PER_SERVICE_WINDOW", 512*1024),
638+
// 3 MiB, not 4 (#201 Q2). 4 MiB/window consumes the entire 4.5 GiB
639+
// main tier under the optimistic 2x amplification assumption and
640+
// leaves no operational margin; it stays configurable, it is not the
641+
// default until the seven-day gate (#202) proves it fits.
642+
ExemplarBytesGlobalWindow: getEnvInt("EXEMPLAR_BYTES_GLOBAL_WINDOW", 3*1024*1024),
615643
ExemplarHealthyRate: getEnvFloat("EXEMPLAR_HEALTHY_RATE", 0.005),
616644
ExemplarStratumTopK: getEnvInt("EXEMPLAR_STRATUM_TOP_K", 5),
617645
ExemplarLogsErrorPerServiceWindow: getEnvInt("EXEMPLAR_LOGS_ERROR_PER_SERVICE_WINDOW", 50),
618646
ExemplarLogsWarnEnabled: getEnvBool("EXEMPLAR_LOGS_WARN_ENABLED", false),
619647
ExemplarLogsWarnPerServiceWindow: getEnvInt("EXEMPLAR_LOGS_WARN_PER_SERVICE_WINDOW", 20),
620648
ExemplarMaxSpansPerTrace: getEnvInt("EXEMPLAR_MAX_SPANS_PER_TRACE", 500),
621649
ExemplarMaxBytesPerTrace: getEnvInt("EXEMPLAR_MAX_BYTES_PER_TRACE", 256*1024),
650+
ExemplarRetentionDays: getEnvInt("EXEMPLAR_RETENTION_DAYS", 2),
651+
ExemplarSynthLogsPerSpan: getEnvInt("EXEMPLAR_SYNTH_LOGS_PER_SPAN", 8),
652+
ExemplarSynthLogsPerTrace: getEnvInt("EXEMPLAR_SYNTH_LOGS_PER_TRACE", 64),
653+
// 8 GiB data budget (#201 Q1).
654+
DataDiskBudgetMB: getEnvInt("DATA_DISK_BUDGET_MB", 8192),
655+
DataDiskPath: getEnv("DATA_DISK_PATH", "./data"),
622656
}
623657

624658
// Parse AGGREGATE_METRIC_DIMS config
@@ -1049,6 +1083,21 @@ func (c *Config) Validate() error {
10491083
if c.ExemplarMaxBytesPerTrace < 1024 {
10501084
return fmt.Errorf("EXEMPLAR_MAX_BYTES_PER_TRACE must be >= 1024, got %d", c.ExemplarMaxBytesPerTrace)
10511085
}
1086+
if c.ExemplarRetentionDays < 1 || c.ExemplarRetentionDays > c.HotRetentionDays {
1087+
return fmt.Errorf("EXEMPLAR_RETENTION_DAYS must be between 1 and HOT_RETENTION_DAYS (%d), got %d: the exemplar tier is a shorter-lived subset of hot retention, never a longer-lived one", c.HotRetentionDays, c.ExemplarRetentionDays)
1088+
}
1089+
if c.ExemplarSynthLogsPerSpan < 1 {
1090+
return fmt.Errorf("EXEMPLAR_SYNTH_LOGS_PER_SPAN must be >= 1, got %d", c.ExemplarSynthLogsPerSpan)
1091+
}
1092+
if c.ExemplarSynthLogsPerTrace < c.ExemplarSynthLogsPerSpan {
1093+
return fmt.Errorf("EXEMPLAR_SYNTH_LOGS_PER_TRACE (%d) must be >= EXEMPLAR_SYNTH_LOGS_PER_SPAN (%d): a per-trace cap below the per-span cap makes the per-span cap unreachable", c.ExemplarSynthLogsPerTrace, c.ExemplarSynthLogsPerSpan)
1094+
}
1095+
if c.DataDiskBudgetMB < 64 {
1096+
return fmt.Errorf("DATA_DISK_BUDGET_MB must be >= 64, got %d", c.DataDiskBudgetMB)
1097+
}
1098+
if strings.TrimSpace(c.DataDiskPath) == "" {
1099+
return fmt.Errorf("DATA_DISK_PATH must not be empty")
1100+
}
10521101

10531102
// Sum-of-caps validation: sub-caps must fit under global cap
10541103
sumSubCaps := c.AggregateMaxSeriesMetrics + c.AggregateMaxSeriesTraces + c.AggregateMaxSeriesEdges + c.AggregateMaxSeriesLogs + c.AggregateMaxSeriesSystem

0 commit comments

Comments
 (0)