Skip to content

flowctl: preview-next fixture feeder panics with "disk_backlog_bytes underflow" when the consumer is slower than COMPRESS_AFTER #3443

Description

@mdibaiee

Since cd18939 (#3408), flowctl raw preview-next --fixture panics once a connector takes longer than COMPRESS_AFTER (2.5s) per transaction:

thread 'tokio-runtime-worker' panicked at crates/flowctl/src/raw/preview_next/fixture.rs:680:10:
disk_backlog_bytes underflow
ERROR flowctl::raw::preview_next: fixture feeder ended before end-of-stream; stopping active session

Reproduced with tests/benchmark/materialize/run.sh --connector materialize-redshift --scenario large-docs.yaml in estuary/connectors (flowctl v0.6.13-55-gfef96c502c7): the panic lands in round 2, while the connector is busy with a multi-second COPY.

Cause. The fixture charges disk_backlog_bytes with each rolled segment's size. Blocks above the writer's compress threshold are already LZ4-compressed, so when a segment survives COMPRESS_AFTER and SealedSegment::serve compresses the whole file, the file can grow: serve yields old.saturating_sub(new) == 0, sets sealed.size = new_size, and the eventual unlink yields that larger size. on_reclaimed then credits more than it charged and checked_sub(...).expect(...) panics. A consumer faster than 2.5s unlinks before compression and never sees it.

LogActor::on_reclaimed (crates/shuffle/src/log/actor.rs) has the same checked_sub.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions