Skip to content

LMDB: TTL eviction of a record with a Blob attribute orphans the blob file on disk (unbounded disk growth; RocksDB reclaims correctly) #1606

Open

Description

@kriszyp

Summary

On the LMDB storage engine, when a record with a Blob attribute expires via TTL, LMDB removes the row but leaves the blob file on disk. The orphaned files are never reclaimed → unbounded disk growth for any LMDB table combining TTL (expiration) with blob attributes. RocksDB reclaims correctly — this is engine-divergent.

Reproduction (harper main 228eacc0f, v5.1.15)

  • Table @table(expiration: 5) with a Blob attribute, threads: { count: 4 }.
  • Insert 5 records, each with a distinct ~200 KB blob.
  • Confirm eviction: point-read → 404 (~4 s after the deadline) before counting.
  • Count blob files under {dataRootDir}/blobs/data before vs after eviction.
Engine before → after
RocksDB 5 → 0 (clean); replace-then-expire 1 → 0 (both v1 + v2 reclaimed)
LMDB 5 → 5 (leak)

Re-ran the LMDB case with a 30 s settle → 5 → 1, not 0 — so this is not eviction lag; LMDB's unlink is partial/racy and permanently orphans blob files.

Root-cause hypothesis (unverified, medium confidence)

Table.evict() → removeEntry() gates blob deletion on entry.value && (metadataFlags & HAS_BLOBS), but the LMDB eviction path's getEntry(id) doesn't reliably return a populated value/flags, so deleteBlobsInObject never fires. Candidate fix: feed the scan's already-decoded record into evict() rather than re-fetching it.

Severity

MEDIUM. LMDB is the deprecated engine (replication is RocksDB-only), so blast radius is limited to LMDB deployments — but it is a silent, unbounded-disk-growth footgun for anyone on LMDB with blob + TTL tables (sessions, caches, media with expiry).

Found via exploratory QA; local repro test available (qa-scratch/qa-blob-ttl.test.ts).

— KrAIs 🤖 (exploratory QA, on Kris's behalf)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:storageStorage engine, LMDB/RocksDB, compactionbugSomething isn't working

    Type

    No type

    Fields

    Priority

    P2

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions