Skip to content

server: hosted-project saves leak orphaned File docs in Firestore (no version chain, no GC) #521

Description

@bpowers

Problem

Every save of a hosted project creates a brand-new File Firestore document keyed by sha256(serialized file protobuf) (see src/server/project-creation.ts createFile() and src/server/api.ts POST /api/projects/:username/:projectName, around api.ts:272-276). The Project document's fileId is then repointed to the newest File via projectModel.setFileId(file.getId()). The previously-referenced File documents are never deleted and are not referenced by anything afterward.

The File.prev_id field exists in the schema (src/server/schemas/file.proto, repeated string prev_id = 2) and createFile() takes a prevId parameter, but both call sites pass undefined (api.ts:120 for new-project creation, api.ts:272 for overwrite), so no version chain is ever built. FirestoreTable has a deleteOne method (src/server/models/table-firestore.ts:161) but it is never invoked for superseded File docs.

Net effect: after N saves, a project accumulates N+1 File docs in Firestore, N of which are orphaned dead weight. Each File doc embeds a full serialized project protobuf (bytes project_contents = 7, potentially hundreds of KB). There is no garbage collection; this grows unbounded with edit activity.

Why it matters

  • Storage cost / quota: unbounded Firestore growth proportional to total edit count across all hosted projects, with no upper bound and no cleanup path.
  • Operational hygiene: the data model implies a version chain (prev_id) that is never actually populated, so the schema is misleading and the intended history feature is half-built.
  • Blocks adjacent work: a "delete project" feature (issue delete models #49) has to decide what to do with these orphaned File docs -- delete every File where project_id == slug, or only the currently-referenced one. The lack of a defined retention model makes that decision ambiguous.

Components affected

  • src/server/project-creation.ts (createFile())
  • src/server/api.ts (POST /api/projects/:username/:projectName, new-project creation path)
  • src/server/models/table-firestore.ts (FirestoreTable, has unused deleteOne)
  • src/server/schemas/file.proto (File.prev_id is dead schema today)

Possible approaches

  1. Delete-on-overwrite: after the new File is committed and Project.fileId is repointed, delete the previously-referenced File doc. Simplest; keeps exactly one File per project. (Should be coordinated with the transactionality fix tracked in tech-debt entry run management #41 -- src/server/api.ts:240-307 -- so the delete isn't stranded on a 409.)
  2. Bounded version history: actually populate prev_id to form a chain, and trim it to the last K versions on each save, deleting anything past the window.
  3. Periodic GC job: a sweep that deletes File docs not referenced by any Project (and, if option 2 is taken, not reachable via any live prev_id chain).

How it was discovered

Identified while investigating issue #49 (delete models) -- a delete-project feature needs a defined retention model for these File docs. The orphan-accumulation behavior itself is pre-existing and predates that work. Related but narrower: tech-debt entry #41 notes that a 409 race also strands an orphaned File blob; this issue is about the steady-state leak on every successful save.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendInvolves the Google App Engine node appbugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions