Skip to content

Bound browser-upload chunk reads before allocation #585

Description

@BrandonRobare

Parent: #582

backend/routers/uploads.py:289-302 reads the whole chunk into memory, then checks whether it was too big:

state = _state(upload_id)
data = await chunk.read()
with _lock_for(upload_id):
    ...
    limits = _limits()
    if len(data) > limits["chunk_size_bytes"]:
        raise HTTPException(status_code=413, detail="Chunk exceeds configured size")

An oversized part is fully allocated before the 413.

Memory and disk are different problems here

The memory bound is easy: await chunk.read(limit + 1) and reject on overflow.

The disk bound cannot be fixed in this handler. Starlette exempts file parts from max_part_size, so by the time the handler body runs the part has already been spooled to a SpooledTemporaryFile. Bounding that needs a Content-Length pre-check or a custom parser.

An earlier draft asked to "reject excess before any file write", which is not achievable at this layer. Either descope it or accept a Content-Length guard.

The Starlette spool behaviour was confirmed against 1.0.1; this project locks 1.3.1 (uv.lock:2337-2346). It is long-standing, but worth re-confirming against the pinned version before relying on it.

Reuse the helper that already exists

backend/routers/flight_log.py:29 and backend/routers/srt.py:18 both define:

async def _read_upload_with_limit(file: UploadFile, max_bytes: int) -> bytes

Consolidate onto one instead of adding a third copy.

Acceptance criteria

  • The handler reads at most the configured chunk size plus one byte
  • Planned-size, offset, and path validation still run
  • One shared bounded-read helper, used by all three call sites
  • A test fake records the requested read size

Checks

pytest tests/backend/test_uploads_router.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendFastAPI/backend service workbugSomething isn't workingperformancePerformance, bundle size, scalabilitypriority: highHigh-priority correctness or safety issuepythonPull requests that update python codesecurity

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions