Skip to content

fix(storage): close the upload file handle when the request fails - #1575

Open
tushardev-365 wants to merge 1 commit into
supabase:mainfrom
tushardev-365:fix/storage-upload-handle-leak
Open

fix(storage): close the upload file handle when the request fails#1575
tushardev-365 wants to merge 1 commit into
supabase:mainfrom
tushardev-365:fix/storage-upload-handle-leak

Conversation

@tushardev-365

Copy link
Copy Markdown
Contributor

What

When a caller passes a path instead of an already-open file, storage3 opens the handle itself:

files = {"file": (filename, open(file, "rb"), content_type)}

_request then closed it after the except HTTPStatusError block, so the close only ran when the request succeeded. Any non-2xx raised StorageApiError straight past it and the handle stayed open — and since it only ever existed inside _request's local files dict, the caller has no way to close it either.

It leaks whenever the exception is retained: logging with exc_info, collecting failures in a batch-upload loop, raise ... from, an error reporter. The traceback keeps the frame holding the reader alive. Python also emits ResourceWarning: unclosed file on that path.

Counting open fds against a mock transport returning 409 (the most common upload failure), with the error retained each time:

baseline: 0
after failed upload #1: 1
after failed upload #2: 2
...
after failed upload #5: 5
after successful upload: 0     <- success path was always fine

Both upload() / update() (via _upload_or_update) and upload_to_signed_url() are affected.

Fix

Move the close into a finally so it runs on both paths. The condition is unchanged — still only BufferedReader, i.e. only handles storage3 opened itself, so a caller-supplied stream is still left alone to close as they see fit.

Tests

test_upload_closes_file_handle_on_error (async + sync): mock the transport into an error, then assert the handle storage3 opened is closed. Reverting only the source change fails it on .closed is False.

Note

The diff on file_api.py looks larger than it is — wrapping the body in try reindents it. The only behavioural change is where the close happens.

_sync is generated, so the change was made in _async and regenerated with make build-sync; unrelated regenerated files are excluded.

This does not touch the resp.text line a few lines above (#1563), which has PRs open already — different defect, different code path.

When a caller passes a path rather than an open file, _upload_or_update
and upload_to_signed_url open the handle themselves and hand it to
_request inside the files dict. The close lived after the
except HTTPStatusError block, so it only ran when the request succeeded:
any non-2xx raised StorageApiError straight past it and the handle was
left open, with no way for the caller to reach it.

It leaks whenever the exception is retained (logging with exc_info,
collecting failures in a batch upload loop, an error reporter), because
the traceback keeps the frame holding the reader alive. Python also
emits ResourceWarning: unclosed file on that path.

Move the close into a finally so it runs on both paths. The condition is
unchanged, so only handles storage3 opened itself are closed and a
caller-supplied stream is still left alone.
@tushardev-365
tushardev-365 requested review from a team and o-santi as code owners August 15, 2026 05:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant