Skip to content

File handle resource leak in upload_file_chunked method #2708

@Showmick119

Description

@Showmick119

Confirm this is an issue with the Python library and not an underlying OpenAI API

  • This is an issue with the Python library

Describe the bug

The upload_file_chunked method in both sync and async implementations fails to properly close file handles when uploads complete successfully, causing a resource leak.

Location: src/openai/resources/uploads/uploads.py

In the sync version (lines 145-164):

if isinstance(file, builtins.bytes):
    buf: io.FileIO | io.BytesIO = io.BytesIO(file)
else:
    buf = io.FileIO(file)

try:
    while True:
        data = buf.read(part_size)
        if not data:
            # EOF
            break

        part = self.parts.create(upload_id=upload.id, data=data)
        log.info("Uploaded part %s for upload %s", part.id, upload.id)
        part_ids.append(part.id)
except Exception:
    buf.close()  # Only closes on exception!
    raise

return self.complete(upload_id=upload.id, part_ids=part_ids, md5=md5)

Problem: The file handle buf is only closed when an exception occurs. On successful uploads, buf.close() is never called, leaking the file descriptor.

The async version has the same issue (lines 456-470) for the BytesIO case:

else:
    buf = io.BytesIO(file)

    try:
        while True:
            data = buf.read(part_size)
            if not data:
                break

            part = await self.parts.create(upload_id=upload.id, data=data)
            log.info("Uploaded part %s for upload %s", part.id, upload.id)
            part_ids.append(part.id)
    except Exception:
        buf.close()  # Only closes on exception!
        raise

To Reproduce

from openai import OpenAI
from pathlib import Path

client = OpenAI()

# Upload multiple files - file handles are never closed
for i in range(100):
    client.uploads.upload_file_chunked(
        file=Path("my-file.pdf"),
        mime_type="application/pdf",
        purpose="assistants",
    )
# On systems with limited file descriptors, this will eventually fail
# with "OSError: [Errno 24] Too many open files"

Code snippets

The bug is in src/openai/resources/uploads/uploads.py:
Sync version: lines 145-164
Async version: lines 456-470

OS

10.0.26100 (but affects all platforms)

Python version

Python v3.11.4

Library version

openai v1.0.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions