-
Couldn't load subscription status.
- Fork 4.4k
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Confirm this is an issue with the Python library and not an underlying OpenAI API
- This is an issue with the Python library
Describe the bug
The upload_file_chunked method in both sync and async implementations fails to properly close file handles when uploads complete successfully, causing a resource leak.
Location: src/openai/resources/uploads/uploads.py
In the sync version (lines 145-164):
if isinstance(file, builtins.bytes):
buf: io.FileIO | io.BytesIO = io.BytesIO(file)
else:
buf = io.FileIO(file)
try:
while True:
data = buf.read(part_size)
if not data:
# EOF
break
part = self.parts.create(upload_id=upload.id, data=data)
log.info("Uploaded part %s for upload %s", part.id, upload.id)
part_ids.append(part.id)
except Exception:
buf.close() # Only closes on exception!
raise
return self.complete(upload_id=upload.id, part_ids=part_ids, md5=md5)
Problem: The file handle buf is only closed when an exception occurs. On successful uploads, buf.close() is never called, leaking the file descriptor.
The async version has the same issue (lines 456-470) for the BytesIO case:
else:
buf = io.BytesIO(file)
try:
while True:
data = buf.read(part_size)
if not data:
break
part = await self.parts.create(upload_id=upload.id, data=data)
log.info("Uploaded part %s for upload %s", part.id, upload.id)
part_ids.append(part.id)
except Exception:
buf.close() # Only closes on exception!
raise
To Reproduce
from openai import OpenAI
from pathlib import Path
client = OpenAI()
# Upload multiple files - file handles are never closed
for i in range(100):
client.uploads.upload_file_chunked(
file=Path("my-file.pdf"),
mime_type="application/pdf",
purpose="assistants",
)
# On systems with limited file descriptors, this will eventually fail
# with "OSError: [Errno 24] Too many open files"
Code snippets
The bug is in src/openai/resources/uploads/uploads.py:
Sync version: lines 145-164
Async version: lines 456-470OS
10.0.26100 (but affects all platforms)
Python version
Python v3.11.4
Library version
openai v1.0.1
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working