AttributeError: 'SpooledTemporaryFile' object has no attribute 'readinto'

[ramtinabadi]

I get the following error while trying backup a Postgres db and upload it to the S3. The S3 setup works fine.

Writing file to default-something-2020-12-18-002957.psql
AttributeError: 'SpooledTemporaryFile' object has no attribute 'readinto'
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/utils.py", line 118, in wrapper
    func(*args, **kwargs)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/management/commands/dbbackup.py", line 61, in handle
    self._save_new_backup(database)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/management/commands/dbbackup.py", line 88, in _save_new_backup
    self.write_to_storage(outputfile, filename)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/management/commands/_base.py", line 88, in write_to_storage
    self.storage.write_file(file, path)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/storage.py", line 82, in write_file
    self.storage.save(name=filename, content=filehandle)
  File "/home/django/venv/lib/python3.8/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/home/django/venv/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 446, in _save
    obj.upload_fileobj(reader, ExtraArgs=params)
  File "/home/django/venv/lib/python3.8/site-packages/boto3/s3/inject.py", line 619, in object_upload_fileobj
    return self.meta.client.upload_fileobj(
  File "/home/django/venv/lib/python3.8/site-packages/boto3/s3/inject.py", line 539, in upload_fileobj
    return future.result()
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 558, in _submit
    self._submit_multipart_request(
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 622, in _submit_multipart_request
    for part_number, fileobj in part_iterator:
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 270, in yield_upload_part_bodies
    fileobj, full_size = self._get_upload_part_fileobj_with_full_size(
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 337, in _get_upload_part_fileobj_with_full_size
    data = fileobj.read(kwargs['part_size'])
  File "/home/django/venv/lib/python3.8/site-packages/django/core/files/utils.py", line 17, in <lambda>
    readinto = property(lambda self: self.file.readinto)

Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    main()
  File "manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/home/django/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/home/django/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/django/venv/lib/python3.8/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/django/venv/lib/python3.8/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/utils.py", line 118, in wrapper
    func(*args, **kwargs)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/management/commands/dbbackup.py", line 61, in handle
    self._save_new_backup(database)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/management/commands/dbbackup.py", line 88, in _save_new_backup
    self.write_to_storage(outputfile, filename)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/management/commands/_base.py", line 88, in write_to_storage
    self.storage.write_file(file, path)
  File "/home/django/venv/lib/python3.8/site-packages/dbbackup/storage.py", line 82, in write_file
    self.storage.save(name=filename, content=filehandle)
  File "/home/django/venv/lib/python3.8/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/home/django/venv/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 446, in _save
    obj.upload_fileobj(reader, ExtraArgs=params)
  File "/home/django/venv/lib/python3.8/site-packages/boto3/s3/inject.py", line 619, in object_upload_fileobj
    return self.meta.client.upload_fileobj(
  File "/home/django/venv/lib/python3.8/site-packages/boto3/s3/inject.py", line 539, in upload_fileobj
    return future.result()
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 558, in _submit
    self._submit_multipart_request(
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 622, in _submit_multipart_request
    for part_number, fileobj in part_iterator:
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 270, in yield_upload_part_bodies
    fileobj, full_size = self._get_upload_part_fileobj_with_full_size(
  File "/home/django/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 337, in _get_upload_part_fileobj_with_full_size
    data = fileobj.read(kwargs['part_size'])
  File "/home/django/venv/lib/python3.8/site-packages/django/core/files/utils.py", line 17, in <lambda>
    readinto = property(lambda self: self.file.readinto)
AttributeError: 'SpooledTemporaryFile' object has no attribute 'readinto'

Settings.py

DBBACKUP_CONNECTORS = {
    'default': {
        'USER': os.environ['DB_USER'],
        'PASSWORD': os.environ['DB_PASSWORD'],
        'HOST': os.environ['DB_HOST']
    }
}

DBBACKUP_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
DBBACKUP_STORAGE_OPTIONS = {
    "access_key": os.environ['AWS_ACCESS_KEY_ID'],
    "secret_key": os.environ['AWS_SECRET_ACCESS_KEY'],
    "bucket_name": os.environ['AWS_BUCKET_NAME'],
    'default_acl': 'private'
}

Versions

Django-dbbackup

• pypi: 3.30

External tools

• Python: 3.8
• Django: 3.1
• OS: Ubuntu 18.4

[jonathan-s]

It looks like you're bumping into this issue described in the PR on comment here: python/cpython#3249 (comment) As described in the comment a viable solution could be to monkeypatch SpooledTemporaryFile. If you've got time to test that I'd be keen to hear back.

[jonathan-s]

A longer term fix would fix this in django itself..

[Archmonger]

Appears to be resolved in python/cpython#29560