Problem with saving file on s3 storage. How save works?

[darkowic]

Since version 1.1.9+ (after fork) you changed S3BotoStorageFile _file attribute from StringIO to SpooledTemporaryFile.

In my code I use PIL for generate picture thumbnail like this:

img = Image.open(default_storage.open(instance.uploaded.name, 'r'))
img.thumbnail(meta_data['size'], Image.ANTIALIAS)
f = default_storage.open(str('%s' + meta_data['prefix'] + '%s') % (file_path, filename,), 'w')
img.save(f, ext, quality=90)
f.close()

and weird think was that it works for png but not for jpg files.
After some investigation (python-pillow/Pillow#1940) I found that there is two ways of saving files in Pillow

# file ImageFile.py
    try:
        fh = fp.fileno()
        fp.flush()
    except (AttributeError, io.UnsupportedOperation):
        # compress to Python file-compatible object
        for e, b, o, a in tile:
            e = Image._getencoder(im.mode, e, a, im.encoderconfig)
            if o > 0:
                fp.seek(o, 0)
            e.setimage(im.im, b)
            while True:
                l, s, d = e.encode(bufsize)
                fp.write(d)
                if s:
                    break
            if s < 0:
                raise IOError("encoder error %d when writing image file" % s)
            e.cleanup()
    else:
        # slight speedup: compress to real file object
        for e, b, o, a in tile:
            e = Image._getencoder(im.mode, e, a, im.encoderconfig)
            if o > 0:
                fp.seek(o, 0)
            e.setimage(im.im, b)
            s = e.encode_to_file(fh, bufsize)
            if s < 0:
                raise IOError("encoder error %d when writing image file" % s)
            e.cleanup()

First way (when error is throw) call python's file write method and second no.

It works for version 1.1.8 where was use StringIO and doesn't work for newer versions where SpooledTemporaryFile is used.

Is this bug? How to fix it?

[behconsci]

how is ext defined in your save?

[EssaAlshammri]

have you found a solution ?

[darkowic]

@behconsci ext was simply string e.g. 'JPEG' (depends on input file)

@EssaAlshammri I do not use default_storage.open(). Instead of it I use default_storage.save(). But we removed our solution for thumbnails and we are using library easy_thumbnails which rocks! :)

[EssaAlshammri]

@darkowic I'm having the same problem but in another scenario storages.backends.gs.GSBotoStorage and on google app engine, I'm getting

ImportError at /accounts/pictureUpload/
cannot import name SpooledTemporaryFile

when I try to upload an image using ImageField

I think google app engine doesn't have tempfile.py and I don't know how to include it

any idea ?

[darkowic]

@EssaAlshammri I don't think its google's app engine problem. Its's rather on you side. But I don't have experience with this. And I guess it isn't topic of this issue.

[EssaAlshammri]

@darkowic I'm sorry but I thought it would be easier than opening another issue

• yes it's a google app engine problem because it work on my machine and it upload to google storage

thanks

[jschneier]

Ah this is #281, I didn't understand what this meant before.

[darkowic]

But I don't use Google app engine but boto.

[EssaAlshammri]

I have solved it already. 👍

[Cealtea]

@EssaAlshammri What was your solution on google appengine? I am facing the same issue only with large files it seems