Uploads repeating for large file sizes

[danwax]

I have an issue whereby large files, those I've found to be around 1GB (one in question I've been trialing most is 1.3GB) will upload, then upload again.

Factors involved:-

• Only seems to be on large files - 1GB+
• Only seems to be on Firefox - replicated on Linux and PC versions - 16.0.1 and 15. Haven't managed to replicate on Chrome or Firefox 12.
• This happens both on drag and drop - for which I'm using (dropBoxObj).bind('fileuploadadd', function (e, data) {}); and the input caller.
• May or may not be related. I'm having it getting 'stuck' on such large files on Chrome 22.0.1229.79 m - same place everytime. And then on a linux Chromium (20.0.1132.47) it stalls at the finish.

Note; this particular file is not alone and the file itself is fine.

Are there any known problems with large file sizes - any stability issues. Should I throttle uploads to a certain size for safety?

[blueimp]

I'm not aware of and could not reproduce an issue that would result in repeating (?) file uploads.
The only known issue with large file sizes is when you reach the limit of 4GB:
#1694

I will close this issue for now, if you can reproduce this problem with an unmodified repository setup, please post again here with information on how to reproduce it.

[mediadonkey]

I thought it worth adding to this, I'm constantly able to re-produce this with the S3 input script ('Source for Amazon S3 Upload with PHP SDK ', by @robertocolonello) which is something I hadn't mentioned before.

Have you a default setup with that plugin that reproduces? I will try if not, just wanted to ask first.

I have a feeling it might be on the events 'fileuploaddone' and 'fileuploadprogress' - which, with a larger file - can take a long time to report back, as that script is uploading it to local /tmp then doing a create-object on S3 API to transfer the data. Don't know, I'm playing with it.

[blueimp]

Thanks for coming back with further info, @mediadonkey
You might want to contact Robert directly since I never tested his setup.

[mediadonkey]

Will do, I'll update with any findings.

[mediadonkey]

Just a quick question since you're here (I've emailed Robert btw)... I've confirmed the S3 API is reporting back success a good 2 minutes later than the 'upload' has finished. Is there a timeout trigger or anything on file-uploader side which would say 'no further progress after x seconds, restart'?

It seems to wait 60 seconds, from what I can see. Potentially this is PHP INI standard config of default_socket_timeout? Is that used at all in uploads?

[blueimp]

There's no default timeout setting in the JS library, although you can make use of the timeout option which is passed to jQuery.ajax.

[JaapMoolenaar]

Might this be related to a windows server?
I am having the same issue on a locally installed Wamp server.

Windows XP sp3
Wamp v2.2
PHP v5.3.13
Tested in Chrome 22.0.1229.94

I ran the demo ( only added the maxFileSize: 10000000 on line 23 in main.js ) in the package downloaded yesterday (04-12-2012). Every file uploads to 2.099.166 Kb and then starts over.

I uploaded the same script to a fedora server on which it runs perfectly.

[mediadonkey]

Hi Satsume,

I'm running on linux machines. I've now actually rebuilt my dropbox with the standard PHP uploaderHandler - uploading locally and then transferring over to s3... the problem remains.

I'm just continuing to try everything under the sun.

I'll check my Kb max before it starts over, see if any correlation.

[JaapMoolenaar]

To bad, I thought it might have helped. I forgot to mention I'm running Apache/2.2.22. If I think of anything else, I'll comment.

[mediadonkey]

Hey Satsume,

How did you log where the start over happens? I'm debugging to find that point but never get a complete file so nothing reports on file size. Are you getting that back in the JS

[mediadonkey]

I've now resolved this for my situation.

It was related to the PHP file having a process in it which held up the response. I pushed this process out to another file which carried on the process and it's now working.

So, basically the PHP input/upload file needs to respond IMMEDIATELY on complete otherwise this repeating upload starts over and over.

[AvantGardeDreams]

Mediadonkey, could you provide some more details regarding your solution? Was your system running https? I am having a very similar problem on an apache server using mod_ssl.

Apache/2.2.16 (Debian)
PHP 5.3.3

[nehapungavkar]

hi ,
I am facing the same issue,could you please give some idea about the solution you tried.

[mediadonkey]

Hi there,

I haven't got much else to add - the issue was solved for my case by assuring that any processing or file transfer time was pushed out to a separate process.

That is to say when a user uploaded a file to our service, the file was then transferred to another server. This time it took to transfer the file held up the POST response to the dropbox. It became the dropbox requried a 200 OK response as soon as the file had reached 100% (i.e. it wouldn't wait for the upload PHP to transfer the file) otherwise it repeated the upload.

This was an older version of this plugin, I'm not sure whether it's resolved in later versions.