Resume support is not working as it should

[hellais]

I am trying to get jQuery-File-Upload to work with resume support however I believe the functions using FileAPI and Blob are not seeking to the correct position in the file.

an example of what is happening.

In the first request we can see that the locally stored file has a size of 4504332, where the file that is supposed to be uploaded is 10104332.

INFO:Rocket.Requests:127.0.0.1 - "GET /globaleaks/submission/fileupload?file=abigfile.txt HTTP/1.1" - 200 202
filepath: //xxx//applications/globaleaks//uploads/abigfile.txt 
append: True file.size: 10104332 getsize: 4504332

These are the following chunks that are of size 1000000:

INFO:Rocket.Requests:127.0.0.1 - "POST /globaleaks/submission/fileupload HTTP/1.1" - 200 224
filepath: //xxx//applications/globaleaks//uploads/abigfile.txt 
append: True file.size: 10104332 getsize: 5504332

INFO:Rocket.Requests:127.0.0.1 - "POST /globaleaks/submission/fileupload HTTP/1.1" - 200 224
filepath: //xxx//applications/globaleaks//uploads/abigfile.txt 
append: True file.size: 10104332 getsize: 6504332

INFO:Rocket.Requests:127.0.0.1 - "POST /globaleaks/submission/fileupload HTTP/1.1" - 200 224
filepath: //xxx//applications/globaleaks//uploads/abigfile.txt 
append: True file.size: 10104332 getsize: 7504332

INFO:Rocket.Requests:127.0.0.1 - "POST /globaleaks/submission/fileupload HTTP/1.1" - 200 224
filepath: //xxx//applications/globaleaks//uploads/abigfile.txt 
append: True file.size: 10104332 getsize: 8504332

INFO:Rocket.Requests:127.0.0.1 - "POST /globaleaks/submission/fileupload HTTP/1.1" - 200 224
filepath: //xxx//applications/globaleaks//uploads/abigfile.txt 
append: True file.size: 10104332 getsize: 9504332

In this request however we see that the size of the file stored locally is more than the size of the file being uploaded.

INFO:Rocket.Requests:127.0.0.1 - "POST /globaleaks/submission/fileupload HTTP/1.1" - 200 225
filepath: //xxx//applications/globaleaks//uploads/abigfile.txt 
append: False file.size: 10104332 getsize: 10504332

Also I did some checks at intermediate steps and I noticed that the even before completion when the file is resumed it does not match the original.

bla=204050;cat ~/Documents/abigfile.txt | head -n $bla | md5; cat abigfile.txt | head -n $bla | md5
726f25273d6d4ecb7f137eb59d1c9143
d5fc97458542eae12b5a6ed0d35bfc6c

Has the upload function been thoroughly tested?
Has someone actually used it?

[hellais]

Also, for doing seeking, shouldn't you be using FileReader()?

That is the new FileAPI inside HTML5. I did not find any reference to that in the .js code.

[blueimp]

The seeking is done using the Blob API's slice method (search for "file.webkitSlice || file.mozSlice || file.slice" in the source code).
The correct bytes offset used for the slicing is set via the uploadedBytes option, which is retrieved with a call to the server.
Please have a look at the Chunked Uploads tutorial for more information:
https://github.com/blueimp/jQuery-File-Upload/wiki/Chunked-file-uploads

Although no unit-test case has been set up for this feature, I've tested it thoroughly with supported browsers (Chrome, Firefox) and it worked as expected in my tests.

[hellais]

Yes, I did check out the Chunked file uploads tutorial and I do understand how it is supposed to work, however it does not work as it should.

If you look at the logs I posted, you will see that the client (JS) is sending bytes past the file file size. This means that the options UploadedBytes is not working as it should.

[blueimp]

Can you give more information on the plugin settings, browser versions and server environment you've tested this with?

I just tested chunked uploads/resuming aborted uploads with the latest stable versions of Firefox (7.0.1) and Chrome (14.0.835.202) and they definitely worked as expected.

Steps to reproduce my results:

• Download the plugin archive, extract it and upload the extracted folder (you may rename it) to your PHP-enabled server (PHP version >= 5.2).
• Set discard_aborted_uploads in example/upload.php to false.
• Replace $('#fileupload').fileupload(); in example/application.js with the code snippet from the "Resuming file uploads" section of the Chunked-file-uploads tutorial.
• Upload a file bigger than 10MB, cancel the upload and reselect the same file to resume the upload process.

[hellais]

Ok, so I have been testing this on Chrome 14.0.835.202 and FF 6.0.2.

This is the source of the plugin.

// Initialize the jQuery File Upload widget:
$('#fileupload').fileupload({
    url: "/globaleaks/submission/fileupload",
    autoUpload: true,
    maxChunkSize: 1000000,
    multipart: false,
    add: function (e, data) {
        var that = this;
        $.getJSON('/globaleaks/submission/fileupload', {file: data.files[0].name}, function (file) {
            data.uploadedBytes = file && file.size;
            $.blueimpUI.fileupload.prototype
                .options.add.call(that, e, data);
    });
}

});

Steps to reproduce:

1. get a copy of GlobaLeaks.
   git clone github.com/globaleaks/globaleaks.git
2. ./startglobaleaks
3. Go to the submission interface and upload a file
4. If the file upload is interrupted, it will not resume.

[blueimp]

Just to sort out if it's an issue with the client-side plugin library or a server-side issue, could you please try to reproduce my results with the steps outlined above?
If you don't have a PHP environment set up, XAMPP is an easy way to test it out:
http://www.apachefriends.org/en/xampp.html

[hellais]

I will test this thoroughly tomorrow, because what I suspect is that the file does not resume, but in fact the file get's uploaded twice (this means bytes_previously_uploaded -> EOF -> ALL_FILE).

I ran some basic tests and it appears to be the case also with the PHP version, but as I said I need to cross check this.

The reason why I am noticing this and why it's a big deal for me is that I plan on using this to do file uploads to a Tor Hidden Service. Since there is a lot of latency and it could happen that the file upload get's interrupt, you don't want the user restarting it every time.

[pntkl]

I was able to replicate this issue, with the same setup you are describing, blueimp. A user testing the upload on Mac OSX Lion with Safari 5.1. She reported back to me with, "The problem is that that was a 135mb video I uploaded and it only showed like .1kb or whatever. It uploaded for like 2 seconds, showed it as done and then showed it as .1kb."

I was able to fix that, with the comments in issue #700. Following that, I tested resume on my out of date hackintosh. It looks like in both Safari 5.0 and Firefox 3.6, file resume does not work as expected. This was initially apparent, after testing resume, on a file that had already successfully uploaded. The upload would start at 3.42MB out of 3.28MB and was corrupt, after finishing.

[blueimp]

Safari 5.1 doesn't support non-multipart file uploads anymore (this issue is documented in the Options docs).
I've removed the multipart: false settings from the chunked uploads doc, as it was only required for Firefox 4-6, which are unsupported since Mozilla released versions 7 and 8.
Firefox 3.6 has no XHR file upload support since version 5 of the plugin - see why here:
https://github.com/blueimp/jQuery-File-Upload/wiki/Browser-support

[almash]

At http://blueimp.github.com/jQuery-File-Upload/ file does not upload and if one is uploaded then does not show any icon to pause / resume. It also does not show progress bar. I tested it on FireFox 9.0.1 and Chrome 16.0.912.63 m.

[blueimp]

The demo has no support for chunked uploads and therefore no pause/resume support.
However, file uploads should work fine on all browsers and do so in my tests.

[almash]

Hi Sebastian,
Thanks for your prompt response.
I just tried it but did not able to upload. If I could I would have showed you the screenshot. I don't know how to add image to this comment editor.

It shows error like: Error true. Got it?
Thanks

[blueimp]

Don't worry, halmashossain. I believe every error report is genuine, so there must be an issue somewhere.

"Error: true" means there is an error without a detailed error message.
You might want to check the Network tab of the Chrome/Safari developer console or Firebug to retrieve more information.

As long as I cannot reproduce the issue, I cannot debug it. So far you have to try to identify the issue yourself or provide enough details for me or another user to reproduce it.

However, you might want to open another issue, as this doesn't seem to have anything to do with the resumable uploads feature.

[almash]

I get this error in firefox console:

"Component returned failure code: 0x80004001 (NS_ERROR_NOT_IMPLEMENTED) [nsIDOMLSProgressEvent.input]"

FYI: this error occurs when I try to upload any image(jpg, png, gif)

Thanks

[blueimp]

Thanks halmashossain, I think your issue is the same as this bug:
#924