Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added retry capability for uploading files #1233

Merged
merged 1 commit into from
Jun 18, 2024

Conversation

Koc
Copy link
Contributor

@Koc Koc commented Jun 10, 2024

You can observe how it works on the video below. I've tested library changes using npm link. For failure simulation I've added next snippet to remote.php:

	if ($_SERVER['REQUEST_METHOD'] === 'PUT' && random_int(0, 4) === 0) {
		throw new RemoteException('Service unavailable', 503);
	}

	sleep(2);

Also as you can see sha256 hash is the same for the uploaded and original files. So, files are not corrupted during retries. Later on we can configure how many retries available.

nextcloud-retry-upload.mp4

lib/utils/upload.ts Outdated Show resolved Hide resolved
@susnux susnux added the enhancement New feature or request label Jun 11, 2024
@susnux
Copy link
Contributor

susnux commented Jun 12, 2024

Probably we also need to retry folder creation of recursive uploads

Copy link

codecov bot commented Jun 12, 2024

Codecov Report

Attention: Patch coverage is 88.88889% with 2 lines in your changes missing coverage. Please review.

Project coverage is 22.52%. Comparing base (0ae7a81) to head (2571ef9).
Report is 346 commits behind head on main.

Files Patch % Lines
lib/uploader.ts 50.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1233      +/-   ##
==========================================
+ Coverage   16.85%   22.52%   +5.66%     
==========================================
  Files           9       12       +3     
  Lines         445     2482    +2037     
  Branches       77       47      -30     
==========================================
+ Hits           75      559     +484     
- Misses        367     1923    +1556     
+ Partials        3        0       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Koc
Copy link
Contributor Author

Koc commented Jun 12, 2024

@susnux I will try to do that.

Any idea why Cypress is failing? How can I debug/fix that?

  0 passing (368ms)
  1 failing

  1) An uncaught error was detected outside of a test:
     TypeError: The following error originated from your test code, not from Cypress.

  > Failed to fetch dynamically imported module: http://localhost:5173/__cypress/src/@fs/home/runner/work/nextcloud-upload/nextcloud-upload/cypress/components/UploadPicker.cy.ts

@susnux
Copy link
Contributor

susnux commented Jun 12, 2024

Any idea why Cypress is failing? How can I debug/fix that?

Its flaky just needs to be restarted, known issue of Cypress (but if you dont pay money they do not care 😔 ).

@skjnldsv
Copy link
Contributor

Any idea why Cypress is failing? How can I debug/fix that?

Its flaky just needs to be restarted, known issue of Cypress (but if you dont pay money they do not care 😔 ).

5 restarts and still failing? 🤔
Might actually be a bug?

@Koc
Copy link
Contributor Author

Koc commented Jun 15, 2024

@skjnldsv but it works on my machine 🤷

image

@susnux I've added retry for MKCOL as well. But seems like we should add retries for PROPFIND call (not in nextcloud-upload but in core)

@Koc
Copy link
Contributor Author

Koc commented Jun 16, 2024

maybe failure caused by adding of a new dependency. But I didn't get how to fix that by googling. Also as you can see from the video - it works on my machine.

cypress-local.mp4

Please help to debug it 🙏

Here more info cypress-io/cypress#25913

Signed-off-by: Konstantin Myakshin <molodchick@gmail.com>
@susnux
Copy link
Contributor

susnux commented Jun 17, 2024

Yes this is that mentioned error, one long standing Cypress bug...

@skjnldsv skjnldsv merged commit fd8e048 into nextcloud-libraries:main Jun 18, 2024
18 checks passed
@Koc Koc deleted the feature/retry-upload branch June 18, 2024 13:16
@Koc
Copy link
Contributor Author

Koc commented Jun 18, 2024

Thank you for merging 🤝

@susnux @skjnldsv can we bump a new version? Because I need integrate it into https://github.com/nextcloud/server as well

@susnux
Copy link
Contributor

susnux commented Jun 18, 2024

can we bump a new version?

Yes there is another feature we will release, so would schedule a release tomorrow or Thursday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Retry chunk if failed
5 participants