Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stash upload #2505

Merged
merged 2 commits into from
Mar 18, 2019
Merged

Stash upload #2505

merged 2 commits into from
Mar 18, 2019

Conversation

whym
Copy link
Collaborator

@whym whym commented Feb 22, 2019

Description (required)

Fixes #860 - Use stash to upload

This decouples (1) the uploading of the file content and (2) the fine-tuning of the file name. It's a first step before moving forward the idea of background uploading while the user entering the file name (this is possible because stash upload doesn't require a proper file name in the first part).

No UI changes included here, just changes in the backend part of the upload process. It seemed like a good idea to fix potential bugs before moving onto the next step (the forementioned background uploading).

Tests performed (required)

Tested ProdDebug on emulator with API level 24.

Screenshots showing what changed (optional - for UI changes)

No UI changes.

@codecov-io
Copy link

codecov-io commented Feb 22, 2019

Codecov Report

Merging #2505 into master will increase coverage by 3.17%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #2505      +/-   ##
=========================================
+ Coverage    2.69%   5.87%   +3.17%     
=========================================
  Files         258     259       +1     
  Lines       12266   12366     +100     
  Branches     1108    1112       +4     
=========================================
+ Hits          331     726     +395     
+ Misses      11909   11581     -328     
- Partials       26      59      +33
Impacted Files Coverage Δ
...in/java/fr/free/nrw/commons/mwapi/UploadStash.java 0% <0%> (ø)
...in/java/fr/free/nrw/commons/mwapi/CustomMwApi.java 7.52% <0%> (+7.52%) ⬆️
...java/fr/free/nrw/commons/upload/UploadService.java 0% <0%> (ø) ⬆️
...rw/commons/mwapi/ApacheHttpClientMediaWikiApi.java 3.86% <0%> (+3.86%) ⬆️
.../fr/free/nrw/commons/utils/StringSortingUtils.java 75% <0%> (-10.72%) ⬇️
...ree/nrw/commons/upload/ImageProcessingService.java 84.21% <0%> (-0.54%) ⬇️
...n/java/fr/free/nrw/commons/upload/UploadModel.java 31.81% <0%> (-0.03%) ⬇️
...ons/explore/categories/SearchCategoryFragment.java 0% <0%> (ø) ⬆️
...va/fr/free/nrw/commons/campaigns/CampaignView.java 0% <0%> (ø) ⬆️
...n/java/fr/free/nrw/commons/MediaDataExtractor.java 0% <0%> (ø) ⬆️
... and 85 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 561f9ea...7349dc3. Read the comment docs.

@commons-app commons-app deleted a comment Feb 24, 2019
@maskaravivek
Copy link
Member

@whym Is the PR ready for review or is it still WIP? :)

@whym
Copy link
Collaborator Author

whym commented Feb 27, 2019

@maskaravivek It's ready, as far as I'm concerned. Sorry about the confusing commit summaries - they should be removed when the commits are squashed.

@whym
Copy link
Collaborator Author

whym commented Mar 2, 2019

Conflict resolved and tested again with betaDebug.

@maskaravivek
Copy link
Member

@whym Can you add the test plan for the changes you made.

I tried picking one image from the gallery and I was expecting to see the upload begin(based on logcat) but it doesn't seem to happen.

@whym
Copy link
Collaborator Author

whym commented Mar 6, 2019

Thank you for testing @maskaravivek. I tried uploading from camera last time, but when I tried it today it got stuck at "finishing uploading", and here was the log:

2019-03-06 12:34:59.857 4961-5484/fr.free.nrw.commons.beta D/CustomApiResult: API response for method uploadToStash is
See https://commons.wikimedia.beta.wmflabs.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&amp;gt; for notice of API deprecations and breaking changes.
2019-03-06 12:34:59.858 4961-5484/fr.free.nrw.commons.beta E/ApacheHttpClientMediaWikiApi: Result: fr.free.nrw.commons.mwapi.CustomApiResult@c761595
2019-03-06 12:34:59.868 4961-5484/fr.free.nrw.commons.beta E/ApacheHttpClientMediaWikiApi: uploadstash-exception
2019-03-06 12:34:59.869 4961-5490/fr.free.nrw.commons.beta D/UploadService: Stash upload response 1 is UploadStash(errorCode=uploadstash-exception, resultStatus=, filename=Temp_254220987CommonsAppTest.jpg, filekey=)
2019-03-06 12:34:59.869 4961-5490/fr.free.nrw.commons.beta D/UploadService: Contribution upload failed. Wikidata entity won't be edited

It sounds like a server side probem ("Could not connect to storage backend"), hopefully a temporary one, but it's also possible that I might have introduced bugs during when I resolved conflicts with recent changes in master (which were a bit complex). Did you get the same error?

I tried prodDebug and upload attempts went through with this. Prod uses a different (and more stable) server. Before today's testing, I included latest master (again) using "git rebase master" into this branch.

@maskaravivek
Copy link
Member

For me, the upload wasn't initiated at all. Let me try again today with both camera and gallery.

@whym
Copy link
Collaborator Author

whym commented Mar 11, 2019

I tested with two real devices I have (API 27 and API 23). In the end both worked, but I had some (non-reproducible) issues.

  • API 27 device: in one attempt among three, the app crashed somewhere around the end of the upload process.
  • API 23 device: the app halted right after starting the upload process, and I had to clean the app's cache to make it work. However, I cannot reproduce the issue after cleaning the cache.

The tests were done for both the Camera and Gallery methods.

@whym
Copy link
Collaborator Author

whym commented Mar 13, 2019

Resolved conflicts with recent changes to master.

  • API 27, real device: Uploade from gallery succeeded
  • API 24, emulator: Upload from camera succeeded

- split upload process and use stash
- resolve filename conflict after upload not before
- use NotificationManagerCompat; add notification tag; assign temporaty stash file name
@whym
Copy link
Collaborator Author

whym commented Mar 17, 2019

Conflicts resolved again. (Updates for AndroidX #2594)

It passed the same (manual) tests as above.

@maskaravivek
Copy link
Member

I tested it once again and the upload worked for me.

But I am not sure about the testing steps. This is what I did:

  • picked an image from gallery
  • put in title/desc
  • choose category
  • select license

After these steps, the upload begins. Is this expected?

From the description, it seems that a stashed upload should begin in the background as soon as I pick the image and meanwhile I can fine-tune the title and other details. So I was expecting upload to begin as soon as I landed on upload activity.

@whym It would be great if you could briefly explain how to test the changes. It is sad that this PR has been waiting for so many days and you have already rebased the PR multiple times.

@whym
Copy link
Collaborator Author

whym commented Mar 18, 2019

From the description, it seems that a stashed upload should begin in the background as soon as I pick the image and meanwhile I can fine-tune the title and other details. So I was expecting upload to begin as soon as I landed on upload activity.

That is my intention, but most of the plan will be done in subsequent PR(s) I am still preparing. It seems like I should have been clearer - the expected behavior for now is to be able to upload pictures as normally as before, nothing more. There is no noticeable change for users, including the timing to start uploading. At this time, I just wanted to make sure we can upload stably with the stash API first. I'm glad that it worked for you.

The only change here is replacing the traditional upload API with two API calls - 1. API call to upload a file without a real title, 2. API call to give a name to the file. Both happen after the user finishes entering image details (for now).

@whym
Copy link
Collaborator Author

whym commented Mar 18, 2019

To think about it, the question is whether I should really ask for merging this, or I should combine it into the larger and more substantial change and resubmit. (This review allowed more tests than I did before, so I don't think it was wasted anyway.)

  1. Get this merged now and see if any unexpected issue with the stash API arises while I work on the background upload for a next PR.
  2. Abandon this and comeback later with a full background upload implementation.

I initially preferred separate PRs (=1) since this is something new to a core functionality of the app. But maybe merging this partial solution is too confusing?

@nicolas-raoul
Copy link
Member

nicolas-raoul commented Mar 18, 2019 via email

@maskaravivek
Copy link
Member

I am also in favor of merging it first. Am merging it so that alpha testers can report issues if any.

@maskaravivek maskaravivek merged commit ee7af37 into commons-app:master Mar 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use stash to upload
4 participants