Repository navigation
Conversation
…smaller documents
Member
|
Thanks for this, I had this on my agenda as an alternative for #3713 (which is why I stopped considering it for 0.3.17) as I think it is much better to bundle our own implementation rather than loading Google's proprietary one as a module. I'm currently rather busy, so I'll do a more thorough review at a later point |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Apps using the ML Kit Document Scanner API (
GmsDocumentScanning), such as Google Drive, currently show"Update Google Play services – This update is needed to use scanner" on microG.
Fixes #3228, fixes #3587. Related: #3078, #2018.
Cause
ModuleInstall.installModulesformlkit.docscan.ui,mlkit.docscan.shadowandmlkit.docscan.stain; microG answeredStatus.CANCELED, so the client shows the update dialog.com.google.android.gms.mlkit.ACTION_SCAN_DOCUMENTincom.google.android.gms,for which microG had no activity.
uri_array_extra_result_image_uris,uri_extra_result_pdf_uri,int_extra_result_page_count); the client copies the files and buildsGmsDocumentScanningResultitself.Logcat before (Google Drive 2.26.387, microG 0.3.17.252432-hw)
Changes
Open-source implementation building on @DaVinci9196's earlier work: it follows the activity pattern and uses
the CameraX dependency from
BarcodeScanningActivity(#2787), and the OpenCV dependency from the facedetection module (#2793). No proprietary modules or ML models are downloaded and no new dependencies are added.
The activity implements the public ML Kit Document Scanner contract rather than anything specific to Drive, so
any app using
play-services-mlkit-document-scannerworks without changes (the test client below is one suchapp). The edge detection, perspective correction, filters and PDF writer in
org.microg.gms.vision.documentdo not depend on the activity and can be reused by other features.
ACTION_SCAN_DOCUMENTresolves; other requests are unchanged.DocumentScanningActivityhandles the request and returns JPEG pages and a PDF viaFileProviderwith read grants. Cancel returnsRESULT_CANCELED.org.microg.gms.vision.document.Scanner features
The scanner UI and camera are implemented in microG itself. Options sent by the caller are respected:
documents and documents taking up only a small part of the frame
int_extra_default_capture_modeboolean_extra_flash_mode_change_allowedis setstring_extra_camera_id, falling back to the default back camerawith a magnifier, plus Automatic crop / No crop / Rotate, then keep or retake
boolean_extra_filter_allowed): Original, Auto, Grayscale, Black & white. Auto is the defaultwhen
boolean_extra_enable_auto_enhancementsis setboolean_extra_shadow_removal_allowed/boolean_extra_stain_removal_allowed):background estimation and normalization with classical OpenCV operations, not Google's ML models, so
it evens out shadows and light stains but not heavy stains
int_extra_page_limit_maxboolean_extra_gallery_import_allowedis set, with automatic edge detectionand EXIF orientation applied
int_array_extra_result_formats: JPEG per page and a PDF with the JPEG pagesembedded directly (about the size of the JPEGs combined)
Not implemented: the page edit callback (
boolean_extra_page_edit_listener_enabled/bundle_binder_extra_callbacks), as its binder interface is unknown, and image hashes for gallery imports(Drive sets
boolean_extra_enable_compute_hash_for_gallery_imageto false).Testing
A debug-signed build cannot use signature spoofing on EMUI, so I could not sign in to Drive with it.
I tested with a minimal client: https://github.com/grenadin/mlkit-docscan-test
It uses the official ML Kit library
play-services-mlkit-document-scanneras described in theML Kit documentation and the official sample
googlesamples/mlkit/android/documentscanner.
Device: Huawei MatePad (MLR-W09), Android 12, huawei flavor
string_extra_camera_id=1→ front camera; unknown id → back cameraRESULT_CANCELEDKnown limitations: edge detection may fail on white or grey backgrounds and when a corner is outside the
frame (corners can be adjusted manually); stain removal is weaker than Google's ML-based one; not tested
inside Google Drive itself.
Note
#3713 also handles
ACTION_SCAN_DOCUMENT, using Google's scanner module. Both PRs register the same action,so only one entry point can be active if both are merged. I'm happy to adjust this PR to fit whichever
direction is preferred.
This is my largest code contribution to microG so far. I've been following and learning from the project for
about four years, but I'm sure there are conventions I haven't picked up yet, so any feedback on the code or on
how things are usually done here is very welcome.