Use gradle configure plugin
#3661
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jkmassel
added
the
category: tooling
jkmassel
force-pushed
the
use/binary-configure
branch
5 times, most recently
from
f7a11f2 to
472b9ff
Compare
|
You can test the changes on this Pull Request by downloading the APK here. |
AliSoftware
reviewed
Mar 5, 2021
| @@ -1,45 +1,35 @@ | |||
| { | |||
| "project_name": "woocommerce-android", | |||
| "project_name": "wc-android", | |||
There was a problem hiding this comment.
I have to admit I don't know what this key is used for in the configure tooling… but why this change?
There was a problem hiding this comment.
This key defines which encryption key from keys.json to use – it needs to be changed here in order to avoid breaking mobile secrets while I do this work.
At some point after this PR lands (once any branches using the old project_name land, we can roll it back to the proper name if we like
jkmassel
force-pushed
the
use/binary-configure
branch
4 times, most recently
from
cb7a7cf to
697c47b
Compare
jkmassel
force-pushed
the
use/binary-configure
branch
from
b804587 to
75a7c88
Compare
AliSoftware
requested changes
Mar 8, 2021
There was a problem hiding this comment.
Tried this today and on the first try I was able to make rust be "panicked"…
10:56:58 [DEBUG] (1) configure::fs: Reading keys from "/Users/olivier/.mobile-secrets/keys.json"
thread 'main' panicked at 'Unable to decrypt and copy files: MissingProjectKey', src/configure.rs:193:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
I quickly figured that this was due to my mobile-secrets not being up-to-date and not having pulled the latest keys (though a more explicit error message could have been nice), so doing a git pull in ~/.mobile-secrets then re-trying finally made the task pass…
But then when I looked at .configure-files, only one file (upload.jks) were un-encrypted, all the other files were .enc files (despite the logs saying that it stored other decrypted files during the process). So seems like it didn't work all the way?
$ ls -l .configure-files
total 64
-rw-r--r-- 1 olivier staff 2144 Mar 8 11:47 automattic_upload.jks.enc
-rw-r--r-- 1 olivier staff 1264 Mar 8 11:47 debug.keystore.enc
-rw-r--r-- 1 olivier staff 3903 Mar 8 11:48 google-services.json.enc
-rw-r--r-- 1 olivier staff 2402 Mar 8 11:48 google-upload-credentials.json.enc
-rw-r--r-- 1 olivier staff 2123 Mar 8 11:48 gradle.properties.enc
-rw-r--r-- 1 olivier staff 170 Mar 8 11:48 sentry.properties.enc
-rw-r--r-- 1 olivier staff 2142 Mar 8 12:02 upload.jks
-rw-r--r-- 1 olivier staff 2182 Mar 8 11:48 upload.jks.enc
I also think it could be nice to improve a couple of things here:
- Probably reduce the verbosity of the output, as it's printing a lot of
[DEBUG] (1) configure:…lines when running that could make it a bit scary for end-users. Maybe only print those debug lines ifDEBUG=1env var is set or something like that? - Would be very nice to have a better error message when the decryption key is not found, to suggest users to update their
mobile-secretsrepo and retry.
Log Details
Step 1: Fresh clone the repo.
$ git clone git@github.com:woocommerce/woocommerce-android.git woocommerce-android-clean
…
$ cd woocommerce-android-clean
$ git checkout use/binary-configure
Step 2: ./gradlew applyConfiguration fail as expected.
$ ./gradlew applyConfiguration
Starting a Gradle Daemon (subsequent builds will be faster)
> Configure project :
Copying git-hooks scripts from tools/team-props/git-hooks to .git/hooks
Checking whether `configure` binary is present
Downloading `configure` binary
Detected current OS: macos
Detected plugin version: 0.5.0
200
OK
FAILURE: Build failed with an exception.
* Where:
Build file '/Users/olivier/Documents/Dev/woocommerce-android-clean/WooCommerce/build.gradle' line: 347
* What went wrong:
A problem occurred evaluating project ':WooCommerce'.
> Build configuration file gradle.properties doesn't exist, follow README instructions
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 8s
Step 3: Creating gradle.properties then re-running applyConfiguration: panic!
$ cp gradle.properties-example gradle.properties
$ ./gradlew applyConfiguration
Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use --status for details
> Configure project :
Copying git-hooks scripts from tools/team-props/git-hooks to .git/hooks
> Configure project :WooCommerce
WARNING: You're using the example google-services.json file. Google login will fail.
WARNING: You're using the example google-services.json file. Google login will fail.
Could not find google-services.json while looking in [src/wasabi/debug, src/debug/wasabi, src/wasabi, src/debug, src/wasabiDebug]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/jalapeno/debug, src/debug/jalapeno, src/jalapeno, src/debug, src/jalapenoDebug]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/vanilla/debug, src/debug/vanilla, src/vanilla, src/debug, src/vanillaDebug]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/wasabi/release, src/release/wasabi, src/wasabi, src/release, src/wasabiRelease]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/jalapeno/release, src/release/jalapeno, src/jalapeno, src/release, src/jalapenoRelease]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/vanilla/release, src/release/vanilla, src/vanilla, src/release, src/vanillaRelease]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
WARNING: API 'variant.getMappingFile()' is obsolete and has been replaced with 'variant.getMappingFileProvider()'.
It will be removed in version 5.0 of the Android Gradle plugin.
For more information, see https://d.android.com/r/tools/task-configuration-avoidance.
To determine what is calling variant.getMappingFile(), use -Pandroid.debug.obsoleteApi=true on the command line to display more information.
> Configure project :
Checking whether `configure` binary is present
> Task :applyConfiguration FAILED
10:56:58 [DEBUG] (1) configure: libconfigure initialized
10:56:58 [DEBUG] (1) configure: libConfigure initializing encryption
10:56:58 [DEBUG] (1) configure: libConfigure encryption initialization successful
10:56:58 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
10:56:58 [DEBUG] (1) configure::fs: Configure file found at: "/Users/olivier/Documents/Dev/woocommerce-android-clean/.configure"
10:56:58 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
10:56:58 [DEBUG] (1) configure::fs: Keys file found at: "/Users/olivier/.mobile-secrets/keys.json"
10:56:58 [DEBUG] (1) configure::fs: Reading keys from "/Users/olivier/.mobile-secrets/keys.json"
thread 'main' panicked at 'Unable to decrypt and copy files: MissingProjectKey', src/configure.rs:193:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':applyConfiguration'.
> Process 'command '/Users/olivier/Documents/Dev/woocommerce-android-clean/vendor/configure/configure'' finished with non-zero exit value 101
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 7s
1 actionable task: 1 executed
Step 4: Updating ~/mobile-secrets and re-running applyConfiguration: task succeeds...
$ cd ~/.mobile-secrets
$ git pull
…
$ cd -
$ ./gradlew applyConfiguration
> Configure project :WooCommerce
WARNING: You're using the example google-services.json file. Google login will fail.
WARNING: You're using the example google-services.json file. Google login will fail.
Could not find google-services.json while looking in [src/wasabi/debug, src/debug/wasabi, src/wasabi, src/debug, src/wasabiDebug]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/jalapeno/debug, src/debug/jalapeno, src/jalapeno, src/debug, src/jalapenoDebug]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/vanilla/debug, src/debug/vanilla, src/vanilla, src/debug, src/vanillaDebug]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/wasabi/release, src/release/wasabi, src/wasabi, src/release, src/wasabiRelease]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/jalapeno/release, src/release/jalapeno, src/jalapeno, src/release, src/jalapenoRelease]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
Could not find google-services.json while looking in [src/vanilla/release, src/release/vanilla, src/vanilla, src/release, src/vanillaRelease]
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
WARNING: API 'variant.getMappingFile()' is obsolete and has been replaced with 'variant.getMappingFileProvider()'.
It will be removed in version 5.0 of the Android Gradle plugin.
For more information, see https://d.android.com/r/tools/task-configuration-avoidance.
To determine what is calling variant.getMappingFile(), use -Pandroid.debug.obsoleteApi=true on the command line to display more information.
> Configure project :
Copying git-hooks scripts from tools/team-props/git-hooks to .git/hooks
Checking whether `configure` binary is present
> Task :applyConfiguration
11:02:17 [DEBUG] (1) configure: libconfigure initialized
11:02:17 [DEBUG] (1) configure: libConfigure initializing encryption
11:02:17 [DEBUG] (1) configure: libConfigure encryption initialization successful
11:02:17 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
11:02:17 [DEBUG] (1) configure::fs: Configure file found at: "/Users/olivier/Documents/Dev/woocommerce-android-clean/.configure"
11:02:17 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
11:02:17 [DEBUG] (1) configure::fs: Keys file found at: "/Users/olivier/.mobile-secrets/keys.json"
11:02:17 [DEBUG] (1) configure::fs: Reading keys from "/Users/olivier/.mobile-secrets/keys.json"
11:02:17 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
11:02:17 [DEBUG] (1) configure::fs: "/Users/olivier/Documents/Dev/woocommerce-android-clean/gradle.properties" already exists – making a backup at "/Users/olivier/Documents/Dev/woocommerce-android-clean/gradle-2021-03-08-11-02-17.properties.bak"
11:02:17 [DEBUG] (1) configure::fs: Encrypting file at "/Users/olivier/Documents/Dev/woocommerce-android-clean/.configure-files/gradle.properties.enc" and storing contents at "/Users/olivier/Documents/Dev/woocommerce-android-clean/gradle.properties"
11:02:17 [DEBUG] (1) configure::fs: Original File Hash: Ok("7e29gZCJ4n8ShY7NVNm8H5pTbywfUK6ZyS6GFdjM2zQ=")
11:02:17 [DEBUG] (1) configure::fs: New File hash: Ok("KwZ2aI7uyjloXi3kvdh4bsztdX0pP01/N9zME7reNLY=")
11:02:17 [DEBUG] (1) configure::fs: Keeping backup file because it differs from the original
11:02:17 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
11:02:17 [DEBUG] (1) configure::fs: "/Users/olivier/Documents/Dev/woocommerce-android-clean/WooCommerce/google-services.json" already exists – making a backup at "/Users/olivier/Documents/Dev/woocommerce-android-clean/WooCommerce/google-services-2021-03-08-11-02-17.json.bak"
11:02:17 [DEBUG] (1) configure::fs: Encrypting file at "/Users/olivier/Documents/Dev/woocommerce-android-clean/.configure-files/google-services.json.enc" and storing contents at "/Users/olivier/Documents/Dev/woocommerce-android-clean/WooCommerce/google-services.json"
11:02:17 [DEBUG] (1) configure::fs: Original File Hash: Ok("9wyjUznr4ktWFCnVSIHFrzPV7nJdk0ydRI39Fry1eS8=")
11:02:17 [DEBUG] (1) configure::fs: New File hash: Ok("RDY9r9jRjWh2CDME2u4cyCAzb6fKzncHPvRviTrXiX4=")
11:02:17 [DEBUG] (1) configure::fs: Keeping backup file because it differs from the original
11:02:17 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
11:02:17 [DEBUG] (1) configure::fs: Encrypting file at "/Users/olivier/Documents/Dev/woocommerce-android-clean/.configure-files/sentry.properties.enc" and storing contents at "/Users/olivier/Documents/Dev/woocommerce-android-clean/sentry.properties"
11:02:17 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
11:02:17 [DEBUG] (1) configure::fs: Encrypting file at "/Users/olivier/Documents/Dev/woocommerce-android-clean/.configure-files/upload.jks.enc" and storing contents at "/Users/olivier/Documents/Dev/woocommerce-android-clean/.configure-files/upload.jks"
11:02:17 [DEBUG] (1) configure::fs: Discovered Repository at "/Users/olivier/Documents/Dev/woocommerce-android-clean"
11:02:17 [DEBUG] (1) configure::fs: Encrypting file at "/Users/olivier/Documents/Dev/woocommerce-android-clean/.configure-files/google-upload-credentials.json.enc" and storing contents at "/Users/olivier/Documents/Dev/woocommerce-android-clean/google-upload-credentials.json"
11:02:17 [DEBUG] (1) configure::configure: All Files Copied!
11:02:17 [ INFO] Done
BUILD SUCCESSFUL in 987ms
1 actionable task: 1 executed
… but only one decrypted file in .configure-files 😞
$ ls -l .configure-files
total 64
-rw-r--r-- 1 olivier staff 2144 Mar 8 11:47 automattic_upload.jks.enc
-rw-r--r-- 1 olivier staff 1264 Mar 8 11:47 debug.keystore.enc
-rw-r--r-- 1 olivier staff 3903 Mar 8 11:48 google-services.json.enc
-rw-r--r-- 1 olivier staff 2402 Mar 8 11:48 google-upload-credentials.json.enc
-rw-r--r-- 1 olivier staff 2123 Mar 8 11:48 gradle.properties.enc
-rw-r--r-- 1 olivier staff 170 Mar 8 11:48 sentry.properties.enc
-rw-r--r-- 1 olivier staff 2142 Mar 8 12:02 upload.jks
-rw-r--r-- 1 olivier staff 2182 Mar 8 11:48 upload.jks.enc
PS: I tried to git clean -dxn the working copy, even rm -rf .configure-files then git restore .configure-files, then re-run ./gradlew applyConfiguration again, and got the same result (= upload.jks being the only unencrypted file present in .configure-files after that).
|
@jkmassel any luck on making progress on the issues highlighted above? Wondering if I could get this tested during next code freeze on Monday 🤔 |
|
@jkmassel: moved the milestone of this PR once again (since I'm doing the code freeze of 6.4 today). |
|
Another sprint, another milestone move 😅 (We'll get there! 🤞 ) |
malinajirka
requested review from
malinajirka
and removed request for
malinajirka
|
@jkmassel Should we move this to a draft status? |
|
I'll allow myself to close this PR - it updates CircleCI config that we don't use anymore. Tasks removed here are also discussed in #12934 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR uses the gradle
configureplugin instead ofbundler, which should remove the need for a ruby toolchain for day-to-day development of WCAndroid.To Test:
./gradlew applyConfiguration. Note that it fails with the error "Build configuration file gradle.properties doesn't exist, follow README instructions". Copygradle.properties-exampletogradle.properties, then run./gradlew applyConfiguration. This should replace the default values with the proper ones.Note: @oguzkocer and I discussed having a more automatic solution that doesn't require duplicating the template file, but there are a lot of technical hurdles to overcome there, so for now this is likely the simplest solution for both OSS and internal contributors.
Update release notes:
RELEASE-NOTES.txtif necessary.