Gutenberg/no jitpack updates to gutenberg 1.28 branch #11921

mchowning · 2020-05-14T17:19:40Z

Related gutenberg-mobile PR: wordpress-mobile/gutenberg-mobile#2264

Bringing the changes to not use jitpack for Gutenberg to develop.

To test

Insure you can build the app successfully under these three scenarios:

wp.BUILD_GUTENBERG_FROM_SOURCE	build flavor
true	wasabiDebug
false	wasabiDebug
false	vanillaRelease

Ensure that when wp.BUILD_GUTENBERG_FROM_SOURCE is false, changes to gutenberg's package.json file rerun the yarn task.
We should also ensure that rebuilding the wasabiDebug flavor when wp.BUILD_GUTENBERG_FROM_SOURCE is false is reasonably fast since that is the build that will be generally used for development by non-Gutenberg Android devs.
Lastly, we should insure that a clean regenerates the bundle:
1. With wp.BUILD_GUTENBERG_FROM_SOURCE set to false, run ./gradlew installWasabiDebug and verify that you can load the Gutenberg editor from resulting app
2. Run that build again and observe that it runs much faster (~10 seconds)
3. Run ./gradlew clean installWasabiDebug and verify that you can load the Gutenberg editor from the resulting app. This build should take longer again because clean deletes the bundle so the build should detect that and rebuild it.

PR submission checklist

I have considered adding unit tests where possible.
I have considered adding accessibility improvements for my changes.
I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

peril-wordpress-mobile · 2020-05-14T17:19:46Z

You can trigger optional UI/connected tests for these changes by visiting CircleCI here.

peril-wordpress-mobile · 2020-05-14T17:29:00Z

You can test the changes on this Pull Request by downloading the APK here.

mchowning · 2020-05-14T21:15:10Z

Added bf7215f on top of @hypest's original avoid-jitpack changes. This change has CircleCI cache each JS bundle it generates against a key based on the gutenberg-mobile submodule hash. This means that CircleCI only generates the JS bundle the first time it builds a given gutenberg-mobile submodule hash. On all subsequent builds, it just reuses the cached JS bundle.

This means that instead of the "gutenberg-bundle-build" job taking 2.5 minutes to generate a bundle every time, it now only does that the first time. For all subsequent builds where the gutenberg-mobile submodule hash has not changed, the "gutenberg-bundle-build" job only takes ~10 seconds to confirm that a cached JS bundle exists (for example: first build, subsequent build).

This change depends on the assumption we should never need to rebuild the JS bundle if the gutenberg-mobile submodule hash has not changed. Let me know if you can think of any scenario where that is not true.

SergioEstevao · 2020-05-15T09:40:19Z

When trying to build the wasabiDebug with no value set in BUILD_GUTENBERG_FROM_SOURCE I got this error

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':react-native-gutenberg-bridge:nodeSetup'.
> Could not list contents of '/var/folders/0m/lcxn02xn1rn_xy88_0mlj9hc0000gn/T/jsbundle/null/nodejs/node-v12.13.1-darwin-x64/bin/npm'. Couldn't follow symbolic link.

hypest · 2020-05-15T09:51:13Z

When trying to build the wasabiDebug with no value set in BUILD_GUTENBERG_FROM_SOURCE I got this error

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':react-native-gutenberg-bridge:nodeSetup'.
> Could not list contents of '/var/folders/0m/lcxn02xn1rn_xy88_0mlj9hc0000gn/T/jsbundle/null/nodejs/node-v12.13.1-darwin-x64/bin/npm'. Couldn't follow symbolic link.

I experience the same issue Sérgio encountered.

maxme · 2020-05-15T10:22:38Z

Right after cloing the repository (with its submodules):

# Using wp.BUILD_GUTENBERG_FROM_SOURCE=true
$ ./gradlew buildWasabiDebug
...
BUILD SUCCESSFUL in 1m 20s

# Using wp.BUILD_GUTENBERG_FROM_SOURCE=false
$ ./gradlew buildWasabiDebug
...
BUILD SUCCESSFUL in 1m 15s

# Using wp.BUILD_GUTENBERG_FROM_SOURCE=false
$ ./gradlew buildVanillaRelease
...
BUILD SUCCESSFUL in 1m 30s

# Unset wp.BUILD_GUTENBERG_FROM_SOURCE
$ ./gradlew buildWasabiDebug
...
BUILD SUCCESSFUL in 57s

mchowning · 2020-05-15T12:11:22Z

When trying to build the wasabiDebug with no value set in BUILD_GUTENBERG_FROM_SOURCE I got this error... Could not list contents of '/var/folders/0m/lcxn02xn1...

I experience the same issue Sérgio encountered.

Thanks @SergioEstevao and @hypest . I've seen that same kind of error 2 times myself. I haven't been able to recreate it though, so I was hoping it was just an issue with my local machine since I've been testing a lot of different things out lately with these builds.

I was able to fix it by deleting the referenced folders in var/folders/..... One time I had to do that two times because after the first failure I got the same error with a different folder. Nothing jumped out as weird or suspicious about those folders for me, but do either of you see anything suspicious?

I'm a bit stumped on this one, so let me know if anyone has any guesses about what might be the issue. Also, if anyone else encounters this please leave a comment indicating things like:

The error message, in particular the folder being referenced;
What build command you were using;
Whether there is anything unusual or suspicious about the folder the error is referencing on your machine
have you previously successfully built WPAndroid with the no-jitpack changes and BUILD_GUTENBERG_FROM_SOURCE set to false; and
If you have had previously had a successful no-jitpack build, what branch was that build from (most likely either the gutenberg/integrate_1.28.0 or gutenberg/embedded-build-no-jitpack branches).

UPDATE: Looks like there may be some promising things to try here. Investigating.

SergioEstevao · 2020-05-15T13:18:39Z

After deleting the following paths:

rm /var/folders/0m/lcxn02xn1rn_xy88_0mlj9hc0000gn/T/jsbundle/null/yarn/yarn-v1.10.1/bin/yarn

rm /var/folders/0m/lcxn02xn1rn_xy88_0mlj9hc0000gn/T/jsbundle/null/yarn/yarn-v1.10.1/bin/yarnpkg

rm /var/folders/0m/lcxn02xn1rn_xy88_0mlj9hc0000gn/T/jsbundle/null/nodejs/node-v12.13.1-darwin-x64/bin/npm

The build started to work

hypest · 2020-05-15T14:00:19Z

I'm currently trying the following diff on top of release/14.8 (commit 9d18612) that updates the node gradle plugin, and the Node/npm/Yarn versions (to what I have locally installed):

diff --git a/react-native-gutenberg-bridge/android/build.gradle b/react-native-gutenberg-bridge/android/build.gradle
index 033357a..671487e 100644
--- a/react-native-gutenberg-bridge/android/build.gradle
+++ b/react-native-gutenberg-bridge/android/build.gradle
@@ -20,7 +20,7 @@ buildscript {
         classpath 'com.github.dcendents:android-maven-gradle-plugin:2.1'

         if (buildGutenbergMobileJSBundle) {
-            classpath "com.moowork.gradle:gradle-node-plugin:1.2.0"
+            classpath "com.moowork.gradle:gradle-node-plugin:1.3.1"
         }
     }
 }
@@ -40,13 +40,13 @@ if (buildGutenbergMobileJSBundle) {

     node {
         // Version of node to use.
-        version = '12.13.1'
+        version = '12.16.1'

         // Version of npm to use.
-        npmVersion = '6.3.0'
+        npmVersion = '6.14.2'

         // Version of Yarn to use.
-        yarnVersion = '1.10.1'
+        yarnVersion = '1.22.4'

         // Base URL for fetching node distributions (change if you have a mirror).
         distBaseUrl = 'https://nodejs.org/dist'

Update: run No1 worked without having to delete any paths like above. Runs No2 and No3 also succeeded. On to trying the diff on this PR itself. Edit: run No1 with the diff on this PR succeeded. Will try a couple more. Succeeded as well, and were super fast at that 🎉.

Also, @SergioEstevao , can you try building again a couple of time (without changing or touching anyway) to see if the same symlink issue will occur anyway?

SergioEstevao · 2020-05-15T14:27:03Z

Also, @SergioEstevao , can you try building again a couple of time (without changing or touching anyway) to see if the same symlink issue will occur anyway?

After the first successful build all the follow up run without problems.

hypest · 2020-05-15T15:39:50Z

I tried the PR by removing the diff I shared above and the nodeSetup issue happens again for me. To me that shows that the diff consistently fixes it, at least on my end.

mchowning · 2020-05-15T15:57:59Z

Thanks for investigating @hypest !

I tried the PR by removing the diff I shared above and the nodeSetup issue happens again for me. To me that shows that the diff consistently fixes it, at least on my end.

That's certainly encouraging. I'm worried though that your change might only be fixing it because you're no longer looking for the same, problematic node artifacts since you changed the version numbers. I'm worried that whatever caused the initial artifacts to get messed up could still exist and eventually, maybe, mess up the new ones the same way. I'm obviously just speculating though. I certainly don't see any harm in making your change.

This comment seemed interesting since we have nested subprojects, but I haven't been able to investigate since I can't recreate the problem on my machine currently.

In the effort to fix the "Couldn't follow symbolic link." issue reported here: wordpress-mobile/WordPress-Android#11921 (comment)

cameronvoell

wp.BUILD_GUTENBERG_FROM_SOURCE	build flavor	success?
true	wasabiDebug	✅
false	wasabiDebug	✅
false	vanillaRelease	✅

In addition to successfully building the above three, I also ran the other tests in the description, and everything behaved as expected.

After a fresh clone of WPAndroid (and submodules) I saw 1m 22 seconds for the first installWasabiDebug and 9 seconds for the second installWasabiDebug. Looks good!

cameronvoell · 2020-05-15T20:04:45Z

build.gradle

+if (project.ext.buildGutenbergFromSource) {
+    def nodeModulesDir = new File('libs/gutenberg-mobile/node_modules')
+    if (!nodeModulesDir.exists()) {
+        def message = "Attempting to build Gutenberg from source without a libs/gutenberg-mobile/node_modules directory. \


definitely appreciate this message as a reminder! 🙏

Nothing like tripping over something multiple times yourself to make you think, "Hmmm, maybe I should give a helpful message here." 😆

In the effort to fix the "Couldn't follow symbolic link." issue reported here: wordpress-mobile/WordPress-Android#11921 (comment)

together

…is missing

wip CircleCI

cameronvoell · 2020-05-15T22:09:55Z

wp.BUILD_GUTENBERG_FROM_SOURCE build flavor success?
true wasabiDebug ✅
false wasabiDebug ✅
false vanillaRelease ✅
In addition to successfully building the above three, I also ran the other tests in the description, and everything behaved as expected.

After a fresh clone of WPAndroid (and submodules) I saw 1m 22 seconds for the first installWasabiDebug and 9 seconds for the second installWasabiDebug. Looks good!

Did another round of checks after retargeting and rebase with release branch. Looks good!

mchowning added the gutenberg-mobile label May 14, 2020

mchowning added this to the 14.9 milestone May 14, 2020

mchowning requested review from cameronvoell and hypest May 14, 2020 17:19

mchowning self-assigned this May 14, 2020

mchowning mentioned this pull request May 14, 2020

Improve avoid-jitpack workflow wordpress-mobile/gutenberg-mobile#2264

Merged

2 tasks

mchowning force-pushed the gutenberg/no-jitpack-to-develop branch 5 times, most recently from 7b59cb8 to bf7215f Compare May 14, 2020 20:58

mchowning marked this pull request as ready for review May 15, 2020 18:45

cameronvoell approved these changes May 15, 2020

View reviewed changes

mchowning changed the base branch from develop to gutenberg/integrate_release_1.28.0 May 15, 2020 20:45

mchowning changed the title ~~Gutenberg/no jitpack to develop~~ Gutenberg/no jitpack updates to gutenberg 1.28 branch May 15, 2020

mchowning force-pushed the gutenberg/no-jitpack-to-develop branch from 24e3dd3 to b82f10f Compare May 15, 2020 21:18

mchowning added 5 commits May 15, 2020 17:23

Insure BUILD_GUTENBERG_FROM_SOURCE and buildGutenbergMobileJSBundle work

7220511

together

Prevent CI from failing if directories already exist

23d5954

Clarify error message when building gb from source with node_modules …

599730e

…is missing

Use Circle CI cache to avoid rebuilding Gutenberg JS bundle

bdf3354

wip CircleCI

Update gutenberg ref with new JS bundle location

afb45aa

mchowning force-pushed the gutenberg/no-jitpack-to-develop branch from b82f10f to afb45aa Compare May 15, 2020 21:23

oguzkocer mentioned this pull request May 15, 2020

Merge 14.8 final to develop #11933

Merged

mchowning merged commit 469c837 into gutenberg/integrate_release_1.28.0 May 15, 2020

mchowning deleted the gutenberg/no-jitpack-to-develop branch May 15, 2020 22:25

hypest mentioned this pull request May 18, 2020

Gutenberg/embedded build no jitpack #11896

Closed

3 tasks

cameronvoell mentioned this pull request May 20, 2020

[Gutenberg] Enable build from monorepo #11956

Merged

3 tasks

Gutenberg/no jitpack updates to gutenberg 1.28 branch #11921

Gutenberg/no jitpack updates to gutenberg 1.28 branch #11921

Uh oh!

Conversation

mchowning commented May 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

To test

PR submission checklist

Uh oh!

peril-wordpress-mobile bot commented May 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

peril-wordpress-mobile bot commented May 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mchowning commented May 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SergioEstevao commented May 15, 2020

Uh oh!

hypest commented May 15, 2020

Uh oh!

maxme commented May 15, 2020

Uh oh!

mchowning commented May 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SergioEstevao commented May 15, 2020

Uh oh!

hypest commented May 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SergioEstevao commented May 15, 2020

Uh oh!

hypest commented May 15, 2020

Uh oh!

mchowning commented May 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cameronvoell left a comment

Choose a reason for hiding this comment

Uh oh!

cameronvoell May 15, 2020

Choose a reason for hiding this comment

Uh oh!

mchowning May 15, 2020

Choose a reason for hiding this comment

Uh oh!

cameronvoell commented May 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mchowning commented May 14, 2020 •

edited

Loading

peril-wordpress-mobile bot commented May 14, 2020 •

edited

Loading

peril-wordpress-mobile bot commented May 14, 2020 •

edited

Loading

mchowning commented May 14, 2020 •

edited

Loading

mchowning commented May 15, 2020 •

edited

Loading

hypest commented May 15, 2020 •

edited

Loading

mchowning commented May 15, 2020 •

edited

Loading