feat(react-native-github): a script to automate patch version bumping of packages #35767

hoxyq · 2023-01-03T11:21:14Z

Summary:
Changelog: [Internal]

Introducing a script, which can be used to identify all packages inside /packages, which contain any changes after the last time its version was changed

How it works step by step:

check that no git changes are present

for each package:
    if package is private -> skip

    grep id of the last commit that changed package
    grep id of the last commit that changed version of the package

    if these ids are different:
        bump package patch version

commit changes if required

Can be executed only in git environment and by running: node ./scripts/bump-all-updated-packages

Also adding a separate script align-package-versions.js, which can be used to update versions of packages inside consumer packages

check that no git changes are present

for each package x:
   for each package y:
       if y has x as dependency:
           validate that y uses the latest version of x

if some changes were made:
   run yarn

Q: Why run_yarn step was removed from CircleCI flow?
A: For *-stable branches, there are no yarn workspaces and all packages are specified as direct dependencies, so if we update @react-native/assets-registry to the next version, we won't be able to run yarn for react-native root package, because updated version is not yet published to npm

To avoid this, we first need publish new versions and then update them in consumer packages

The final flow:

Developer uses node ./scripts/bump-all-updated-packages to bump versions of all updated packages.
Commit created from step 1 being merged or directly pushed to main or *-stable branches
A workflow from CircleCI publishes all updated versions to npm
Developer can use align-package-versions.js script to create required changes to align all packages versions

Differential Revision: D42295344

facebook-github-bot · 2023-01-03T11:21:51Z

This pull request was exported from Phabricator. Differential Revision: D42295344

hoxyq · 2023-01-03T11:25:21Z

Demo:

Screen.Recording.2022-12-30.at.13.36.28.mov

analysis-bot · 2023-01-03T11:44:43Z

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	8,493,101	+0
android	hermes	armeabi-v7a	7,813,178	+0
android	hermes	x86	8,967,624	+0
android	hermes	x86_64	8,827,011	+0
android	jsc	arm64-v8a	9,679,094	+0
android	jsc	armeabi-v7a	8,412,803	+0
android	jsc	x86	9,742,206	+0
android	jsc	x86_64	10,220,829	+0

Base commit: 88a1b8e
Branch: main

analysis-bot · 2023-01-03T12:18:38Z

Platform	Engine	Arch	Size (bytes)	Diff
ios	-	universal	n/a	--

Base commit: 292268e
Branch: main

pull-bot · 2023-01-03T12:30:13Z

PR build artifact for aa26c5c is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

pull-bot · 2023-01-03T12:30:38Z

PR build artifact for aa26c5c is ready.
To use, download tarball from "Artifacts" tab in this CircleCI job then run yarn add <path to tarball> in your React Native project.

kelset

leaving a couple comments so that we can discuss them further, overall LGTM but one improvement I think is very important

scripts/monorepo/bump-all-updated-packages/index.js

kelset · 2023-01-04T12:36:45Z

scripts/monorepo/bump-all-updated-packages/bump-package-version.js

+  }
+};
+
+const bumpPackageVersion = (packageAbsolutePath, packageManifest) => {


this method seems "comprehensive" enough that should work also on -stable branches, but just to be sure, could you create a separate branch on top of 0.71 with the same script and verify that it works fine in there too? (usual difference of root package not being workspace, and repo-config's package.json and root package.json having different content VS when they are on main)

AFAIK it should just work fine but worth double checking

This is just utility function, I've extracted it to avoid keeping one file with 300+ lines. This is not supposed to work from cli (node ./scripts/monorepo/...)

If you find it useful, I could make it available through cli and make a separate PR against 0.71-stable branch

my point here is that I want to make sure that the overall script works fine even in a 0.XX-stable branch without us having to reach the point of cutting 0.72-stable and finding out only then if it works proper.

my point here is that I want to make sure that the overall script works fine even in a 0.XX-stable branch without us having to reach the point of cutting 0.72-stable and finding out only then if it works proper.

I've just tested it on 0.71-stable branch. Works as expected, except for yarn: root level package.json does not contain workspaces, so running yarn install fails after version bump, because packages with updated version is not yet published

I can remove this step at all (running yarn), what do you think?

except for yarn: root level package.json does not contain workspaces, so running yarn install fails after version bump, because packages with updated version is not yet published

that actually raises a good point! Will CI be fault-resistant enough to be able to get to the step of doing the npm publish for these new versions if root yarn install fail? I'm concerned CI will fail on like, one of the early steps of setting up the CI VMs because yarn install fails, so it will never get to publishing the new versions.

What do you think?

except for yarn: root level package.json does not contain workspaces, so running yarn install fails after version bump, because packages with updated version is not yet published

that actually raises a good point! Will CI be fault-resistant enough to be able to get to the step of doing the npm publish for these new versions if root yarn install fail? I'm concerned CI will fail on like, one of the early steps of setting up the CI VMs because yarn install fails, so it will never get to publishing the new versions.

What do you think?

Yes, it will fail. I can remove all external depedencies (like chalk and shelljs) from find-and-publish-all-bumped-packages script. After this, we can remove run_yarn step from CircleCI job.

We will lose fancy output, but this the only option I see if we want to make it work on *-stable branches.

Also there should be other CircleCI jobs for *-stable branches, which will require running yarn and will eventually fail. Should we skip these jobs for such bump commits?

Also there should be other CircleCI jobs for *-stable branches, which will require running yarn and will eventually fail. Should we skip these jobs for such bump commits?

I'm starting to think that splitting the steps 1-2 from 3->5 is probably even more important than expected; we need to update the yarn.lock after the new releases in all cases. So I am not sure "forcing" it all to happen in one go by skipping other jobs will make things ok since then we'll still have to do one more commit with the yarn.lock update.

Okay, I see. I can work on splitting this.

I am proposing this flow:

User runs script, which bumps all relevant packages and creates a commit with a specific message.

User creates PR with this commit and at some point this commit is merged.

We have automated CircleCI workflow, which runs on main and *-stable branches, which does the following:
3.1. publish each bumped package to npm
3.2 commit version bumps for each consumer packages, open PR against target branch of original commit (on top of which workflow was executed)

Do we have an examples of an automated processes where bot creates a PR?

about the proposed flow: I think step 2 is optional in a way, user could also just directly commit (if they are a Meta engineer and doing it in the internal monorepo, or a person with write access on a -stable branch); about step 3.2 are you thinking of 1 PR per each bumped package (ex. if I bumped @react-native/codegen and @react-native/gradle-plugin, will it open 1 or 2 PRs)?

Do we have an examples of an automated processes where bot creates a PR?

There are many like https://github.com/dependabot, if I'm understanding your question (so probably https://github.com/dependabot/dependabot-core is what you want to look into)

are you thinking of 1 PR per each bumped package (ex. if I bumped @react-native/codegen and @react-native/gradle-plugin, will it open 1 or 2 PRs)?

No, I am thinking about opening a PR which contains updated versions for each of the packages, which were bumped in previous commit and then published

I will research about dependabot, but I think we might need something more simple, just to create PR with bumped versions

facebook-github-bot · 2023-01-05T17:19:54Z

This pull request was exported from Phabricator. Differential Revision: D42295344

… of packages (facebook#35767) Summary: Pull Request resolved: facebook#35767 Changelog: [Internal] Introducing a script, which can be used to identify all packages inside `/packages`, which contain any changes after the last time its version was changed How it works step by step: ``` check that no git changes are present for each package: if package is private -> skip grep id of the last commit that changed package grep id of the last commit that changed version of the package if these ids are different: bump package patch version for each package: check if this package is used and update its version run yarn if required commit changes if required ``` Can be executed only in git environment and by running: `node ./scripts/bump-all-updated-packages` Reviewed By: cortinico Differential Revision: D42295344 fbshipit-source-id: c6f17cdd480984d31fdbcad1b4379cfae915fd35

… of packages (facebook#35767) Summary: Pull Request resolved: facebook#35767 Changelog: [Internal] Introducing a script, which can be used to identify all packages inside `/packages`, which contain any changes after the last time its version was changed How it works step by step: ``` check that no git changes are present for each package: if package is private -> skip grep id of the last commit that changed package grep id of the last commit that changed version of the package if these ids are different: bump package patch version for each package: check if this package is used and update its version run yarn if required commit changes if required ``` Can be executed only in git environment and by running: `node ./scripts/bump-all-updated-packages` Reviewed By: cortinico Differential Revision: D42295344 fbshipit-source-id: 326b1b28f20f21252154bf9f3ebf42d71de9112b

facebook-github-bot · 2023-01-06T11:20:53Z

This pull request was exported from Phabricator. Differential Revision: D42295344

kelset · 2023-01-06T11:27:52Z

scripts/monorepo/find-and-publish-all-bumped-packages.js

+        throw new Error(
+          `Package ${packageManifest.name} was updated, but not through CI script`,
+        );
+      }


great error 👏

scripts/monorepo/bump-all-updated-packages/bump-package-version.js

hoxyq · 2023-01-10T12:09:33Z

Current demo:

Screen.Recording.2023-01-05.at.16.35.21.mov

I am now working on splitting this into two distinct scipts: one for version bumps (which will be used to publish packages to npm) and other to bump version in consumer packages. Will update demo after that

… of packages (facebook#35767) Summary: Pull Request resolved: facebook#35767 Changelog: [Internal] Introducing a script, which can be used to identify all packages inside `/packages`, which contain any changes after the last time its version was changed How it works step by step: ``` check that no git changes are present for each package: if package is private -> skip grep id of the last commit that changed package grep id of the last commit that changed version of the package if these ids are different: bump package patch version commit changes if required ``` Can be executed only in git environment and by running: `node ./scripts/bump-all-updated-packages` --- Also adding a separate script `align-package-versions.js`, which can be used to update versions of packages inside consumer packages ``` check that no git changes are present for each package x: for each package y: if y has x as dependency: validate that y uses the latest version of x if some changes were made: run yarn ``` --- Q: Why `run_yarn` step was removed from CircleCI flow? A: For *-stable branches, there are no yarn workspaces and all packages are specified as direct dependencies, so if we update `react-native/assets-registry` to the next version, we won't be able to run `yarn` for react-native root package, because updated version is not yet published to npm To avoid this, we first need publish new versions and then update them in consumer packages --- The final flow: 1. Developer uses `node ./scripts/bump-all-updated-packages` to bump versions of all updated packages. 2. Commit created from step 1 being merged or directly pushed to `main` or `*-stable` branches 3. A workflow from CircleCI publishes all updated versions to npm 4. Developer can use `align-package-versions.js` script to create required changes to align all packages versions Reviewed By: cortinico Differential Revision: D42295344 fbshipit-source-id: 1a4bbe33a996a93af37a6642ed4a11341066e6ee

facebook-github-bot · 2023-01-10T17:20:58Z

This pull request was exported from Phabricator. Differential Revision: D42295344

hoxyq · 2023-01-10T17:21:46Z

bump-all-updated-packages demo:

Screen.Recording.2023-01-10.at.16.12.35.mov

align-package-versions demo:

Screen.Recording.2023-01-10.at.16.14.03.mov

kelset

I think now this is ready to

kelset · 2023-01-10T17:43:48Z

scripts/monorepo/align-package-versions.js

+        packageManifest.version,
+      ),
+    );
+  });


not a super big fan of this double forEach which feels a bit like brute forcing checking everything against everything but honestly hey at least we'll make sure that everything is correctly aligned!

And I think this will work out of the box with -stable branches too so 👍

kelset · 2023-01-10T17:45:11Z

one last thing: can you add the two new scripts to root package.json in the scripts section? so that folks won't have to invoke node directly?

hoxyq · 2023-01-10T17:57:31Z

one last thing: can you add the two new scripts to root package.json in the scripts section? so that folks won't have to invoke node directly?

Sure, I will add this. I will also update our new wiki page with relevant information

facebook-github-bot · 2023-01-10T21:25:28Z

This pull request has been merged in ec28c5b.

… of packages (facebook#35767) Summary: Pull Request resolved: facebook#35767 Changelog: [Internal] Introducing a script, which can be used to identify all packages inside `/packages`, which contain any changes after the last time its version was changed How it works step by step: ``` check that no git changes are present for each package: if package is private -> skip grep id of the last commit that changed package grep id of the last commit that changed version of the package if these ids are different: bump package patch version commit changes if required ``` Can be executed only in git environment and by running: `node ./scripts/bump-all-updated-packages` --- Also adding a separate script `align-package-versions.js`, which can be used to update versions of packages inside consumer packages ``` check that no git changes are present for each package x: for each package y: if y has x as dependency: validate that y uses the latest version of x if some changes were made: run yarn ``` --- Q: Why `run_yarn` step was removed from CircleCI flow? A: For *-stable branches, there are no yarn workspaces and all packages are specified as direct dependencies, so if we update `react-native/assets-registry` to the next version, we won't be able to run `yarn` for react-native root package, because updated version is not yet published to npm To avoid this, we first need publish new versions and then update them in consumer packages --- The final flow: 1. Developer uses `node ./scripts/bump-all-updated-packages` to bump versions of all updated packages. 2. Commit created from step 1 being merged or directly pushed to `main` or `*-stable` branches 3. A workflow from CircleCI publishes all updated versions to npm 4. Developer can use `align-package-versions.js` script to create required changes to align all packages versions Reviewed By: cortinico Differential Revision: D42295344 fbshipit-source-id: 54b667adb3ee5f28d19ee9c7991570451549aac2

facebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner fb-exported labels Jan 3, 2023

kelset suggested changes Jan 4, 2023

View reviewed changes

hoxyq force-pushed the export-D42295344 branch from aa26c5c to bf7af96 Compare January 5, 2023 17:19

hoxyq force-pushed the export-D42295344 branch from bf7af96 to fca5d04 Compare January 6, 2023 11:20

kelset reviewed Jan 6, 2023

View reviewed changes

tido64 reviewed Jan 10, 2023

View reviewed changes

scripts/monorepo/bump-all-updated-packages/bump-package-version.js Outdated Show resolved Hide resolved

scripts/monorepo/bump-all-updated-packages/bump-package-version.js Show resolved Hide resolved

hoxyq force-pushed the export-D42295344 branch from fca5d04 to 104a05f Compare January 10, 2023 17:21

kelset approved these changes Jan 10, 2023

View reviewed changes

facebook-github-bot closed this in ec28c5b Jan 10, 2023

facebook-github-bot added the Merged This PR has been merged. label Jan 10, 2023

kelset mentioned this pull request Jan 11, 2023

RFC: React Native as a monorepo react-native-community/discussions-and-proposals#480

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(react-native-github): a script to automate patch version bumping of packages #35767

feat(react-native-github): a script to automate patch version bumping of packages #35767

hoxyq commented Jan 3, 2023 •

edited

Loading

facebook-github-bot commented Jan 3, 2023

hoxyq commented Jan 3, 2023

analysis-bot commented Jan 3, 2023 •

edited

Loading

analysis-bot commented Jan 3, 2023

pull-bot commented Jan 3, 2023

pull-bot commented Jan 3, 2023

kelset left a comment

kelset Jan 4, 2023

hoxyq Jan 5, 2023

kelset Jan 5, 2023

hoxyq Jan 5, 2023 •

edited

Loading

kelset Jan 6, 2023

hoxyq Jan 6, 2023

kelset Jan 6, 2023

hoxyq Jan 6, 2023 •

edited

Loading

kelset Jan 6, 2023

hoxyq Jan 6, 2023

facebook-github-bot commented Jan 5, 2023

facebook-github-bot commented Jan 6, 2023

kelset Jan 6, 2023

hoxyq commented Jan 10, 2023

facebook-github-bot commented Jan 10, 2023

hoxyq commented Jan 10, 2023

kelset left a comment

kelset Jan 10, 2023

kelset commented Jan 10, 2023

hoxyq commented Jan 10, 2023

facebook-github-bot commented Jan 10, 2023

feat(react-native-github): a script to automate patch version bumping of packages #35767

feat(react-native-github): a script to automate patch version bumping of packages #35767

Conversation

hoxyq commented Jan 3, 2023 • edited Loading

facebook-github-bot commented Jan 3, 2023

hoxyq commented Jan 3, 2023

analysis-bot commented Jan 3, 2023 • edited Loading

analysis-bot commented Jan 3, 2023

pull-bot commented Jan 3, 2023

pull-bot commented Jan 3, 2023

kelset left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hoxyq Jan 5, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hoxyq Jan 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

facebook-github-bot commented Jan 5, 2023

facebook-github-bot commented Jan 6, 2023

Choose a reason for hiding this comment

hoxyq commented Jan 10, 2023

facebook-github-bot commented Jan 10, 2023

hoxyq commented Jan 10, 2023

kelset left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kelset commented Jan 10, 2023

hoxyq commented Jan 10, 2023

facebook-github-bot commented Jan 10, 2023

hoxyq commented Jan 3, 2023 •

edited

Loading

analysis-bot commented Jan 3, 2023 •

edited

Loading

hoxyq Jan 5, 2023 •

edited

Loading

hoxyq Jan 6, 2023 •

edited

Loading