Managing floating V8 patches

[natorion]

As discussed in a previous LTS WG meeting, the V8 team would like help the LTS effort. One concrete area where we could help is how V8 patches are being managed on the LTS (and other) branches.

Node.js currently is maintaining patches in their copy of V8. This copy of V8 is located in a subdirectory without any reference to the imported V8 commit. On top of this import Node.js maintains floating patches in order to make Node.js work with V8 or add more bug fixes and features. The important point is that those floating patches are not in an official V8 branch in the V8 repo.

It would be beneficial for Node.js to have a solution to clearly identify those floating patches from the V8 base import. Additionally it would be helpful to be able to keep track of which patches got merged into which branch.

Some of them are pulled into V8 as long as the branch is still active on the V8 side. It would be beneficial if Node.js didn't have to maintain their in-tree V8 fork anymore but push all the patches upstream into a V8 repo.

The Node.js team at Google outlined two potential solutions. Feel free to add your own ideas!

Maintain all the floating patches in the V8 repository on Chrome's infrastructure

A prerequisite is that the Node.js build is able to pull in a specific V8 commit/ref. This is similar to V8's DEPS format.

This means that V8 creates separate release branches for Node.js in V8's repository: e.g. node-master, node-current, node-lts, node-maintenance. Node is then going to directly import the HEAD of each branch into their respective Node.js branch.

Node.js benefits:

• Node.js gets V8 test coverage for the patches
  • V8 only runs a very limited subset of the ToT waterfall on those branches
  • Node.js recently added ability to run V8 tests in their CI, which could be used.
• It makes it easier for Node.js developers to contact V8 developers for code-reviews
  • On the other side Node.js developers need to use V8's infrastructure
• Tracking of merges works if V8's merge scripts are used.

Node.js drawbacks:

• Node.js maintainers need to heed V8's workflow for patches (including the tools)
• Node.js is completely dependant on V8's infrastructure
  • As a contingency node.js probably still wants to be able to float patches which raises the question why we are doing this?
• Complicates branching/versioning/handling merges because two branches with the same origin might start to differ and e.g. 5.6.333.67 might point to two different commits.

Maintain all the floating patches in a V8 fork on GitHub

A prerequisite is that the Node.js build is able to pull in a specific V8 commit/ref. This is similar to V8's DEPS format.

Another solution would be create a V8 fork in the github nodejs or v8 organizations which automatically gets synced with the official V8 repo (Currently V8 is doing the reverse and maintaining a Node.js fork for integrating a new V8 version.). On the V8 fork the branches node-master, node-current, node-lts, node-maintenance are created. Node.js can then directly import the HEAD of each branch into their respective Node.js branch.

Node.js benefits:

• Node.js developers can use their own merge processes and tools
• Node.js' build does not need to be integrated with another system (Chrome infrastructure)
• Tracking of merges works if V8's merge scripts are used.
  • We should be able to adapt tools/release/merge_to_branch.py to be able service a V8 fork repo

Node.js drawbacks:

• No test coverage from the V8 infrastructure
  • Which probably is not needed because Node.js infrastructure can execute V8 tests
• Complicates branching/versioning/handling merges because two branches with the same origin might start to differ and e.g. 5.6.333.67 might point to two different commits.

It would be great to get some more input on this analysis.

[bnoordhuis]

/cc @nodejs/ctc and @nodejs/v8 - this is relevant to your interests.

If we want to be a good OSS citizen, option 1 is arguably best. If a bug fix or feature is beneficial to node.js, chances are good it's beneficial to other projects that depend on V8. A V8 LTS branch would make a lot of people quite happy, I'm sure.

V8 only runs a very limited subset of the ToT waterfall on those branches

That might be an issue. How limited is very limited?

Node.js maintainers need to heed V8's workflow for patches (including the tools)

IME, the tooling is not terrible once you get the hang of it but the infrastructure is very opaque to people outside the googleplex. I'm still not sure what the difference between the trybots and the main waterfall is, for example.

On a semi-related topic, what about integration testing? Ideally, we apply a change to the V8 tree and the node.js tree and check that both test suites pass. That's something we can make work on our own CI infrastructure but probably not on Google's.

[MylesBorins]

I'm very much in support of an official LTS release stream. I've gotten started with using Depot tools before and would be willing to dive in more if that will help. Do we have individuals from google that are willing to volunteer as mentors to help individuals from the node project get up and running on your stack?

[natorion]

V8 only runs a very limited subset of the ToT waterfall on those branches

That might be an issue. How limited is very limited?

Compare our bots servicing beta and stable branch to essentially all the bots combined on the other of our waterfalls (e.g. main waterfall). This works for us because our branches are close to ToT and we get a lot of coverage from the general Chromium bots downstream.

On a semi-related topic, what about integration testing? Ideally, we apply a change to the V8 tree and the node.js tree and check that both test suites pass. That's something we can make work on our own CI infrastructure but probably not on Google's.

We are already running something similar on our waterfall. This bots tries to continually integrate V8's LKGR into a fork of node master. Doing the same for an LTS is a different beast because one would need infrastructure which is compatible > 30 months backwards.

I'm very much in support of an official LTS release stream.

Just to clarify, you mean the second idea, a V8 fork on GitHub in Node.js' domain, right? If you have specific questions regarding depot_tools, I am sure we can get them answered.

CC @ofrobots

[MylesBorins]

@natorion I was referring to

V8 creates separate release branches

I'm open to whatever has the most traction between the two teams!

[MylesBorins]

@natorion would you like to join us for the next LTS working group meeting?

[ofrobots]

I think @natorion is away this week. He was planning to attend an LTS meeting, but we haven't had one for a while. A Europe friendly meeting time for the next one would be nice, but not a requirement. /cc @jasnell.

[natorion]

I would gladly try to join the next meeting. When is it scheduled?

[MylesBorins]

/cc @nodejs/v8 and @natorion

We'd like to schedule a meeting time / day that works for everyone. @mhdawson is going to be posting a doodle in here in the near future, if you could fill it out with available dates / times that would be great!

[mhdawson]

Link to doodle poll http://doodle.com/poll/dwcpugyzsvitta94

[ofrobots]

I'm out of office during the week of July 4. Could there be options for other weeks?