Repository navigation
RFC: Bazel for build system #464
Description
Activity
So far I don't see any disadvantages of Bazel compared to GN besides the fact that Bazel requires a separate installation.
Node.js native modules require gyp. Migration away would be very difficult.
As far as I know, native modules don't require any special support from Node.js. That is, GYP files could be completely removed from Node.js, and they would still compile and work. Please correct me if I'm wrong.
JavaScript developers may have an irrational aversion against Java.
I don't think anyone would really care as long as they don't have to read or write Java code or install JVM.
If JS devs have to rely on a Java tool, they would have to read or write Java in order to contribute for bugfixes etc; there is definitely an aversion in the community to that (“irrational” is a subjective enough term that perhaps it should be removed from the doc, to avoid debating how rational it might indeed be).
Reacted by Ali Ijaz Sheikh, Jake Niemiec, Tierney Cyren, Toru Nagashima and Gürgün DayıoğluWell in case of gyp that would be Python. In case of cmake and V8 that would be C++.
Reacted by Markus Padourek, Dimiter 'malkia' Stanev, Paul Draper and Gürgün DayıoğluTo clarify, would be particularly compelling if the tool used were implemented in JS.
Don't you then have to come up with a way to bootstrap?
I’m -1 on moving to a Java tool. Python is part of all Linux distributions, so it is not a dependency we need to install in a lot of cases.
I’m +0 to be self hosted, but writing yet another build tool is not a priority. I would prefer to use an existing tool.
Reacted by LeedehaiReacted by Navid RahimiI’m -1 on moving to a Java tool. Python is part of all Linux distributions, so it is not a dependency we need to install in a lot of cases.
The way I understand it Bazel comes packaged with JVM. The fact that it is written in Java is an implementation detail that is transparent to the user.
Reacted by Dimiter 'malkia' StanevFrom https://docs.bazel.build/versions/master/install-ubuntu.html Bazel seems more complicated than using/installing gyp.
Reacted by @brody2consult - Chris J. Brody and Nikita SkovorodaReacted by Navid Rahimi, Dimiter 'malkia' Stanev and Thulio Ferraz AssisGYP files could be completely removed from Node.js, and they would still compile and work. Please correct me if I'm wrong.
Add-ons depend on node's common.gypi: nodejs/node-gyp#1118
It's not unfixable but that will take time and effort.
If JS devs have to rely on a Java tool, they would have to read or write Java in order to contribute for bugfixes etc; there is definitely an aversion in the community to that (“irrational” is a subjective enough term that perhaps it should be removed from the doc, to avoid debating how rational it might indeed be).
FWIW, the surface of Bazel that the JS devs will have to interact with is mostly the Skylark (Python-based) API extension that the rules are implemented in (like rules_node). The core incremental graph processing framework are mostly developed by the core Bazel developers and some rule writers.
Reacted by Navid Rahimi and Dimiter 'malkia' StanevSure. But the JS devs that use it might become those core developers given the chance - and having it be in JS maximizes that.
Reacted by Navid RahimiThat sounds a bit far-fetched though, and considering that there is no mature build system in JS at all right now, purposefully avoiding Java seems unreasonable.
Add-ons depend on node's common.gypi: nodejs/node-gyp#1118
It's not unfixable but that will take time and effort.
That doesn't seem to be a blocker.
- It probably wouldn't take much effort to continue maintaining
common.gypi. - Fixing Don't include common.gypi from node source tree node-gyp#1118 is a good idea whether or not Node.js switches to a different build system.
- It probably wouldn't take much effort to continue maintaining
👍 for fixing nodejs/node-gyp#1118 anyway.
Some notes:
- Will the new build tool be pushed/made available to the ecosystem, or the ecosystem would stick being gyp-only? I don't think we can easily move the ecosystem, so we must keep maintaining gyp anyway and moving to another just for core increases the code we have to rely on.
- Gyp is written in python, we maintain it in-tree and it is very easy to ship and distribute as part of our platform. I think these are good properties, and we should retain them. Asking developers to install XYZ separately will introduce a lot of friction in the ecosystem (and for core developers).
If by ecosystem you mean node modules currently relying on gyp... I don't think that should be within scope of this discussion. Anything that replaces gyp in Node core will have the same effect.
9 remaining items
So far it seems Bazel isn't really a better choice than GN.
- Although Bazel has more public adoption than GN, it's still somewhat obscure.
- Bazel is difficult to install on a number of platforms, so we would probably end up bundling it or providing an installation script - just like with GN.
- Platform support in Bazel isn't any better than GN. According to the document, the only difference is that Bazel supports OpenBSD officially, but GN only unofficially.
Reacted by Navid Rahimi and Thulio Ferraz AssisSo far it seems Bazel isn't really a better choice than GN.
From what you say, Bazel is the same or better in all three of those areas, it's more popular, has more official platform support, and is equally difficult to install.
Also I'd assume Google aren't going to be interested in supporting both long-term, so the chances are that GN will go the way of GYP. What would be the reason(s) to choose GN over Bazel?
What would be the reason(s) to choose GN over Bazel?
- If I understood correctly, Bazel isn't supported officially by V8, it's only for internal use. Also, Google uses Blaze internally, not Bazel, which might result in compatibility issues that we won't get help with.
- With Bazel, we'll have to figure out how to bundle it or provide an installation script. With GN, it's already done for us.
Reacted by Navid RahimiSo far it seems Bazel isn't really a better choice than GN.
FWIW, from my point of view, here are the differences between Bazel and GN:
- Google has been using the technology behind Bazel for around a decade. While Bazel is a Google initiated project, it is very much an open source project with a strong interest in bringing a fast and correct build system to the widest community of developers. IOW, this is a NOT a Google-only tool. Much like the Kubernetes project.
- GN is developed by the Chrome team for building Chrome. There isn't much interest by the Chrome team to make it usable in more scenarios or to be able to support other use cases.
If the comparison was only between these two tools, my recommendation would be strongly towards Bazel.
If I understood correctly, Bazel isn't supported officially by V8, it's only for internal use. Also, Google uses Blaze internally, not Bazel, which might result in compatibility issues that we won't get help with.
For the purposes of this discussion the differences between Bazel and Blaze are academic. Internally at Google we build V8 using Blaze. The V8 team supports this build configuration. I would expect that the V8 team could easily support a bazel build configuration, if necessary.
Reacted by Yang Guo, Conrad Taylor and Navid RahimiReacted by Dimiter 'malkia' Stanev and tomaszstrejczekWhile Bazel is a Google initiated project, it is very much an open source project with a strong interest in bringing a fast and correct build system to the widest community of developers. IOW, this is a NOT a Google-only tool.
But what does it mean for Node.js in practical terms? It seems there is no practical difference in terms of platform support at least.
There isn't much interest by the Chrome team to make it usable in more scenarios or to be able to support other use cases.
Any use cases that are currently supported by GYP and Bazel, but not by GN?
By the way, I don't see any BUILD files in the V8 repo. Where can I find them?
Any use cases that are currently supported by GYP and Bazel, but not by GN?
My understanding, to put it bluntly, is that any use of GN in a project other than Chrome is unsupported.
By the way, I don't see any BUILD files in the V8 repo. Where can I find them?
Those are in our internal-only repo.
Those are in our internal-only repo.
So what's the plan, make them available if we agree to use Bazel? Do we get to experiment with them first?
My understanding, to put it bluntly, is that any use of GN in a project other than Chrome is unsupported.
As in, you won't get help with GN issues unless you're working on Chrome? But will you get help building V8 with Bazel if you don't work for Google?
Everything works fine with gyp and gn - at the moment. It might be, that we can use gyp for the next decades and never run into any problems, it might be, that there's a change in V8 in 6 month that just happens to make it impossible to build with gyp (unlikely).
The issue with gn is the long term outlook. It's build for and by Chrome, and afaik there's no guarantee that it will be supported in the future. Might be that Chrome ends up using it forever, might be that it's replaced next year. We're not looking at other build options because gyp or gn don't work but because we don't want ugly surprises in the future.
@seishun We (V8 team) would make our existing Bazel/Blaze configs and scripts that convert from gn to Bazel available and then they would be an additional build option. If the Node project officially adapts those or makes them the default all depends on how well they work or our users and ecosystem.
Unsupported means, that there's no guarantee that it will be supported in the future or that features don't get removed. Gn is open source, as in, you can use the code, but changes really depend on what's decided in Chromium whereas Bazel has a larger community and a different "goal", so you get all the help the community is willing and able to give.
Reacted by Navid RahimiI have definitive statement from Chrome's GN team to not promote GN outside of Chrome. GN has been developed for Chrome and only Chrome. The team to support GN is very small and will not have the bandwidth to deal with other users.
I'm not entirely clear what you can expect wrt support from Bazel. But given the size of the Bazel team at Google and the fact that Google uses Blaze for everything internally, I'd say that it is here to stay. I will have a meeting with the Bazel team next week to explore options for Node.js. At this point, we have config files for V8 that should be easily adopted to Bazel. We don't have that for Node.js though.
Reacted by Navid RahimiJust looping back to the earlier discussion, what are the advantages of Bazel over cmake ? I guess it is because the v8 team is willing to maintain the Bazel configs ?
Yes.
So far it sounds like the V8 team would be doing Node.js a special favor by providing us with support for building V8 with Bazel. Would it take more effort for the GN team to do us a favor by supporting the use of GN for Node.js instead? Considering there is extensive public documentation for GN, but Bazel build for V8 is effectively non-existent as of today, that doesn't seem to be the case. Or is the GN team completely separate from the V8 team, and the former just doesn't care about Node.js at all?
This discussion appears to have stalled out. What is the next step?
I might experiment with this some time. Not high priority for me though due to lots of other things going on.
Let's close this for now.
This is something I've been brooding over for a while. Elaborated here.
TL; DR: GN doesn't seem like a good option going forward. Maybe Bazel is a viable replacement for GYP? V8 already has, internally at Google, Bazel configs. I'm just suggesting this as option, not trying to convince anyone. The V8 team doesn't really have anything to gain from this.