Repository navigation
Automatic V8 DEPS detection #168
Description
Activity
Note that I intentionally didn't use
depot_toolsin order to only fetch the dependencies that are required to build V8 for node (there are many other dependencies, some of which are large, that are not necessary to us)We can have a
ignore_depsif that's the case. Do you have stats on how much the size would grow with those dependencies?There are also the gitignore updates that are related to the deps that we use.
As a first step, would you be willing to add add
//buildand//buildtoolsdependencies, drop all ignores we have on the script, and adddeps/gnwhich is updates alongside V8? We can probably drop depot_tools from our V8 testing script if we do that, and it opens the door to build V8 with GN in a hybrid setup.Edit: also
third_party/icu, I'm still working on getting this setup running so other dependencies might come up.I'm open to anything that works 😄
Reacted by mary marchiniCool! I'll try to get this working with
node-core-utils, hopefully when we upgrade to V8 8.5 we can also have those deps added and then we can simplify the wholetest-v8setup.Edit: and maybe in the near future we can drop the
gypfiles for V8 and have a gyp/GN hybrid build. What a dream 😄have a gyp/GN hybrid build
Are you ready for the nightmare it will be to make it work on Windows? :D
I'm ready to push super hard for us to drop msvc and support only clang like V8 does 😅
Reacted by Michaël Zasso$ du -sh buildtools build tools third_party/icu 85M buildtools 631M build 32M tools 629M third_party/icuThat's with the full fetch, which includes binary files and the git folder, those folder are considerably smaller when cloning a single commit
This is what I get with
gclient sync. Is there another way to use depot_tools?I don't think so, but maybe we can
git clean -xdfthe repositories. Otherwise we'll need to stick with the approach we have today.those folders are git repo.
git clean -xdflikely not to work.I like the idea. But I have bad experience both with depot_tools and gclient even on linux (setup and sync issue). Also git submodule actually is pretty mess up. Maybe publish related toolchain in a npm module.
those folders are git repo. git clean -xdf likely not to work.
The folders yes, but all the binaries that depot_tools downloads on the side aren't and can be deleted with xdf (and that's where most of the size @targos mentioned is coming from).
But I have bad experience both with depot_tools and gclient even on linux (setup and sync issue).
We all have 😅 but hopefully automating it via ncu would make things easier
Also git submodule actually is pretty mess up
I didn't suggest submodules in this issue :)
Maybe publish related toolchain in a npm module.
That could work, but we would face pushback from folks that want Node.js to be builable offline at any point for any commit after you clone the repository. Also, publishing the toolchain to npm has many challenges as we would need to store all toolchains for any commit. Of course we can reuse a specific toolchain version for many commits, but as the time passed the number of versions we have to store grows. This might not scale well either in terms of storage or developer experience.
I'm not saying we shouldn't do it, it's something I'm even considering as part of the hole "bye bye gyp" initiative, but the hole is a lot deeper than it seems. And what I described is basically us rebuilding depot_tools our own way.
I'm trying to find a way for us to auto sync V8 dependencies without needing to manually list them and without having to manually parse the DEPS file. One idea I had is to add depot_tools as a dependency to
git node v8(which the user can specify a path or we can download if not specified) and then usefetch/gclient syncto ensure dependencies are sync'ed.The main reason I'm looking into this is that I'm looking into the idea of "hybrid builds" where we build V8 with GN (or use GN to generate
gyp/cmakefiles), and for this to work without Node.js collaborators having to installdepot_tools, we need to make sure all dependencies are fetched, includingbuildandbuild_tools(see all submodules on rusty_v8).