Vendor mode does not include everything for an airgapped build

[AlexanderAA]

Description of the bug:

Bazel vendor command produces a directory with dependencies.

However, it is impossible to build or run any targets on an airgapped system with the produced directory.

Bazel still requires access to network (or access to the cache that was in use while building vendor directory).

According to #19563 (comment), airgapped builds should be possible.

Which category does this issue belong to?

Core

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Testing this in 7.3.0rc2:

$ bazel vendor //target
<Completes successfully, produces several gigabytes of dependencies in vendor directory, which will later be copied to an airgapped system>
$ bazel fetch //...  # Is this needed? It does not seem to help.

... Disconnect from network or add --nofetch to .bazelrc ...

$ rm -rf <bazel local cache directory>  
$ bazel test //target
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
ERROR: Error computing the main repository mapping: error during computation of main repo mapping: to fix, run
        bazel fetch //...
External repository @@local_config_platform not found and fetching repositories is disabled.

Sometimes the error is:

External repository @@bazel_tools not found and fetching repositories is disabled.

Same happens when running bazel vendor without a particular target.
Without removing local cache bazel test works fine on local system, but obviously will fail on an airgapped host.

Is there a setting I am missing?

It looks like bazel is having difficulty resolving approximately the following after cache is removed:

@local_config_platform//:all
@local_config_sh//:all
@@local_config_cc_toolchains//:all
@@local_jdk//:all
@@rules_java//:all

Running bazel vendor with each of the above does not help.

Which operating system are you running Bazel on?

any

What is the output of bazel info release?

release 7.3.0rc2

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

Instructions at https://bazel.build/run/build#running-bazel-airgapped do not resolve the problem.

[AlexanderAA]

Same issue with 8.0.0-pre.20240730.1

[fmeum]

@AlexanderAA Could you check whether --noenable_workspace fixes this?

[AlexanderAA]

@AlexanderAA Could you check whether --noenable_workspace fixes this?

It does not, unfortunately.

Extracting Bazel installation...
Starting local Bazel server and connecting to it...
ERROR: Error computing the main repository mapping: error during computation of main repo mapping: to fix, run
        bazel fetch //...
External repository @@local_config_platform not found and fetching repositories is disabled.

[meteorcloudy]

--nofetch shoud not be used. --nofetch means don't fetch any external repo even if fetching the repo doesn't require network access. See my comment here: #22302 (comment)

[AlexanderAA]

@meteorcloudy Thanks, I am fetching everything without --nofetch of course!

I also want to use the fetched vendor directory on an airgapped machine later. This does not work.

To reproduce you would just need to bazel vendor, then turn off your wifi/ethernet connection and try building any target you vendored earlier. Please make sure you remove local bazel cache in your home directory and leave vendor directory only.

P.S. I would be interested to see any practical example of a bzlmod repository that does not require network access. Is this even possible with bcr/bazel_dep?

[fmeum]

I just successfully tried the following on the Bazel repo itself:

1. Set Bazel version to 7.3.0rc2 in .bazelversion.
2. Run bazel vendor --vendor_dir=vendor //src:bazel-dev.
3. Run bazel clean --expunge.
4. Disconnect your machine from the internet.
5. Run bazel build --vendor_dir=vendor --repository_cache= //src:bazel-dev.

@AlexanderAA Does this work for you? What errors are you seeing if not?

[meteorcloudy]

External repository @@local_config_platform not found and fetching repositories is disabled.

This error message clearly shows --nofetch was enabled. @AlexanderAA Can you please make sure to remove it from your bazelrc and try it again?

[meteorcloudy]

P.S. I would be interested to see any practical example of a bzlmod repository that does not require network access. Is this even possible with bcr/bazel_dep?

bazel/src/test/py/bazel/bzlmod/bazel_vendor_test.py

Lines 676 to 686 in 6205568

	# Build and run the target in a clean build with internet blocked and make
	# sure it works
	_, _, _ = self.RunBazel(['clean', '--expunge'])
	_, stdout, _ = self.RunBazel(
	['run', '//:main', '--vendor_dir=vendor', '--repository_cache='],
	env_add={
	'HTTP_PROXY': 'internet_blocked',
	'HTTPS_PROXY': 'internet_blocked',
	},
	)
	self.assertIn('Hello there! => bbb@1.0', stdout)

This test case is an exmaple.

[AlexanderAA]

@fmeum Thanks, I repeated your steps, but I am trying to understand why you did not delete your local cache for this test.

Is there an expectation that local cache will be copied to an airgapped machine/container?

Summary of my testing:

• bazel vendor creates local cache (~/.cache/bazel on linux or /private/var/tmp on macOS).
• --repository_cache= does not prevent local cache directory from being created (and populated with ~200MB of files in my case)
• bazel clean --expunge does not remove local cache, which is why the example works.
• As soon as local cache is not available anymore, the example above stops working. (Just rm -rf the cache directory.)

Errors after removing local cache:

$ bazel build --repository_cache=  //experimental:tests
WARNING: Download from https://github.com/bazelbuild/bazel-skylib/releases/download/1.6.1/bazel-skylib-1.6.1.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: github.com
WARNING: Download from https://github.com/bazelbuild/rules_cc/releases/download/0.0.9/rules_cc-0.0.9.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: github.com
INFO: Repository rules_cc instantiated at:
  /DEFAULT.WORKSPACE.SUFFIX:50:6: in <toplevel>
  /private/var/tmp/_bazel_a/f444419aa1c8c9c8114ef87e7668a230/external/bazel_tools/tools/build_defs/repo/utils.bzl:268:18: in maybe
Repository rule http_archive defined at:
  /private/var/tmp/_bazel_a/f444419aa1c8c9c8114ef87e7668a230/external/bazel_tools/tools/build_defs/repo/http.bzl:382:31: in <toplevel>
INFO: Repository bazel_skylib instantiated at:
  /DEFAULT.WORKSPACE.SUFFIX:29:6: in <toplevel>
  /private/var/tmp/_bazel_a/f444419aa1c8c9c8114ef87e7668a230/external/bazel_tools/tools/build_defs/repo/utils.bzl:268:18: in maybe
Repository rule http_archive defined at:
  /private/var/tmp/_bazel_a/f444419aa1c8c9c8114ef87e7668a230/external/bazel_tools/tools/build_defs/repo/http.bzl:382:31: in <toplevel>
ERROR: An error occurred during the fetch of repository 'rules_cc':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_a/f444419aa1c8c9c8114ef87e7668a230/external/bazel_tools/tools/build_defs/repo/http.bzl", line 131, column 45, in _http_archive_impl
                download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://github.com/bazelbuild/rules_cc/releases/download/0.0.9/rules_cc-0.0.9.tar.gz] to /private/var/tmp/_bazel_a/f444419aa1c8c9c8114ef87e7668a230/external/rules_cc/temp5034424698524737333/rules_cc-0.0.9.tar.gz: Unknown host: github.com
ERROR: BUILD.bazel:149:17: While resolving toolchains for target //experimental:tests (b9f8d89): invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': no such package '@@rules_cc//cc': java.io.IOException: Error downloading [https://github.com/bazelbuild/rules_cc/releases/download/0.0.9/rules_cc-0.0.9.tar.gz] to /private/var/tmp/_bazel_a/f444419aa1c8c9c8114ef87e7668a230/external/rules_cc/temp5034424698524737333/rules_cc-0.0.9.tar.gz: Unknown host: github.com
ERROR: Analysis of target '//experimental:tests' failed; build aborted
INFO: Elapsed time: 0.167s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
FAILED: 
    Fetching repository @@local_jdk; starting

[meteorcloudy]

$ bazel build --repository_cache= //experimental:tests

You are missing --vendor_dir flag, without this flag, the vendor source won't be used.

[meteorcloudy]

@AlexanderAA Did it work? I'll close this one for now, feel free to reopen if you discover any other problem!