Simple way to apply cargo_config to rust_rules bootstrapping?

[sthornington]

In a corporate environment, where crates.io is intermediated by an artifactory mirror, we must have a cargo_config.toml which creates new registries and sources and a replace-with rule. It's not too bad getting this to work for our own rust targets, but to get e.g. rust_analyzer working, this cargo_config also needs to be passed (I suspect) in various places in 3rdparty/deps.bzl sort of stuff which is doing crates_vendor in order to bootstrap the build of rust analyzer.

How do people typically deal with this? If we pull in rules_rust via http_archive then I guess I am supposed to patch it in all the places that I find? Should I instead add rust_analyzer to my project myself, so that it its deps get pulled in using the cargo_config of my crates_repository call in WORKSPACE ?

EDIT: this is all basically in order to build rust-project.json for our LSP.

[UebelAndre]

I'd be happy to review a change that wires all that up. cargo_bootstrap_repository should probably get a cargo_config attribute and ensure it's used when building.

[sthornington]

I'm a little new to bazel, what's the process here - work backwards from every instance of crates_repository/crates_vendor that can take a cargo_config and thread that as a parameter all the way to every instance of things like rules_rust_dependencies and crate_universe_dependencies and rust_analyzer_dependencies ?

[sthornington]

I'm starting on this, but immediately with a clone of latest (up from 0.31) our basic rules are not working:

Repository rule crates_repository defined at:
  /scratch/simont/bazel_base/4fd9a0903ffbf1e501ab97de5c381dcc/external/rules_rust/crate_universe/private/crates_repository.bzl:108:36: in <toplevel>
ERROR: An error occurred during the fetch of repository 'crate_index':
   Traceback (most recent call last):
        File "/scratch/simont/bazel_base/4fd9a0903ffbf1e501ab97de5c381dcc/external/rules_rust/crate_universe/private/crates_repository.bzl", line 28, column 48, in _crates_repository_impl
                generator, generator_sha256 = get_generator(repository_ctx, host_triple.str)
        File "/scratch/simont/bazel_base/4fd9a0903ffbf1e501ab97de5c381dcc/external/rules_rust/crate_universe/private/generate_utils.bzl", line 60, column 13, in get_generator
                fail((
Error in fail: No generator URL was found either in the `CARGO_BAZEL_GENERATOR_URL` environment variable or for the `x86_64-unknown-linux-gnu` triple in the `generator_urls` attribute
ERROR: /scratch/simont/src/quadcap/WORKSPACE:111:18: fetching crates_repository rule //external:crate_index: Traceback (most recent call last):
        File "/scratch/simont/bazel_base/4fd9a0903ffbf1e501ab97de5c381dcc/external/rules_rust/crate_universe/private/crates_repository.bzl", line 28, column 48, in _crates_repository_impl
                generator, generator_sha256 = get_generator(repository_ctx, host_triple.str)
        File "/scratch/simont/bazel_base/4fd9a0903ffbf1e501ab97de5c381dcc/external/rules_rust/crate_universe/private/generate_utils.bzl", line 60, column 13, in get_generator
                fail((
Error in fail: No generator URL was found either in the `CARGO_BAZEL_GENERATOR_URL` environment variable or for the `x86_64-unknown-linux-gnu` triple in the `generator_urls` attribute
ERROR: Error computing the main repository mapping: no such package '@crate_index//': No generator URL was found either in the `CARGO_BAZEL_GENERATOR_URL` environment variable or for the `x86_64-unknown-linux-gnu` triple in the `generator_urls` attribute

Is there something more I need to do to upgrade our WORKSPACE to 0.50 before I can start on this work?

[sthornington]

Never mind, I'm figuring it out reading the DEVELOPMENT.md inside crate_universe

[sthornington]

I'd be happy to review a change that wires all that up. cargo_bootstrap_repository should probably get a cargo_config attribute and ensure it's used when building.

I'm a little unclear what cargo_bootstrap_repository has to do with this? I'm looking a little more deeply, it looks like the crates_vendor call is baking crates.io URLs into defs.bzl here https://github.com/bazelbuild/rules_rust/blob/main/tools/rust_analyzer/3rdparty/crates/defs.bzl#L418 and then they get fetched directly, later, with http_archive?

So, the goal now is to make bazel run @//tools/rust_analyzer/3rdparty:crates_vendor work correctly to rebuild this file with correct URLs, and that requires that cargo_bazel_bootstrap takes a cargo_config in order to correctly back out the final URLs for everything to replace into that defs.bzl ?

[UebelAndre]

Sorry, I may have misunderstood your question. There is no way to have a Cargo config.toml influence already rendered dependencies but if you want to point crates.io links to an internal source you can use --experimental_remote_downloader_config (bazelbuild/bazel#12170). Does that solve for your use case?

[sthornington]

That might help a great deal, I will try it out!

[sthornington]

This seems like it should work (and if I block crates.io in the file, it fails fast), but yet it does not seem to be rewriting the download URLs whilst trying to download crates to build rust_analyzer. I'll keep trying to see where it's going wrong...

[sthornington]

Okay, so I think the examples in that source are wrong - I think the regex is matching only the part of the source URL after https:// onwards.

rewrite static.crates.io/crates/(.*) https://artifactory..../$1

works fine. Just a bit weird that the https:// is not there when you are matching.. is that intentional [@shs96c] ? This is in bazel 7.3.1

[sthornington]

Okay so this was super clutch, thank you! Combined with passing --bazel to gen_rust_project in order to point to our wrapper script, everything now works, no hacking on rules_rust required after all! Closing this out.