Remote Version Definitions

[psionic-k]

Problem

In the early phase of creating several repos for one company, we end up with scattered lock and version definitions. Later on, most poeple will switch to maintaining an alternative registry, but it can be kind of silly otherwise. It is tempting to use a workspace to collect all versions of dependencies, but this also enforces mono-repo.

I am already enjoying remote version definitions with our non-Rust dependencies (and many Rust binaries) with Nix. Each project has a flake, but they delegate their definitions to a single flake I named "pinning". The pinning flake, which lives in its own repo, contains actual versions and its lock file has concrete definitions. The dependent repo lock files instead all point to the concrete definitions in pinning.

How my Nix deps work:

• We have a repo called pinning with a flake that contains actual versions and a lock file with concrete versions
• When I want to update or add a dependency, I do it in pinning
• Dependents will see pinning the next time they update pinning. We know exactly what they will upgrade to.
• Every repo is using some version of the pinning repo, so the sum of all our dependencies remains a tiny set no matter how many repos we use
• Each repo can of course point a specific input to a concrete definition if it needs to, so we can still override in-detail, both for development and if there is a need for two versions at the same time

So basically on the Nix side, we have the best of all worlds. It's central like monorepo. It upgrades and overrides per-repo like multi-repo.

A remote definition would be a fine solution for version scattering while still using crates.io to obtain crates until organizations set up alternative registries.

Proposed Solution

Something like a workspace or remote table, but with a definition that can be specified from a separate repository and then used like a workspace version in dependency tables of dependents.

• Must not depend on version control sub modules. It should include ability to specify via git.
• The local lock may specify a version of the remote definition, but an update to that definition should cascade into the local repo's lockfile.
• If the remote lock and toml would require updating the remote's lockfile to make sense, just error.

We want the dependents to look like:

[dependencies]
zeroize = {workspace=true, features = ["zeroize_derive"]}
thiserror.workspace = true

And the remote table to contain actual versions, just like a workspace.

[workspace.dependencies]
zeroize = "1.8.1"
thiserror = "1.0.61"

Notes

I haven't been able to get this working with submodules unless my workspace includes all my crates as submodules, which kind of brings us back to effectively monorepo with extra steps.

[epage]

So across a many-repo environment, you are trying to keep dependencies in-sync? How much of the focus is on keeping version requirements in-sync vs resolved versions? Version requirements will only focus on direct dependencies. Resolved versions will also help with indirect dependencies.

What is the motivation for keeping things so strictly in lock step?

For resolved dependencies, one workaround would be to have a no-op package with every dependency of interest specified with a = version requirement in a [target.'cfg(any())'.dependencies] so they won't actually be used. You can then either pull this as a git dependency or as a path dependency to a submodule.

[epage]

For anyone following along, there is also an Internals discussion that as been started.

I also remembered another potential way of handling this in the future. With publc/private dependencies, we talked about delegating the source for dependency A to dependency B's use of A. See https://rust-lang.github.io/rfcs/3516-public-private-dependencies.html#caller-declared-relations

[epage]

CC @ericseppanen