use a URI-based scheme for package names like Java

[andrewrk]

Related issues:

• add tooling to deal with build.zig dependency trees #14288
• new package hash format: $name-$semver-$hash #20178
• ability to override packages locally #20180

In order to detect that two different fetched packages represent different versions of the same project, with one intended to supersede the other, we need some kind of stable identifier. Zig package management is decentralized, so there is no global namespace that serves this purpose.

We have the name field of packages, but it is not globally unique.

... or is it? This proposal takes inspiration from Java's solution to this problem which is to use the domain name system and canonical URIs, so you end up with names such as com.sun.foo.bar.ArrayList. Although Zig supports URL mirrors, canonical URIs can still be used as globally unique package identifiers.

Example fully-qualified package names would be:

• com.github.scottredig.JavascriptBridge
• me.andrewkelley.groovebasin
• org.codeberg.river.river
• org.ziglang.zig

In order to discourage people from fighting over unqualified names and causing general chaos, Zig would emit an error if a package fetched by URL was missing a top level domain name.

Whether the person owns the domain names they use is not enforced. When you choose to add a dependency on a package, you can notice whether they use their own domain name, or someone else's and factor that into the human decision of whether to trust them. Maybe someone decides to take over an abandoned project, and continue using the old maintainer's domain name for the project, for a seamless transition. If that new maintainer has built trust in the community, people would generally find this acceptable.

As for #14288, there would no longer be an id field introduced. The name field acts as the globally unique identifier that persists across versions.

As for #20178, only the last path segment in the name would be used. For example:

JavascriptBridge-1.2.3-iDYAACr46GhU
groovebasin-2.0.0-NQ8kAD5eWxrE
river-3.1.0-eTYAAFRBXp0H
zig-0.14.0-cW8vBMOZSHPt

Finally, as for #20180, this could potentially provide a way to override an entire project across the full dependency tree, to quickly find out if a particular patch, for example, could be used to make all the different usage sites of the package satisfied, thus being a candidate for a global override.

[sno2]

Is there any way we could avoid having to read package names backwards and forwards to be able to find the actual URL? We could try a restricted version of Go's approach and use github.com/ziglang/zig where everything after the last / is used in #20178.

[andrewrk]

In this proposed system, there is no actual URL corresponding to a package name. URLs are only a fetch mechanism, and the same package could be fetched via an arbitrary number of different URLs, each with different domain names and schemes. Why would you want such an "actual URL" anyway?

[squeek502]

What's the benefit of the domain name over just a <namespace>.<name> format, so e.g. squeek502.resinator, ziglang.zig, etc?

[mlugg]

One criticism I would have of this is it functionally requires you to own a [sub]domain in order to create a package. I recognise this is pretty easy with services like GitHub Pages, but it's an extra roadblock which I don't think ought to exist. If Layperson Bob wants to make a high-quality package that others can use, which conforms to our typical naming convention, I don't think we should require him to set up a personal domain. Perhaps they could use com.github.username.repo to effectively include the path components of a URL, but then that ties the package name to the version control host the author happened to use when they first created the package.

[andrewrk]

What's the benefit of the domain name over just a <namespace>.<name> format, so e.g. squeek502.resinator, ziglang.zig, etc?

How do you ensure that <namespace> is unique? andrewrk is already taken on many sites I register on, for example.

[andrewrk]

If Layperson Bob wants to make a high-quality package that others can use, which conforms to our typical naming convention, I don't think we should require him to set up a personal domain.

In this example, what URL will users of Layperson Bob's package use to fetch it?

[sno2]

In this proposed system, there is no actual URL corresponding to a package name. URLs are only a fetch mechanism, and the same package could be fetched via an arbitrary number of different URLs, each with different domain names and schemes. Why would you want such an "actual URL" anyway?

I'm not implying that it is better. I just find it an inconvenience to mentally turn your format into a URL. We can say that this format is not restricted to being a valid URL, but in almost every case (and example you wrote in this proposal) I'd argue it'd be one.

[mlugg]

In this example, what URL will users of Layperson Bob's package use to fetch it?

I imagine a GitHub archive URL, like many people already use today (or perhaps a Git URI). However, there's a big difference between using this as the fetch URL and the package name: if Bob later decides that GitHub's pivot to AI is leading to too many poor business decisions, and moves his repos over to GitLab, I don't think he should be stuck with this legacy package name.

[squeek502]

How do you ensure that <namespace> is unique? andrewrk is already taken on many sites I register on, for example.

Why does it need to be unique, or, alternatively, how is the uniqueness of the domain enforced/what is the uniqueness meant to represent? Couldn't I create my own org.ziglang.zig package and distribute it however I want?

[mlugg]

@squeek502 The idea isn't that it would be technically enforced, but that it would just give a unique name you could use. This is effectively a convention (the only enforcement would be the leading TLD requirement), but the benefit of the convention is that when you follow it, by using a [sub]domain you own, you'll get a package name which is unique amongst everyone else following the convention.

Packages need some kind of unique identifier for overrides, deduplication, and version management. This proposal is an alternative to the id field suggested by #14288.

[andrewrk]

if Bob later decides that GitHub's pivot to AI is leading to too many poor business decisions, and moves his repos over to GitLab, I don't think he should be stuck with this legacy package name.

I think this is the main downside of the proposal; people will be tempted to pointlessly rename their projects, causing completely unnecessary churn for package users.

Why does it need to be unique, or, alternatively, how is the uniqueness of the domain enforced/what is the uniqueness meant to represent?

The build system needs a way to find out when the dependency tree has multiple package versions of the same project, so that it can implement features such as:

• version selection
• detecting when an updated version is available
• many other kinds of tooling that I didn't think of right this second

Imagine the situation that would occur if two different projects were both named "abc" and the build system swapped the higher version number in for the lower version one. This is not a problem for centralized package managers because they have a single, global namespace.

The only alternative to this proposal is to generate a unique random identifier and either attach it to the name, or have another id field in the package manifest.