ref-engine-discovery: Add a ref-engine discovery protocol

This is mostly from [1], but pulls in ideas from the rest of the
discussion with Keyang and Aleksa there, as well as the inspirational
specs mentioned in the README.

[1]: #1 (comment)

README.md

@@ -1,13 +1,11 @@
 # README
 
-This repository is to present the [discovery specification](DESIGN-SPEC.md) OCI image discovery functionality, as part of [image-spec](https://github.com/opencontainers/image-spec). 
+This repository contains the [ref-engine discovery specification](ref-engine-discovery.md) as an extention to the [image specification][image-spec].
 
-For OCI image distibution is a complex and wide system, and much implementation should will done, [discovery specification](DESIGN-SPEC.md) document just be separated from that, to achieve only consensual image discovery protocol.
-
-The strategies in this document refer to existing great open implementations.
+The strategies in this specification are inspired by some previous implementations:
 
 * [ABD](https://github.com/appc/abd/blob/master/abd.md)
-
 * [App Container Image Discovery](https://github.com/appc/spec/blob/v0.8.10/spec/discovery.md)
-
 * [parcel](https://github.com/cyphar/parcel)
+
+[image-spec]: https://github.com/opencontainers/image-spec

dns-compatible-image-names.md

@@ -0,0 +1,30 @@
+# DNS-compatible image names
+
+This is version 1 of this specification.
+
+The [X.509 Public Key Infrastructure][X.509] provides a well-established mechanism for trusted namespacing of [domain names][rfc5890].
+Protocols interested in leveraging that infrastructure need to be able to extract domain names from image names.
+This specification provides one approach for that extraction.
+
+This specification defines image names compatible with DNS using the following [ABNF][]:
+
+```ABNF
+dns-compatible-image-name = host "/" path-rootless [ "#" fragment ]
+```
+
+where:
+
+* `host` is defined in [RFC 3986 section 3.2.2][rfc3986-s3.2.2].
+* `path-rootless` is defined in [RFC 3986 section 3.3][rfc3986-s3.3].
+* `fragment` is defined in [RFC 3986 section 3.5][rfc3986-s3.5].
+
+Implementations MAY accept other names, for example, by creating a default `host` for names that match `segment-nz` (defined in [RFC 3986 section 3.3][rfc3986-s3.3]).
+
+Names which are not supported for `dns-compatible-image-name` will not be able to use protocols that rely on this rule, although they may use other protocols.
+
+[ABNF]: https://tools.ietf.org/html/rfc5234
+[rfc3986-s3.2.2]: https://tools.ietf.org/html/rfc3986#section-3.2.2
+[rfc3986-s3.3]: https://tools.ietf.org/html/rfc3986#section-3.3
+[rfc3986-s3.5]: https://tools.ietf.org/html/rfc3986#section-3.5
+[rfc5890]: https://tools.ietf.org/html/rfc5890
+[X.509]: https://tools.ietf.org/html/rfc5280

index-template.md

@@ -0,0 +1,80 @@
+# OCI Index Template Protocol
+
+This is version 1 of this specification.
+
+The index-template protocol is configured via a single [URI Template][rfc6570].
+When configured via a [`refEngines` entry](ref-engine-discovery.md#ref-engines-objects), the `uri` property MUST be set, and its value is the URI Template.
+
+Consumers MUST provide at least the following variables:
+
+* `name`, matching `dns-compatible-image-name` in the [`dns-compatible-image-name` rule](dns-compatible-image-names.md).
+* `host`, matching `host` in the `dns-compatible-image-name` rule.
+* `path`, matching `path-rootless` in the `dns-compatible-image-name` rule.
+* `fragment`, matching `fragment` in the `dns-compatible-image-name` rule.
+    If `fragment` was not provided in the image name, it defaults to an empty string.
+
+and expand the URI Template as defined in [RFC 6570 section 3][rfc6570-s3].
+
+The server providing the expanded URI MUST support requests for media type [`application/vnd.oci.image.index.v1+json`][index].
+Servers MAY support other media types using HTTP content negotiation, as described in [RFC 7231 section 3.4][rfc7231-s3.4] (which is [also supported over HTTP/2][rfc7540-s8]).
+
+Consumers retrieving `application/vnd.oci.image.index.v1+json` SHOULD process it like a [layout's `index.json`][index.json], respecting [`org.opencontainers.image.ref.name` and other annotations which are recommended for `index.json`][annotations].
+
+## Example
+
+An example [`refEngines` entry](ref-engine-discovery.md#ref-engines-objects) using the [registered `oci-index-template-v1` protocol identifier](ref-engine-protocols.md) is:
+
+```json
+{
+  "protocol": "oci-index-template-v1",
+  "uri": "https://{host}/ref/{host}/{path-rootless}"
+}
+```
+
+An image name like `a.b.example.com/c/d#1.0` matches [`dns-compatible-image-name`](dns-compatible-image-names.md) with `a.b.example.com` as `host`, `c/d` as `path-rootless`, and `1.0` as `fragment` so the expanded URI is:
+
+    https://a.b.example.com/ref/a.b.example.com/c/d
+
+Retrieving that URI (with a pretend result, since [`example.com` is reserved][rfc2606-s3]):
+
+```
+$ curl -H 'Accept: application/vnd.oci.image.index.v1+json' https://a.b.example.com/ref/a.b.example.com/c/d
+{
+  "schemaVersion": 2,
+  "manifests": [
+    {
+      "mediaType": "application/vnd.oci.image.manifest.v1+json",
+      "size": 7143,
+      "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
+      "platform": {
+        "architecture": "ppc64le",
+        "os": "linux"
+      },
+      "annotations": {
+        "org.opencontainers.image.ref.name": "1.0"
+      }
+    },
+    {
+      "mediaType": "application/xml",
+      "size": 7143,
+      "digest": "sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e",
+      "annotations": {
+        "org.freedesktop.specifications.metainfo.version": "1.0",
+        "org.freedesktop.specifications.metainfo.type": "AppStream"
+      }
+    }
+  ]
+}
+```
+
+Deciding whether to look for `1.0` (the `fragment`) or the full `a.b.example.com/c/d#1.0` name is left as an exercise for the reader, as is switching based on `platform` entries or [chosing between multiple entries with the same name][duplicate-name-resolution].
+
+[annotations]: https://github.com/opencontainers/image-spec/blob/v1.0.0/annotations.md#pre-defined-annotation-keys
+[duplicate-name-resolution]: https://github.com/opencontainers/image-spec/issues/588#event-1080723646
+[index]: https://github.com/opencontainers/image-spec/blob/v1.0.0/image-index.md
+[index.json]: https://github.com/opencontainers/image-spec/blob/v1.0.0/image-layout.md#indexjson-file
+[rfc2606-s3]: https://tools.ietf.org/html/rfc2606#section-3
+[rfc6570]: https://tools.ietf.org/html/rfc6570
+[rfc6570-s3]: https://tools.ietf.org/html/rfc6570#section-3
+[rfc7231-s3.4]: https://tools.ietf.org/html/rfc7231#section-3.4
+[rfc7540-s8]: https://tools.ietf.org/html/rfc7540#section-8

ref-engine-discovery.md

@@ -0,0 +1,81 @@
+# OCI Ref-engine Discovery
+
+This is version 0.1 of this specification.
+
+To faciliate communication between image publishers and consumers, this specification defines a [ref-engine discovery](#ref-engine-discovery) protocol which publishers can used to direct consumers towards [reference engines](#ref-engine).
+Having retrieved a set of reference engines (via this and other protocols), consumers can recover a set of [Merkle roots](#merkle-root) potentially associated with a given image name.
+Merkle roots may suggest [CAS engines](#cas-engine), e.g. via a `casEngines` entry in their JSON, but that is out of scope for ref-engine discovery.
+
+## Glossary
+
+### Ref-engine discovery
+
+A service that suggests possible [ref engines](#ref-engine).
+Thie specification defienes a ref-engine discovery protocol.
+
+### CAS engine
+
+A service that provides access to [content-addressable storage][cas].
+
+### Merkle root
+
+The root node in a [Merkle tree][Merkle-tree].
+In the OCI ecosystem, Merkle links are made via [descriptors][descriptor].
+The Merkle root may be a descriptor ([media type][media-type] [`application/vnd.oci.descriptor.v1+json`][descriptor]), or it may have a different media type.
+
+### Ref engine
+
+A service that maps an image name to a set of potential [Merkle roots](#merkle-root).
+
+## Well-known URI
+
+For image names which are supported for [`dns-compatible-image-name`](dns-compatible-image-names.md), a ref-engine discovery object MAY be retrieved from the following [well-known URI][well-known]:
+
+    https://{host}/.well-known/oci/ref-engines
+
+If retrieving that resource fails for any reason, consumers SHOULD walk the DNS ancestors of `host`.
+For example, if the `host` extracted from the image name is `a.b.example.com` the well-known URI failed for `a.b.example.com`, the client would fall back to `b.example.com` and, if that too failed, to `example.com`.
+
+The server MUST support requests for media type [`application/vnd.oci.ref-engines.v1+json`](#ref-engines-objects).
+Servers MAY support other media types using HTTP content negotiation, as described in [RFC 7231 section 3.4][rfc7231-s3.4] (which is [also supported over HTTP/2][rfc7540-s8]).
+
+## Ref-engines objects
+
+This section defines the `application/vnd.oci.ref-engines.v1+json` [media type][media-type].
+Content of this type MUST be a JSON object, as defined in [RFC 7159 section 4][rfc7159-s4].
+The object MAY include a `refEngines` entry.
+If set, the `refEngines` entry MUST be an [array][rfc7159-s5].
+Each entry in the `refEngines` array MUST be an [objects][rfc7159-s4] with at least a `protocol` entry specifying the [ref-engine protocol](ref-engine-protocols.md).
+Consumers SHOULD ignore entries which declare unsupported `protocol` values.
+
+### Example
+
+```json
+{
+  "refEngines": [
+    {
+      "protocol": "oci-index-template-v1",
+      "uri": "https://{host}/ref/{name}"
+    },
+    {
+      "protocol": "docker",
+      "uri": "https://index.docker.io/v2",
+      "authUri": "https://auth.docker.io/token",
+      "authService": "registry.docker.io",
+    }
+  ]
+}
+```
+
+The [`oci-index-template-v1` protocol](index-template.md) is [registered](ref-engine-protocols.md).
+The `docker` protocol is currently [unregistered](ref-engine-protocols.md), and is given as sketch of a possible extention protocol.
+
+[CAS]: https://en.wikipedia.org/wiki/Content-addressable_storage
+[descriptor]: https://github.com/opencontainers/image-spec/blob/v1.0.0/descriptor.md
+[media-type]: https://tools.ietf.org/html/rfc6838
+[Merkle-tree]: https://en.wikipedia.org/wiki/Merkle_tree
+[rfc7159-s4]: https://tools.ietf.org/html/rfc7159#section-4
+[rfc7159-s5]: https://tools.ietf.org/html/rfc7159#section-5
+[rfc7231-s3.4]: https://tools.ietf.org/html/rfc7231#section-3.4
+[rfc7540-s8]: https://tools.ietf.org/html/rfc7540#section-8
+[well-known]: https://tools.ietf.org/html/rfc5785

ref-engine-protocols.md

@@ -0,0 +1,12 @@
+## Ref-engine protocols
+
+There are many possible [ref-engine](ref-engine-discovery.md#ref-engine) protocols.
+Having identifiers for the protocols facilitates [ref-engine discovery](ref-engine-discovery.md#ref-engine-discovery) by allowing discovery services to [describe the protocol for suggested ref engines](ref-engine-discovery.md#ref-engines-objects).
+Consumers can then prefer ref engines which implement their favorite protocol and use the appropriate API to connect to them.
+
+This section registers known protocol identifiers and maps them to their specification.
+Anyone may submit new ref-engine protocol identifiers for registration.
+
+| Protocol identifier     | Specification                                               |
+|-------------------------|-------------------------------------------------------------|
+| `oci-index-template-v1` | [OCI index template protocol, version 1](index-template.md) |