Add RFC for component metadata.

[jfng]

Rendered

[jfng]

Modified as a follow-up to the 2023-11-20 Amaranth meeting.

[whitequark]

I propose changing:

Annotation names must be prefixed by a reversed second-level domain name

to:

Annotation names must be prefixed by a reversed domain name

[whitequark]

I propose changing:

To ensure this, schema URLs must have the following structure: <protocol>://<domain>/schema/<version>/<path>. The version of an annotation schema should match the Python package that implements it.

to:

To ensure this, schema URLs must have the following structure: <protocol>://<domain>/schema/<path>/<version>.json. The version of an annotation schema should match the Python package that implements it.

For these reasons:

1. Even within the amaranth-lang organization, we do not have synchronized versioning, and it is weird to have version as the first item.
2. Ideally the schema should be served with the correct MIME type, and adding .json at the end makes things simpler for the case where the web server is provided by a third party (e.g. GitHub pages).

[jfng]

Updated with @whitequark's proposed changes for URLs.

[whitequark]

The start date is the date of the merge.

[whitequark]

We have discussed this RFC on the 2023-11-27 weekly meeting. We did not reach unanimous consensus as two reviewees (galibert and Chips4Makers) abstained, without objecting to merging.

This isn't a situation we had before so for now the RFC is not actually merged while we figure out what to do next.

On the text of the RFC:

@whitequark commented that reset values must be strings to avoid exhausting numeric precision in JSON parsers.

@zyp commented that it is unclear why reset values should be represented in two's complement, and that they could be just as well represented in signed decimal, which could be easier to parse.

[jfng]

I propose changing:

To ensure this, schema URLs must have the following structure: <protocol>://<domain>/schema/<version>/<path>. The version of an annotation schema should match the Python package that implements it.

to:

To ensure this, schema URLs must have the following structure: <protocol>://<domain>/schema/<path>/<version>.json. The version of an annotation schema should match the Python package that implements it.

Would it make sense to enforce package names in schema URLs? The format could become: <protocol>://<domain>/schema/<package>/<version>/<path>.json.

I think this would give better names to files served at those URLs (e.g. "component.json" instead of "4.0.json").

[whitequark]

How would you determine the package name?

[jfng]

The validator would assume that the string between "/schema/" and the next "/" is the package name. I don't think we need to check that it is actually one, though.

[whitequark]

Would the package name for something in amaranth.lib.foo be amaranth, amaranth.lib.foo, something else?

[jfng]

Would the package name for something in amaranth.lib.foo be amaranth, amaranth.lib.foo, something else?

It would be amaranth. lib.foo may appear in the rest of the path, if needed. We wouldn't force symmetry between Python module names and URL paths.

[whitequark]

Sounds reasonable to me.

[stafverhaegen-chipflow]

In previous IRC discussion I abstained when voting on this RFC. Had today a quick chat with @whitequark on this.
My main worry was about bloat of the metadata and abuse of the data to for example trying to describe scripting things like loops etc. in a data serialization format.
I think these worries are not valid for schema that go through the Amaranth RFC process as these aspects are taken into account during that process. I think it should then be made clear in the RFC and/or documentation that standardized metadata schemas are living under the https://amaranth-lang.org URL and that the RFC process is to be used to propose new Amaranth metadata schemas under this URL.

[anuejn]

I also had a couple of questions regarding the schema-urls and asked them on IRC today.
The result of the discussion was, that it would be nice to add an option to the amaranth-cli to also output the schema when building a netlist and a metadata.json when that part is developed.

in @whitequark's words:

we could have an option outputting the schema as well. eg design.v + design.v.json + design.v.json-schema
the .json-schema would be a mapping from URIs to schemas, as a JSON file

[jfng]

Updated to add:

• string representation of reset values (see Add RFC for component metadata. #30 (comment))
• a better format for $id URLs (see Add RFC for component metadata. #30 (comment))
• a pattern for interface port names (see Implement RFC 30: Component metadata. amaranth#978 (comment))
• clarification of the process for https://amaranth-lang.org schemas (see Add RFC for component metadata. #30 (comment))

[jfng]

This RFC was discussed during the 2024-01-15 meeting:

(@wanda-phi) The constraint for $id URLs would require users of Github Pages to have a repository named "schema", because their URLs are formatted as "https://<username>.github.io/<repo>". This is overly restrictive.

(@whitequark) Annotation names should be removed in favor of just using $id URLs:

• $id can uniquely identify an annotation schema;
• not having to retrieve the name of an annotation from $id removes the need for constraining the latter and solves the issue with Github Pages (and similar hosting providers).
• "com.example.foo.serial" isn't much more readable than "https://example.com/schema/foo/0.1/serial.json", and cannot disambiguate between versions of a schema;

(@whitequark) Annotations can originate from mutable objects (such as a SoC memory map) which would be attached to interfaces rather than signatures, which are immutable. To collect annotations from a Component, the Signature.annotations property should be replaced by a Signature.annotations(self, interface) method that returns an iterable of a Annotation instances specific to interface.

[jfng]

This RFC was discussed during the 2024-01-15 meeting:

(@wanda-phi) The constraint for $id URLs would require users of Github Pages to have a repository named "schema", because their URLs are formatted as "https://<username>.github.io/<repo>". This is overly restrictive.

(@whitequark) Annotation names should be removed in favor of just using $id URLs:

• $id can uniquely identify an annotation schema;

• not having to retrieve the name of an annotation from $id removes the need for constraining the latter and solves the issue with Github Pages (and similar hosting providers).

• "com.example.foo.serial" isn't much more readable than "https://example.com/schema/foo/0.1/serial.json", and cannot disambiguate between versions of a schema;

(@whitequark) Annotations can originate from mutable objects (such as a SoC memory map) which would be attached to interfaces rather than signatures, which are immutable. To collect annotations from a Component, the Signature.annotations property should be replaced by a Signature.annotations(self, interface) method that returns an iterable of a Annotation instances specific to interface.

Updated to address these issues.

[whitequark]

We have discussed this RFC on the 2024-01-22 weekly meeting. The disposition was to merge, without changes.

@jfng Please create a tracking issue for this RFC in the amaranth-lang/amaranth repo, add it to RFC text, and set starting date to today. I will hit merge then. Thank you for this work!

[jfng]

Done.

[whitequark]

Thanks!