Allow type declarations at package scope

[yordis]

This PR allows type, record, variant, enum and flags declarations at package scope in WIT, as proposed in #694. Today, sharing a type vocabulary across interfaces requires inventing an interface to hold it, and that container reaches the artifact as an instance import that nothing calls.

resource is deliberately excluded. Resources carry identity rather than structure, so an interface is what gives a resource its identity, and package scope has nothing to offer there. That is why the grammar names a package-typedef-item rather than reusing typedef-item, which already includes resource-item.

[alexcrichton]

For myself this is the main point of question for me of how these top-level types would be encoded in WIT. This component as-is is not valid (doesn't pass wasm-tools validate), and I assume that you don't want to change the validation rules for components, so could this be updated with an encoding that's valid?

[lukewagner]

Great question! Here's one idea I think might work:

For WIT interfaces like the above, we treat these dependencies on package-level types just like use of an interface that contains the analogous type definition, we just strip out the wrapping instance type:

(component $C
  (type $Point (export "point") (record (field "x" u32) (field "y" u32)))
  (type (export "api") (component
    (import "local:demo/point" (type $Point' (eq $Point)))
    (export "local:demo/api" (instance
      (export "move-to" (func (param "p" $Point')))
    ))
  ))
)

and then for the analogous WIT world (that exports move-to), we move the (import "local:demo/point" (type ...)) inside the inner component type so that it works like a type definition inside an imported interface, just not wrapped in an instance type:

(component $C
  (type $Point (export "point") (record (field "x" u32) (field "y" u32)))
  (type (export "api") (component
    (export "local:demo/api" (component
      (import "local:demo/point" (type $Point' (eq $Point)))
      (export "move-to" (func (param "p" $Point')))
    ))
  ))
)

[lukewagner]

This all looks reasonable to me. I'll plan to leave this open until we get some further implementation feedback.

[alexcrichton]

One point I'm personally a bit murky on: how is this expected to handle merging across packages? The componentization process routinely merges worlds together and various WIT packages, and that works out well today because conventionally everyone picks a unique name for their package and has their own version tracks. With this sort of global namespace, however, that breaks this merging property because now every package must agree on that record point has an x and a y field, and not a z field (or similar).

I could put this a different way which is that I don't feel I fully understand the rationale for adding this as a feature. I feel like this breaks encapsulation of type definitions within WIT packagse and makes something that should be local a global decision. Increasing the risk of not being able to merge world together gives rise to the problem of bindings work in isolation and don't work together which is a major composability hazard.

Additionally this feels a bit weird to allow some types but not others. For example resources aren't allowed, and while I understand the rationale for not including them it means that this is a sort of "wart" where pieces don't compose together.

[yordis]

Additionally this feels a bit weird to allow some types but not others.

Right now, I only need the types for the sake of #695 and since the spec can be additive, I am personally Ok focusing on types right now.

Being said, better wait for @lukewagner since he has more context outside the Annotations as well.

[lukewagner]

With this sort of global namespace, ...

There might be some confusing wording that says "global" in the PR to be clarified, but what I believe is being proposed here (and shown in the WIT-to-WAT encodings) is using the existing package-scoped namespace (wherein any two interface and world names already have to be unique; this just adding a 3rd kind of "thing"). So I think it should fit right in to how we currently merge packages.

As I mentioned in the design issue, I think this is a logical feature in isolation but, more importantly, if we want to do "annotations" in the style of #694 (big if! but maybe let's discuss that there) this feature is a natural basis.

[alexcrichton]

If I've got two packages:

package a:b1 {
  record r { a: u32 }

  world w1 {
      export f1: func() -> r;
  }
}

package a:b2 {
  record r { a: f32 }

  world w2 {
      export f2: func() -> r;
  }
}

and I were to merge those two worlds together, what would the resulting component type be? I suppose when I say "global" I mean for the world-merging process here which isn't quite the same as the package-merging process. Merging packages is easy enough since everything is scoped under a package, but merging worlds is where semantic knowledge comes in (e.g. interfaces are injected, union'd, dedup'd by semver, etc).