Protocol definition with language-specific idiomatic bindings #12

This is a feature request to add wit-like protocol definition, from which language-specific bindings with correct function names and signatures can be generated.

Wit comparison

Let's consider a simple example: the protocol will export a function called "calculate", and the host will provide functions "get_x" and "get_y" that the plugin can call during the calculation.

The wit protocol would look something like this:

package example:protocol

world my-world {
  import get-x: func() -> float64
  import get-y: func() -> float64

  export run: calculate() -> float64
}

Plugin side

Currently when writing a Rust plugin, we have to manually import the "get_x" and "get_y" functions, and we have to manually export a function "calculate" with manually created signature.

Instead, extism could generate an idiomatic "binding" based on the protocol, in case of Rust, a trait that looks something like this:

trait MyWorld {
    fn calculate() -> f64;
}

In addition, it could define the external "get_x" and "get_y" functions with correct Rust signature:

fn get_x() -> f64 {
     // call the extern function
}
fn get_y() -> f64 {
     // call the extern function
}

All we would have to do is implement the trait, for example:

struct MyPlugin;
impl MyWorld for MyPlugin {
    fn calculate() -> f64 {
        get_x() + get_y()
    }
}

And done.

Host side

Similarly, when writing a rust host for example, we currently have to specify the name of the function we want to call as string, and pass arguments "dynamically" as a slice of extism::Val. When defining the imported functions, the experience is similar.

Instead, we could generate idiomatic function signatures like so:

trait MyWorldImports {
    fn get_x() -> f64;
    fn get_y() -> f64;
}
struct MyWorldInstance<T> { ... }
impl<T: MyWorldImports> MyWorldInstance<T>{
    fn instantiate(module_bytes: &[u8] import_object: T) -> Result<Self> { ... }
    fn calculate() -> f64 { ... }
}

So instead of relying on manually typing out function names and using dynamic signatures, we get a nice trait where we implement the get_x and get_y method with correct type signatures, and we get an instance from instantiate where we can call the calculate method.

Additional notes

I used a lot of simplification in the code examples, but my basis was this project where I tried to use wit-bindgen, but it didn't work because of some error with the component model version.

More importantly, having idiomatic bindings like this is not only better developer experience to use, but also (and perhaps more importantly), when we change the protocol, we get syntax errors instead of runtime errors in places where we forget to update the code.

And of course, add these idiomatic bindings for every language, not just Rust. For example, in Java/C#, use interface instead of a trait, etc.

What format to use

If you decide that this is something you want for extism, then next question would be what protocol format to use. One option would be to generate the host bindings automatically from the plugin code, or vice versa, but I do not think this is the right solution.

Using .wit format directly as-is might also not be optimal - since I'm not sure if extism supports all features of wit, like resources. Also, if you are using the wit format anyway, then you might as well use wit-bindgen instead.

I think a best solution would be to create a new protocol format designed for extism.