Details on "Reimplement the C# API" for 0.15

[christophwille]

I was just updating the NuGet from 0.12 and literally everything turned red. To clarify the API changes I'll take this as an example:

https://github.com/christophwille/csharp-opa-wasm/blob/master/src/Opa.Wasm/OpaHost.cs

No more IHost, no more Import - basically everything is now runtime on Host?

https://github.com/christophwille/csharp-opa-wasm/blob/master/src/Opa.Wasm.ConsoleSample/Program.cs

No more Engine, no more Store?

Given that the API changed in such a big way, can you describe a bit about why now this way compared to before? (I'd like to fix my code in a way that matches your API). Thanks.

[peterhuene]

Sure, and sorry about the big change, but it should be pretty straight forward to port from 0.12.0.

Reflection (and thus "IHost") is no longer used to link host-provided functions, globals, and memories.

Instead, you create an instance of Wasmtime.Host which has methods for defining functions, globals, and memories, as well as a WASI instance.

Take a quick look at the memory example which showcases most of the changes.

The DefineFunction method takes any Action or Func that has WebAssembly types (i.e. int/long/float/double), as well as a special Caller type that must be the first parameter if used.

The Caller class has methods to get an exported memory from the caller so you can get data being passed to it.

Let me know if you have problems updating your source, I'd be happy to help with anything you run into. Sorry again about the big change, but hopefully this makes the API a little easier to use with increased functionality (specifically, the linking details are now all in Wasmtime itself, so you can easily link in different WASI implementations from the engine as well as provide your own functions).

[peterhuene]

Also, the API docs are now current with 0.15.0.

[christophwille]

I'll go through my codebase later today and let you know if I came across any stumbling blocks.

[christophwille]

So porting is easy christophwille/dotnet-opa-wasm@3f0eac9

One thing I noticed in the benchmark though is that it is slower now. Before, I was loading engine, store, module - and it created the host for the test. Now, the host loads the module, which is like in total 30 times slower, given the old perf https://github.com/christophwille/csharp-opa-wasm/tree/e74ea738214c3d944c6311c5c84dab1186d92290/spikes/BenchmarkOpaWasm this is quite an increase.

What could I properly cache in concurrent scenarios? Eg could I have one Host, but call _host.Instantiate(module) on it multiple times (even in parallel, incl. then calls on those instances in parallel) and this would be safe?

In my case I define & remember a memory in BuildHost(), how would that work with multiple instances?

[peterhuene]

So currently hosts and their instances have thread-affinity, meaning they're not safe to interact with from other threads - this is a documented requirement of the C API and we haven't fully fleshed out the Rust API's Sync/Send on the various engine primitives just yet.

The change to hide the store under the Host should not impact performance, but it does change the use pattern which I think is the source of the problem.

For your use case, I think you should create a Host only once in WasmPolicyExecution which defines the memory and host functions that the modules will use. You should also only load the target module into the host a single time.

Then, in the benchmark function, you would start by zeroing the host's memory (DefineMemory returns a reference to it you can use to get the span to zero it out). Then you would call Host.Instantiate to run the sample application. This would closer approximate what the 0.12.0 code was doing; it might even be slightly faster as we only have to set the Host up once to run multiple instances.

Does that make sense?

[christophwille]

I'll rephrase to see if I got this correct (for simplicity assuming a single wasm):

• the perf I am seeing is for end-to-end newing a Host, defining memory, defining functions, instantiating an instance and running ("one off").
• a shortcut would be to reuse an existing Host after a previous run, zeroing the Memory, and then instantiating a new instance ("reuse")
• also, I could create a pool of similarly configured Host's to dish out available Hosts in a concurrent scenario.
• one wasm only (because others will have differing imports et al).

You might have seen my ASP.NET AuthZ spike in my repo (and ultimately what got me started thinking about OPA in .NET). This will have many different wasms (compiled regos) for different authz policy scenarios - what would be a good scenario then if can't reuse most anything?

The current ~40ms make it more palatable to call out to Gatekeeper.

[peterhuene]

the perf I am seeing is for end-to-end newing a Host, defining memory, defining functions, instantiating an instance and running ("one off").

That's correct. Previously you created a single engine and store, where a single module was loaded into. Most of the perf cost comes from loading the module since that is when the module gets Just-In-Time compiled. With the changes, the host hides the engine and store implementation since most use cases were 1:1 engine and store. As a result, loading the module is done in the inner loop resulting in the noticable slow down.

a shortcut would be to reuse an existing Host after a previous run, zeroing the Memory, and then instantiating a new instance ("reuse")

That's correct. If the host is reused, the module is compiled only once. Instantiating the module is a very cheap operation by comparison.

also, I could create a pool of similarly configured Host's to dish out available Hosts in a concurrent scenario.

I believe that could work.

one wasm only (because others will have differing imports et al).

If you desire different imports, a different host would be required or you can call ClearDefinitions to reset what the host currently defines.

Unfortunately I'm not at all familiar with AuthZ or OPA, so I can't say what would make the most sense for you. However, I feel like this indicates the .NET API isn't quite there yet to cover the case where you want to provide per-instantiation differentiation.

Let me think more on this and see how we can better adapt the API for this use case.

[peterhuene]

At the very least, I think perhaps exposing the store (again) where you can load modules into independent of the host would make sense.

[christophwille]

Being able to inject the Store into a different Host (or something similar) sounds like it would get the most perf given your previous explanations. Unless it isn't thread safe of course.

In my OPA case, getting access to the Memory from the instance would do the trick (in my current implementation, I can only get access to the Memory in BuildHost() and have to keep a reference, because the Instance gives me only a MemoryImport)

[christophwille]

@peterhuene Do you have plans to share/discuss how (if at all) you intend to change the API?

[peterhuene]

Hi @christophwille. Sorry for the slow response. I haven't spent time on designing this yet, but I'm going to work on this now. I'll get back to you soon regarding any potential changes.

[peterhuene]

I propose the following breaking changes:

Split the Host abstraction into two abstractions: Store (this is where you load modules) and Host (this is where you instantiate modules). Hosts would accept a store upon construction.

This would allow for you to create a single store to load one or more modules, but then instantiate the same module into different hosts. Obviously the hosts would need to define the same set of expected imports for the module, but the implementations could then differ.

Would this work for your use case?

[christophwille]

As long as the most expensive part (module compilation) happens only once I am happy. If the Store doesn't do that - then there won't be any real savings, right?