Decompose VM instantiation costs and add module cache #1359

graydon · 2024-02-29T10:10:19Z

This is a sketch of the first step in the plan for addressing #1292 and eventually #827 and #1313, namely:

add a bunch of cost types that decompose the current "worst case" VM instantiation cost type
continue to charge the worst case on initial contract upload
store the decomposed cost-type inputs in the ledger, since we can observe them after the initial upload parse
use those decomposed cost-type inputs when doing runtime instantiation
add a module cache
populate the module cache with all modules on host startup
use cached modules during instantiation

This PR has accompanying changes in XDR and wasmi:

Remaining to do:

determine what the correct set of decomposed cost types even is
~~add more code to wasmi to enable observing more inputs~~
add cost-type runners / calibrations for all the decomposed cost-types
protocol-gate this new behaviour
make the linker-loop do less work to match the tighter cost model (i.e. complete Minimize linkage during VM instantiation #1292)
possibly duplicate the set of cost types added here, so we have a cached and uncached version of each, and implement a solution to Recycle VM components in between host calls #827
possibly decompose the set sufficiently to model what will happen when we take wasmi 0.32 and enable lazy translation (i.e. complete Evaluate and adopt wasmi v0.32+ (register machine) #1313)

soroban-env-host/src/host/data_helper.rs

soroban-env-host/benches/common/cost_types/wasm_insn_exec.rs

soroban-env-host/benches/common/cost_types/vm_ops.rs

graydon · 2024-03-09T07:43:55Z

Updated significantly:

Does intra-tx caching
Does minimal linking
Splits parsing from instantiation to, I think, be more ready for wasmi 0.9
Drives parser manually to get refined costs rather than delegating to wasmi
Added calibration and expanded refined costs a bit more
Added checks for complexity of const exprs
Protocol gated
feature="next" gated

This is getting close to where I want to merge it, I think it's a fairly good set of perf wins.

dmkozh

I've looked at the host code (not benchmarks) and it looks good to me, only left a few small comments.

I think we'll need to be able to re-observe the existing tests with "next" feature enabled and with protocol 21 enabled, but that doesn't need to belong to this PR.

soroban-env-host/src/host/data_helper.rs

soroban-env-host/src/vm/parsed_module.rs

soroban-env-host/src/budget/model.rs

soroban-env-host/src/host/data_helper.rs

soroban-env-host/src/budget.rs

soroban-env-host/src/vm/parsed_module.rs

soroban-env-host/src/vm.rs

soroban-env-host/src/vm/module_cache.rs

graydon · 2024-03-15T05:51:11Z

Updated to address all comments (and all revisions from CAP review process today)

soroban-env-host/src/vm/parsed_module.rs

jayz22

Just pointed out a few minor things that are can be addressed later.

jayz22 · 2024-03-26T00:22:49Z

soroban-env-host/benches/worst_case_linear_models.rs

+    // nearly-the-same) nonzero constant term in each `CostComponent`. We
+    // correct this here by zeroing out the constant term in all but the first
+    // `CostComponent` of each set, (and attempting to confirm that they all
+    // have roughly-the-same constant term).


This is an interesting approach. Glad it works.
I would've suggested using the new_baseline_case to account for these types of high constant overhead. Right now the baseline case is used for eliminating measurement noise, but it could be repurposed for here too.

jayz22 · 2024-03-26T15:48:27Z

soroban-env-host/benches/common/cost_types/wasm_insn_exec.rs

+    let mut me = ModEmitter::default().add_bench_import();
+    let names = Vm::get_all_host_functions();
+    for (module, name, arity) in names.iter().take(n) {
+        if *module == "t" {


Nit: if you don't call the add_bench_import before, then you don't have to skip "t" here. Then you get a smooth linear increase in number of imported functions.

soroban-env-host/src/vm/parsed_module.rs

soroban-env-host/src/budget.rs

jayz22 · 2024-03-26T18:19:37Z

soroban-env-host/src/cost_runner/cost_types/vm_ops.rs

 #[derive(Clone)]
 pub struct VmInstantiationSample {
    pub id: Option<Hash>,
    pub wasm: Vec<u8>,
+    pub module: Rc<ParsedModule>,


Nit: it would be more clear to split this into VmParseSample and VmInstantiationSample. I was a bit confused seeing the Parse measurements using the instantiation sample at first glance.

soroban-env-host/src/vm/module_cache.rs

jayz22 · 2024-03-26T18:40:28Z

soroban-env-host/src/test/budget_metering.rs

@@ -383,7 +383,8 @@ fn total_amount_charged_from_random_inputs() -> Result<(), HostError> {
    }

    let actual = format!("{:?}", host.as_budget());
-    expect![[r#"
+    #[cfg(not(feature = "next"))]
+    let expected = expect![[r#"


Nit: the tracker above should include new rows corresponding to the newly added cost types, such that the budget doesn't charge 0 for them (for the checksum to be nontrivial).

jayz22 · 2024-03-27T15:39:26Z

Added #1374 to follow up on setting up ModuleCache, calibrating its cost, and charging for it in the hit path.

This is a set of followups to #1359 to cover missing testing and wiring around the module cache. It includes calibration of the `VmCachedInstantiation` cost type, a shift to using a single shared linker when using a module cache (this turns out to be critical -- the existing code deadlocks on cross-contract calls) and some fairly subtle wiring to patch up costs in recording mode, along with some tests to check that it's mostly working. Several tests remain broken with --feature=next, so I wouldn't say it's quite ready yet. I tried, but am out of time for tonight.

graydon requested review from sisuresh, dmkozh and a team as code owners February 29, 2024 10:10

graydon marked this pull request as draft February 29, 2024 10:10

This was referenced Feb 29, 2024

Add ContractCodeCostInputs and new cost types stellar/stellar-xdr#172

Closed

Take cheaper-instantiation XDR stellar/rs-stellar-xdr#346

Merged

Expose some Module APIs that measure various lengths stellar/wasmi#26

Closed

dmkozh reviewed Feb 29, 2024

View reviewed changes

soroban-env-host/src/host/data_helper.rs Outdated Show resolved Hide resolved

graydon force-pushed the cheaper-instantiation branch from c09eb57 to 21f4af5 Compare March 1, 2024 07:48

jayz22 reviewed Mar 1, 2024

View reviewed changes

graydon force-pushed the cheaper-instantiation branch 5 times, most recently from a702bd5 to b325786 Compare March 9, 2024 07:39

dmkozh reviewed Mar 12, 2024

View reviewed changes

jayz22 reviewed Mar 12, 2024

View reviewed changes

soroban-env-host/src/budget/model.rs Show resolved Hide resolved

soroban-env-host/src/host/data_helper.rs Outdated Show resolved Hide resolved

soroban-env-host/src/budget.rs Show resolved Hide resolved

soroban-env-host/src/vm/parsed_module.rs Outdated Show resolved Hide resolved

sisuresh reviewed Mar 14, 2024

View reviewed changes

soroban-env-host/src/vm.rs Show resolved Hide resolved

sisuresh reviewed Mar 14, 2024

View reviewed changes

soroban-env-host/src/vm/module_cache.rs Outdated Show resolved Hide resolved

graydon force-pushed the cheaper-instantiation branch from b325786 to 641e14c Compare March 15, 2024 05:37

dmkozh reviewed Mar 15, 2024

View reviewed changes

soroban-env-host/src/vm/parsed_module.rs Outdated Show resolved Hide resolved

graydon force-pushed the cheaper-instantiation branch 5 times, most recently from bf8bcf2 to b25cdac Compare March 22, 2024 07:30

graydon mentioned this pull request Mar 22, 2024

Add ContractCodeCostInputs and new cost types stellar/stellar-xdr#177

Merged

Sketch of cheaper instantiation change

6081e24

graydon added 3 commits March 22, 2024 18:44

Fix initial page size maximum

2713f51

Add tests and calibration for new VM instantiation cost model

3e52bdd

Fix vnext test deviations

728c1be

graydon force-pushed the cheaper-instantiation branch from b25cdac to 728c1be Compare March 23, 2024 05:41

graydon added 6 commits March 22, 2024 23:31

fix some clippy issues

2e314e6

Switch back to release wasmi

d9d5c7d

Fix semver violation

5d68c09

Fix latest-clippy issue

b5ed3a0

Revert Cargo.toml churn

dc409fd

Bump cargo-deny, old version appears to be crashing?

dd55466

graydon changed the title ~~Sketch of cheaper instantiation change~~ Decompose VM instantiation costs and add module cache Mar 23, 2024

graydon marked this pull request as ready for review March 23, 2024 07:24

Fix latest-clippy issue

041473f

graydon enabled auto-merge March 23, 2024 07:31

Fix Rust 1.77 size-change in test_expected_size

61bd6f5

jayz22 approved these changes Mar 26, 2024

View reviewed changes

jayz22 mentioned this pull request Mar 26, 2024

Add a new method in Budget that charges several cost types in a group #1370

Closed

sisuresh approved these changes Mar 26, 2024

View reviewed changes

graydon added this pull request to the merge queue Mar 26, 2024

This was referenced Mar 26, 2024

The default constant cost parameter in WasmDataSegmentBytes is non-zero #1371

Closed

Charge for allocation of new ParsedModule #1372

Closed

Merged via the queue into main with commit 41b4ee3 Mar 26, 2024
13 checks passed

graydon deleted the cheaper-instantiation branch March 26, 2024 19:47

jayz22 mentioned this pull request Mar 27, 2024

Properly set up ModuleCache and use cached instantiation model #1374

Closed

graydon mentioned this pull request Mar 28, 2024

Module cache followup #1375

Merged

This was referenced Apr 8, 2024

Recycle VM components in between host calls #827

Closed

Minimize linkage during VM instantiation #1292

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Decompose VM instantiation costs and add module cache #1359

Decompose VM instantiation costs and add module cache #1359

graydon commented Feb 29, 2024 •

edited

Loading

graydon commented Mar 9, 2024

dmkozh left a comment

graydon commented Mar 15, 2024

jayz22 left a comment •

edited

Loading

jayz22 Mar 26, 2024

jayz22 Mar 26, 2024

jayz22 Mar 26, 2024

jayz22 Mar 26, 2024 •

edited

Loading

jayz22 commented Mar 27, 2024

Decompose VM instantiation costs and add module cache #1359

Decompose VM instantiation costs and add module cache #1359

Conversation

graydon commented Feb 29, 2024 • edited Loading

graydon commented Mar 9, 2024

dmkozh left a comment

Choose a reason for hiding this comment

graydon commented Mar 15, 2024

jayz22 left a comment • edited Loading

Choose a reason for hiding this comment

jayz22 Mar 26, 2024

Choose a reason for hiding this comment

jayz22 Mar 26, 2024

Choose a reason for hiding this comment

jayz22 Mar 26, 2024

Choose a reason for hiding this comment

jayz22 Mar 26, 2024 • edited Loading

Choose a reason for hiding this comment

jayz22 commented Mar 27, 2024

graydon commented Feb 29, 2024 •

edited

Loading

jayz22 left a comment •

edited

Loading

jayz22 Mar 26, 2024 •

edited

Loading