Skip to content

Add gas parameters for event validation + calibration harnesses. #1635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Feb 3, 2023

Conversation

raulk
Copy link
Member

@raulk raulk commented Feb 3, 2023

Closes #1109.


Changes

  • Sets the gas parameters for event validation starting from nv18. See below for more details on the choice of gas parameters.
  • Adds a harness for event gas calibration under testing/calibration.
    • It offers two modes: (a) target a specific encoded event size, (b) generate specific event shapes determined by the number of entries, key length, value length.
    • We currently use (b) to calibrate gas parameters by generating events in the same shape as the EVM runtime actor does for LOG{0..4} opcodes.
      • That is, events with 1-5 entries, with 32 byte values, and with 2-char keys (although the data entry carries a 1-char key: d).
      • Note that the EVM runtime drops leading zeros in values, so our calibration targets the worst-case scenario where all topics and the payload have no leading zeros.
    • There's a case for simply dropping (a), but I've kept it because it may come in handy later stages of the FVM roadmap. It compiles but it never runs.
  • Caps the maximum event length to 1024 bytes. This limit can be disarmed by activating the gas-calibration feature.

Choice of gas parameters

These parameters hit the 10gas/ns baseline consistently, with a slight overestimation to cover for the worst case scenario reasonably well (just under 10gas/ns). These parameters do lead to a larger overestimation on the 1-entry case (LOG0 -- pretty rare anyway), but I'm happy living with that.


entries

size

compute_gas
count
mean
std
min
25%
50%
75%
max
1entries 40 2846000 490.0 14.264802 2.191890 8.131429 14.978947 14.978947 15.811111 16.741176
2entries 79 3914600 490.0 12.859819 1.223028 9.103721 12.233125 13.498621 13.498621 14.498519
3entries 118 4983200 490.0 12.285889 1.019456 9.384557 12.458000 12.777436 12.777436 13.113684
4entries 157 6051800 490.0 11.927226 0.987889 9.019076 12.103600 12.350612 12.607917 12.876170
5entries 196 7120400 490.0 11.596974 0.932503 8.466587 11.847587 12.048054 12.048054 12.255422

Here is a visual depiction of the above as emitted from the calibration tools:

image

Here's a closer look at the gas distribution in a violin plot:

gas_distribution

Future direction

This exercise has highlighted the fact that CBOR is not a good choice for an exchange format at the ABI level dealing with (future) untrusted inputs. We found that the costs scale with the number of entries (CBOR list items), which we can't learn prior to parsing the event in some way (we could do an O(n) prevalidation though). It's worth revising this choice going forward. Any changes to the ABI will not have externally visible impacts.

Err(syscall_error!(IllegalArgument; "panic when decoding event cbor from actor").into())
}
};
t.stop();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now capturing a reading even on deserialization errors.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be careful about that. We usually don't time failures because it can mess up the number's we're looking for (e.g., we can stop half-way through parsing something).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes sense here as long as the test harness is controlling for when it's sending malformed input and when it isn't. This allows us to calibrate the costs for invalid CBOR input, although that will be less relevant if/when we move away from the CBOR-based ABI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, fair enough. And the gas charges here are specifically about parsing.

std::iter::repeat_with(move || {
seed = a * seed + c;
seed
seed = Wrapping(a) * seed + Wrapping(c);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure why, but the lack of wrapping arithmetics here caused weird side effects during ASCII string generation, which led to weird characters. This is more correct.

@@ -24,7 +24,7 @@ use num_traits::Zero;
use serde::Serialize;

pub const WASM_COMPILED_PATH: &str =
"../../target/debug/wbuild/fil_gas_calibration_actor/fil_gas_calibration_actor.compact.wasm";
"../../target/release/wbuild/fil_gas_calibration_actor/fil_gas_calibration_actor.compact.wasm";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the culprit for some major headaches. We run the harness in --release profile, which means that we recompile contract changes under the release profile. Unfortunately the calibration harness was picking the actor up from the debug profile, which usually contains a stale version.

@Stebalien -- this might have or might have not affected your readings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't matter. I was timing FVM operations, not things within the actors.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, it should not affect the readings. However, in my case this ended up picking stale actor code more than once, leading to wrong conclusions. So wanted to give you a heads-up just in case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh.... Damn, I see. I think rust-analyzer's background "let me recompile everything for you and melt a hole in your lap" helped there.

@raulk raulk force-pushed the raulk/events-calibration-alternative branch from 11dcd5b to e7c386c Compare February 3, 2023 12:20
@raulk raulk changed the title add gas calibration harness for events. Add gas parameters for event validation + calibration harnesses. Feb 3, 2023
@raulk raulk marked this pull request as ready for review February 3, 2023 14:58
@raulk raulk requested review from Stebalien and arajasek February 3, 2023 14:58
@codecov-commenter
Copy link

codecov-commenter commented Feb 3, 2023

Codecov Report

Merging #1635 (814a60a) into master (7a11399) will increase coverage by 1.61%.
The diff coverage is 0.00%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1635      +/-   ##
==========================================
+ Coverage   51.72%   53.33%   +1.61%     
==========================================
  Files         146      146              
  Lines       14619    14177     -442     
==========================================
  Hits         7561     7561              
+ Misses       7058     6616     -442     
Impacted Files Coverage Δ
fvm/src/gas/price_list.rs 26.38% <0.00%> (-1.74%) ⬇️
fvm/src/kernel/default.rs 14.97% <0.00%> (+3.20%) ⬆️
fvm/src/syscalls/event.rs 0.00% <0.00%> (ø)
shared/src/event/mod.rs 0.00% <ø> (ø)
fvm/src/call_manager/default.rs 0.00% <0.00%> (ø)
fvm/src/executor/default.rs 5.26% <0.00%> (+0.65%) ⬆️
fvm/src/call_manager/mod.rs 27.27% <0.00%> (+10.12%) ⬆️
fvm/src/state_tree.rs 84.47% <0.00%> (+20.01%) ⬆️

@raulk raulk force-pushed the raulk/events-calibration-alternative branch from 735bf86 to d216ed0 Compare February 3, 2023 22:11
@raulk raulk force-pushed the raulk/events-calibration-alternative branch from d216ed0 to b70e55b Compare February 3, 2023 22:13
@raulk raulk force-pushed the raulk/events-calibration-alternative branch from ef1f69e to be21bef Compare February 3, 2023 22:32
@raulk
Copy link
Member Author

raulk commented Feb 3, 2023

I've updated the benchmarks to cover power of two byte array lengths in the range 32..8KiB in the last entry in the event, which in EVM land will correspond to the event data (which is of arbitrary length). The longer that value gets, the higher the overestimation. We will retain these parameters because the award us good security, and are not too oversized for reasonable inputs (and we prefer to encourage reasonable inputs!).

validate

Furthermore, @Stebalien and I revised the theoretical accept costs and made some adjustments (increased to 2 memory allocs, and added cost of hashing). These costs are also somewhat overestimated but award good security (this diagram shows only 1 memcpy and 1 alloc; hiding all deferred costs that happen when committing the events).

accept

@raulk raulk enabled auto-merge (squash) February 3, 2023 23:42
@raulk raulk merged commit c4c808a into master Feb 3, 2023
@raulk raulk deleted the raulk/events-calibration-alternative branch February 3, 2023 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Events: parameterize gas for events
4 participants