Authenticator Fuzzer #14869

zi0Black · 2024-10-04T09:55:15Z

Description

An early stage fuzzer for Authenticators. This PR exists only to push it on OSS-Fuzz, while working on the other strategies and refactoring the code.
Brought back coverage report generation in fuzz.sh.

New:

aptosvm_authenticators.rs
./fuzz.sh coverage
Arbitrary implementations for custom types

How Has This Been Tested?

Local build and runs.

Key Areas to Review

N/A

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-10-04T09:55:19Z

⏱️ 3h 27m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
execution-performance / single-node-performance	25m	🟩
dispatch_event	16m	🟩
dispatch_event	16m	🟩
dispatch_event	15m	🟩
dispatch_event	15m	🟩
rust-cargo-deny	13m	🟩 🟩 🟩 🟩 🟩 (+2 more)
check-dynamic-deps	11m	🟩 🟩 🟩 🟩 🟩 (+2 more)
rust-move-tests	10m	🟩
rust-move-tests	10m	🟩
rust-move-tests	10m	🟩
rust-move-tests	10m	🟩
rust-move-tests	10m	🟩
rust-move-tests	10m	🟩
execution-performance / test-target-determinator	6m	🟩
rust-move-tests	6m	⬜

🚨 2 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
execution-performance / single-node-performance	25m	16m
execution-performance / test-target-determinator	6m	4m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

testsuite/fuzzer/fuzz.sh

Co-authored-by: Gerardo Di Giacomo <gerardo@aptoslabs.com>

alinush · 2024-10-08T17:45:23Z

crates/aptos-crypto/src/ed25519/ed25519_keys.rs

@@ -38,6 +38,15 @@ impl Clone for Ed25519PrivateKey {
 #[derive(DeserializeKey, Clone, SerializeKey)]
 pub struct Ed25519PublicKey(pub(crate) ed25519_dalek::PublicKey);

+#[cfg(any(test, feature = "fuzzing"))]


So this just picks uniform random bytes, right?

If so, this will not be a valid PK.

The Q is: do you want it to be?

I think I should either move to a custom implementation of arbitrary or get rid of it. It comes in handy for starting a really dumb fuzzer without having to recreate too many types.

alinush · 2024-10-08T17:46:46Z

crates/aptos-crypto/src/ed25519/ed25519_sigs.rs

@@ -18,6 +18,15 @@ use std::{cmp::Ordering, fmt};
 #[derive(DeserializeKey, Clone, SerializeKey)]
 pub struct Ed25519Signature(pub(crate) ed25519_dalek::Signature);

+#[cfg(any(test, feature = "fuzzing"))]
+impl<'a> arbitrary::Arbitrary<'a> for Ed25519Signature {
+    fn arbitrary(u: &mut arbitrary::Unstructured<'a>) -> arbitrary::Result<Self> {


Same point here as for the PK.

It's even trickier here because a "valid" signature is defined w.r.t. to a PK and a message, so you cannot even sample it randomly without having those.

alinush · 2024-10-08T17:50:15Z

testsuite/fuzzer/fuzz/fuzz_targets/move/aptosvm_authenticators.rs

+        FuzzerTransactionAuthenticator::Keyless {
+            sender: _,
+            style,
+            any_keyless_public_key,


How is this sampled? Via the Arbitrary trait?

How do you plan on sampling keyless public keys and signatures that are at least partially-valid (e.g., the EphemeralCertificate in the KeylessSignature verifies under the KeylessPublicKey?)

I guess you will modify them appropriately inside your match style cases, as needed for that fuzzing strategy?

Yes, I can override within the match what I want done in a specific way.

gedigi · 2024-10-04T14:36:39Z

testsuite/fuzzer/fuzz.sh

            echo "    add               adds a new fuzz target"
            echo "    build             builds fuzz targets"
            echo "    build-oss-fuzz    builds fuzz targets for oss-fuzz"
+            echo "    coverage          generates coverage for a fuzz target"


clean-coverage missing?

gedigi · 2024-10-04T14:38:21Z

testsuite/fuzzer/fuzz/fuzz_targets/move/aptosvm_authenticators.rs

+    )
+    .set_not_parallel();
+
+    let sender_acc = if true {


is the if statement needed?

github-actions · 2024-10-11T14:15:27Z

✅ Forge suite `realistic_env_max_load` success on `cbe5413246f81ab0053e2df37ba8d46a1676892b`

two traffics test: inner traffic : committed: 13750.65 txn/s, latency: 2890.99 ms, (p50: 2700 ms, p70: 3000, p90: 3000 ms, p99: 3600 ms), latency samples: 5228340
two traffics test : committed: 100.08 txn/s, latency: 2386.44 ms, (p50: 2400 ms, p70: 2500, p90: 2700 ms, p99: 3000 ms), latency samples: 1840
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.246, avg: 0.219", "QsPosToProposal: max: 0.326, avg: 0.271", "ConsensusProposalToOrdered: max: 0.320, avg: 0.299", "ConsensusOrderedToCommit: max: 0.494, avg: 0.469", "ConsensusProposalToCommit: max: 0.794, avg: 0.768"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.72s no progress at version 2511504 (avg 0.21s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.39s no progress at version 2511502 (avg 8.39s) [limit 15].
Test Ok

github-actions · 2024-10-11T14:16:12Z

✅ Forge suite `framework_upgrade` success on `beff51858b445401e49d5be352feadcf05652cc0` ==> `cbe5413246f81ab0053e2df37ba8d46a1676892b`

Compatibility test results for beff51858b445401e49d5be352feadcf05652cc0 ==> cbe5413246f81ab0053e2df37ba8d46a1676892b (PR)
Upgrade the nodes to version: cbe5413246f81ab0053e2df37ba8d46a1676892b
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1250.81 txn/s, submitted: 1253.57 txn/s, failed submission: 2.76 txn/s, expired: 2.76 txn/s, latency: 2560.89 ms, (p50: 2100 ms, p70: 2600, p90: 4500 ms, p99: 6000 ms), latency samples: 108780
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1220.16 txn/s, submitted: 1222.60 txn/s, failed submission: 2.43 txn/s, expired: 2.43 txn/s, latency: 2664.59 ms, (p50: 2100 ms, p70: 2900, p90: 4500 ms, p99: 7200 ms), latency samples: 100220
5. check swarm health
Compatibility test for beff51858b445401e49d5be352feadcf05652cc0 ==> cbe5413246f81ab0053e2df37ba8d46a1676892b passed
Upgrade the remaining nodes to version: cbe5413246f81ab0053e2df37ba8d46a1676892b
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1290.05 txn/s, submitted: 1292.42 txn/s, failed submission: 2.37 txn/s, expired: 2.37 txn/s, latency: 2629.05 ms, (p50: 2400 ms, p70: 2700, p90: 4500 ms, p99: 6000 ms), latency samples: 108940
Test Ok

github-actions · 2024-10-11T14:18:26Z

✅ Forge suite `compat` success on `beff51858b445401e49d5be352feadcf05652cc0` ==> `cbe5413246f81ab0053e2df37ba8d46a1676892b`

Compatibility test results for beff51858b445401e49d5be352feadcf05652cc0 ==> cbe5413246f81ab0053e2df37ba8d46a1676892b (PR)
1. Check liveness of validators at old version: beff51858b445401e49d5be352feadcf05652cc0
compatibility::simple-validator-upgrade::liveness-check : committed: 11820.14 txn/s, latency: 2872.69 ms, (p50: 1900 ms, p70: 2100, p90: 3100 ms, p99: 27200 ms), latency samples: 452340
2. Upgrading first Validator to new version: cbe5413246f81ab0053e2df37ba8d46a1676892b
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 5834.23 txn/s, latency: 4917.77 ms, (p50: 5500 ms, p70: 5800, p90: 6300 ms, p99: 6400 ms), latency samples: 112540
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 5433.92 txn/s, latency: 5979.05 ms, (p50: 6400 ms, p70: 6600, p90: 6900 ms, p99: 8000 ms), latency samples: 179440
3. Upgrading rest of first batch to new version: cbe5413246f81ab0053e2df37ba8d46a1676892b
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7462.08 txn/s, latency: 3875.64 ms, (p50: 4300 ms, p70: 4500, p90: 4700 ms, p99: 4700 ms), latency samples: 143380
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6670.52 txn/s, latency: 4824.35 ms, (p50: 4800 ms, p70: 4900, p90: 7000 ms, p99: 7200 ms), latency samples: 233840
4. upgrading second batch to new version: cbe5413246f81ab0053e2df37ba8d46a1676892b
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 10639.43 txn/s, latency: 2559.91 ms, (p50: 2800 ms, p70: 3000, p90: 3100 ms, p99: 3200 ms), latency samples: 187580
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 9522.98 txn/s, latency: 3256.88 ms, (p50: 2900 ms, p70: 3000, p90: 5900 ms, p99: 7500 ms), latency samples: 326560
5. check swarm health
Compatibility test for beff51858b445401e49d5be352feadcf05652cc0 ==> cbe5413246f81ab0053e2df37ba8d46a1676892b passed
Test Ok

zi0Black added 2 commits September 27, 2024 10:39

Initial authenticator fuzzer commit

45255b4

1/3 fuzzing style for autheticators

31fe26b

zi0Black assigned gedigi Oct 4, 2024

Merge branch 'main' into authenticator-hell-of-types-fuzz

4153a15

zi0Black marked this pull request as ready for review October 4, 2024 09:57

zi0Black requested review from alinush, mstraka100, heliuchuan, zjma and rex1fernando as code owners October 4, 2024 09:57

gedigi reviewed Oct 4, 2024

View reviewed changes

testsuite/fuzzer/fuzz.sh Outdated Show resolved Hide resolved

Fix missing command in helper

1dac3b8

Co-authored-by: Gerardo Di Giacomo <gerardo@aptoslabs.com>

alinush reviewed Oct 8, 2024

View reviewed changes

alinush approved these changes Oct 8, 2024

View reviewed changes

gedigi approved these changes Oct 10, 2024

View reviewed changes

zi0Black added 4 commits October 10, 2024 10:37

Merge branch 'main' into authenticator-hell-of-types-fuzz

979d76b

Remove unused import

572523f

fmt

2274f7b

fmt cargo.toml

8aff6fd

zi0Black enabled auto-merge (squash) October 10, 2024 15:07

This comment has been minimized.

Sign in to view

Merge branch 'main' into authenticator-hell-of-types-fuzz

cbe5413

This comment has been minimized.

Sign in to view

zi0Black merged commit a6ad9d3 into main Oct 11, 2024
49 checks passed

zi0Black deleted the authenticator-hell-of-types-fuzz branch October 11, 2024 14:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Authenticator Fuzzer #14869

Authenticator Fuzzer #14869

zi0Black commented Oct 4, 2024

trunk-io bot commented Oct 4, 2024 •

edited

Loading

alinush Oct 8, 2024 •

edited

Loading

zi0Black Oct 9, 2024

alinush Oct 8, 2024

alinush Oct 8, 2024

alinush Oct 8, 2024

zi0Black Oct 9, 2024

gedigi Oct 4, 2024

gedigi Oct 4, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Oct 11, 2024

github-actions bot commented Oct 11, 2024

github-actions bot commented Oct 11, 2024

Authenticator Fuzzer #14869

Authenticator Fuzzer #14869

Conversation

zi0Black commented Oct 4, 2024

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

trunk-io bot commented Oct 4, 2024 • edited Loading

alinush Oct 8, 2024 • edited Loading

Choose a reason for hiding this comment

zi0Black Oct 9, 2024

Choose a reason for hiding this comment

alinush Oct 8, 2024

Choose a reason for hiding this comment

alinush Oct 8, 2024

Choose a reason for hiding this comment

alinush Oct 8, 2024

Choose a reason for hiding this comment

zi0Black Oct 9, 2024

Choose a reason for hiding this comment

gedigi Oct 4, 2024

Choose a reason for hiding this comment

gedigi Oct 4, 2024

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Oct 11, 2024

✅ Forge suite realistic_env_max_load success on cbe5413246f81ab0053e2df37ba8d46a1676892b

github-actions bot commented Oct 11, 2024

✅ Forge suite framework_upgrade success on beff51858b445401e49d5be352feadcf05652cc0 ==> cbe5413246f81ab0053e2df37ba8d46a1676892b

github-actions bot commented Oct 11, 2024

✅ Forge suite compat success on beff51858b445401e49d5be352feadcf05652cc0 ==> cbe5413246f81ab0053e2df37ba8d46a1676892b

trunk-io bot commented Oct 4, 2024 •

edited

Loading

alinush Oct 8, 2024 •

edited

Loading

✅ Forge suite `realistic_env_max_load` success on `cbe5413246f81ab0053e2df37ba8d46a1676892b`

✅ Forge suite `framework_upgrade` success on `beff51858b445401e49d5be352feadcf05652cc0` ==> `cbe5413246f81ab0053e2df37ba8d46a1676892b`

✅ Forge suite `compat` success on `beff51858b445401e49d5be352feadcf05652cc0` ==> `cbe5413246f81ab0053e2df37ba8d46a1676892b`