fix: blob size validation #562

avichalp · 2025-03-06T23:39:59Z

Summary

This PR addresses issue #458 where malicious users could upload large blobs while declaring smaller sizes in the SDK, resulting in validators storing more data than users are charged for.

Problem

In the current implementation, users first stage data in a validator (either using Iroh's reverse upload or directly). Without proper validation, a user could:

Upload a large blob (e.g., 1GB)
But override the size parameter in the SDK to a smaller value (e.g., 1MB)
Result: User is charged for storing only 1MB while validators are forced to store the full 1GB

Solution

The PR implements size validation at multiple checkpoints throughout the blob lifecycle:

Size Verification During Set Pending: Added a validation check in set_blob_pending that verifies the blob's actual size matches the claimed size before changing status to pending

if blob.size != size {
    return Err(ActorError::assertion_failed(format!(
        "blob {} size mismatch (expected: {}; actual: {})",
        hash, size, blob.size
    )));
}

Size Validation During Download: Added validation in the Iroh resolver to ensure downloaded blob size matches the expected size

if res.local_size + res.downloaded_size != size {
    return Err(anyhow!(
        "downloaded blob size {} does not match expected size {}",
        res.local_size + res.downloaded_size,
        size
    ));
}

Size Information Propagation: Modified relevant data structures to carry size information throughout the resolution process:
- Added size to BlobRequest, PendingBlob, and FinalizedBlob structures
- Updated BlobPoolItem to include size parameter
- Modified Iroh resolver interfaces to include size validation

Key Changes

Data Structure Updates:
- Modified BlobRequest to include blob size: (Hash, u64, HashSet<(Address, SubscriptionId, PublicKey)>)
- Added size field to FinalizedBlob and PendingBlob structures
- Updated BlobPoolItem to include size information
Function Modifications:
- Updated get_added_blobs and get_pending_blobs to return blob size
- Modified set_blob_pending to accept and validate size parameter
- Updated Iroh resolver to verify downloaded sizes
Validation Points:
- Blob actor validates size matches during pending status changes
- Iroh download process validates actual downloaded size matches claimed size

This fix ensures users cannot exploit the system by misrepresenting blob sizes, protecting validators from storing more data than they're compensated for.

Resolves #458

avichalp · 2025-03-07T20:24:51Z

fendermint/actors/blobs/src/state.rs

+                let blob = blobs
+                    .get(&hash)?
+                    .ok_or_else(|| ActorError::not_found(format!("blob {} not found", hash)))?;
+                Ok((hash, blob.size, sources))


Gets the size of the newly added blobs and return it to Fendermint

avichalp · 2025-03-07T20:26:07Z

fendermint/actors/blobs/src/state.rs

+                let blob = blobs
+                    .get(&hash)?
+                    .ok_or_else(|| ActorError::not_found(format!("blob {} not found", hash)))?;
+                Ok((hash, blob.size, sources))


Returns the size of pending blobs to Fendermint.

avichalp · 2025-03-07T21:05:49Z

fendermint/vm/interpreter/src/chain.rs

                        env.blob_pool.add(BlobPoolItem {
                            subscriber: blob.subscriber,
                            hash: blob.hash,
+                            size: blob.size,


Added size in the pool item so it can be checked against the download blob.

avichalp · 2025-03-07T21:11:23Z

fendermint/actors/blobs/src/actor.rs

 /// The return type used when fetching "added" or "pending" blobs.
 /// See `get_added_blobs` and `get_pending_blobs` for more information.
-type BlobRequest = (Hash, HashSet<(Address, SubscriptionId, PublicKey)>);
+type BlobRequest = (Hash, u64, HashSet<(Address, SubscriptionId, PublicKey)>);


Returns the size of the blob as u64 so that it could be added to the resolution pool

avichalp · 2025-03-07T21:21:19Z

fendermint/vm/message/src/ipc.rs

    /// The blake3 hash of the blob.
    pub hash: Hash,
+    /// The size of the blob.
+    pub size: u64,


FinalizedBlob needs to have size to construct BlobPoolItem again so it can be removed from the pool.

Signed-off-by: avichalp <hi@avichalp.me>

sanderpick

very nice! big improvement. i left one question about changing the toolchain version

sanderpick · 2025-03-12T14:36:32Z

fendermint/actors/blobs/src/state.rs

 use recall_ipld::hamt::{BytesKey, MapKey};

+type BlobSourcesResult =
+    anyhow::Result<Vec<(Hash, u64, HashSet<(Address, SubscriptionId, PublicKey)>)>, ActorError>;


at some point we should make these giant types into structs

sanderpick · 2025-03-12T14:41:43Z

rust-toolchain.toml

@@ -1,4 +1,4 @@
 [toolchain]
-channel = "1.81.0"
+channel = "1.82.0"


do we need to do this on this PR? I think CI will need to be changed as well... esp. the docker build and publish

i will revert this for now

This reverts commit f238fa4. Signed-off-by: avichalp <hi@avichalp.me>

avichalp force-pushed the avichalp/size-quorum branch from 46cfc09 to 9f1e487 Compare March 7, 2025 04:53

avichalp changed the title ~~feat: blob size validation~~ fix: blob size validation Mar 7, 2025

avichalp commented Mar 7, 2025

View reviewed changes

avichalp force-pushed the avichalp/size-quorum branch from 1f5a5f0 to 48a1a1a Compare March 7, 2025 22:00

avichalp added 2 commits March 7, 2025 16:12

Re-enable direct uploads in objects API

c8a1350

Signed-off-by: avichalp <hi@avichalp.me>

Update rust tool chain

f238fa4

Signed-off-by: avichalp <hi@avichalp.me>

avichalp force-pushed the avichalp/size-quorum branch from 00cac73 to f238fa4 Compare March 7, 2025 23:14

avichalp marked this pull request as ready for review March 8, 2025 00:08

avichalp requested a review from sanderpick March 8, 2025 00:09

sanderpick approved these changes Mar 12, 2025

View reviewed changes

avichalp mentioned this pull request Mar 13, 2025

Refactor big tuples in Blob actor's state into structs #571

Open

Revert "Update rust tool chain"

bbdf1f1

This reverts commit f238fa4. Signed-off-by: avichalp <hi@avichalp.me>

avichalp merged commit 1cfd68d into main Mar 13, 2025
13 checks passed

avichalp deleted the avichalp/size-quorum branch March 13, 2025 03:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: blob size validation #562

fix: blob size validation #562

Uh oh!

avichalp commented Mar 6, 2025 •

edited

Loading

Uh oh!

avichalp Mar 7, 2025 •

edited

Loading

Uh oh!

avichalp Mar 7, 2025

Uh oh!

avichalp Mar 7, 2025

Uh oh!

avichalp Mar 7, 2025

Uh oh!

avichalp Mar 7, 2025

Uh oh!

sanderpick left a comment

Uh oh!

sanderpick Mar 12, 2025

Uh oh!

avichalp Mar 13, 2025

Uh oh!

sanderpick Mar 12, 2025

Uh oh!

avichalp Mar 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fix: blob size validation #562

fix: blob size validation #562

Uh oh!

Conversation

avichalp commented Mar 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

Solution

Key Changes

Uh oh!

avichalp Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanderpick left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

avichalp commented Mar 6, 2025 •

edited

Loading

avichalp Mar 7, 2025 •

edited

Loading