Tiered Storage #1832
Tiered Storage #1832
Conversation
RodrigoVillar
added
the
DO NOT MERGE
chain/transaction.go
Outdated
| @@ -300,7 +302,15 @@ func (t *Transaction) PreExecute( | |||
| if err != nil { | |||
| return err | |||
| } | |||
| return bh.CanDeduct(ctx, t.Auth.Sponsor(), im, fee) | |||
|
|
|||
| var st state.Immutable | |||
There was a problem hiding this comment.
Does this detail need to be handled by the transaction execution?
There was a problem hiding this comment.
Nope; changed it so now callers are responsible for translating their values
chain/transaction.go
Outdated
| // We refund here | ||
| refund, err := scope.Refund(r) | ||
| if err != nil { | ||
| return nil, ErrRefundDimensions | ||
| } | ||
| refundFee, err := feeManager.Fee(refund) | ||
| if err != nil { | ||
| return nil, ErrFailedToComputeRefund | ||
| } | ||
| if err := bh.AddBalance(ctx, t.Auth.Sponsor(), ts, refundFee); err != nil { | ||
| return nil, ErrRefundFailed | ||
| } |
There was a problem hiding this comment.
Don't we have all the information needed to handle the refund immediately after fetching, so we can differentiate between tiered storage accessed? Do we need to wait until after execution for some reason?
There was a problem hiding this comment.
Keeping the refund logic inside of transaction.Execute() would be consistent with how we're currently manipulating with the sponsor's balance. It's true that once we've fetched all values from the fetcher (in both the block builder and the validator) that we can immediately compute/apply the refund. However, this would require us to make a call to BalanceHandler outside of transaction.Execute(), which could be considered an antipattern as the sponsor's balance is manipulated only in transaction.Execute().
As where the refund logic is placed inside transaction.Execute(), we could apply the refund after deducting. A property we gain from the current implementation is that refunds are applied only to successful transactions, but I wouldn't mind removing it.
| func (*VM) removeSuffixes(values [][]byte, errors []error) ([][]byte, []error) { | ||
| returnValues := make([][]byte, 0, len(values)) | ||
| returnErrors := make([]error, 0, len(values)) | ||
|
|
||
| // Invariant: len(values) == len(errors) | ||
| for i, v := range values { | ||
| if errors[i] != nil { | ||
| returnErrors = append(returnErrors, errors[i]) | ||
| returnValues = append(returnValues, nil) | ||
| continue | ||
| } | ||
| // Time to unsuffix | ||
| if len(v) < consts.Uint64Len { | ||
| returnErrors = append(returnErrors, ErrValueTooShortForPrefix) | ||
| returnValues = append(returnValues, nil) | ||
| continue | ||
| } | ||
| returnValues = append(returnValues, v[:len(v)-consts.Uint64Len]) | ||
| returnErrors = append(returnErrors, nil) | ||
| } | ||
|
|
||
| return returnValues, returnErrors | ||
| } | ||
|
|
There was a problem hiding this comment.
Does this need to be coupled to the VM? In effect, we are grabbing the state from the VM's merkledb, could we apply the same transformation we normally do to a merkledb view before giving actions read access to it?
This function gives raw access to the statedb, should we leave to the caller (that knows where the information is used) to strip the suffixes / transform the data as needed?
| @@ -397,6 +405,26 @@ func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent | |||
| c.metrics.emptyBlockBuilt.Inc() | |||
| } | |||
|
|
|||
| // Before continuing, we append the executing block's suffix to all existing | |||
| @@ -290,7 +291,12 @@ func (c *Builder) BuildBlock(ctx context.Context, parentView state.View, parent | |||
| } | |||
|
|
|||
| // Execute block | |||
| tsv := ts.NewView(stateKeys, storage) | |||
| unsuffixedStorage, hotkeys, err := state.ExtractSuffixes(storage, height, c.config.Epsilon) | |||
This PR introduces tiered storage into the HyperSDK. By adding tiered storage, users are now refunded if any KV-pairs their TX touches was actually in memory in contrast to the current assumption that data always lives on disk.
TODO:
e2etests