Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

avoid key collision on child trie and proof on child trie #2209

Closed
wants to merge 142 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
142 commits
Select commit Hold shift + click to select a range
f6c4bb2
Straight forward move of trie related only change from child-trie-soft
cheme Mar 26, 2019
878a7ae
merge backend stuff from old branch, add insertion of child trie root in
cheme Mar 26, 2019
a464936
commit before wasm_executor change (we will keep old interface for now:
cheme Mar 26, 2019
4de73d7
tabify
cheme Mar 26, 2019
be0e340
Things compile (account change is unoptimized), not for wasm.
cheme Mar 26, 2019
7f3a282
Make SutrieField non public
cheme Mar 27, 2019
4d208db
Compile no std child trie with subtrie query (super costy: do we trust
cheme Mar 27, 2019
6d7f355
Make the single child trie test pass.
cheme Mar 27, 2019
25bcb4c
Proving synch child trie content (TODO it requires deletion tests).
cheme Mar 28, 2019
9eaef35
restore test
cheme Apr 1, 2019
37d9536
Merge branch 'master' into child-trie-soft-min
cheme Apr 1, 2019
67c03a2
Fix test error (wrong vec alloc).
cheme Apr 2, 2019
99dbb5a
tests for no key collision
cheme Apr 3, 2019
3be1802
Merge branch 'master' into child-trie-soft-min
cheme Apr 3, 2019
fec73d0
Revert `storage_root` using child, create a variant for it: see
cheme Apr 4, 2019
20d168d
indentation and remove comment
cheme Apr 4, 2019
d797bd0
Remove some TODOs, fix compile error
cheme Apr 4, 2019
27096aa
plaindb does not need to be keyspaceddb
cheme Apr 4, 2019
3560acc
Merge branch 'master' into child-trie-soft-min
cheme Apr 5, 2019
c3fc432
bump impl_version
cheme Apr 8, 2019
3e417eb
Merge branch 'master' into child-trie-soft-min
cheme Apr 16, 2019
5af490f
fix and 2299
cheme Apr 16, 2019
5a8dddd
Merge branch 'master' into child-trie-soft-min
cheme Apr 16, 2019
8f04f00
Merge branch 'master' into child-trie-soft-min
cheme Apr 26, 2019
8051179
fix keyspace insert and start proof (missing in higher level test case a
cheme Apr 29, 2019
8eac118
remove todo of test
cheme Apr 30, 2019
a65f9f2
Merge branch 'master' into child-trie-soft-min
cheme Apr 30, 2019
d495ef5
Merge branch 'master' into child-trie-soft-min &
cheme May 14, 2019
197d77a
- refactor api, use noderef of read only query (require and additional
cheme May 15, 2019
2ea3c89
Remove SubTrieNode struct
cheme May 16, 2019
99c45ea
put extension in encoding/decoding properly
cheme May 16, 2019
3553ab2
SubTrie struct renaming (and single encode point)
cheme May 16, 2019
47984fe
remove contract prefix, and make use of a child trie prefix.
cheme May 16, 2019
2bf2d7b
fix for overlay before reverting
cheme May 16, 2019
bc7165c
fix2 for prefix in overlay
cheme May 16, 2019
21c3acf
revert (will probably need two variant, but for rpc pr)
cheme May 16, 2019
98b2fc3
break some lines
cheme May 16, 2019
432cb10
Merge branch 'master' into child-trie-soft-min
cheme May 16, 2019
e266dfe
Add module specific seed to generator (accountid is undefined)
cheme May 16, 2019
7f64652
Update core/primitives/src/subtrie.rs
cheme May 17, 2019
999a26e
Merge branch 'master' into child-trie-soft-min and adapt rpc test
cheme May 21, 2019
7bbd681
Merge branch 'child-trie-soft-min' of github.com:cheme/substrate into…
cheme May 21, 2019
87f03b7
child trie additional prefix is bad design for primitive, removing it.
cheme May 21, 2019
f18e002
safer encoding of version.
cheme May 24, 2019
0eaeca0
Merge branch 'master' into child-trie-soft-min (untested)
cheme May 24, 2019
b57319d
missing import
cheme May 24, 2019
1f848d2
bump impl version
cheme May 24, 2019
403df51
Merge branch 'master' into child-trie-soft-min
cheme May 24, 2019
305b60a
Merge branch 'master' into child-trie-soft-min
cheme May 28, 2019
423cfb1
correct version
cheme May 28, 2019
a0ffa31
fix typing
cheme May 28, 2019
6e3bed7
Merge branch 'master' into child-trie-soft-min
cheme May 29, 2019
7411146
misnamed storage key variables.
cheme May 29, 2019
331be51
fix error messages.
cheme May 29, 2019
bc2935c
remove convenience constructor, remove unneeded pub on field.
cheme May 29, 2019
5a87b6a
Merge branch 'master' into child-trie-soft-min
cheme May 29, 2019
f40400b
minor tweaks before reworking docs
cheme May 30, 2019
9bc1ab7
First pass of documentation improvement.
cheme May 30, 2019
c073b21
do not refer to parent_key when it is ParentTrie:
cheme May 30, 2019
d15ca49
child trie renaming did exceed 100 character line.
cheme May 30, 2019
0736b96
Link github issue for TODOs
cheme May 31, 2019
9e0485d
Merge branch 'master' into child-trie-soft-min
cheme May 31, 2019
586b50e
comment incorrect variable
cheme May 31, 2019
65d7485
Merge branch 'master' into child-trie-soft-min
cheme Jun 6, 2019
95a69b2
Makes child trie instantiation a bisschen more explicit.
cheme Jun 6, 2019
d089693
Merge branch 'master' into child-trie-soft-min
cheme Jun 12, 2019
ec69ae0
rename fetch and init trie function.
cheme Jun 12, 2019
76ea14d
moved_child is a bad idea, just delete in top, if it is reinserted this
cheme Jun 14, 2019
b2050c8
moved map is correct (fix usage of child trie to return option<option<)
cheme Jun 14, 2019
5a0cbe1
Add missing wasm boilerplate.
cheme Jun 14, 2019
6e84810
remove moved_ct from overlay in favor to optional in pending.
cheme Jun 14, 2019
7f5694f
Merge branch 'master' into child-trie-soft-min
cheme Jun 14, 2019
5006d73
Use keyspace for storage_cache.
cheme Jun 17, 2019
b85508e
Comment of unsafety of `set_child` if directly exposed.
cheme Jun 17, 2019
0c14777
Merge branch 'master' into child-trie-soft-min
cheme Jun 17, 2019
10c4f58
use named fields.
cheme Jun 17, 2019
453927b
Merge branch 'master' into child-trie-soft-min
cheme Jun 21, 2019
349f9a5
Merge branch 'master' into child-trie-soft-min
cheme Jun 25, 2019
fc034fb
Refacto of keyspace to use static creation unique info.
cheme Jun 27, 2019
45cfbd6
put child trie content under their own prefix (with common root with
cheme Jun 28, 2019
4f9717e
Revert to storing keyspace in state approach.
cheme Jun 28, 2019
7cb2d84
invalid asumption for `child_delta_trie_root`.
cheme Jun 28, 2019
c821a08
clean unused child storage key relative methods.
cheme Jun 28, 2019
c3ba830
Merge branch 'master' into child-trie-soft-min and
cheme Jun 28, 2019
acf9641
invalid comment
cheme Jun 28, 2019
0050457
indent and remove deprecated comments/struct
cheme Jun 28, 2019
c36b91b
Make top trie write through keyspace db, this is not going fine because
cheme Jul 2, 2019
9d3d9e2
Fix issue: prefixed_key already got the keyspace in prefix.
cheme Jul 2, 2019
3db4da8
Merge branch 'master' into child-trie-soft-min
cheme Jul 2, 2019
4348d70
Address review comment:
cheme Jul 4, 2019
134a4bf
Merge branch 'master' into child-trie-soft-min
cheme Jul 4, 2019
5325621
Restore auth version.
cheme Jul 4, 2019
97118e8
Avoid confusing Option<Option< in result return of overlay.
cheme Jul 4, 2019
0ed7f80
Missing fix from review on testing ext for set child trie (code from
cheme Jul 4, 2019
8a6986a
Use fields for change trie storage.
cheme Jul 4, 2019
7652de9
switch MapTransaction to a struct.
cheme Jul 4, 2019
94c629e
Merge branch 'master' into child-trie-soft-min
cheme Jul 4, 2019
7675740
Apply same encoding for parent root (compatible with implementation of a
cheme Jul 1, 2019
90fba8a
Fix previous cherry pick
cheme Jul 4, 2019
d8c58c6
Addressing nits.
cheme Jul 4, 2019
661ba2e
Avoid unused creation of child trie.
cheme Jul 4, 2019
833e9ff
Create child trie id from additional storage state.
cheme Jul 5, 2019
71bda12
update default encoding.
cheme Jul 5, 2019
08b3062
Handle possible switch to multiple trie layer.
cheme Jul 5, 2019
89f3cd5
Remove a redundancy on keyspaceddb instantiation.
cheme Jul 5, 2019
834f52a
actually remove redundancy correctly
cheme Jul 5, 2019
0445228
comment
cheme Jul 5, 2019
75add99
Merge branch 'child-trie-soft-min-old' into child-trie-soft-min
cheme Jul 5, 2019
e70edab
Merge branch 'master' into child-trie-soft-min and apply review
cheme Jul 9, 2019
b8a0cd4
Merge branch 'master' into child-trie-soft-min
cheme Jul 10, 2019
1514171
Merge branch 'master' into child-trie-soft-min
cheme Jul 10, 2019
a7da811
Fix new test cases.
cheme Jul 10, 2019
b96c523
Merge branch 'master' into child-trie-soft-min
cheme Jul 11, 2019
865672d
Merge branch 'master' into child-trie-soft-min
cheme Jul 12, 2019
b6d7705
bump runtime version
cheme Jul 12, 2019
b45344c
Merge branch 'master' into child-trie-soft-min
cheme Jul 18, 2019
517f95c
Merge branch 'master' into child-trie-soft-min
cheme Jul 31, 2019
79a07de
Merge branch 'master' into child-trie-soft-min
cheme Aug 2, 2019
a027fb0
Some doc enhancement.
cheme Aug 2, 2019
b92655c
ChildRead correctly defined (previous definition was wrong).
cheme Aug 2, 2019
8a875f4
child_trie 'is_updatable_with' to avoid already broken redundancy.
cheme Aug 2, 2019
fcd8bdb
fix system bench error.
cheme Aug 2, 2019
3241fc4
Fix client doc rust code.
cheme Aug 2, 2019
ecac03b
Merge branch 'master' into child-trie-soft-min
cheme Aug 2, 2019
10369b9
Merge branch 'master' into child-trie-soft-min
cheme Aug 7, 2019
15a2967
Merge branch 'master' into child-trie-soft-min
cheme Aug 8, 2019
acc94e4
Merge branch 'master' into child-trie-soft-min
cheme Aug 8, 2019
7d96338
Remove unused method (in favor of assimilate).
cheme Aug 8, 2019
ba7bdcb
Merge branch 'child-trie-soft-min' of github.com:cheme/polkadot into …
cheme Aug 14, 2019
aacea85
Merge branch 'master' into child-trie-soft-min
cheme Aug 14, 2019
6789641
merge fix.
cheme Aug 14, 2019
2b37160
Rename MapTransaction to StorageContent.
cheme Aug 14, 2019
b1183bd
Get subscription stuff from child-deletion branch:
cheme Aug 14, 2019
bc5653c
Merge branch 'master' into child-trie-soft-min
cheme Aug 22, 2019
5ecfd19
update to master.
cheme Aug 22, 2019
baf89c8
Cast explicitelly for readability.
cheme Aug 26, 2019
0f8bff9
bump spec version.
cheme Aug 26, 2019
5a8576b
Merge branch 'master' into child-trie-soft-min
cheme Aug 29, 2019
045ee32
Merge branch 'master' into child-trie-soft-min
cheme Aug 29, 2019
58e6e41
Fix compilation.
cheme Aug 29, 2019
70555c5
Merge branch 'master' into child-trie-soft-min with conflicts.
cheme Sep 5, 2019
130e5e4
build passing, regression on extrinsics for a set_child (see new TODO).
cheme Sep 6, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add missing wasm boilerplate.
  • Loading branch information
cheme committed Jun 14, 2019
commit 5a0cbe15d71ca73e2293516dfe2cc8fbe8b6f342
68 changes: 68 additions & 0 deletions core/executor/src/wasm_executor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,74 @@ impl_function_executor!(this: FunctionExecutor<'e, E>,
debug_trace!(target: "sr-io", "free {}", addr);
Ok(())
},
ext_child_trie(
storage_key_data: *const u8,
storage_key_len: u32,
a: *mut *mut u8,
b: *mut u32,
c: *mut *mut u8,
d: *mut u32,
e: *mut *mut u8,
f: *mut u32,
g: *mut *mut u8,
h: *mut u32
) -> u32 => {
let storage_key = this.memory.get(storage_key_data, storage_key_len as usize)
.map_err(|_| UserError("Invalid attempt to determine storage_key in ext_get_child_trie"))?;

Ok(if let Some(ct) = this.ext.child_trie(&storage_key) {

let mut alloc_vec =
|value: Option<&[u8]>, dest_vec, dest_len| -> std::result::Result<(), UserError> {
if let Some(value) = value {
let offset = this.heap.allocate(value.len() as u32)? as u32;
this.memory.set(offset, &value)
.map_err(|_| UserError("Invalid attempt to set memory in ext_child_trie"))?;
this.memory.write_primitive(dest_len, value.len() as u32)
.map_err(|_| UserError("Invalid attempt to write length in ext_child_trie"))?;
this.memory.write_primitive(dest_vec, offset)
.map_err(|_| UserError("Invalid attempt to write vec ptr in ext_child_trie"))?;
} else {
this.memory.write_primitive(dest_len, u32::max_value())
.map_err(|_| UserError("Invalid attempt to write failed length in ext_child_trie"))?;
}
Ok(())
};
let p = ct.unsafe_to_ptr_vec();
alloc_vec(Some(p.0), a, b)?;
alloc_vec(p.1, c, d)?;
alloc_vec(Some(p.2), e, f)?;
alloc_vec(Some(p.3), g, h)?;
1
} else {
0
})
},
ext_set_child_trie(
a: *const u8,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Please use descriptive parameter names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I should have made the change already (this is non consistent with existing code).
It is funny how I really like it better without descriptive name, is it me, but I find it less error prone with number or single letter (for boilerplate code of course).
Long descriptive name tend to be difficult to read when checking (I remember a bug where the xxx_len did not match the xxx name and it only got clear when debugging).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, but now when a code reader has to make sense what is the difference between

let f4 = this.memory.get(g, h as usize)

and

let f3 = this.memory.get(e, f as usize)

b: u32,
c: *const u8,
d: u32,
e: *const u8,
f: u32,
g: *const u8,
h: u32
) -> u32 => {
let f1 = this.memory.get(a, b as usize)
.map_err(|_| UserError("Invalid attempt to determine f1 in ext_set_child_trie"))?;
let f2 = if d == u32::max_value() {
None
} else {
Some(this.memory.get(c, d as usize)
.map_err(|_| UserError("Invalid attempt to determine f2 in ext_set_child_trie"))?)
};
let f3 = this.memory.get(e, f as usize)
.map_err(|_| UserError("Invalid attempt to determine f3 in ext_set_child_trie"))?;
let f4 = this.memory.get(g, h as usize)
.map_err(|_| UserError("Invalid attempt to determine f4 in ext_set_child_trie"))?;
let ct = ChildTrie::unsafe_from_ptr_vecs(f1, f2, f3, f4);
Ok(if this.ext.set_child_trie(ct) { 1 } else { 0 })
},
ext_set_storage(key_data: *const u8, key_len: u32, value_data: *const u8, value_len: u32) => {
let key = this.memory.get(key_data, key_len as usize)
.map_err(|_| UserError("Invalid attempt to determine key in ext_set_storage"))?;
Expand Down
20 changes: 17 additions & 3 deletions core/primitives/src/child_trie.rs
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,17 @@ impl ChildTrie {
self.extension.len() as u32,
)
}
/// Function to rebuild child trie accessed from
/// Function to access child trie field unsafely (for memcopy).
pub fn unsafe_to_ptr_vec(&self) -> (&[u8], Option<&[u8]>, &[u8], &[u8]) {
(
self.keyspace.as_ref(),
self.root.as_ref().map(|r| r.as_ref()),
self.parent.as_ref(),
self.extension.as_ref(),
)
}

/// Function to rebuild child trie accessed from.
pub fn unsafe_from_ptr_child_trie(pct: PtrChildTrieMut) -> Self {
let (
keyspace,
Expand All @@ -302,9 +312,13 @@ impl ChildTrie {
ChildTrie { keyspace, root, parent, extension }
}
}
/// Function to rebuild child trie accessed from mem copied field.
pub fn unsafe_from_ptr_vecs(a: Vec<u8>, b: Option<Vec<u8>>, c: Vec<u8>, d: Vec<u8>) -> Self {
ChildTrie { keyspace: a, root: b , parent: c, extension: d }
}

}

// this is redundant with runtime io without_std TODO EMCH move to some util crate
unsafe fn from_raw_parts(ptr: *mut u8, len: u32) -> Option<Vec<u8>> {
if len == u32::max_value() {
None
Expand Down Expand Up @@ -337,12 +351,12 @@ type PtrChildTrieMut = (
u32,
);


impl AsRef<ChildTrie> for ChildTrie {
fn as_ref(&self) -> &ChildTrie {
self
}
}

/// Builder for `KeySpace`.
/// Implementation of this trait must ensure unicity of generated `KeySpace` over the whole runtime context.
/// In the context of deterministic generation this can be difficult, so
Expand Down
20 changes: 15 additions & 5 deletions core/sr-io/without_std.rs
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,12 @@ pub mod ext {
/// Set value for key in storage.
fn ext_set_storage(key_data: *const u8, key_len: u32, value_data: *const u8, value_len: u32);
/// Get child trie at a storage location.
fn ext_get_child_trie(
///
/// # Returns
///
/// - `1` if child trie found and content written.
/// - `0` otherwhise.
fn ext_child_trie(
storage_key_data: *const u8,
storage_key_len: u32,
a: *mut *mut u8,
Expand All @@ -214,8 +219,12 @@ pub mod ext {
f: *mut u32,
g: *mut *mut u8,
h: *mut u32
) -> bool;
) -> u32;
/// Set child trie return false if there is an attempt to change non empty root.
/// # Returns
///
/// - `1` if set successfull
/// - `0` if not
fn ext_set_child_trie(
a: *const u8,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Use descriptive variable names.

b: u32,
Expand All @@ -225,7 +234,7 @@ pub mod ext {
f: u32,
g: *const u8,
h: u32
) -> bool;
) -> u32;
/// Remove key and value from storage.
fn ext_clear_storage(key_data: *const u8, key_len: u32);
/// Checks if the given key exists in the storage.
Expand Down Expand Up @@ -618,7 +627,7 @@ impl StorageApi for () {
let mut g = ptr::null_mut();
let mut h = 0u32;
unsafe {
if ext_get_child_trie.get()(
if ext_child_trie.get()(
storage_key.as_ptr(),
storage_key.len() as u32,
&mut a as *mut _,
Expand All @@ -629,7 +638,7 @@ impl StorageApi for () {
&mut f,
&mut g as *mut _,
&mut h,
) {
) == 1 {
Some(ChildTrie::unsafe_from_ptr_child_trie((a, b, c, d, e, f, g, h)))
} else {
None
Expand All @@ -642,6 +651,7 @@ impl StorageApi for () {
unsafe {
let p = ct.unsafe_ptr_child_trie();
ext_set_child_trie.get()(p.0, p.1, p.2, p.3, p.4, p.5, p.6, p.7)
== 1
}
}

Expand Down