Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

accounts/abi: Abi binding support for nested arrays, fixes #15648, including nested array unpack fix #15676

Merged
merged 12 commits into from
Mar 4, 2018

Conversation

protolambda
Copy link
Contributor

@protolambda protolambda commented Dec 14, 2017

The binding generator only matched one level of the multi-dimensional array types that were parsed by the other binding code.

Fix:

  • Method that reads the unmatched characters
  • Looks for a valid array declaration
  • Returns array-sizes of each level (empty if no array)
  • Each language wraps their old (but cleaner) type-translator with the array reader.
  • Each language has a method that translates the combination of inner-type and array-sizes into a typed array declaration.
  • Minor fix: Java used "bool" while this should have been a "boolean". Not that anyone would pick Java bindings over Go 😄
  • Another minor fix: Java used int, also for non-32 bit numbers. I changed this to byte/short/int/long (8/16/32/64 bits) but I'm not sure about the implications. However, it should be better than previously in any case, as a Java int doesn't have sufficient space for the (u)int64.
  • Edit: another bugfix: argument unpacking shifts to the correct index after nested arrays now.

Before merging: This is my first PR here, suggestions & reviews are very much appreciated 👍

@GitCop
Copy link

GitCop commented Dec 14, 2017

Thank you for your contribution! Your commits seem to not adhere to the repository coding standards

  • Commit: b5c402824e901a57a904477727dbf65ea3e33bf6

  • Commit subjects should be kept under 100 characters

  • Commit: aa5257580233c9dab10e5b2de7b770caeb9c50ae

  • Commits must be prefixed with the package(s) they modify

Please check the contribution guidelines for more details.


This message was auto-generated by https://gitcop.com

@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from aa52575 to ac6ff7e Compare December 14, 2017 20:12
@GitCop
Copy link

GitCop commented Dec 14, 2017

Thank you for your contribution! Your commits seem to not adhere to the repository coding standards

  • Commit: b5c402824e901a57a904477727dbf65ea3e33bf6

  • Commit subjects should be kept under 100 characters

  • Commit: ac6ff7e6a278a3f3211125c80b6a3c9728c258c4

  • Commits must be prefixed with the package(s) they modify

Please check the contribution guidelines for more details.


This message was auto-generated by https://gitcop.com

@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from ac6ff7e to 68e3cef Compare December 14, 2017 20:16
@protolambda
Copy link
Contributor Author

@GitCop Added an ":" and split that one commit message, quality suggestions bot 👍

@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from 68e3cef to 0d5e1d1 Compare December 14, 2017 22:47
@protolambda
Copy link
Contributor Author

For no particular good reason, the OSX Go 1.9 travis CI jobs fails:

--- FAIL: TestDeposit (0.24s)

	cheque_test.go:323: expected balance 40, got 42

FAIL

coverage: 77.0% of statements

FAIL	github.com/ethereum/go-ethereum/contracts/chequebook	1.261s

This happens in a package that wasn't touched in this PR, nor uses the accounts/abi/bind package. I suspect this is because of the test being prone to a race condition (Execution literally depends on time.Sleep(...)). @mexskican You worked on this test most recently, do you know what's up?

@fjl fjl requested a review from gballet December 19, 2017 13:11
@holiman
Copy link
Contributor

holiman commented Dec 21, 2017

Thanks for the PR!
Since this adds some new capabilties, could you please provide some testcases that utilise the new features, and verifies the correctness?

I assume you already have examples that you based the implementation on, which you perhaps can place in the bind_test.go file?

@protolambda
Copy link
Contributor Author

@holiman Sure, I'll make some tests. But that'll probably be next week. 🎅

Copy link
Member

@gballet gballet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @holiman said, please add more tests to make sure that nothing got broken, and also address the minor nitpicks in my review. Nice job otherwise!

case "32":
return len(parts[0]), "int"
case "64":
return len(parts[0]), "long"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple remarks here:

  • Instead of a switch, it would make more sense to have the type mapping in some map and write it return len(parts[0]), sizeMap[parts[2]]. On top of being more concise, it would be less confusing to find out that BingInt is the fallback.
  • Can we update the regex to only accept between 1 and 2 digits?

Copy link
Contributor Author

@protolambda protolambda Dec 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gballet

  1. The switch isn't pretty, a map lookup may work better yes. It's not that this code is performance-critical anyway. On the other hand though, the mapping to go keywords is constant, and shouldn't change. And Go doesn't offer a constant map (afaik). So we either have a pseudo-constant map, or a (slightly less verbose) switch case.

  2. No. From the solidity docs: "uint and int are aliases for uint256 and int256, respectively.". And these are mapped to a BigInt. So it's 0 or more digits to parse, and I don't see a reason to limit it with a maximum. (Soldity has a theoretical limit of 2^256, but I don't see why this should be enforced in this regex)

I'll see if I can improve the clarity when I write the tests. 👍

For now, happy holidays 🎄

@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from eb6618d to 7207fc5 Compare December 28, 2017 23:42
@protolambda
Copy link
Contributor Author

protolambda commented Dec 29, 2017

@gballet When preparing the tests, I found another more serious problem related to nested-arrays.

So I was writing the tests for the nested arrays, but discovered that the unpacking of nested arrays happened incorrectly. Packing of nested arrays works correctly, but unpacking only works for non-nested arrays/slices. The problem is that the unpacker assumes that each element of an array is 32 bytes (because of padding), but elements of a (nested) array-type can be different sizes.

Note that the current code still unpacks nested arrays, but the data is read wrongly; it only reads the first entry correctly, and starts incrementing with too small increments from there, misaligning & duplicating the read data.

I fixed it, but I'm not fond of my solution. Unpacking an individual element doesn't tell us the size of the consumed bytes, so I had to find another solution... For now, it calculates the size of the element by packing it, and it works correctly now, but the solution could be way nicer if we let accounts/abi/unpack.go#toGoType return the amount of consumed bytes.

You can find my debugging code here: https://github.com/protolambda/go-eth-nested-array-debug
(I'll move some of it to bind_test.go once the unpacking bug is fixed, and a nested-array test case in unpack_test.go is probably a good idea too).

Edit: I read the ABI spec more extensively, and found a problem with my previously solution of packing; slices can be "seen" as 32 bytes elements, since the contents are located and accessed differently than arrays. However, arrays are still packed in-place, and can be more than 32 bytes. I removed the packing, and found a far more elegant solution to fix the array problem 👍 .

if err != nil {
return nil, err
}
i += len(interPacked)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By packing it, and using the length as the increment, it guarantees correct element alignment (given that packing works correctly). However, getting a "consumed" size from the toGoType call would be much more efficient. I decided to go with this inefficient solution first, since changing toGoType affects other code.

@GitCop
Copy link

GitCop commented Dec 30, 2017

Thank you for your contribution! Your commits seem to not adhere to the repository coding standards

  • Commit: e08174f87ecb6d2342cae5c9175c6b77598664b8
  • Your commit message body contains a line that is longer than 80 characters

Please check the contribution guidelines for more details.


This message was auto-generated by https://gitcop.com

@protolambda
Copy link
Contributor Author

Fixed unpacking of multi-dimensional arrays, without the packing step I took previously, and more efficient. 👍

Added a test for the bindings generated for the deeply nested (more than one level of nesting) arrays, as well as a test for both the unpacking and packing of the arrays.

@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from e08174f to 5d42cf6 Compare December 30, 2017 17:32
@GitCop
Copy link

GitCop commented Dec 30, 2017

Thank you for your contribution! Your commits seem to not adhere to the repository coding standards

  • Commit: 5d42cf60fa1db42bf537983c25946a85fba19422
  • Your commit message body contains a line that is longer than 80 characters

Please check the contribution guidelines for more details.


This message was auto-generated by https://gitcop.com

@protolambda
Copy link
Contributor Author

@GitCop Great. :I

@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from 5d42cf6 to b274e9b Compare December 30, 2017 17:36
@protolambda
Copy link
Contributor Author

protolambda commented Dec 30, 2017

Rebased to resolve merge conflicts, and rebased again to resolve GitCop.

PR should be good to go now.

@gballet Can you take a look?

Edit: noticed that one of my tests still used an import from my test repo, which made it fail on travis CI. This, along with some improved naming, is fixed now.

Edit 2: Forgot to go-fmt 2 lines... Fixed it.

Edit 3: triggered a rebuild, the CI failed to download the linter module

@protolambda protolambda changed the title accounts/abi/bind: Abi binding support for nested arrays, fixes #15648 accounts/abi: Abi binding support for nested arrays, fixes #15648, including nested array unpack fix Dec 30, 2017
@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch 3 times, most recently from 5951552 to c5db3c5 Compare January 3, 2018 00:27
@protolambda
Copy link
Contributor Author

Well, one test, in non-related code, fails in one of the CI jobs because of a race condition... :(

--- FAIL: TestDeposit (0.26s)
	cheque_test.go:323: expected balance 40, got 42
FAIL
coverage: 77.0% of statements

Copy link
Member

@gballet gballet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few comments inline, and a more general one: since there is a lot of code repeats in your bindUnnestedTypes* / arrayBind* , see if instead you can use a general regexp like (address|uint|int|bool...)(\d)* and then use a name conversion map depending on the output language.

//
// Returned array sizes are in the same order as solidity signatures; inner array size first.
// Array sizes may also be "", indicating a dynamic array.
func wrapArray(stringKind string, innerLen int, innerMapping string) (string, []string) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be simpler, more readable and arguably more efficient to use the following regular expression \[(\d*)\] along with FindAllStringSubmatch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, probably a good idea. I'll simplify the implementation 👍

//append a "[]" for each array size.
// (Full arraySizes is used in signature for consistency with the go-lang version)
for i := 0; i < len(arraySizes); i++ {
//normally, like with Go, you could declare an array size. Not in Java.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update this comment to make it more factual like "java array type declarations do not include the length".

We are not interested in starting yet another religious war on the theme of whose language is better. Also, your comment is inaccurate: Java has arrays while Go has slices, and the typing in Java is more dynamic than what Go offers, so not enforcing the array size in the type declaration makes sense (and is therefore 'normal'). You don't like Java? That makes two of us. Still, this is not the place to vent out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, I'll change it. And I wasn't meaning to start a "religious war", it was written jokingly (i.e., you can in Java, but not in the same compact way).

out := inner
//append a "[]" for each array size.
// (Full arraySizes is used in signature for consistency with the go-lang version)
for i := 0; i < len(arraySizes); i++ {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This loop can be rewritten: inner + strings.Repeat("[]", len(arraySizes)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good spot, I started of with the Go implementation, which has changing additions, instead of repeating "[]".

@@ -93,6 +93,17 @@ func readFixedBytes(t Type, word []byte) (interface{}, error) {

}

func getDeepSizeForType(t *Type) int {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking here: I would prefer getTotalArraySize rather than "deep" size.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's like a deep copy; it repeats the same operation up to the deepest array level. Total array size would work too, but it misses the aspect of nesting imho.

elemSize := 32
if t.T == ArrayTy {
elemSize = getDeepSizeForType(t.Elem)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just write those 4 lines: elemSize := getDeepSizeForType(t.Elem) since getDeppSizeForType will return 32 if this is not an array.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it doesn't. It is the element type that is being passed. So it returns 32 when the element is not array. Say you call getDeepSizeForType(t.Elem) with t being a slice, then we need the elemSize to be 32 bytes, since it's only the pointer to the contents of a slice-slot being counted, but t.Elem may be ArrayTy, resulting in something different than the 32 bytes.

{
def: `[{"type": "uint32[2][3][4]"}]`,
enc: "000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000003000000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000050000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000700000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000009000000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000b000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000d000000000000000000000000000000000000000000000000000000000000000e000000000000000000000000000000000000000000000000000000000000000f000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000110000000000000000000000000000000000000000000000000000000000000012000000000000000000000000000000000000000000000000000000000000001300000000000000000000000000000000000000000000000000000000000000140000000000000000000000000000000000000000000000000000000000000015000000000000000000000000000000000000000000000000000000000000001600000000000000000000000000000000000000000000000000000000000000170000000000000000000000000000000000000000000000000000000000000018",
want: [4][3][2]uint32{{{1, 2}, {3, 4}, {5, 6}}, {{7, 8}, {9, 10}, {11, 12}}, {{13, 14}, {15, 16}, {17, 18}}, {{19, 20}, {21, 22}, {23, 24}}},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should also check that errors get caught if e.g. the string is shorter or longer than the total size, as well as if the type specification is wrong.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only fixed the iteration step being incorrect, not anything about the unpacking of types themselves. When the input length doesn't match up by accident, toGoType will handle it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright

@protolambda
Copy link
Contributor Author

there is a lot of code repeats in your bindUnnestedTypes* / arrayBind*

Well, there is, but they're not mine. I dislike the switch cases too (and yes, a mapping for each conversion language would be nicer) but the switch-structure works as the previous author intended, without problems, and I'm fine with that. No need to refactor it even more.

@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from 1dbb74c to 7eb7ef5 Compare January 6, 2018 13:21
@gballet
Copy link
Member

gballet commented Jan 6, 2018

Well, there is, but they're not mine.

I'm aware, I only meant that you gave it a thought. Just because it's been done by someone else doesn't mean they're better at it than you are ^^ This being said, I heard that there is a refactoring on the way so I'll just fix it when it's rolled in. Thank you for your work so far, let me know when you're done with the regexp part and I'll review again.

@protolambda
Copy link
Contributor Author

Well, I found another bug related to nested arrays. When unpacking arguments, it only considered the outer-size of arrays. This resulted in binary contents of nested arrays "overflowing" (effectively) into the contents of the arguments following after the array.

Also see the explanation in the commit message.

@gballet Would you be so kind to re-review again, to include this small bugfix?

Copy link
Member

@gballet gballet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setting it to 'request changes' to make sure the PR doesn't get merged before I've had a chance to look at the new commit

Copy link
Member

@gballet gballet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@protolambda
Copy link
Contributor Author

@karalabe @gballet Is this PR still being looked at?

@holiman
Copy link
Contributor

holiman commented Feb 21, 2018

Now that 1.8.0 is out the door, we're lifting the "pr-embargo" and will start merging a bit more. I'm hoping to merge #15770 soon, and then some other abi-related PRs.

@@ -80,6 +80,21 @@ func (arguments Arguments) Unpack(v interface{}, data []byte) error {
return arguments.unpackAtomic(v, data)
}

// Computes the full size of an array;
// i.e. counting nested arrays, which count towards size for unpacking.
func getArraySize(arr *Type) int {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've done a rebase on master, and find that if don't even use this method, but leave it as is, I get no test failures. I'll push my rebased branch, with the method still present (but unused), and you can take a look and check if

  • The method is not needed, or,
  • An additional testcase is needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check out my my branch: https://github.com/ethereum/go-ethereum/compare/master...holiman:pr_15676?expand=1 . It's rebased on master. PTAL that I didn't miss something, and why it's still working without using getArraySize (I guess the corresponding place in the master-code is here: https://github.com/ethereum/go-ethereum/compare/master...holiman:pr_15676?expand=1#diff-04b076f4fee5b5afe22e6efa614e4aceR200 )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need tests for the argument unpacking then, it may be very uncommon, but there was definitely a bug.

How to reproduce it (will test later with your rebase, this is what needed to be fixed in the original PR starting point):

  1. Some argument value needs to be a nested array. The abi format packs the array pointers in-place, hence no standard 32 byte argument length, but a variable length based on the array.
  2. There needs to be another argument value, after the nested array, that will be offset because of the miscalculated length of the previous argument (the nested array). If there isn't any, then there is (effectively) nothing wrong. But when there is, the argument is read from the wrong position, causing array data from the array argument to be read into content of following argument(s).

So it shows up very rarely, doesn't cause a panic, and only affects multi-valued arguments.

Also consider that there can be multiple levels of nested arrays, all packed together. Hence the for-loop that multiplies them. For completeness: multiplying in the size calculation works because of the array-size symmetry, no need to recursively sum all sizes. Other collections such as slices are just pointers, same size as any other argument, so nothing wrong there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/ethereum/go-ethereum/compare/master...holiman:pr_15676?expand=1#diff-04b076f4fee5b5afe22e6efa614e4aceR200
Is indeed the relevant part, but it doesn't cover arrays with multiple levels of nesting, only the surface. I'll rebase it with the multi-level size calculation, and add a test for it 👍

Also:
- reduce usage of regexes a bit.
- fix minor Java syntax problems

Fixes ethereum#15648
The code previously assumed the arrays/slices were always 1 level
deep. While the packing supports nested arrays (!!!).

The current code for unpacking doesn't return the "consumed" length, so
this fix had to work around that by calculating it (i.e. packing and
 getting resulting length) after the unpacking of the array element.
It's far from ideal, but unpacking behaviour is fixed now.
Removed the temporary workaround of packing to calculate size, which was
incorrect for slice-like types anyway.
Full size of nested arrays is used now.
Test unpacking of an array nested more than one level.
Same as the deep nested array unpack test, but the other way around.
Test the usage of bindings that were generated
for methods with multi-dimensional (and not
just a single extra dimension, like foo[2][3])
array arguments and returns.

edit: trigger rebuild, CI failed to fetch linter module.
wrapArray uses a regex now, and arrayBindingJava is improved.
The full step size for unpacking an array
 is now retrieved with "getFullElemSize".
Previously, the code only considered the outer-size of the array,
ignoring the size of the contents. This was fine for most types,
but nested arrays are packed directly into it, and count towards
the total size. This resulted in arguments following a nested
array to replicate some of the binary contents of the array.

The fix: for arrays, calculate their complete contents size:
 count the arg.Type.Elem.Size when Elem is an Array, and
 repeat when their child is an array too, etc.
The count is the number of 32 byte elements, similar to how it
 previously counted, but nested.
@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from 05e164a to 1566af2 Compare February 21, 2018 22:24
Arguments with a deeply nested array should not cause the next arguments
to be read from the wrong position.
@protolambda protolambda force-pushed the abi-binding-nested-array-fix branch from 1566af2 to c788f13 Compare February 21, 2018 22:25
@holiman
Copy link
Contributor

holiman commented Feb 24, 2018

I missed that you had done this -- and I won't be available to review for a week. Maybe @gballet can review/merge?

Copy link
Member

@gballet gballet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @protolambda Before I merge, I have a final request: could you please find a more descriptive and accurate name than virtualArgs ? This is an offset, and I think the name should reflect that, because this code is complicated (thank you for your effort and patience!) and it will help people reading it to better understand what it is about.

@protolambda
Copy link
Contributor Author

@gballet The name may be a bit confusing yes, but I didn't introduce it. The virtualArgs name was introduced here, by @holiman: protolambda@1ede683#diff-04b076f4fee5b5afe22e6efa614e4aceR212

And if you read closer, it sort of makes sense, it counts things hidden from the surface argument count. Hence the "virtual". I do think they're not quite "arguments" however, more like partial arguments that just so happen to be exactly as long.

Also, I added some extra comments just above the first usage to explain what is happening, but kept the old naming (constantly renaming things is something I try to avoid).

@holiman Do you think it's a good idea to change it?

@gballet gballet merged commit 0b814d3 into ethereum:master Mar 4, 2018
@holiman
Copy link
Contributor

holiman commented Mar 5, 2018

Yup, that was my 'bad'. I also think it sort of makes sense, but I'm not opposed to changing it to something better. Thanks for the hard work, @protolambda , and sorry that it took so long to get it in!

@holiman holiman added this to the 1.8.2 milestone Mar 5, 2018
prestonvanloon pushed a commit to prestonvanloon/go-ethereum that referenced this pull request Apr 2, 2018
…5648, including nested array unpack fix (ethereum#15676)

* accounts/abi/bind: support for multi-dim arrays

Also:
- reduce usage of regexes a bit.
- fix minor Java syntax problems

Fixes ethereum#15648

* accounts/abi/bind: Add some more documentation

* accounts/abi/bind: Improve code readability

* accounts/abi: bugfix for unpacking nested arrays

The code previously assumed the arrays/slices were always 1 level
deep. While the packing supports nested arrays (!!!).

The current code for unpacking doesn't return the "consumed" length, so
this fix had to work around that by calculating it (i.e. packing and
 getting resulting length) after the unpacking of the array element.
It's far from ideal, but unpacking behaviour is fixed now.

* accounts/abi: Fix unpacking of nested arrays

Removed the temporary workaround of packing to calculate size, which was
incorrect for slice-like types anyway.
Full size of nested arrays is used now.

* accounts/abi: deeply nested array unpack test

Test unpacking of an array nested more than one level.

* accounts/abi: Add deeply nested array pack test

Same as the deep nested array unpack test, but the other way around.

* accounts/abi/bind: deeply nested arrays bind test

Test the usage of bindings that were generated
for methods with multi-dimensional (and not
just a single extra dimension, like foo[2][3])
array arguments and returns.

edit: trigger rebuild, CI failed to fetch linter module.

* accounts/abi/bind: improve array binding

wrapArray uses a regex now, and arrayBindingJava is improved.

* accounts/abi: Improve naming of element size func

The full step size for unpacking an array
 is now retrieved with "getFullElemSize".

* accounts/abi: support nested nested array args

Previously, the code only considered the outer-size of the array,
ignoring the size of the contents. This was fine for most types,
but nested arrays are packed directly into it, and count towards
the total size. This resulted in arguments following a nested
array to replicate some of the binary contents of the array.

The fix: for arrays, calculate their complete contents size:
 count the arg.Type.Elem.Size when Elem is an Array, and
 repeat when their child is an array too, etc.
The count is the number of 32 byte elements, similar to how it
 previously counted, but nested.

* accounts/abi: Test deep nested arr multi-arguments

Arguments with a deeply nested array should not cause the next arguments
to be read from the wrong position.
mariameda pushed a commit to NiluPlatform/go-nilu that referenced this pull request Aug 23, 2018
…5648, including nested array unpack fix (ethereum#15676)

* accounts/abi/bind: support for multi-dim arrays

Also:
- reduce usage of regexes a bit.
- fix minor Java syntax problems

Fixes ethereum#15648

* accounts/abi/bind: Add some more documentation

* accounts/abi/bind: Improve code readability

* accounts/abi: bugfix for unpacking nested arrays

The code previously assumed the arrays/slices were always 1 level
deep. While the packing supports nested arrays (!!!).

The current code for unpacking doesn't return the "consumed" length, so
this fix had to work around that by calculating it (i.e. packing and
 getting resulting length) after the unpacking of the array element.
It's far from ideal, but unpacking behaviour is fixed now.

* accounts/abi: Fix unpacking of nested arrays

Removed the temporary workaround of packing to calculate size, which was
incorrect for slice-like types anyway.
Full size of nested arrays is used now.

* accounts/abi: deeply nested array unpack test

Test unpacking of an array nested more than one level.

* accounts/abi: Add deeply nested array pack test

Same as the deep nested array unpack test, but the other way around.

* accounts/abi/bind: deeply nested arrays bind test

Test the usage of bindings that were generated
for methods with multi-dimensional (and not
just a single extra dimension, like foo[2][3])
array arguments and returns.

edit: trigger rebuild, CI failed to fetch linter module.

* accounts/abi/bind: improve array binding

wrapArray uses a regex now, and arrayBindingJava is improved.

* accounts/abi: Improve naming of element size func

The full step size for unpacking an array
 is now retrieved with "getFullElemSize".

* accounts/abi: support nested nested array args

Previously, the code only considered the outer-size of the array,
ignoring the size of the contents. This was fine for most types,
but nested arrays are packed directly into it, and count towards
the total size. This resulted in arguments following a nested
array to replicate some of the binary contents of the array.

The fix: for arrays, calculate their complete contents size:
 count the arg.Type.Elem.Size when Elem is an Array, and
 repeat when their child is an array too, etc.
The count is the number of 32 byte elements, similar to how it
 previously counted, but nested.

* accounts/abi: Test deep nested arr multi-arguments

Arguments with a deeply nested array should not cause the next arguments
to be read from the wrong position.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants