Decoding logs with bytes data fields broken in 1.2.8 for solc 0.4.x / external

[bingen]

Expected behavior

Using decodeLogs should parse logs correctly, see here an example

Actual behavior

Throws with an error like this if there’s a bytes field in the log:

     Error: data out-of-bounds (length=36, offset=64, code=BUFFER_OVERRUN, version=abi/5.0.0-beta.153)
      at Logger.makeError (node_modules/@ethersproject/logger/lib/index.js:178:21)
      at Logger.throwError (node_modules/@ethersproject/logger/lib/index.js:187:20)
      at Reader._peekBytes (node_modules/@ethersproject/abi/lib/coders/abstract-coder.js:135:20)
      at Reader.readBytes (node_modules/@ethersproject/abi/lib/coders/abstract-coder.js:146:26)
      at BytesCoder.DynamicBytesCoder.decode (node_modules/@ethersproject/abi/lib/coders/bytes.js:30:23)
      at BytesCoder.decode (node_modules/@ethersproject/abi/lib/coders/bytes.js:41:81)
      at node_modules/@ethersproject/abi/lib/coders/array.js:77:31
      at Array.forEach (<anonymous>)
      at Object.unpack (node_modules/@ethersproject/abi/lib/coders/array.js:71:12)
      at TupleCoder.decode (node_modules/@ethersproject/abi/lib/coders/tuple.js:39:49)
      at AbiCoder.decode (node_modules/@ethersproject/abi/lib/abi-coder.js:93:22)
      at ABICoder.decodeParameters (node_modules/web3-eth-abi/src/index.js:322:30)
      at ABICoder.decodeLog (node_modules/web3-eth-abi/src/index.js:376:52)
      at eventLogs.map.log (node_modules/@aragon/contract-helpers-test/events.js:18:20)
      at Array.map (<anonymous>)
      at decodeEvents (node_modules/@aragon/contract-helpers-test/events.js:16:20)
      at getDeepEventLogs (test/lock_and_call.js:9:10)
      at getDeepEventArgument (test/lock_and_call.js:12:16)
      at checkCallbackLog (test/lock_and_call.js:50:18)
      at Context.it (test/lock_and_call.js:59:7)
      at process._tickCallback (internal/process/next_tick.js:68:7)

Steps to reproduce the behavior

Call that function mentioned above with a log like this one:

event LogLockCallback(uint256 amount, uint256 allowance, bytes data);

Logs

Copied above.

Environment

npm: 6.13.4
Node: v10.15.0
web3: 1.2.8
OS: PureOS 9.0

Reference

Related issue: aragon/contract-helpers#32

[ryanio]

Hey @bingen thanks for reporting, I will take a closer look this afternoon to see if we have tests for decoding logs with bytes data and if it’s working properly.

Meanwhile, could you share what the raw data is for the bytes field in your example above? It seems the ethers abi package is erroring out on “data out-of-bounds”.

[bingen]

Yes, it was:

0x00000000000000000000000000000000000000000000000000000000000000280000000000000000000000000000000000000000000000000000000000000078000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000040cbd8c2f

Thanks!

[ryanio]

hey @bingen, it does look like we have tests for decodeLog that include bytes examples and they are passing ok: https://github.com/ethereum/web3.js/blob/1.x/test/eth.abi.decodeLog.js

When I try to add a test with your example data:

{
    params: [[{
        type: 'bytes',
        name: 'data'
    }], '0x00000000000000000000000000000000000000000000000000000000000000280000000000000000000000000000000000000000000000000000000000000078000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000040cbd8c2f'],
    result: {
        'data': '0x00000000000000000000000000000000000000000000000000000000000000280000000000000000000000000000000000000000000000000000000000000078000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000040cbd8c2f',
        "__length__": 1
    }
}

I get:

  1) decodeLog
       should convert correctly:
     data: overflow (fault="overflow", operation="toNumber", value="2213609288845146193920", code=NUMERIC_FAULT, version=bignumber/5.0.0-beta.138)

[bingen]

Hm, but that data is for the whole log, which has 2 uints and 1 bytes fields, so it’s:

0x
uint  -> 0000000000000000000000000000000000000000000000000000000000000028
uint  -> 0000000000000000000000000000000000000000000000000000000000000078
bytes -> 000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000040cbd8c2f

[ryanio]

@bingen ah ok thanks that makes more sense. Ok I can reproduce the same error now with data out-of-bounds for the bytes data. Do you know what your bytes data is supposed to represent or convert to?

[bingen]

It’s supposed to be a function signature: https://github.com/aragon/staking/blob/master/test/lock_and_call.js#L27

[ryanio]

@bingen great thanks, ok looks like there may be some issue in the transport of the event because the function signature for receiveLock(uint256,uint256,bytes) is 0xabe985cb which doesn't seem to line up with the bytes data above, I am seeing it encode as:

{
    params: ['bytes', '0xabe985cb'],
    result: '0x00000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000004abe985cb00000000000000000000000000000000000000000000000000000000'
}

It could be an encoding/decoding issue in the middle, will have to keep digging deeper on this one. We upgraded our abi coder dependency in 1.2.8 from ethers v4 to v5 so we are especially sensitive to any issues.

[bingen]

Oh, yes, sorry, that’s because I changed its name, it used to be lockCallback(uint256,uint256,bytes)
😬

[cgewecke]

@bingen It's possible this is a compatibility issue with Truffle which remains on Web3 1.2.1 (and Ethers V4).

In your example CALLBACK_DATA is passed to a truffle-contract method (and encoded there) but decoded using 1.2.8 in the test helpers. No this is fine.

[cgewecke]

@bingen

Have tried to make a reproduction case for this using code from aragon contract-helpers, Truffle and Web3 1.2.8 and was not able to trigger the error.

Could you look at https://github.com/cgewecke/web3-issue-3544 and see if it models the bug correctly?

One notable difference from what's reported here is that when I have a solidity event like

event lockCallback(uint256 amount, uint256 allowance, bytes data);

...and inspect the receipt from a transaction that fires it, the raw log data looks like this:

0x0000000000000000000000000000000000000000000000000000000000000005000000000000000000000000000000000000000000000000000000000000000500000000000000000000000000000000000000000000000000000000000000600000000000000000000000000000000000000000000000000000000000000004abe985cb00000000000000000000000000000000000000000000000000000000

It's right padded. Is it possible your data was getting mangled somewhere after you got the receipt?

[cgewecke]

@bingen Apologies, did you have a chance to look at this again? We'd really like to fix this ASAP if it's happening but at the moment are having difficulty reproducing it from a real example.

[bingen]

Hey, sorry, I didn’t have time to look at this, I’ll try tomorrow.

[bingen]

Hm, this is weird, I tired a fresh clone of the repo using this new branch where I just upgrade to web3 1.2.8 and it worked:

git clone git@github.com:aragon/staking.git
cd staking 
git checkout web3_issue
npm i
npm t test/lock_and_call.js

Besides, your example above looks good to me.
Maybe it’s some other package that I had in my local node_modules that was causing the issue, so I guess we can close this issue.
Sorry for the distraction!