sendmany with alternating asset input produces invalid tx

[ixje]

Current behavior

with sendmany you can create multiple transfers in a single tx. In a very specific scenario a tx is created that is not accepted by the network. I've done the following tests

# of tx | description
2 | gas to 2 different addresses                                            success
2 | neo to 2 different addresses                                            success
2 | neo & gas to same address                                               success
4 | neo to 4 diferent addresses                                             success
4 | gas to 4 different addresses                                            success
4 | neo to address_a, neo to address_b, gas to address_a, gas to address_b  success
4 | 1x neo + gas to address_a, 1 x neo_gas + address b                      fail
4 | neo/gas/neo/gas to 4 different addresses                                fail
3 | neo to address_a, gas to address_a, neo to address_b                    fail
3 | gas to address_a, neo to _address_a, gas to address_b                   fail

From these tests I was able to conclude that whenever we switch between neo/gas/neo or gas/neo/gas we end up calculating the wrong return value for the output that holds the "change funds". To eleborate; in a neo->gas->neo scenario, the first neo vout is not taken into account when calculating the change. So only neo vout 2 is deduced from the vin. This leads to the tx asking to return more funds as change than actually available.

Expected behavior

the correct return value should be calculated resulting in a valid tx.

How to reproduce

see the table above.

Your environment

Let us know in what environment you're running into the issue:

• OS: osx mojave
• neo-python version: 0.8.3-dev
• Python version: 3.6

[ixje]

I fixed the change return value and the funds are processed. What now fails is that we somehow give the wrong tx id that was sent to the network.

neo> wallet open wallet796
Opened wallet at wallet796
neo> wallet
Wallet {
    "path": "wallet796",
    "addresses": [
        {
            "version": 0,
            "address": "APCAGn7U472L6L57dxTWA5poHoigXzhqn6",
            "script_hash": "0b1c5f897151407b0cc0ce349975df49eabd6051",
...
            "balances": [
                {
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "156.0"
                },
                {
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "45.0"
                }
            ],
            ...
        },
        {
            "version": 0,
            "address": "ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT",
            "script_hash": "5904f99d4e381654ebafa9eb8db7877f63c92877",
...
            "balances": [
                {
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "95.0"
                },
                {
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "64.0"
                }
            ],
            "is_watch_only": false
        },
        {
            "address": "AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y",
            "script_hash": "e9eed8dc39332032dc22e5d6e86332c50327ba23",
            "balances": [
                {
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "99999567.0"
                },
                {
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "15845.0"
                }
            ],
            "is_watch_only": false
        },
    ],
    "height": 17995,
    "percent_synced": 100,
    "synced_balances": [
        "[NEO]: 99999852.0 ",
        "[NEOGas]: 16024.0 "
    ],
    "public_keys": [
...
    ],
    "tokens": [],
    "claims": {
        "available": "127231.93773448",
        "unavailable": "672.00830296"
    }
}
neo> wallet sendmany 3 --from-addr=AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y
Outgoing Number  1
Outgoing Number  2
Outgoing Number  3
sending with fee: 0.0
[<neo.Core.TX.Transaction.ContractTransaction object at 0x10c120ac8>, <neocore.UInt160.UInt160 object at 0x10c120c88>, None, <neocore.Fixed8.Fixed8 object at 0x10c120e48>, None, []]
Validate your transaction details
---------------------------------
Sending 1.0 NEO from AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y to ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT
Sending 2.0 GAS from AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y to ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT
Sending 3.0 NEO from AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y to APCAGn7U472L6L57dxTWA5poHoigXzhqn6
Returning 99999563.0 NEO as change to AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y
Returning 15843.0 GAS as change to AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y

Enter your password to send to the network
{'txid': '0xd189cca4e0e35ce1a2d50e8d3abaf90195858917111f9dcff7be351f65577f32', 'size': 374, 'type': 'ContractTransaction', 'version': 0, 'attributes': [], 'vout': [{'n': 0, 'asset': '0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b', 'value': '1', 'address': 'ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT'}, {'n': 1, 'asset': '0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7', 'value': '2', 'address': 'ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT'}, {'n': 2, 'asset': '0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b', 'value': '3', 'address': 'APCAGn7U472L6L57dxTWA5poHoigXzhqn6'}, {'n': 3, 'asset': '0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b', 'value': '99999563', 'address': 'AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y'}, {'n': 4, 'asset': '0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7', 'value': '15843', 'address': 'AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y'}], 'vin': [{'txid': '0x83c2acf80dddf4a20001780a2ed5f7fce94fbb7ecfa75d6b7bea7d25c077493e', 'vout': 3}, {'txid': '0x83c2acf80dddf4a20001780a2ed5f7fce94fbb7ecfa75d6b7bea7d25c077493e', 'vout': 4}], 'sys_fee': '0', 'net_fee': '0', 'scripts': []}
[Password]> **********
[I 190107 18:57:22 Transaction:619] Verifying transaction: b'd189cca4e0e35ce1a2d50e8d3abaf90195858917111f9dcff7be351f65577f32'
Relayed Tx: d189cca4e0e35ce1a2d50e8d3abaf90195858917111f9dcff7be351f65577f32
neo> show tx d189cca4e0e35ce1a2d50e8d3abaf90195858917111f9dcff7be351f65577f32
Could not find transaction for hash d189cca4e0e35ce1a2d50e8d3abaf90195858917111f9dcff7be351f65577f32
neo> wallet
Wallet {
    "path": "wallet796",
    "addresses": [
        {
            "version": 0,
            "address": "APCAGn7U472L6L57dxTWA5poHoigXzhqn6",
            "script_hash": "0b1c5f897151407b0cc0ce349975df49eabd6051",
            "frozen": false,
            "votes": [],
            "balances": [
                {
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "159.0"
                },
                {
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "45.0"
                }
            ],
            "is_watch_only": false
        },
        {
            "version": 0,
            "address": "ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT",
            "script_hash": "5904f99d4e381654ebafa9eb8db7877f63c92877",
            "frozen": false,
            "votes": [],
            "balances": [
                {
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "96.0"
                },
                {
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "66.0"
                }
            ],
            "is_watch_only": false
        },
        {
            "version": 0,
            "address": "AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y",
            "script_hash": "e9eed8dc39332032dc22e5d6e86332c50327ba23",
            "frozen": false,
            "votes": [],
            "balances": [
                {
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "99999563.0"
                },
                {
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "15843.0"
                }
            ],
            "is_watch_only": false
        },
        {
            "version": 0,
            "address": "AXjHnbYdHBJnNtdk8sc4qzXKoc5cw7uBxz",
            "script_hash": "b4f68b3f35664c6ce4abbefa62a7e27557ea04af",
            "frozen": false,
            "votes": [],
            "balances": [
                {
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "24.0"
                },
                {
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "30.0"
                }
            ],
            "is_watch_only": false
        },
        {
            "version": 0,
            "address": "ANCWtWVcBbs7mic2wYwKfvVTfCH6vnWfQW",
            "script_hash": "0c0e353f06d50f970f142208cbae02c552d37946",
            "frozen": false,
            "votes": [],
            "balances": [
                {
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "10.0"
                },
                {
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "40.0"
                }
            ],
            "is_watch_only": false
        }
    ],
    "height": 17999,
    "percent_synced": 100,
    "synced_balances": [
        "[NEO]: 99999852.0 ",
        "[NEOGas]: 16024.0 "
    ],
    "public_keys": [
...
    ],
    "tokens": [],
    "claims": {
        "available": "127919.93475544",
        "unavailable": "16.01123464"
    }
}

before

APCAGn7U472L6L57dxTWA5poHoigXzhqn6 has 156 NEO, 45 GAS
ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT has 95 NEO, 64 GAS
AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y has 99999567.0 NEO, 15845.0 GAS

validation

Sending 1.0 NEO from AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y to ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT
Sending 2.0 GAS from AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y to ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT
Sending 3.0 NEO from AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y to APCAGn7U472L6L57dxTWA5poHoigXzhqn6
Returning 99999563.0 NEO as change to AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y
Returning 15843.0 GAS as change to AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y

after

ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT has 96 (95+1) NEO, 66 (64+2) GAS
APCAGn7U472L6L57dxTWA5poHoigXzhqn6 has 159 (156+3) NEO

The funds changed correctly.

================
However, neo-python thought the TX id was is d189cca4e0e35ce1a2d50e8d3abaf90195858917111f9dcff7be351f65577f32
in reality it turned out to be 0x5e823a1f972e99075292c4cfc392787bd7850c0e1a548c0722e6f4eb2b4fab85

as seen here

neo> show block 17997
{
    "hash": "0xe89fad5ddb4146d50de1a0d54f67ce879071f271952cc13aea4471c7dfdec979",
    "size": 1051,
    "version": 0,
    "previousblockhash": "0x85973d2ef92307fe7dd0c204f49cd6dc5a4bbaddf0cb7190de2e5ec0e0a74e52",
    "merkleroot": "0xb26d4cc0e10b63268a2eb8b10d0e20e7bb59c4b161284b1031cd21aceb03be69",
    "time": 1546883873,
    "index": 17997,
    "nonce": "e3d4ed35310db015",
    "nextconsensus": "AZ81H31DMWzbSnFDLFkzh9vHwaDLayV7fU",
    "script": {
        "invocation": "40f0adf3c6c99739f86d3c19d2080690501d83373a532501d62f95e3bd2ac9013292921011fd1180f9b77122b0eca0c88267073a432dd301ddb58959691fa26856402d80d396251a395773c3cac18dfc8c073dbe895bb462b6d3a04c9cbf5346c20aaefd5ad7365105d358768c1f0325b0bb75e903a3b5baaddc1b749c7f18bc7cf6408db014b06e0479f05607ddc9ce40ed107767c993a487577c72776345de30fbeb04b60af1196f1db095e93956e5e5859909c901b3a79f6355000ab7a921b9881c",
        "verification": "532102103a7f7dd016558597f7960d27c516a4394fd968b9e65155eb4b013e4040406e2102a7bc55fe8684e0119768d104ba30795bdcc86619e864add26156723ed185cd622102b3622bf4017bdfe317c58aed5f4c753f206b7db896046fa7d774bbc4bf7f8dc22103d90c07df63e690ce77912e10ab51acc944b66860237b608c4f8f8309e71ee69954ae"
    },
    "tx": [
        {
            "txid": "0x1903db31cdef51a6ab99f69d4627b16269e20762de4e656f0231c99b517feb56",
            "size": 10,
            "type": "MinerTransaction",
            "version": 0,
            "attributes": [],
            "vout": [],
            "vin": [],
            "sys_fee": "0",
            "net_fee": "0",
            "scripts": [],
            "nonce": 822980629
        },
        {
            "txid": "0x5e823a1f972e99075292c4cfc392787bd7850c0e1a548c0722e6f4eb2b4fab85",
            "size": 599,
            "type": "ContractTransaction",
            "version": 0,
            "attributes": [
                {
                    "usage": 32,
                    "data": "5160bdea49df759934cec00c7b405171895f1c0b"
                }
            ],
            "vout": [
                {
                    "n": 0,
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "1",
                    "address": "ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT"
                },
                {
                    "n": 1,
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "2",
                    "address": "ASdvvEXLJ4a5GGeo7oYiMVVb9NRSw4kDeT"
                },
                {
                    "n": 2,
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "3",
                    "address": "APCAGn7U472L6L57dxTWA5poHoigXzhqn6"
                },
                {
                    "n": 3,
                    "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
                    "value": "99999563",
                    "address": "AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y"
                },
                {
                    "n": 4,
                    "asset": "0x602c79718b16e442de58778e148d0b1084e3b2dffd5de6b7b16cee7969282de7",
                    "value": "15843",
                    "address": "AK2nJJpJr6o664CWJKi1QRXjqeic2zRp8y"
                }
            ],
            "vin": [
                {
                    "txid": "0x83c2acf80dddf4a20001780a2ed5f7fce94fbb7ecfa75d6b7bea7d25c077493e",
                    "vout": 3
                },
                {
                    "txid": "0x83c2acf80dddf4a20001780a2ed5f7fce94fbb7ecfa75d6b7bea7d25c077493e",
                    "vout": 4
                }
            ],
            "sys_fee": "0",
            "net_fee": "0",
            "scripts": [
                {
                    "invocation": "408e54ecd1ea4ab0d1905383e4d4bb1d81859078db698cc30c44117ff7a32d99b97acc82782233ced83d93c0801e5e92803df218a5d4dd6a1465ece1c897254832",
                    "verification": "2102eda3141750ebf0efd17734e9903e01aa4025057b8d43575e4ff463a660584d27ac"
                },
                {
                    "invocation": "4006cc7ae8cce1b695576a04ffbafda1672a20114e4a7a783674d764ddfb182b765e8202ab5655792dd5e73f3182e9d64279731a23cc8d079529fe9938c15066ea",
                    "verification": "21031a6c6fbbdf02ca351745fa86b9ba5a9452d785ac4f7fc2b7548ca2a46c4fcf4aac"
                }
            ]
        }
    ]
}

So need to figure out why this happens

[ixje]

found the issue of a wrong hash. The Hash function of a TX tries to be smart by not recomputing itself

neo-python/neo/Core/TX/Transaction.py

Lines 283 to 287 in e378012

	if not self.__hash:
	ba = bytearray(binascii.unhexlify(self.GetHashData()))
	hash = Crypto.Hash256(ba)
	self.__hash = UInt256(data=hash)
	return self.__hash

We triggered a hash computation before prompting for the password for debugging purposes. Then when we relay the tx and print tx.Hash its hash was calculated before appending other attributes, thus getting a wrong hash.