Linux binary for version 0.4.26 in solc-bin produces different bytecode than other platforms

[cameel]

I detected this while comparing bytecode reports produced by binaries in solc-bin. This happens only for a single source file (out of about 1840) and only when using the --optimize flag. asm.js, wasm and macOS binaries all produce identical output for that file and their output is different from the one produced by the Linux binary.

The output is deterministic - I was able to reproduce it on my system too.

I haven't checked if this difference happens via Standard JSON too but I don't see why it wouldn't.

The differences are significant but the bytecode is still similar. At the beginning of the bytecode there are only single-byte differences. Later there are longer differing sequences interspersed with identical ones

Full bytecode reports

• report-solc-emscripten-asmjs-v0.4.26+commit.4563c3fc.js.txt
• report-solc-emscripten-wasm32-v0.4.26+commit.4563c3fc.js.txt
• report-solc-linux-amd64-v0.4.26+commit.4563c3fc.txt
• report-solc-macosx-amd64-v0.4.26+commit.4563c3fc.txt

Input file

                pragma experimental ABIEncoderV2;
                contract C {
                        function f0() public pure returns (bytes) {
                                return abi.encodeWithSelector(0x12345678);
                        }
                        function f1() public pure returns (bytes) {
                                return abi.encodeWithSelector(0x12345678, "abc");
                        }
                        function f2() public pure returns (bytes) {
                                bytes4 x = 0x12345678;
                                return abi.encodeWithSelector(x, "abc");
                        }
                        function f3() public pure returns (bytes) {
                                bytes4 x = 0x12345678;
                                return abi.encodeWithSelector(x, uint(-1));
                        }
                        struct S { uint a; string b; uint16 c; }
                        function f4() public pure returns (bytes) {
                                bytes4 x = 0x12345678;
                                S memory s;
                                s.a = 0x1234567;
                                s.b = "Lorem ipsum dolor sit ethereum........";
                                s.c = 0x1234;
                                return abi.encodeWithSelector(x, uint(-1), s, uint(3));
                        }
                }

Bytecode (Linux)

608060405234801561001057600080fd5b50610510806100206000396000f30060806040526004361061006c5763ffffffff7c01000000000000000000000000000000000000000000000000000000006000350416639942ec6f8114610071578063a58504751461009c578063aaf05f3d146100b1578063c27fc305146100c6578063c3f90202146100db575b600080fd5b34801561007d57600080fd5b506100866100f0565b60405161009391906103fb565b60405180910390f35b3480156100a857600080fd5b50610086610161565b3480156100bd57600080fd5b50610086610194565b3480156100d257600080fd5b506100866101bc565b3480156100e757600080fd5b5061008661021e565b6040516060906000805160206104b783398151915290819061011490602401610413565b60408051601f19818403018152919052602081018051600160e060020a03167bffffffffffffffffffffffffffffffffffffffffffffffffffffffff199093169290921790915291505090565b604080516004815260248101909152602081018051600160e060020a03166000805160206104b783398151915217905290565b6040516060906000805160206104b78339815191529081906101149060001990602401610429565b606063123456786040516024016101d290610413565b60408051601f19818403018152919052602081018051600160e060020a03167c010000000000000000000000000000000000000000000000000000000090930292909217909152905090565b6060600061022a61030f565b6301234567815260408051606081018252602681527f4c6f72656d20697073756d20646f6c6f722073697420657468657265756d2e2e6020808301919091527f2e2e2e2e2e2e00000000000000000000000000000000000000000000000000008284015283015261123481830152516000805160206104b7833981519152925082906102c190600019908490600390602401610437565b60408051601f19818403018152919052602081018051600160e060020a03167bffffffffffffffffffffffffffffffffffffffffffffffffffffffff19909316929092179091529250505090565b6060604051908101604052806000815260200160608152602001600061ffff1681525090565b60006103408261046e565b80845261035481602086016020860161047c565b61035d816104ac565b9093016020019392505050565b600381527f6162630000000000000000000000000000000000000000000000000000000000602082015260400190565b805160009060608401906103ae85826103f2565b50602083015184820360208601526103c68282610335565b91505060408301516103db60408601826103e3565b509392505050565b6103ec81610472565b82525050565b6103ec81610479565b6020808252810161040c8184610335565b9392505050565b602080825281016104238161036a565b92915050565b6020810161042382846103f2565b6060810161044582866103f2565b8181036020830152610457818561039a565b905061046660408301846103f2565b949350505050565b5190565b61ffff1690565b90565b60005b8381101561049757818101518382015260200161047f565b838111156104a6576000848401525b50505050565b601f01601f19169056001234567800000000000000000000000000000000000000000000000000000000a265627a7a72305820c8f3e1bb5d84f2e8dbfc88da40512c57bb13096b771faa5a6763b890fc7a0ac26c6578706572696d656e74616cf50037

Bytecode (asm.js, wasm, macOS)

608060405234801561001057600080fd5b506104e0806100206000396000f30060806040526004361061006c5763ffffffff7c01000000000000000000000000000000000000000000000000000000006000350416639942ec6f8114610071578063a58504751461009c578063aaf05f3d146100b1578063c27fc305146100c6578063c3f90202146100db575b600080fd5b34801561007d57600080fd5b506100866100f0565b60405161009391906103eb565b60405180910390f35b3480156100a857600080fd5b5061008661015d565b3480156100bd57600080fd5b5061008661018c565b3480156100d257600080fd5b506100866101b0565b3480156100e757600080fd5b50610086610212565b60405160609060e360020a6302468acf0290819061011090602401610403565b60408051601f19818403018152919052602081018051600160e060020a03167bffffffffffffffffffffffffffffffffffffffffffffffffffffffff199093169290921790915291505090565b604080516004815260248101909152602081018051600160e060020a031660e360020a6302468acf0217905290565b60405160609060e360020a6302468acf029081906101109060001990602401610419565b606063123456786040516024016101c690610403565b60408051601f19818403018152919052602081018051600160e060020a03167c010000000000000000000000000000000000000000000000000000000090930292909217909152905090565b6060600061021e6102ff565b6301234567815260408051606081018252602681527f4c6f72656d20697073756d20646f6c6f722073697420657468657265756d2e2e6020808301919091527f2e2e2e2e2e2e000000000000000000000000000000000000000000000000000082840152830152611234818301525160e360020a6302468acf02925082906102b190600019908490600390602401610427565b60408051601f19818403018152919052602081018051600160e060020a03167bffffffffffffffffffffffffffffffffffffffffffffffffffffffff19909316929092179091529250505090565b6060604051908101604052806000815260200160608152602001600061ffff1681525090565b60006103308261045e565b80845261034481602086016020860161046c565b61034d8161049c565b9093016020019392505050565b600381527f6162630000000000000000000000000000000000000000000000000000000000602082015260400190565b8051600090606084019061039e85826103e2565b50602083015184820360208601526103b68282610325565b91505060408301516103cb60408601826103d3565b509392505050565b6103dc81610462565b82525050565b6103dc81610469565b602080825281016103fc8184610325565b9392505050565b602080825281016104138161035a565b92915050565b6020810161041382846103e2565b6060810161043582866103e2565b8181036020830152610447818561038a565b905061045660408301846103e2565b949350505050565b5190565b61ffff1690565b90565b60005b8381101561048757818101518382015260200161046f565b83811115610496576000848401525b50505050565b601f01601f1916905600a265627a7a72305820c8f3e1bb5d84f2e8dbfc88da40512c57bb13096b771faa5a6763b890fc7a0ac26c6578706572696d656e74616cf50037

Metadata

{
    "compiler": {
        "version": "0.4.26+commit.4563c3fc"
    },
    "language": "Solidity",
    "output": {
        "abi": [
            {
                "constant": true,
                "inputs": [],
                "name": "f2",
                "outputs": [
                    {
                        "name": "",
                        "type": "bytes"
                    }
                ],
                "payable": false,
                "stateMutability": "pure",
                "type": "function"
            },
            {
                "constant": true,
                "inputs": [],
                "name": "f0",
                "outputs": [
                    {
                        "name": "",
                        "type": "bytes"
                    }
                ],
                "payable": false,
                "stateMutability": "pure",
                "type": "function"
            },
            {
                "constant": true,
                "inputs": [],
                "name": "f3",
                "outputs": [
                    {
                        "name": "",
                        "type": "bytes"
                    }
                ],
                "payable": false,
                "stateMutability": "pure",
                "type": "function"
            },
            {
                "constant": true,
                "inputs": [],
                "name": "f1",
                "outputs": [
                    {
                        "name": "",
                        "type": "bytes"
                    }
                ],
                "payable": false,
                "stateMutability": "pure",
                "type": "function"
            },
            {
                "constant": true,
                "inputs": [],
                "name": "f4",
                "outputs": [
                    {
                        "name": "",
                        "type": "bytes"
                    }
                ],
                "payable": false,
                "stateMutability": "pure",
                "type": "function"
            }
        ],
        "devdoc": {
            "methods": {}
        },
        "userdoc": {
            "methods": {}
        }
    },
    "settings": {
        "compilationTarget": {
            "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol": "C"
        },
        "evmVersion": "byzantium",
        "libraries": {},
        "optimizer": {
            "enabled": true,
            "runs": 200
        },
        "remappings": []
    },
    "sources": {
        "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol": {
            "keccak256": "0x8037494985eabf57d0f48c42678cde5b3b8789cad1e13bbb615e7cc4083856a0",
            "urls": [
                "bzzr://76929172b1599501a4e9a24799df13c94b20bcf81dfaf418ec7dca2ca2a978a2"
            ]
        }
    },
    "version": 1
}

[chriseth]

My first suspicion is that this might be related to the optimizer. Maybe it is somehow related to the order of functions being inlined?

One thing that caught my eye is that the last difference is that the linux version contains the number 0012345678000... and the others don't. This looks like the number is included using datacopy on linux while it is used directly (or computed) on the other platforms.

[cameel]

It's definitely the optimizer. The differences are only in the optimized bytecode. I have also just diffed unoptimized bytecode from 0.4.25, 0.4.25 and 0.4.26 and it's identical (the only difference is the metadata hash at the end).

Also, 0.4.24 and 0.4.25 produce identical optimized bytecode (except for metadata). I checked the diff between 0.4.25 and 0.4.26 and I don't see anything that would account for the change. I haven't analyzed the changes in depth but at a glance I see only refactors that should not affect the output (splitting the optimizer rule list, adding asserts for the ABI encoder, etc).

One thing that caught my eye is that the last difference is that the linux version contains the number 0012345678000... and the others don't. This looks like the number is included using datacopy on linux while it is used directly (or computed) on the other platforms.

Yeah. Both contain the 12345678 part though.

So anyway, do we want to find out the reason for the difference or just do something about the binary (remove, rebuild, etc)? We could probably get to the bottom of it by bisecting between 0.4.25 and 0.4.26, the question is just whether it'll help us in any way.

[chriseth]

If you want, you can create a diff of the assembly produced by the two versions. Also, trying to re-build might be a solution - maybe the binary was not built from the right version? But if that does not help, I would just leave it like that.

[cameel]

Good idea. Here's the diff between 0.4.25 optimized (red) and 0.4.26 optimized (green):

191c191,201
<       mul(0x2468acf, exp(0x2, 0xe3))
---
>       0x0
>       dup1
>       mload
>       0x20
>       data_12e0d8f4db034b03ff34330aa35e4797be790557b8b07077164f0ac403fb49cd
>       dup4
>       codecopy
>       dup2
>       mload
>       swap2
>       mstore
281c291,301
<       mul(0x2468acf, exp(0x2, 0xe3))
---
>       0x0
>       dup1
>       mload
>       0x20
>       data_12e0d8f4db034b03ff34330aa35e4797be790557b8b07077164f0ac403fb49cd
>       dup4
>       codecopy
>       dup2
>       mload
>       swap2
>       mstore
297c317,327
<       mul(0x2468acf, exp(0x2, 0xe3))
---
>       0x0
>       dup1
>       mload
>       0x20
>       data_12e0d8f4db034b03ff34330aa35e4797be790557b8b07077164f0ac403fb49cd
>       dup4
>       codecopy
>       dup2
>       mload
>       swap2
>       mstore
428c458,468
<       mul(0x2468acf, exp(0x2, 0xe3))
---
>       0x0
>       dup1
>       mload
>       0x20
>       data_12e0d8f4db034b03ff34330aa35e4797be790557b8b07077164f0ac403fb49cd
>       dup4
>       codecopy
>       dup2
>       mload
>       swap2
>       mstore
950a991,992
>     stop
>     data_12e0d8f4db034b03ff34330aa35e4797be790557b8b07077164f0ac403fb49cd 1234567800000000000000000000000000000000000000000000000000000000
952c994
<     auxdata: 0xa265627a7a723058206c5a54efd39356c047dc90bbb52f4f3c630cd9621cb405ff99c03b3c62f4788b6c6578706572696d656e74616cf50037
---
>     auxdata: 0xa265627a7a72305820b8f19b548f266b63f73ec24d270bcfa9a4f62596e1d2ea4bdda869e3f7f0fe246c6578706572696d656e74616cf50037

Whole output:

• test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp-0.4.25.asm.txt
• test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp-0.4.26.asm.txt

Weird. They don't even seem to be doing the same thing. One read/writes memory and the other just does some arithmetic on the stack.

[cameel]

Here's the differing fragment with a bit more context (whole part between two labels):

0.4.25 optimized

    tag_9:
        /* "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol":586:618  abi.encodeWithSelector(x, "abc") */
      mload(0x40)
        /* "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol":483:488  bytes */
      0x60
      swap1
      mul(0x2468acf, exp(0x2, 0xe3))
        /* "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol":524:545  bytes4 x = 0x12345678 */
      swap1
      dup2
      swap1
        /* "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol":586:618  abi.encodeWithSelector(x, "abc") */
      tag_29
      swap1
      0x24
      add
      jump(tag_30)

0.4.26 optimized

    tag_9:
        /* "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol":586:618  abi.encodeWithSelector(x, "abc") */
      mload(0x40)
        /* "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol":483:488  bytes */
      0x60
      swap1
      0x0
      dup1
      mload
      0x20
      data_12e0d8f4db034b03ff34330aa35e4797be790557b8b07077164f0ac403fb49cd
      dup4
      codecopy
      dup2
      mload
      swap2
      mstore
        /* "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol":524:545  bytes4 x = 0x12345678 */
      swap1
      dup2
      swap1
        /* "test_9a0896f58138a03af502be9bd35a9c6faa4fa505e633536ebe0478dacec3ce88_solidityendtoendtest_cpp.sol":586:618  abi.encodeWithSelector(x, "abc") */
      tag_29
      swap1
      0x24
      add
      jump(tag_30)

[chriseth]

They do the same things. These are two different strategies to optimize constants. They are selected through the "runs" parameter (and some other factors).

[cameel]

Oh, you're right! I quickly calculated 0x2468acf * (0x2 ** 0xe3) and got a value different from 123455678... but I missed that it's actually a hex number. So they're equal after all.

This might indicate that the problem is with optimizer defaults? We don't set any optimizer parameters explicitly when generating the report. It's not --optimize-runs though - I just tried to run it with --optimize-runs 200 and the difference is still there.