Feature: Add bytes building method

[SilentCicero]

Can we have a simple method that nicely and tightly packs things properly like what .call does and returns it in bytes.

A literal copy and past job of what the call method does. Just expose the subroutine for call, that aids in the building of bytes data.

That way I could build transactions by running a sequence like this:

pack here being the bytes packing method.

forward_transaction(destination, value, pack(bytes4(sha3('somemethod(address)'), msg.sender));

The only alternative right now that I can see is doing crazy things like this:

function bytes4and20ToBytes(bytes4 z, bytes32 x) constant returns (bytes) {
        bytes memory bytesString = new bytes(37);
        uint charCount = 0;
        byte char;
        
        for (uint u = 0; u < 4; u++) {
            char = byte(bytes4(uint(z) * 2 ** (8 * u)));
            
            bytesString[charCount] = char;
            charCount++;
        }
        
        for (uint k = 0; k < 1; k++) {
            char = byte(bytes4(uint(0) * 2 ** (8 * k)));

            bytesString[charCount] = char;
            charCount++;
        }
        
        for (uint a = 0; a < 32; a++) {
            char = byte(bytes32(uint(x) * 2 ** (8 * a)));

            bytesString[charCount] = char;
            charCount++;
        }

        return bytesString;
    }

I think the building of bytes data will be critical going forward, and we should have reliable and simple ways to build bytes data for proxy forwarding.

[chriseth]

This was planned under the name encode as a template function. Perhaps we should just make it an arbitrary-argument function just like sha3 just that it returns a bytes and does not apply the actual sha3 operation. Is that what you have in mind?
Perhaps we could have two functions: encode that does ABI-encoding without tight packing and pack than does the same but packs tightly.

[SilentCicero]

Yes, this would be amazing. Just think: proxy contracts, multisig wallets, anything with a forward method would need and use this.

[chriseth]

Could you formalize your specification a bit more? I actually see three different ways of doing this.

[SilentCicero]

So I would say just like call:

Definition:
Can take in multi-type fixed length inputs (bytes32, bytes4, address, uint256, uin8 etc...), returns a single bytes output.

Usage:

bytes txData = encode(bytes4(sha3('mymethod(address, uint256)')), msg.sender, uint256(37287328));

Perhaps that first argument needs to be different or special for the proper bytes4 packing, like with call. But everything else can just be properly packed (with padding in the front and not in the back).

pack can be reserved for tightly packing or encodePacked.

[SilentCicero]

encode returns a single bytes output.

[chriseth]

So everything is ABI-encoded, but if the first argument is a bytes4, it is not padded and the offset of the ABI-encoding starts from the second argument?

[SilentCicero]

@chriseth yes, similar to call, although would there be a better way to do this that is more general but allows me to form the proper 4 byte method sig at the beginning?

[pipermerriam]

Alternate or additional idea:

contract MyContract {
  function myFunction(uint a, bytes b) {
    ...
  }
}

contract UsesMyContract {
  MyContract myContract;

  function doit() {
    uint a = 3;
    uint b = "abcd";
    bytes preparedCallData = myContract.myFunction.callData(a, b);
  }
}

How about having a function that can be called on function types that returns the prepared bytes that would be used as the call data? This could be a convenience function that is really just wrapping the encode function previously referenced.

[pipermerriam]

Also, I see two distinct functions here.

1. encode: Performs proper ABI encoding of the arguments. Returns bytes
2. pack: Performs tight packing of the arguments in the same manner as .call and .delegatecall. Returns bytes

[axic]

Back at the proposal of macros, one of the motivations was to have all of these abstracted and today with the intermediate language, it seems very feasible. Also it always comes up as a feature request when implementing proxies :)

It would be nice to have the following:

a) inline assembly methods:

• abi encode item
• abi decode item
• tight encode item (this is basically the cleaned up stack item)
• tight decode item

b) higher level methods:

• all the previous exposed to Solidity under the abi. builtin "library"
• plus these versions taking variable arguments:
  • abi.encode(<varargs>) -> bytes
  • abi.encodePacked(<varargs>) -> bytes
  • abi.encodeCalldata(<bytes4 sig>, <varargs>) -> bytes

We could implement keccak256() and call() as a combination of the above macros and calls.

I am also open to only expose the low level methods under the abi. library, or only as low-level assembly methods, and implement the rest as a standard library.

[SilentCicero]

@axic any ETA on these features. I'm dying for these. Super priority for multi-sig wallet/governance building on Eth. Also, a general decode would be amazing. Right now I need to decode by hand, which is very dangerous.

[chriseth]

We could already do that without inline assembly, actually I don't see the big benefit anyway. We might use the new "callLowLevelFunction" feature and easily turn these into assembly functions at a later point in time. Also we might do without tight encoding for now (tight encoding is very different for dynamic data).

[SilentCicero]

@chriseth could you expand? Also, what is the easiest way to pull and compare a method signature form a bytes data store?

I want to get the method sig from the first 4 bytes. What do you recommend? In fact, what is the best way to pull the first 4 bytes in general and store in a bytes4 store (if possible).

[chriseth]

@SilentCicero expand on what? The comment above was basically an idea about how to implement these changes internally. About extracting the method signature:

bytes memory data;
bytes4 sig;
assembly { sig := mload(add(data, 0x20)) }

If you just want to have the signature of the current call, you can also use msg.sig.

[SilentCicero]

@chriseth thanks for expanding, makes sense. Assembly is nice and simple =D

[izqui]

Regarding this issue, could I please get your input on this?

What should be the correct behavior for a function like approveAndCall? Should it expect its 'extraData' parameter to be ABI encoded already or encode it before making the call?

Consensys/Tokens#45

[axic]

Here's an old Pivotaltracker version of the issue: https://www.pivotaltracker.com/n/projects/1189488

add template functions to encode and decode calldata:
var (x, y, z) = decode[uint, uint, string](encode[uint, uint, string](7, 8, "abc"))
The template parameters for encode are optional.

[chriseth]

We are still missing the changelog entry and the documentation.