"skip" method to generate a null value in prepared queries

[benelgiac]

Hi

as stated in this discussion, I think prepared queries with unnamed variables are very limited.

In my use case, I have a CQL table with many fields and a massive amount of INSERT to perform, where only some fields per query will be present. I want to do this with prepared queries for performance reasons.

At present, prepared queries with '?' bound variables are not usable in real life. Best solution would be to have named bound variables, but I understand this is not immediate to accomplish (also, very new feature in cassandra and CQL).

So, temporarily, I would just love to have a "skip" method that I could use instead of a "push_back" every time I do not have a value to push to a given field.

This is my attempt to implement this. It is based on the following description of a "null" byte in binary protocol specifications. I'm not sure this is the correct way to do it, however it was quick enough to implement that I just tried it

    [bytes]        A [int] n, followed by n bytes if n >= 0. If n < 0,
                   no byte should follow and the value represented is `null`.

I have tested this with a simple table.

Notice that the entry for key4 is generated manually by a cqlsh query where I just insert a value for v3. I correctly see "null" for v2 field.

key2 and key3 entries are generated by a prepared query using the "skip" method. I don't see the "null" representation so I guess my method is not correct

 key  | v2   | v3
------+------+-----
 key1 |    a |  b
 key4 | null |   b
 key3 |    a |
 key2 |      |  b

Any pointers about this? Is this even possible at all with my (simple) approach?

Thanks

Giacomo

P.S. i just realized I committed also the exposure of "str()" method from cql_message_execute_impl_t. That is unrelated to the "skip" method

[mhfrantz]

Without using the hton* function, this won't be portable, will it?

Also, how would this be different from encode_int(output, -1)? If you're trying to encode a NULL string, then encode_short(output, -1) might do what you want, if you're right about negative length values having special significance. If you're trying to encode a NULL integer (or floating point), then how do you distinguish NULL from any valid value? I don't know the binary spec, but based on this implementation, there is no room in the representation for NULL integer values.

[benelgiac]

Definitely right about the hton, I have just missed it. (initially I though len was a cql_byte_t)

Please consider that I'm just trying to speculate about the binary protocol, that I just don't know well enough. However, my point is based, besides the lines already quoted, on the following ones too (4.1.4)

4.1.4. QUERY

Performs a CQL query. The body of the message must be:
    <query><query_parameters>
where <query> is a [long string] representing the query and
    <query_parameters> must be
    <consistency><flags>[<n><value_1>...<value_n>][<result_page_size>][<paging_state>][<serial_consistency>]
where:
   - <consistency> is the [consistency] level for the operation.
   - <flags> is a [byte] whose bits define the options for this query and
     in particular influence what the remainder of the message contains.
     A flag is set if the bit corresponding to its `mask` is set. Supported
     flags are, given there mask:
       0x01: Values. In that case, a [short] <n> followed by <n> [bytes]
             values are provided. Those value are used for bound variables in
             the query.

If I get this correctly, values are always encoded as [bytes]. So, I guess the server takes care of correctly interpreting the byte stream, by casting the appropriate amount of bytes to the right data type based on the corresponding type of the bound variable.

In that case, a 'null' byte should always lead to a 'null' value for any possible type (if that is taken care of on the server side!).

For the purpose of what I'm trying to do, yes, I could have used any other encode_* method that leads to encoding a negative cql_int_t value. I thought it was just more clear to add a new one, given the specific purpose.

[benelgiac]

A similar feature has been correctly added on #32