Support for org.apache.cassandra.db.marshal-namespaced custom types (See #151)

[nemosupremo]

I was unable to get a word from the cassandra developer mailing list (http://www.mail-archive.com/dev@cassandra.apache.org/msg07178.html) so I can't say I am bringing any sort of official functionality. As stated in my issue (#151) instead of seeing the native type over the wire for timestamps, I was seeing this custom type. Seeing how we were unable to continue without a fix in the Go driver I have simply opted to develop this fix.

The issue didn't occur in the official Python driver, so I have opted to duplicate the functionality there (Can be seen @ https://github.com/datastax/python-driver/blob/master/cassandra/cqltypes.py and https://github.com/datastax/python-driver/blob/master/tests/unit/test_types.py).

The first commit fixes what appears to be a type in the String function of TypeInfo. The next two are the relevant implementations and tests.

[0x6e6562]

On a first pass, this looks OK to me, but it would be good to know what version of Cassandra that you tested this against. Also, what was the CQL schema definition that prompts the relevant server side behavior?

[nemosupremo]

This was tested against Cassandra 2.0.6.

Schema in question:

CREATE TABLE video_daily (
  videoTag text,
  ts timestamp,
  stats blob,
  delta blob,
  PRIMARY KEY (videoTag, ts)
) WITH CLUSTERING ORDER BY (ts DESC);

This same schema however on my most recent tests returns the native type timestamp.

[0x6e6562]

I assume you meant to post

WITH CLUSTERING ORDER BY (ts DESC);

?

[nemosupremo]

Yes, its ts DESC. Was sloppy typing it out.

[0x6e6562]

What is bugging me is that there is a virtually equivalent schema definition in the cqlc test suite, which I have been running regularly against 2.0.6. The relevant test case for this is here. So I'm a bit baffled as to what the actual difference is here (I'm not discounting what you're actually seeing in practice, it's just a curious one but it seems hard to reproduce).

[0x6e6562]

BTW, I may have said this before, but I get the impression that posting to the Cassandra dev list is like writing to /dev/null, but I'm not sure because I don't subscribe to it. There is, however, more visible life on the regular Cassandra user list.

[0x6e6562]

Is there any movement on this?

[nemosupremo]

Still no feedback from Cassandra proper. On another note for 2.1, custom types will be handled through this mechanism, so you think this shouldn't be included in the driver, there should be some way to expose the custom type from the driver rather than just "Failed to unmarshal custom"

[0x6e6562]

That's an interesting point about 2.1. Currently we don't have any explicit support for 2.1 other than try it and see, but we are trying to put through a patch to regression test the driver against multiple versions of Cassandra (see #159). This could serve as basis to test out any handling of 2.1 specific stuff - if you can get a test case to fail against 2.1, that this patch would resolve, then that would definitely be a strong reason to merge this.

[0x6e6562]

BTW, we're not stalling on this patch, but I was wondering how your example schema differs from the test suite that I referenced last week?

[nemosupremo]

The schema doesn't differ. After an upgrade from 1.2 to 2.0, a column family of mine that had been defined timestamp, had become custom<org.apache.cassandra.db.marshal.datetype> for reasons unclear to me - which had caused gocql to reject my queries, but the Java and Python java seemed to handle it fine.

I'll try again in finding more information about the issue through JIRA or emailing a committer.

[nemosupremo]

Finally got some info on the error I was seeing on the IRC channel, its related to this JIRA issue - https://issues.apache.org/jira/browse/CASSANDRA-5723.

Basically from 1.2 to 2.0, they changed the cassandra time type from org.apache.cassandra.db.marshal.datetype to org.apache.cassandra.db.marshal.datetimetype` and the bug I'm experiencing actually seems to be a problem with cassandra.

[nemosupremo]

So here is a response I got over IRC from a datastax dev:

[08:02:43]  <pcmanus>    You have 2 choices: You can update your tables 
(with cqlsh, do a ALTER .. ALTER TYPE x timestamp, to switch to the new TimestampType). 
This is fine if you know you never inserted pre-epoch timestamp. 
Or you can ask the go driver authors to special case for it (which is what the python and java driver do).
[08:02:51]  <pcmanus>    Or both really.

So again this behavior was caused by a change in the way Cassandra handles timestamps, so a test against cassandra isn't really possible. A fix for me seems to be just doing ALTER TYPE against all my tables. So finally the only reason to really continue with this merge is if you want to match the behavior of the Java/Python drivers in regards to handling this case.

[0x6e6562]

OK, on that basis, I'm down with this change - if @phillipCouto, @tux21b or @Zariel agrees, then we should get on and merge it.

[Zariel]

LGTM, thanks very much!

[0x6e6562]

So it turns out that somebody has gone to the effort of identifying the root cause of this behavior, so we might not need this workaround going forwards: https://issues.apache.org/jira/browse/CASSANDRA-7576