Support user provided timestamps

[sagar0]

A basic implementation to support user specified timestamps in RocksDB.
This allows to fetch older versions of the keys from RocksDB. RocksDB already provides MVCC support via snapshots and internal sequence numbers, but allowing RocksDB users to plugin their own timestamps is going one step further to allow users on top of RocksDB to build their own MVCC-aware services.

See the tests added in db_basic_test.cc : DBBasicTest.Timestamp1 and DBBasicTest.Timestamp2, but here's a simplified version:

WriteOptions wo;
wo.timestamp = 100;
Put(wo, "k1", "v1");
// overwrite k1
wo.timestamp = 200;
Put(wo, "k1", "v2");

ReadOptions ro;
ro.timestamp = 0; // this is the default anyway
ASSERT_EQ("v2", Get(ro, "k1"));  //gets the latest version

// Request a version older than or equal to timestamp 150
ro.timestamp = 150;
ASSERT_EQ("v1", Get(ro, "k1"));

Sending this out so that we can start a discussion on the API. This is not ready for committing, yet.

In the current implementation:

• a new option timestamp is added to WriteOptions and ReadOptions.
• the timestamp is of type uint64_t, but we can decide to make it a user pluggable type to be more generic and if is so desired.
• the timestamp is added after the sequence number at the end of the internal key.

Still need to think about / implement:

• The timestamp could be zeroed out similar to the sequence number at the bottom-most level, or based on some condition.
• A user callback function that tells RocksDB when a key older than some timestamp can be dropped.
• Would it be cleaner to add a new ValueType ?
• Should we add new overloaded Get / Put overloaded APIs with timestamp as an argument instead of putting it in Read/Write Options?

[adamretter]

@sagar this looks very interesting to me, could you comment on how you see this being used?

[sagar]

Buddy I wish I could but don’t have any context :)

On Sat, 2 Feb 2019 at 5:01 PM, Adam Retter ***@***.***> wrote: @sagar <https://github.com/sagar> this looks very interesting to me, could you comment on how you see this being used? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4942 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABI7d_f304O6mRJ694-4YmfyMa1o_7m2ks5vJXcugaJpZM4afeV1> .

-- Sagar

[adamretter]

@sagar so would this allow users to implement MVCC aware structures on top of RocksDB? For example if I wanted to cache some key/value pairs in-memory, I would need to know their version (or timestamp) so that my transaction is reading the correct version.

[sagar0]

This PR was a prototype to show how user-provided timestamps feature can be implemented in RocksDB.
@riversand963 is working on user-provided timestamps in #5079 . So closing this now.