README

lrucache_client - foundation package for VoltDB LRU Cache

What is this repository for?

This repo contains the core classes used by VoltDB's LRU cache implementation.

How do I get set up?

Configuration

You will need the latest VoltDB client library.

Dependencies

voltseutil

To use the lrucache you will also need an implementation project, such as lrucache_client_cassandra . To add a new database you will find [lrucache_sdk]https://bitbucket.org/sr_mad_science/lrucache_sdk) very useful.

Database configuration

Both voltDB and the target database will have copies of the same table. So for example:

VoltDB:

CREATE TABLE subscriber  ( 
  s_id    BIGINT NOT NULL PRIMARY KEY
, sub_nbr VARCHAR(80) NOT NULL
, f_tinyint tinyint
, f_smallint smallint
, f_integer integer
, f_bigint bigint
, f_float float
, f_decimal decimal
, f_geography geography
, f_geography_point GEOGRAPHY_POINT
, f_varchar varchar(80)
, f_varbinary varbinary(1024)
, flush_date TIMESTAMP 
, last_use_date TIMESTAMP NOT NULL) ;

Cassandra:

CREATE TABLE vtest.subscriber  (
  s_id    BIGINT
, sub_nbr text
, f_tinyint tinyint
, f_smallint smallint
, f_integer int
, f_bigint bigint
, f_float float
, f_decimal decimal
, f_geography text
, f_geography_point text
, f_varchar varchar
, f_varbinary blob
, flush_date TIMESTAMP
, last_use_date TIMESTAMP
, primary key (s_id)) ;

Usage guidelines

Jar files

Make sure you have JAR files for

• lrucache_client
• voltseutil - general utilities
• Your chosen implementation (e.g. lrucache_client_cassandra)
• Database support classes (e.g. voltseutil_cassandra)

Connecting to VoltDB

Use LRUCacheClient

Getting missing data

Use a rematerializer class such as CassandraRematerializerImpl that implements LruFillMissingCallbackIFace.

Note that each class will have its own config information:

Properties config = new Properties();
		
config.put(AbstractSqlRematerializer.DATABASE_HOSTNAME, "192.168.0.20");
config.put(AbstractSqlRematerializer.DATABASE_PORT, "9042");

The class name and Properties object ('m_rematConfig') are passed in to the constructor for LRUCacheClient, along with a config for a VoltDB client:

client = new LRUCacheClient(config, m_classname, schema, tableName, m_rematConfig);

Defining a stored procedure that will work with the LRU cache

Your read procedure must follow the following rules:

• Must take PK as leading arguments
• Must call 'setAppStatus' with known values for FOUND and NOT_FOUND

An example:

	private static final byte NOT_FOUND = 42;  
	private static final byte FOUND = 0;  

	private static final byte NOT_FOUND = 42;  
	private static final byte FOUND = 0;  

	...	

	public VoltTable[] run(long subscriberId, long maxSizePerPart) throws VoltAbortException {

	setAppStatusCode(FOUND);

or

	setAppStatusCode(NOT_FOUND);

Defining a 'purge' stored procedure

The most efficient way to purge data is to add the logic to an existing procedure that is called all the time, with a read procedure being an ideal candidate.

Putting it all together

Use the additional 'callProcedure' in LRUCacheClient to call your FOUND-enabled procedure:

Object[] procArgs = { param, limit };
GetByIdCallback callback = new GetByIdCallback(System.currentTimeMillis());

client.callProcedure(callback, theMissingDataCallback, (byte) 42, "GetById", 1,procArgs);

Behind the scenes we:

• Call 'GetById'

• Check the status code - if it's '42' we assume the PK is the first field (the '1'in the call) and that we need to ask the rematerializer to replace the missing data. We ask the other DB for a value and if it's found we insert it into VoltDB and they retry the transction

• Your callback 'callback' 's 'clientCallback' method is invoked.

Copious stats for how this is working are available from 'client.getStatsInstance().toString()'

Contribution guidelines

Use:

• The testing harness : lrucache_sdk

• The test DB schema lrucache_server

Who do I talk to?

• drolfe@voltdb.com