Add shared cache field to CachedState #878
Description
We need a way to keep our cache alive across transactions. This can be achieved by adding a field of type Arc<RwLock<HashMap<ClassHash, CompiledClass>>> to CachedState. This field would be accessed read-only by the execution logic, with the write being managed by the caller so they can decide the eviction and update policies, or lack thereof.
A helper function to update the cache from an owned instance should be provided, which would work much like the following pseudo-Rust:
fn merge_caches(shared: Arc<RwLock<...>>, mut private: CompiledClassCache) {
// Take a read lock to access `shared` fields.
// This is slightly more work for the core but avoids contention.
// The scope is important to drop the read lock when finished, otherwise we'll deadlock later.
{
let shared = shared.read().unwrap();
private.extend(shared.iter().cloned()); // Cloning the elements is relatively cheap if they're inside an `Arc<_>`
}
// Take a write lock to replace the old instance. This one contains all elements now.
// The scope is important to minimize the critical section.
{
let shared = shared.write().unwrap();
// Use `swap` to avoid performing an expensive `drop` with the lock taken.
std::mem::swap(&mut private, &mut shared);
}
// Here the old instance gets dropped without need for locking.
}The suggested approach ensures we only block for a copy of 48 bytes, minimizing contention.
We also need a method to get the private cache for a finished transaction, such as:
// NOTE: type names are bogus.
// The name starts with `take_` to signal it takes ownership away from `tx` rather than
// cloning or borrowing.
fn take_cache(mut tx: Transaction) -> CompiledClassCache {
std::mem::take(tx.cached_state.compiled_class_cache);
}We should also add some tracing with timestamps for the merge_caches calls.
Doc-wise, we should include examples for:
- Using uniquely private caches to avoid any kind of blocking (e.g. create an empty shared cache per transaction);
- FIFO or LRU based eviction policies. Look for cache crates.
Also consider adding an example for the following use cases:
- Updating the cache in some kind of orchestrator;
- Updating a cache with a dedicated thread by passing the private caches via channels;
Or create another PR/issue for that.