KonfigStore - A cache backed multi-tenant config store

A multi tenant configuration management. Configuration are like key/value pairs for a tenant.These configurations are specific to feature/workflow that a tenant uses in your system.

Note: Configurations in this case are not application specific configurations, these are configurations specific to a tenant(tenant can be any entity in the ecosystem)

Features:

• Values are of JSON format
• Developers/users can define their custom values as structs.
• default TTL and custom TTL for each config
• Register/De-Register a configuration using a hook based mechanism. Registration of a configuration will simply register a configuration of specific type.
• Caching of configuration in redis
• Persistent storage backed by rdbms/nosql databases(mysql)
• Support of different cache write policies(write-through, write-around, write-back)
• Cache stampede protection
• Settings/Options at each configuration
  • ttl of the config in cache
  • distributed cache mode
  • db persistence mode
  • db timeout
  • cache timeout
  • eager refresh
  • write policy
  • custom configKey
  • custom cacheKey
• Monitoring
• Logging
• Custom Table name of Config
• shell script to create the DDL
• Benchmarking, performance etc
• Current state of a group/category of configuration in cache/db.
• Grouping/Categorisation of config to ease the fetch of config of similar category/groups(do we need multiple groups per config?)
• Lineage of a configuration with time(will include changes in configuration, timestamp, updatedBy etc)
• Web based UI for managing the configurations per tenant(RBAC for listing, editing, viewing configurations per tenant). It is optional.

Usage

go get github.com/rag594/konfig-store

How to use it

NOTE: Currently for database MYSQL v8.0 or greater and Redis as a distributed cache

NOTE: sqlx(https://github.com/jmoiron/sqlx) is currently supported for Db and go-redis/v9(https://github.com/redis/go-redis) for redis

Config Schema needs to be created

First, create config table (please note that currently entityId)

# MySQL, MariaDB
CREATE TABLE Config
(
  id        BIGINT AUTO_INCREMENT PRIMARY KEY,
  entityId  BIGINT      NOT NULL,
  configKey VARCHAR(50) NOT NULL,
  value     JSON        NOT NULL DEFAULT (JSON_OBJECT()),
  addedOn   DATETIME             DEFAULT CURRENT_TIMESTAMP,
  updatedOn DATETIME             DEFAULT CURRENT_TIMESTAMP,
  CONSTRAINT `uc_entityId_configKey` UNIQUE (`entityId`, `configKey`)
);

Import the konfigStore package

import "github.com/rag594/konfig-store"

Initialise the KonfigStore

konfigStore := konfigStore.New(
		konfigStore.WithDatabase(&konfigStore.Database{Connection: dbConn}),
		konfigStore.WithRedisCache(&konfigStore.RedisCache{
			Connection:         redisConn,
			ClusterModeEnabled: false,
		}))

Write

type IWritePolicy[T config.TenantId, V any] interface {
	SetConfig(ctx context.Context, cacheKey string, entityId T, value *V) error
}

Suppose our configuration of below type per tenant

type SmartFeatureConfig struct {
    SmartSetting string
}

Register your new configuration

Three write policies are supported, we can select as per the needs per configuration level:

1. write-through
2. write-around
3. write-back

smartFeatConfigRegister := configRegister.RegisterConfig[int, SmartFeatureConfig](
      kStore,
      configRegister.WithWritePolicy(writePolicy.WriteBack),
      configRegister.WithTTL(time.Minute),
)

Define your cache key

NOTE: Currently cacheKey is a function of entityId, in coming release we can provide custom func as well

// define your new cache key(it is a function of entityId along with other options)
    cacheKeyForEntityC := cache.NewCacheKey[int, SmartFeatureConfig](20)

Write/Set the config

    err := smartFeatConfigRegister.WritePolicy.SetConfig(context.Background(), cacheKeyForEntityC.DefaultValue(), 20, val)
    
    if err != nil {
        // handle error
    }

Read

type IReadPolicy[T config.TenantId, V any] interface {
	GetConfig(ctx context.Context, cacheKey string, entityId T) (*V, error)
}

Suppose our configuration of below type per tenant

type ComplexFeatureConfig struct {
	Enable string
}

Register your new configuration

complexFeatConfigRegister := configRegister.RegisterConfig[int, ComplexFeatureConfig](
	kStore, 
	configRegister.WithTTL(time.Minute), 
	)

Define your cache key

NOTE: Currently cacheKey is a function of entityId, in coming release we can provide custom func as well

// define your new cache key(it ius a function of entityId along with other options)
	cacheKeyForEntityA := cache.NewCacheKey[int, ComplexFeatureConfig](11)

Read the config

// get the config for any entity
	config, err := complexFeatConfigRegister.ReadPolicy.GetConfig(context.Background(), cacheKeyForEntityA.DefaultValue(), 11)
	if err != nil {
		// handle error
	}