🚀 RediServe

RediServe is a distributed cache + DB fetcher microservices playground, built with:

• Java 17 + Spring Boot 3
• Spring Cloud Config (centralized configuration)
• Postgres (for persistence)
• Redis (for caching)
• Apache Kafka (for async cache policy + invalidation events)
• Docker Compose (for local infra)

This is a pet project to learn and experiment with:

• Horizontal scalability
• Cache-aside and read-through patterns
• Centralized cache policy management
• Event-driven orchestration with Kafka
• Configurable caching behavior
• Hot-reloading configs without redeploy

---

📂 Project Structure

RediServe/
├── pom.xml                   # Parent Maven POM (manages versions + dependency management)
├── config-repo/              # Centralized configs (picked up by Spring Cloud Config Server)
│   ├── cache-gateway.yml
│   ├── db-fetcher.yml
│   └── orchestrator.yml
├── config-server/            # Spring Cloud Config Server
├── services/
│   ├── cache-gateway/        # Entrypoint for clients (talks to Redis + DB fetcher, listens to Kafka)
│   ├── db-fetcher/           # Fetches from Postgres, source of truth for data
│   └── orchestrator/         # Publishes cache policy + invalidation events via Kafka
└── docker-compose.yml        # Infra: Redis + Postgres + Kafka + Zookeeper

---

🔧 Prerequisites

• Java 17+
• Maven 3.9+
• Docker + Docker Compose
• (Optional) Homebrew for redis-cli / psql
• curl or Postman for testing

---

▶️ Running Infra

Bring up Postgres, Redis, Kafka, and Zookeeper:

docker compose up -d

Check running containers:

docker ps

You should see:

• rediserve-postgres
• rediserve-redis
• rediserve-kafka
• rediserve-zookeeper

---

🧪 Testing Infra

Redis

docker exec -it rediserve-redis redis-cli ping
# PONG

Postgres

docker exec -it rediserve-postgres psql -U postgres -d rediserve
\conninfo;

Kafka

docker exec -it rediserve-kafka /usr/bin/kafka-topics --list --bootstrap-server localhost:9092

You should see topics like:

• cache.policy.updates
• cache.namespace.invalidate

---

▶️ Running Services

Start Config Server first:

mvn spring-boot:run -pl config-server

Then run the apps:

mvn spring-boot:run -pl services/db-fetcher
mvn spring-boot:run -pl services/cache-gateway
mvn spring-boot:run -pl services/orchestrator

Check configs being served:

http://localhost:8888/cache-gateway/default
http://localhost:8888/db-fetcher/default
http://localhost:8888/orchestrator/default

---

📡 REST Endpoints

1. Fetch Product (Cache-Gateway)

curl -X GET "http://localhost:8081/cache/shop/products/1"

• Reads from Redis if present.
• Falls back to db-fetcher on cache miss (read-through).

---

2. Update Policy (via Orchestrator → Kafka → Cache-Gateway)

curl -X POST "http://localhost:8083/admin/namespaces/shop/policy" \
  -H "Content-Type: application/json" \
  -d '{"ttlSeconds": 5, "consistencyMode": "READ_THROUGH"}'

• Orchestrator publishes to cache.policy.updates.
• Cache-Gateway listens and updates its in-memory policy registry dynamically.
• No restart needed.

---

3. Invalidate Namespace (via Orchestrator → Kafka → Cache-Gateway)

curl -X POST "http://localhost:8083/admin/namespaces/shop/invalidate"

• Orchestrator publishes to cache.namespace.invalidate.
• Cache-Gateway listens and deletes all matching Redis keys.

---

✅ Deliverables

Phase 0 – Foundations

• Infra running (docker compose up works, Redis + Postgres healthy)
• Empty Spring Boot services build and start
• Verified connectivity (PONG from Redis, \conninfo from Postgres)

Phase 1 – Config Server + Cache Basics

• Config Server running (config-server/)
• Centralized config files in config-repo/
• db-fetcher exposes REST endpoint to fetch products from Postgres
• cache-gateway fetches from Redis, falls back to db-fetcher on cache-miss
• Redis caching with type-safe serialization using GenericJackson2JsonRedisSerializer
• TTL is configurable per namespace/entity via policy registry
• Health endpoints exposed (/actuator/health)

Phase 2 – Orchestrator + Kafka

• Added orchestrator service for centralized cache admin
• Kafka topics for cache policy + invalidation events
• Gateway listens on Kafka and applies changes dynamically
• REST → Kafka → Gateway flow verified with curl

Phase 3 – Config on the Fly

• Hot reload configs with Spring Cloud Config + @RefreshScope
• Policies updated at runtime via Orchestrator
• Cache-Gateway applies new TTLs / consistency modes without restart
• Deliverable: Change caching rules dynamically

---

🏗️ Next Steps (Phase 4)

• Add Docker Compose definitions for all services (cache-gateway, db-fetcher, orchestrator, config-server)
• Add integration tests (Redis + Postgres + Kafka + services)
• Explore service discovery (Eureka / Consul) instead of hardcoding hostnames
• Add metrics + dashboards (Prometheus + Grafana)
• Explore write-through and write-behind caching modes
• Add sequence diagrams to README for better architecture visibility