Spring Reactive Logging Service

This project is a Spring WebFlux-based Logging Service designed to
consume messages from Kafka, persist them into a PostgreSQL
database in batches, and expose metrics via Micrometer + Prometheus
for monitoring in Grafana.

It also integrates with the custom Integration Library,
which provides reusable components for Kafka, Redis, and Prometheus
integration across microservices.

---

🚀 Features

• ✅ Reactive programming with Spring WebFlux.
• ✅ Integration Library for seamless Kafka, Redis, and Prometheus setup.
• ✅ Kafka consumer for log messages.
• ✅ Batch inserts into PostgreSQL for efficiency.
• ✅ Exposes /actuator/prometheus metrics endpoint.
• ✅ Docker Compose setup for Prometheus + Grafana monitoring.
• ✅ Prebuilt Grafana dashboard for visualizing metrics.

---

📂 Project Structure

spring-reactive-logging-service/
├── src/
│   ├── main/java/com/collicode/logging_service
│   │   ├── config/          # Integration + App configuration
│   │   ├── consumer/        # Kafka consumer logic
│   │   ├── controller/      # REST endpoints (if needed)
│   │   ├── model/           # DTOs and Entities
│   │   ├── repository/      # Reactive DB repository
│   │   └── service/         # Core business logic
│   └── resources/
│       ├── application.yml  # Spring configuration
│       └── db/migration/    # Flyway migrations
├── docker-compose.yml        # Prometheus + Grafana stack
├── prometheus/
│   └── prometheus.yml        # Prometheus scrape config
├── grafana/
│   └── dashboard.json        # Prebuilt Grafana dashboard
└── README.md                 # Documentation

---

⚙️ Setup

1️⃣ Prerequisites

• Java 17+
• Docker + Docker Compose
• Kafka + Zookeeper (or Confluent Cloud)
• PostgreSQL database

---

2️⃣ Run Prometheus & Grafana

docker-compose up -d

• Prometheus → http://localhost:9090
• Grafana → http://localhost:3000 (login: admin/admin)

---

3️⃣ Configure Grafana

• Add a Prometheus data source:
  • URL: http://prometheus:9090
• Import dashboard from grafana/dashboard.json.

---

4️⃣ Run the Service

./mvnw spring-boot:run

or build with Docker:

docker build -t reactive-logging-service .
docker run -p 8080:8080 reactive-logging-service

---

5️⃣ Send Test Logs

kafka-console-producer --broker-list localhost:9092 --topic logs
> {"level":"INFO","message":"Hello from Reactive Logging Service!"}

---

📊 Monitoring Metrics

Exposed metrics endpoint:

http://localhost:8080/actuator/prometheus

Metrics tracked include:

• Kafka consumer lag
• Database batch insert rate
• Error counts per minute
• Log throughput

---

📈 Grafana Dashboard

The included dashboard (grafana/dashboard.json) displays:

• Kafka consumer lag
• Log ingestion rate
• Database insert latency
• Error trends

---

🛠️ Development Notes

• application.yml configures DB, Kafka, and batch settings.
• The Integration Library automatically provides Kafka, Redis, and Prometheus beans via IntegrationConfig.
• Run Kafka + PostgreSQL locally using Docker for full-stack testing.

---

🤝 Contributing

Feel free to open issues or PRs to improve the project.

---

📜 License

This project is licensed under the MIT License.