monc is a containerized stack for observability, which includes exposing, storing and visualization of logs and metrics. The stack is completely free from any dependencies and its abstracted to absolute simplification.
- node-exporter - for exporting metrics
- promtail - for exporting logs
- prometheus - for storing metrics
- loki - for storing logs
- grafana - for visualizations
The node exporter containerised setup mounts sys, proc and root file system on the container (in a secure manner). sys and proc for system generated metrics and root for disk monitoring. A process runs in the background which exposes all the collected metrics to {IP}:9100/metrics endpoint. matter fact the metrics generated by the container itself are flexiable but are ignored by default.
The promtail containerized setup mounts /var/log to container. A process runs in he background which pushed these logs to the loki instance, in a configured manner where jobs and other are defined.
Prometheus scraps all the exposed metrics from exposed endpoints and store them in a time series data base. from this tsdb queries specificially PromQL are made to retrive data in specific order to visualize them (yes prometheus can visualize metrics but is not yet powerful as grafana)
All the logs pushed by promtail are stored in the loki's chunks database from which they can be retrived via its own LogQL queries and visualize lateron.
Grafana is used for visualizations [the end], The data stored in loki and prometheus are treated as datasources thy which using queries dashboards can be built [they are beautiful]
The stack is simplified being three tier it is modular in nature where user have flexibility to either setup only metrics monitoring or logs or both at the same time. matter fact the exporters both combined take about 40mb of memory and store services take about 100mb max combined. with docker volumes are mounted which provide flexibility to backup and manage the data stored.
The requirements for the setup is docker-compose thats it .. just clone the repo on the systems , entry the ip and config files paths [basic ones are included in the repo] in the .env file and spin the services.
The services are divided into profiles
monc → all services combined [tier1 + tier2 + tier3]
export → export logs and metrics [tier1]
store → store logs and metrics [tier2]
metrics → setup export and store of metrics
logs → setup export and store of logs
visualize → setup visualizations [tier3]
[assuming user is inside the repo and .env file is filled]
Start the stack
docker-compose -f monc.yml --profile monc --env-file=.env up