From 66bddc53e0399c79741bf782be4918b876b8e58f Mon Sep 17 00:00:00 2001 From: Alan Clucas Date: Wed, 21 Feb 2024 19:13:10 +0000 Subject: [PATCH] docs: Improve archive log docs as to why it's not recommended (#12597) Signed-off-by: Alan Clucas Co-authored-by: Anton Gilgur <4970083+agilgur5@users.noreply.github.com> (cherry picked from commit 11b701a4f9ba45145623d0914c10d61541b9d2c1) --- docs/configure-archive-logs.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/configure-archive-logs.md b/docs/configure-archive-logs.md index 023f66366c4b..44f8be295816 100644 --- a/docs/configure-archive-logs.md +++ b/docs/configure-archive-logs.md @@ -1,6 +1,8 @@ # Configuring Archive Logs -⚠️ We do not recommend you rely on Argo Workflows to archive logs. Instead, use a conventional Kubernetes logging facility. +!!! Warning "Not recommended" + We do not recommend relying on Argo to archive logs as it is naive and not purpose-built for indexing, searching, and storing logs. + This feature is provided as a convenience to quickly view logs of garbage collected Pods in the Argo UI, but we [recommend](#suggested-alternatives) you integrate a dedicated, Kubernetes-aware logging facility. To enable automatic pipeline logging, you need to configure `archiveLogs` at workflow-controller config-map, workflow spec, or template level. You also need to configure [Artifact Repository](configure-artifact-repository.md) to define where this logging artifact is stored. @@ -59,3 +61,19 @@ spec: archiveLocation: archiveLogs: true ``` + +## Suggested alternatives + +Argo's log storage is naive and will not reach feature parity with purpose-built facilities optimized for indexing, searching, and storing logs. Some open-source tools include: + +* [`fluentd`](https://github.com/fluent/fluentd) for collection +* [ELK](https://www.elastic.co/elastic-stack/) as storage, querying and a UI +* [`promtail`](https://grafana.com/docs/loki/latest/send-data/promtail/) for collection +* [`loki`](https://grafana.com/docs/loki/latest/) for storage and querying +* [`grafana`](https://grafana.com/docs/grafana/latest/) for a UI + +You can add [links](links.md) to connect from the Argo UI to your logging facility's UI. See examples in the [`workflow-controller-configmap.yaml`](workflow-controller-configmap.yaml). + +* Link `scope: workflow` to the logs of a Workflow +* Link `scope: pod-logs` to the logs of a specific Pod of a Workflow +* Parametrize the link with `${metadata.name}`, `${metadata.namespace}`, `${metadata.labels}`, and other available metadata