Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Improve archive log docs as to why it's not recommended #12597

Merged
merged 7 commits into from
Feb 21, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion docs/configure-archive-logs.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Configuring Archive Logs

⚠️ We do not recommend you rely on Argo Workflows to archive logs. Instead, use a conventional Kubernetes logging facility.
!!! Warning "Not recommended"
We do not recommend relying on Argo to archive logs as it is naive and not purpose-built for indexing, searching, and storing logs.
This feature is provided as a convenience to quickly view logs of garbage collected Pods in the Argo UI, but we [recommend](#suggested-alternatives) you integrate a dedicated, Kubernetes-aware logging facility.

To enable automatic pipeline logging, you need to configure `archiveLogs` at workflow-controller config-map, workflow spec, or template level. You also need to configure [Artifact Repository](configure-artifact-repository.md) to define where this logging artifact is stored.

Expand Down Expand Up @@ -59,3 +61,19 @@ spec:
archiveLocation:
archiveLogs: true
```

## Suggested alternatives

Argo's log storage is naive and will not reach feature parity with purpose-built facilities optimized for indexing, searching, and storing logs. Some open-source tools include:

* [`fluentd`](https://github.com/fluent/fluentd) for collection
* [ELK](https://www.elastic.co/elastic-stack/) as storage, querying and a UI
* [`promtail`](https://grafana.com/docs/loki/latest/send-data/promtail/) for collection
* [`loki`](https://grafana.com/docs/loki/latest/) for storage and querying
* [`grafana`](https://grafana.com/docs/grafana/latest/) for a UI

You can add [links](links.md) to connect from the Argo UI to your logging facility's UI. See examples in the [`workflow-controller-configmap.yaml`](workflow-controller-configmap.yaml).

* Link `scope: workflow` to the logs of a Workflow
* Link `scope: pod-logs` to the logs of a specific Pod of a Workflow
* Parametrize the link with `${metadata.name}`, `${metadata.namespace}`, `${metadata.labels}`, and other available metadata
Loading