Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add warning around using logstash plugin for new deployments #14058

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

slim-bean
Copy link
Collaborator

What this PR does / why we need it:

Over the years we have really grown to loathe the logstash plugin for the reasons described in the text of this PR. We aren't deprecating or removing it yet but really want to discourage its use for any new deployments.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@slim-bean slim-bean requested a review from a team as a code owner September 5, 2024 12:17
@github-actions github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Sep 5, 2024
Signed-off-by: Edward Welch <edward.welch@grafana.com>
Copy link
Contributor

@JStickler JStickler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[docs team] You also might want to edit /docs/sources/send-data/_index.md where the text says Logstash is the easiest way to start.

docs/sources/send-data/logstash/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/logstash/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/logstash/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/logstash/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/logstash/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/logstash/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/logstash/_index.md Outdated Show resolved Hide resolved
docs/sources/send-data/logstash/_index.md Outdated Show resolved Hide resolved
slim-bean and others added 8 commits September 24, 2024 08:28
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>

* It is very difficult to configure labels correctly. Conceptually Elasticsearch is a very different database from Loki and users almost always end up sending too many high cardinality labels to Loki, which makes getting started with Loki unnecessarily complicated and confusing vs. using other clients.
* Logstash and the upstream Beats components implement backoff and flow control which we've found hard to observe, leading to ingestion delays into Loki which are extremely difficult to address.
* We at Grafana Labs have no expertise at configuring Logstash or understanding its configuration language, so we cannot provide support for it.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* We at Grafana Labs have no expertise at configuring Logstash or understanding its configuration language, so we cannot provide support for it.
* We at Grafana Labs have no expertise at configuring Logstash or understanding of its configuration language, so we cannot provide support for it.

Copy link
Contributor

@JStickler JStickler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[docs team] One small suggestion, but otherwise LGTM.

@loki-gh-app
Copy link
Contributor

loki-gh-app bot commented Sep 24, 2024

This PR must be merged before a backport PR will be created.

2 similar comments
@loki-gh-app
Copy link
Contributor

loki-gh-app bot commented Sep 24, 2024

This PR must be merged before a backport PR will be created.

@loki-gh-app
Copy link
Contributor

loki-gh-app bot commented Sep 24, 2024

This PR must be merged before a backport PR will be created.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport release-3.0.x backport release-3.1.x backport release-3.2.x size/S type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants