Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Created migration guide from SSD -> Distributed #16389

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Jayclifford345
Copy link
Contributor

@Jayclifford345 Jayclifford345 commented Feb 20, 2025

What this PR does / why we need it:
This PR includes a migration guide from Loki SSD to Loki distributed for users using the Loki helm.

I would also like to give credit to @lindeskar for initially running through this process in the dark:
#13726

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@Jayclifford345 Jayclifford345 requested a review from a team as a code owner February 20, 2025 15:07
@github-actions github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Feb 20, 2025
Copy link
Contributor

github-actions bot commented Feb 20, 2025

Copy link
Contributor

@lindeskar lindeskar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice guide, thanks for putting it together!

chunk_encoding: snappy
# Add this to ingester; this will force ingesters to flush before shutting down
wal:
flush_on_shutdown: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be removed after a finished migration?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I can leave it as optional. It depends if the operator wants to force ingesters to flush on a shutdown. I will ask Ed for his opinion aswell

Co-authored-by: Alexander Lindeskär <lindeskar@users.noreply.github.com>
Signed-off-by: Jay Clifford <45856600+Jayclifford345@users.noreply.github.com>
@mwtzzz
Copy link

mwtzzz commented Feb 21, 2025

Hi, going through your doc today, the only problem I have noticed so far, is that the ruler sidecar is not migrated properly.

If using a SimpleScaleable mode with a config like the following:

loki:
  rulerConfig:
    enable_alertmanager_v2: true
    alertmanager_url: http://xxx
    storage:
      type: local
      local:
        directory: /rules
sidecar:
  rules:
    enabled: true
    folder: /rules/1
    logLevel: debug

Then when applying the SSDM->Distributed migration, there are two problems with the new ruler pod:

  1. the new ruler pod does not contain any sidecar
  2. the new ruler pod throws an error:
level=error ts=2025-02-21T21:11:16.53272825Z caller=log.go:216 msg="error running loki" err="mkdir /rules: read-only file system

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/L type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants