Skip to content

docs(self-hosted): monitor self-hosted instance#12660

Merged
BYK merged 4 commits intogetsentry:masterfrom
aldy505:docs/self-hosted/monitoring
Mar 10, 2025
Merged

docs(self-hosted): monitor self-hosted instance#12660
BYK merged 4 commits intogetsentry:masterfrom
aldy505:docs/self-hosted/monitoring

Conversation

@aldy505
Copy link
Copy Markdown
Collaborator

@aldy505 aldy505 commented Feb 11, 2025

DESCRIBE YOUR PR

We don't have any good article or explanation on how to monitor self-hosted Sentry.

"Monitoring self-hosted Sentry" means 3 things:

  1. Monitor the actual services made by Sentry (sentry, snuba, symbolicator, relay) by using Statsd
  2. Monitor the dependencies (Kafka, Postgres, etc) by using Prometheus exporters, or similar things
  3. Monitor the host machine or Docker daemon

cc @getsentry/dev-infra

IS YOUR CHANGE URGENT?

Help us prioritize incoming PRs by letting us know when the change needs to go live.

  • Urgent deadline (GA date, etc.):
  • Other deadline:
  • None: Not urgent, can wait up to 1 week+

SLA

  • Teamwork makes the dream work, so please add a reviewer to your PRs.
  • Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it.
    Thanks in advance for your help!

PRE-MERGE CHECKLIST

Make sure you've checked the following before merging your changes:

  • Checked Vercel preview for correctness, including links
  • PR was reviewed and approved by any necessary SMEs (subject matter experts)
  • PR was reviewed and approved by a member of the Sentry docs team

LEGAL BOILERPLATE

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

EXTRA RESOURCES

@vercel
Copy link
Copy Markdown

vercel bot commented Feb 11, 2025

@aldy505 is attempting to deploy a commit to the Sentry Team on Vercel.

A member of the Team first needs to authorize it.

Comment on lines +124 to +129
prometheus-kafka-exporter:
<<: *restart_policy
image: danielqsj/kafka-exporter
ports:
- "127.0.0.1:9308:9308"
command: ["--kafka.server=kafka:9092"]
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a bit hesitant to use this, since most people recommend this: https://github.com/prometheus/jmx_exporter

Personally I don't use both, since I use Redpanda, and they already got built-in Prometheus metrics https://docs.redpanda.com/current/manage/monitoring/

</clickhouse>
```

Up to this point, all metrics are exposed only on `localhost`. If you don't want your Sentry installation to expose any other ports, you can remove every `ports` directive on the `docker-compose.yml` file and route everything through the `nginx` container, by modifying your `nginx.conf` file to the following:
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also one reason why I don't provide monitoring the Nginx container, because I don't think that the stub_status module is not included on the Nginx's Alpine Docker image: https://nginx.org/en/docs/http/ngx_http_stub_status_module.html

@aldy505
Copy link
Copy Markdown
Collaborator Author

aldy505 commented Feb 14, 2025

Ping @bc-sentry @hubertdeng123 @BYK

@aldy505
Copy link
Copy Markdown
Collaborator Author

aldy505 commented Mar 1, 2025

@bc-sentry @hubertdeng123 @BYK another ping

Copy link
Copy Markdown
Member

@BYK BYK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty good but I have two issues with it:

  1. Looks like there's a preference of using Prometheus for most things but then since Sentry uses statsd clients by default it keeps referring to those. I think the article should take a clear stance and voice: go all in with Prometheus, or just mention it once and be done with it. I think I'd lean towards all-in Prometheus but I'll leave the final decision to you.
  2. The code examples for monitoring are hard to navigate. I think we should just move these over to their respective files, commented out with a link to this article. WDYT?

Comment thread develop-docs/self-hosted/experimental/monitoring.mdx Outdated
Comment thread develop-docs/self-hosted/experimental/monitoring.mdx Outdated
Comment thread develop-docs/self-hosted/experimental/monitoring.mdx
Comment thread develop-docs/self-hosted/experimental/monitoring.mdx Outdated
Comment thread develop-docs/self-hosted/experimental/monitoring.mdx Outdated
Comment thread develop-docs/self-hosted/experimental/monitoring.mdx Outdated
Comment thread develop-docs/self-hosted/experimental/monitoring.mdx Outdated
Co-authored-by: Burak Yigit Kaya <ben@byk.im>
@aldy505
Copy link
Copy Markdown
Collaborator Author

aldy505 commented Mar 3, 2025

  1. Looks like there's a preference of using Prometheus for most things but then since Sentry uses statsd clients by default it keeps referring to those. I think the article should take a clear stance and voice: go all in with Prometheus, or just mention it once and be done with it. I think I'd lean towards all-in Prometheus but I'll leave the final decision to you.

I'm not sure either, as I don't think most of the people who deploy self-hosted Sentry are devops person. I think it'd be best if we provide a rough idea on how they can monitor anything other than what Sentry made (meaning monitoring Kafka, Postgres, etc). But if they have gone this far, I guess they should already know on how to monitor those.

  1. The code examples for monitoring are hard to navigate. I think we should just move these over to their respective files, commented out with a link to this article. WDYT?

I would prefer to have it this way. I think this is a part that doesn't easily get outdated anyway.

@BYK
Copy link
Copy Markdown
Member

BYK commented Mar 5, 2025

I would prefer to have it this way. I think this is a part that doesn't easily get outdated anyway.

It's not about getting outdated but putting the contextually relevant things close together. If these live on docs, then one need to go back and forth between the docs and the config files. If they are in config files with clear references, one can simply read the docs and then switch to configs and code completely to implement it without backreferencing.

Makes sense?

@BYK
Copy link
Copy Markdown
Member

BYK commented Mar 5, 2025

I'm not sure either, as I don't think most of the people who deploy self-hosted Sentry are devops person. I think it'd be best if we provide a rough idea on how they can monitor anything other than what Sentry made (meaning monitoring Kafka, Postgres, etc). But if they have gone this far, I guess they should already know on how to monitor those.

Bu we already take sides so either make it fully neutral (which is very hard) or just pick your battle and expect other people to understand how to adapt it to their needs. A half-baked or middle-of-the-road doc like this is more confusing. At least you'll be internally consistent and easier to transform if you just pick a side.

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 5, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
develop-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 10, 2025 8:46pm
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
changelog ⬜️ Ignored (Inspect) Visit Preview Mar 10, 2025 8:46pm
sentry-docs ⬜️ Ignored (Inspect) Visit Preview Mar 10, 2025 8:46pm

@aldy505
Copy link
Copy Markdown
Collaborator Author

aldy505 commented Mar 10, 2025

Okay let's just provide the docs to monitor Sentry-stuff using statsd. Let's ignore everything else and put a disclaimer.

aldy505 added a commit to aldy505/sentry-self-hosted that referenced this pull request Mar 10, 2025
@aldy505 aldy505 requested a review from BYK March 10, 2025 04:30
@BYK BYK enabled auto-merge (squash) March 10, 2025 20:41
BYK pushed a commit to getsentry/self-hosted that referenced this pull request Mar 10, 2025
@BYK BYK merged commit 94124b0 into getsentry:master Mar 10, 2025
10 checks passed
codyde added a commit that referenced this pull request Mar 13, 2025
…om/getsentry/sentry-docs into codyde/javascript-tracing-refactor

* 'codyde/javascript-tracing-refactor' of https://github.com/getsentry/sentry-docs: (104 commits)
  Update docs/concepts/key-terms/tracing/index.mdx
  Update docs/concepts/key-terms/tracing/index.mdx
  Update docs/concepts/key-terms/tracing/index.mdx
  Update docs/concepts/key-terms/tracing/index.mdx
  Update docs/concepts/key-terms/tracing/index.mdx
  Update docs/concepts/key-terms/tracing/index.mdx
  Update index.mdx (#12928)
  docs(self-hosted): monitor self-hosted instance (#12660)
  fix(feedback): rm per-feedback attach limit (#12968)
  fix(404): php integrations (#12966)
  docs(js): Update stale reference (#12964)
  fix(solidstart): Remove outdated `sentrySolidStartVite` option (#12962)
  chore(sandbox): change sandbox link (#12814)
  feat: Godot debug symbols guide (#12843)
  fix 404s (#12961)
  Fix SDK getting started pages (onboarding options) (#12960)
  fix(platform): Update onboarding options on tab switch (#12913)
  Fix Apollo links (#12952)
  chore(Android): Update docs to match actual SDK behavior (#12954)
  docs(js): Add example docs for non-HTTP protocols (#12933)
  ...
@github-actions github-actions bot locked and limited conversation to collaborators Mar 26, 2025
@aldy505 aldy505 deleted the docs/self-hosted/monitoring branch June 6, 2025 12:44
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants