Skip to content

Conversation

le0pard
Copy link
Contributor

@le0pard le0pard commented Mar 28, 2023

Self hosted sentry limited in sourcemap external size, which can fetch and cache to 1Mb.

Remote file too large for caching

This is because memcached the maximum size of a value you can store in is 1 megabyte by default. But this can be fixed by -I param.

This PR allow to fetch and cache external sourcemaps for sentry up to 25Mb.

image

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

@le0pard le0pard changed the title Support external sitemaps bigger, than 1Mb (25Mb limit) Support external sourcemaps bigger, than 1Mb (25Mb limit) Mar 28, 2023
Co-authored-by: Burak Yigit Kaya <ben@byk.im>
@@ -107,6 +107,9 @@ def get_internal_network():
"BACKEND": "django.core.cache.backends.memcached.MemcachedCache",
"LOCATION": ["memcached:11211"],
"TIMEOUT": 3600,
"OPTIONS": {
"server_max_value_length": 1024 * 1024 * 25,
Copy link
Member

@BYK BYK Mar 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this value needs to be in sync with the other value, I feel like this should be in a shared env variable and here we should parse out that value. Something like:

SENTRY_MAX_EXTERNAL_SOURCEMAP_SIZE="25M"

and then in this file

UNIT_SIZES={
  "K": 1024,
  "M": 1024**2,
  "G": 1024**3,
}
def unit_text_to_bytes(text):
  multiplier = UNIT_SIZES[text[-1]]
  return float(text[:-1])*multiplier

...

"OPTIONS": {
  "server_max_value_length": unit_text_to_bytes(env("SENTRY_MAX_EXTERNAL_SOURCEMAP_SIZE", "1M")),

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BYK changed in 6117ec8

@le0pard le0pard requested a review from BYK March 28, 2023 16:14
@le0pard le0pard changed the title Support external sourcemaps bigger, than 1Mb (25Mb limit) Support external sourcemaps bigger, than 1Mb Mar 28, 2023
Copy link
Member

@BYK BYK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rejecting for the environment docker compose config placement

le0pard and others added 2 commits March 28, 2023 20:20
@le0pard le0pard requested a review from BYK March 28, 2023 17:22
Copy link
Member

@BYK BYK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@hubertdeng123 hubertdeng123 merged commit 1ddb182 into getsentry:master Mar 29, 2023
@le0pard le0pard deleted the support-big-sitemaps branch March 29, 2023 21:46
@github-actions github-actions bot locked and limited conversation to collaborators Apr 14, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants