Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Save tag_cardinality_limit cache to disk before shutdown #2031

Open
binarylogic opened this issue Mar 11, 2020 · 0 comments
Open

Save tag_cardinality_limit cache to disk before shutdown #2031

binarylogic opened this issue Mar 11, 2020 · 0 comments
Labels
domain: performance Anything related to Vector's performance domain: reliability Anything related to Vector's reliability have: nice This feature is nice to have. It is low priority. meta: idea Anything in the idea phase. Needs further discussion and consensus before work can begin. needs: approval Needs review & approval before work can begin. transform: tag_cardinality_limit Anything `tag_cardinality_limit` transform related type: enhancement A value-adding code change that enhances its existing functionality.

Comments

@binarylogic
Copy link
Contributor

Currently, the tag_cardinality_limit transform's cache is held in memory. This means that the cache is reset between Vector restarts. For most cases, this should be sufficient, since this transform is designed as a protection mechanism for long-lived Vector processes. But I could see this being a problem for very large setups where data is continually violating the cardinality limit.

I'm opening this issue for feedback from users. If you find this valuable please comment and 👍 .
@binarylogic binarylogic added type: enhancement A value-adding code change that enhances its existing functionality. meta: idea Anything in the idea phase. Needs further discussion and consensus before work can begin. needs: approval Needs review & approval before work can begin. transform: tag_cardinality_limit Anything `tag_cardinality_limit` transform related labels Mar 11, 2020
@binarylogic binarylogic added domain: performance Anything related to Vector's performance domain: reliability Anything related to Vector's reliability have: nice This feature is nice to have. It is low priority. labels Aug 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
domain: performance Anything related to Vector's performance domain: reliability Anything related to Vector's reliability have: nice This feature is nice to have. It is low priority. meta: idea Anything in the idea phase. Needs further discussion and consensus before work can begin. needs: approval Needs review & approval before work can begin. transform: tag_cardinality_limit Anything `tag_cardinality_limit` transform related type: enhancement A value-adding code change that enhances its existing functionality.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@binarylogic