perf: Enable compression of audit logs to increase retention, without changing disk space requirements #1270
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jimmidyson
reviewed
Sep 1, 2025
|
With the current configuration, we have 1 log file of 100MB, and 10 rotated log files of 100MB each, for a total of 1100MB. That's our target disk space requirement. We want to keep this disk space usage, but enable compression. We can't just make the maximum file size larger. For example, if we increase it to 1000MB, then we'll end up with 1 1000MB log, plus 10 compressed, rotated logs of approximately 100MB each, for a total of 2000MB, much higher than our target. To increase retention, we should reduce the size of the uncompressed log. For example, we could keep |
dlipovetsky
force-pushed
the
dlipovetsky/audit-logs-compress
branch
from
4e92564 to
55978d5
Compare
dlipovetsky
force-pushed
the
dlipovetsky/audit-logs-compress
branch
from
55978d5 to
a904f0b
Compare
jimmidyson
approved these changes
Sep 10, 2025
jimmidyson
reviewed
Sep 10, 2025
dlipovetsky
changed the title
We enable compression of audit logs. Whenever a log file is rotated, it is compressed. Experiments show that gzip yields a compression factor of approximately 12 for audit logs, so we increase the maximum file size by that factor. With the previous configuration, we had 1 log file of 100MB, and 10 rotated log files of 100MB each, for a total of 1100MB. That's our target disk space requirement. We want to keep this disk space usage, but enable compression. We can't just make the maximum file size larger. For example, if we increase it to 1000MB, then we'll end up with 1 1000MB log, plus 10 compressed, rotated logs of approximately 100MB each, for a total of 2000MB, much higher than our target. To increase retention, we reduce the size of the uncompressed log. We keep --audit-log-maxsize unchanged at 100, but increase --audit-log-maxbackup to 90. We end up with 1 100MB uncompressed log, plus 90 compressed, rotated logs of approximately 10MB each, for a total of 1000MB. That's a little below our target!
dlipovetsky
force-pushed
the
dlipovetsky/audit-logs-compress
branch
from
a904f0b to
5910b9c
Compare
jimmidyson
approved these changes
Sep 10, 2025
Merged
This was referenced Oct 22, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?:
Given our disk space requirements (about 1GB), the audit logs hold about 1-2 hours. By enabling compression, we can increase this by a factor of 12, i.e. 12-24 hours.
In this PR, we enable compression of audit logs. Whenever a log file is rotated, it is compressed.
Experiments show that gzip yields a compression factor of approximately 12 for audit logs, so we increase the maximum file size by that factor. We expect disk space requirements to remain the same.
Which issue(s) this PR fixes:
Fixes #
How Has This Been Tested?:
Special notes for your reviewer:
fluent-bit should continue to work, because it supports reading log files compressed using gzip as of fluent/fluent-bit#8585, released in https://github.com/fluent/fluent-bit/releases/tag/v3.0.0.
However, I need to verify this!