Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable writes sync in persistent cache #21754

Merged
merged 1 commit into from
Oct 14, 2020
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Disable writes sync in persistent cache
With default configuration, every write is synced to disk. For the
persistent cache, Beats not need to guarantee that everything is on disk,
as the data can be recovered from the source endpoints, but the database
was not being properly closed, so there could happen that most of the
recent data was lost if the beat is stopped.
Now, the processors can close their resources, so the database is
properly closed in an normal shutdown, so we can go on without syncs on
writes, what gives a much better performance.
  • Loading branch information
jsoriano committed Oct 13, 2020
commit 9001c73dec8e1da90169dd701a37b989d1b60a95
5 changes: 1 addition & 4 deletions x-pack/libbeat/persistentcache/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,7 @@ func newStore(logger *logp.Logger, dir, name string) (*Store, error) {
// Opinionated options for the use of badger as a store for metadata caches in Beats.
options := badger.DefaultOptions(dbPath)
options.Logger = badgerLogger{logger.Named("badger")}
// TODO: Disabling sync writes gives better performance, and data loss wouldn't
// be a problem for caches. But we are not properly closing processors yet, so let
// sync on writes by now.
// options.SyncWrites = false
options.SyncWrites = false

db, err := badger.Open(options)
if err != nil {
Expand Down