Add log pipeline for RDS PostgreSQL prefix #21371
Open
+65
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
RDS only allows two values for log line prefix, so relying on the one suggested in Datadog documentation will be insufficient. This adds new parsing rules for the "fancier" one allowed by RDS (see here).
What does this PR do?
Adds two new patterns to the Grok parser in the first processor of the PostgreSQL pipeline.
Motivation
DBM is working to enhance our
EXPLAIN
plan gathering capabilities by subscribing to the log EVP track type and capturing the JSON plans emitted to log lines by theauto_explain
extension.In order to tie these records into DBM offerings, we need more information than is captured in the default log line.
Though the
log_line_prefix
setting is highly configurable, in the case of RDS (a case we will support), only two values are permitted. Of these, only one has the rich information we will need:The new patterns match this (and capture every emitted value) as well as the variant of the above which contains a
duration
section (modeled after an existing rule).Review checklist (to be filled by reviewers)
qa/skip-qa
label if the PR doesn't need to be tested during QA.backport/<branch-name>
label to the PR and it will automatically open a backport PR once this one is merged