Skip to content

Conversation

jasonmp85
Copy link
Contributor

RDS only allows two values for log line prefix, so relying on the one suggested in Datadog documentation will be insufficient. This adds new parsing rules for the "fancier" one allowed by RDS (see here).

What does this PR do?

Adds two new patterns to the Grok parser in the first processor of the PostgreSQL pipeline.

Motivation

DBM is working to enhance our EXPLAIN plan gathering capabilities by subscribing to the log EVP track type and capturing the JSON plans emitted to log lines by the auto_explain extension.

In order to tie these records into DBM offerings, we need more information than is captured in the default log line.

Though the log_line_prefix setting is highly configurable, in the case of RDS (a case we will support), only two values are permitted. Of these, only one has the rich information we will need:

%m:%r:%u@%d:[%p]:%l:%e:%s:%v:%x:%c:%q%a

The new patterns match this (and capture every emitted value) as well as the variant of the above which contains a duration section (modeled after an existing rule).

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Copy link

⚠️ Recommendation: Add qa/skip-qa Label

This PR does not modify any files shipped with the agent.

To help streamline the release process, please consider adding the qa/skip-qa label if these changes do not require QA testing.

@jasonmp85 jasonmp85 added the qa/skip-qa Automatically skip this PR for the next QA label Sep 17, 2025
RDS only allows two values for log line prefix, so relying on the one
suggested in Datadog documentation will be insufficient. This adds new
parsing rules for the "fancier" one allowed by RDS (see [here]).

[here]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.overview.parameter-groups.html#USER_LogAccess.Concepts.PostgreSQL.log_rotation
@jasonmp85 jasonmp85 force-pushed the PIE-12-Augment-logs-pipelines-for-RDS-prefix branch from 125eaa9 to 4d6f3ad Compare September 17, 2025 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant