[Postgresql] [Bugfix] create new field query_id field to replace query field as dimension field (#8248)

The change helps to prevent the errors of storing data in ES when the query field in pg_stat_activity table exceeds 1024 byte length. The length of the values stored in the query field can extend beyond 1024 bytes, which is the limit of dimension field length after a limit override

Author: agithomas

packages/postgresql/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "1.17.1"
+  changes:
+    - description: Replaced `postgresql.activity.query` with `postgresql.activity.query_id` as TSDS dimension field to support query length greater than 1024.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/8248
 - version: 1.17.0
   changes:
     - description: Update the package format_version to 3.0.0.

packages/postgresql/data_stream/activity/elasticsearch/ingest_pipeline/default.yml

@@ -0,0 +1,8 @@
+---
+description: Pipeline for processing postgres activity datastream
+processors:
+    - fingerprint:
+        fields: ["postgresql.activity.query"]
+        target_field: postgresql.activity.query_id
+        ignore_failure: true
+        ignore_missing: true

packages/postgresql/data_stream/activity/fields/fields.yml

@@ -99,7 +99,10 @@
           * disabled: This state is reported if track_activities is disabled in this backend.
     - name: query
       type: keyword
-      dimension: true
-      # Reason to add as a dimension field: pg_stat_bgwriter and pg_stat_database returns same fields. The length of the query is very small and hence content hashing is not needed.
       description: |
         Text of this backend's most recent query. If state is active this field shows the currently executing query. In all other states, it shows the last query that was executed.
+    - name: query_id
+      type: keyword
+      dimension: true
+      description: |
+        Unique ID representing the query executed

packages/postgresql/docs/README.md

@@ -291,6 +291,7 @@ An example event for `activity` looks as following:
 | postgresql.activity.database.oid | OID of the database this backend is connected to. | long |
 | postgresql.activity.pid | Process ID of this backend. | long |
 | postgresql.activity.query | Text of this backend's most recent query. If state is active this field shows the currently executing query. In all other states, it shows the last query that was executed. | keyword |
+| postgresql.activity.query_id | Unique ID representing the query executed | keyword |
 | postgresql.activity.query_start | Time when the currently active query was started, or if state is not active, when the last query was started. | date |
 | postgresql.activity.state | Current overall state of this backend. Possible values are:    \* active: The backend is executing a query.   \* idle: The backend is waiting for a new client command.   \* idle in transaction: The backend is in a transaction, but is not     currently executing a query.   \* idle in transaction (aborted): This state is similar to idle in     transaction, except one of the statements in the transaction caused     an error.   \* fastpath function call: The backend is executing a fast-path function.   \* disabled: This state is reported if track_activities is disabled in this backend. | keyword |
 | postgresql.activity.state_change | Time when the state was last changed. | date |

packages/postgresql/manifest.yml

@@ -1,7 +1,7 @@
 format_version: "3.0.0"
 name: postgresql
 title: PostgreSQL
-version: "1.17.0"
+version: "1.17.1"
 description: Collect logs and metrics from PostgreSQL servers with Elastic Agent.
 type: integration
 categories: