[proofpoint_essentials] Initial Release of Proofpoint Essentials #16073

brijesh-elastic · 2025-11-21T07:10:16Z

Proposed commit message

proofpoint_essentials: Initial Release of Proofpoint Essentials

The Proofpoint Essentials integration with Elastic enables the
collection of threats for monitoring and analysis. This valuable data
can be leveraged within Elastic to analyze potential threat signals,
including spam, phishing, business email compromise (BEC), imposter
emails, ransomware, and malware.

The integration collects threats using the (Essentials Threat API)
[1], after which they are rerouted to different data streams based on
their types.
- clicks_blocked
- clicks_permitted
- message_blocked
- message_delivered

Sanitized test case inputs were obtained from live Proofpoint Essentials
instance using the Essentials Threat API.

[1] https://help.proofpoint.com/Essentials/Additional_Resources/API_Documentation/Essentials_Threat_API

Note

The integration uses routing rules to route events to a different data streams from a central threat data stream. System test were not feasible due to this. The elastic-package issue is tracked at Allow data stream overrides in system tests for specific integration packages elastic-package#1917

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Clone integrations repo.
Install elastic package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/proofpoint_essentials directory.
Run the following command to run tests.

elastic-package test

Related issues

Screenshots

elasticmachine · 2025-11-21T07:10:22Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

elastic-vault-github-plugin-prod · 2025-11-21T07:29:36Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elasticmachine · 2025-11-21T07:29:39Z

💚 Build Succeeded

Buildkite Build
Commit: 1f52a15

cc @brijesh-elastic

efd6 · 2025-11-24T02:38:53Z

packages/proofpoint_essentials/data_stream/clicks_permitted/fields/fields.yml

+          description: The rewrite status of the message.
+        - name: customer_eid
+          type: keyword
+          description: The customers entity ID.


Suggested change

description: The customers entity ID.

description: The customer's entity ID.

(throughout)

efd6 · 2025-11-24T02:40:20Z

packages/proofpoint_essentials/data_stream/clicks_blocked/fields/fields.yml

+          fields:
+            - name: content_type
+              type: keyword
+              description: The true, detected Content-Type of the messagePart. This may differ from the oContentType value.


Suggested change

description: The true, detected Content-Type of the messagePart. This may differ from the oContentType value.

description: The true, detected Content-Type of the messagePart. This may differ from the `o_content_type` value.

(throughout)

I think probably that generally the camelCase names in the descriptions should be converted to snake_case to avoid confusion since all the fields have been converted to snake_case.

efd6 · 2025-11-24T02:53:54Z

..._essentials/kibana/dashboard/proofpoint_essentials-6ce9c1fd-381f-4757-9747-b8fdabdde163.json

Currently the filters are acting at the dashboard level. Can they be moved to the visualisation level so that users are not able to accidentally remove them?

efd6 · 2025-11-24T03:07:25Z

packages/proofpoint_essentials/data_stream/threat/elasticsearch/ingest_pipeline/default.yml

+          if (ctx.proofpoint_essentials.threat.threats_info_map instanceof List) {
+            for (item in ctx.proofpoint_essentials.threat.threats_info_map) {
+              if (item?.threat_time instanceof String && Instant.parse(item.threat_time).isAfter(ts)) {
+                ctx['@timestamp'] = item.threat_time;


Suggested change

ctx['@timestamp'] = item.threat_time;

ts = item.threat_time;

efd6 · 2025-11-24T03:07:44Z

packages/proofpoint_essentials/data_stream/threat/elasticsearch/ingest_pipeline/default.yml

+        }
+        if (ctx.proofpoint_essentials?.threat?.event_type == 'clicks_blocked' || ctx.proofpoint_essentials?.threat?.event_type == 'clicks_permitted') {
+          if (ctx.proofpoint_essentials.threat.threat_time instanceof String && Instant.parse(ctx.proofpoint_essentials.threat.threat_time).isAfter(ts)) {
+            ctx['@timestamp'] = ctx.proofpoint_essentials.threat.threat_time;


Suggested change

ctx['@timestamp'] = ctx.proofpoint_essentials.threat.threat_time;

ts = ctx.proofpoint_essentials.threat.threat_time;

efd6 · 2025-11-24T03:08:05Z

packages/proofpoint_essentials/data_stream/threat/elasticsearch/ingest_pipeline/default.yml

+          if (ctx.proofpoint_essentials.threat.threat_time instanceof String && Instant.parse(ctx.proofpoint_essentials.threat.threat_time).isAfter(ts)) {
+            ctx['@timestamp'] = ctx.proofpoint_essentials.threat.threat_time;
+          }
+        }


Suggested change

}

}

ctx['@timestamp'] = ts;

Initial Release

71fa0a1

brijesh-elastic self-assigned this Nov 21, 2025

brijesh-elastic requested a review from a team as a code owner November 21, 2025 07:10

Update changelog entry

1f52a15

brijesh-elastic requested a review from a team November 21, 2025 07:14

efd6 reviewed Nov 24, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[proofpoint_essentials] Initial Release of Proofpoint Essentials #16073

[proofpoint_essentials] Initial Release of Proofpoint Essentials #16073

brijesh-elastic commented Nov 21, 2025 •

edited by efd6

Loading

Uh oh!

elasticmachine commented Nov 21, 2025

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 21, 2025

Uh oh!

elasticmachine commented Nov 21, 2025

Uh oh!

efd6 Nov 24, 2025

Uh oh!

efd6 Nov 24, 2025

Uh oh!

efd6 Nov 24, 2025

Uh oh!

efd6 Nov 24, 2025

Uh oh!

efd6 Nov 24, 2025

Uh oh!

efd6 Nov 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	description: The customers entity ID.
	description: The customer's entity ID.

	description: The true, detected Content-Type of the messagePart. This may differ from the oContentType value.
	description: The true, detected Content-Type of the messagePart. This may differ from the `o_content_type` value.

	ctx['@timestamp'] = ctx.proofpoint_essentials.threat.threat_time;
	ts = ctx.proofpoint_essentials.threat.threat_time;

[proofpoint_essentials] Initial Release of Proofpoint Essentials #16073

Are you sure you want to change the base?

[proofpoint_essentials] Initial Release of Proofpoint Essentials #16073

Conversation

brijesh-elastic commented Nov 21, 2025 • edited by efd6 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

elasticmachine commented Nov 21, 2025

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 21, 2025

🚀 Benchmarks report

Uh oh!

elasticmachine commented Nov 21, 2025

💚 Build Succeeded

Uh oh!

efd6 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

efd6 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

efd6 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

efd6 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

efd6 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

efd6 Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

brijesh-elastic commented Nov 21, 2025 •

edited by efd6

Loading