Skip to content

Conversation

kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Jun 23, 2025

Proposed commit message

error.message is undefined in some Threat Intel integrations.
If the field is undefined, the default keyword mapping is applied.
This causes issues when searching on data and also impacts 
custom rules. 

Update error.message field mapping as per ECS inside 
Threat Intel destination indices. The change impacts following 
integrations:
- ti_eclectiqiq
- ti_opencti
- ti_threatconnect

Since the mapping needs to be applied to destination indices, 
the index version suffix is also incremented along with upgrading 
transform's fleet version, only when the transform is updated.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

@kcreddy kcreddy self-assigned this Jun 23, 2025
@kcreddy kcreddy added Integration:ti_opencti OpenCTI Integration:ti_eclecticiq EclecticIQ (Partner supported) Integration:ti_threatconnect ThreatConnect (Partner supported) bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Jun 23, 2025
@kcreddy kcreddy changed the title ti_*: Update error.message field mapping as per ECS inside Threat Intel destination indices. {ti_eclectiqiq,ti_opencti,ti_threatconnect}: Update error.message field mapping as per ECS inside Threat Intel destination indices. Jun 23, 2025
@kcreddy kcreddy marked this pull request as ready for review June 23, 2025 07:57
@kcreddy kcreddy requested a review from a team as a code owner June 23, 2025 07:57
@elasticmachine
Copy link

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Jun 23, 2025

🚀 Benchmarks report

Package ti_threatconnect 👍(0) 💚(0) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
indicator 902.53 497.27 -405.26 (-44.9%) 💔

To see the full report comment with /test benchmark fullreport

# newer versions go on top
- version: "1.4.1"
changes:
- description: Update error.message field mapping as per ECS inside transform destination indices.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- description: Update error.message field mapping as per ECS inside transform destination indices.
- description: Update `error.message` field mapping as per ECS inside transform destination indices.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e20b1d1

# newer versions go on top
- version: "2.6.1"
changes:
- description: Update error.message field mapping as per ECS inside transform destination indices.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- description: Update error.message field mapping as per ECS inside transform destination indices.
- description: Update `error.message` field mapping as per ECS inside transform destination indices.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e20b1d1

# WARNING: this version number needs to be kept up to date in the transform!
- version: "1.10.1"
changes:
- description: Update error.message field mapping as per ECS inside transform destination indices.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- description: Update error.message field mapping as per ECS inside transform destination indices.
- description: Update `error.message` field mapping as per ECS inside transform destination indices.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e20b1d1

dest:
index: "logs-ti_threatconnect_latest.dest_indicator-7"
index: "logs-ti_threatconnect_latest.dest_indicator-8"
pipeline: "1.10.0-tactics_compatibility"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pipeline: "1.10.0-tactics_compatibility"
pipeline: "1.10.1-tactics_compatibility"

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! System tests also failing due to this.
Addressed in e20b1d1

@elasticmachine
Copy link

💚 Build Succeeded

History

cc @kcreddy

Copy link

@kcreddy kcreddy merged commit 85bfc89 into elastic:main Jun 24, 2025
7 checks passed
@elastic-vault-github-plugin-prod

Package ti_eclecticiq - 1.4.1 containing this change is available at https://epr.elastic.co/package/ti_eclecticiq/1.4.1/

@elastic-vault-github-plugin-prod

Package ti_opencti - 2.6.1 containing this change is available at https://epr.elastic.co/package/ti_opencti/2.6.1/

@elastic-vault-github-plugin-prod

Package ti_threatconnect - 1.10.1 containing this change is available at https://epr.elastic.co/package/ti_threatconnect/1.10.1/

shmsr pushed a commit to shmsr/integrations that referenced this pull request Jun 30, 2025
…ld mapping as per ECS inside Threat Intel destination indices. (elastic#14290)

error.message is undefined in some Threat Intel integrations.
If the field is undefined, the default keyword mapping is applied.
This causes issues when searching on data and also impacts 
custom rules. 

Update error.message field mapping as per ECS inside 
Threat Intel destination indices. The change impacts following 
integrations:
- ti_eclectiqiq
- ti_opencti
- ti_threatconnect

Since the mapping needs to be applied to destination indices, 
the index version suffix is also incremented along with upgrading 
transform's fleet version, only then the transform is updated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:ti_eclecticiq EclecticIQ (Partner supported) Integration:ti_opencti OpenCTI Integration:ti_threatconnect ThreatConnect (Partner supported) Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants