Make enrich project-aware #124099
Make enrich project-aware #124099
Conversation
Makes the execution and use of enrich policies project-aware. Note: this does not make the enrich cache project-aware. That is to be handled in a follow-up PR.
nielsbauman
added
the
:Data Management/Ingest Node
elasticsearchmachine
added
Team:Data Management
|
Pinging @elastic/es-data-management (Team:Data Management) |
ywangd
left a comment
ywangd
left a comment
There was a problem hiding this comment.
LGTM
I don't fully understand how ingest processor works. But the changes related to passing project-id makes sense to me. I think it might be worthwhile to create a placeholder ticket to remind relevant teams owning the processors to review whether the new project-id parameter should be leveraged. Probably also worth a separate ticket of the similar nature for logstash since it is a separate product.
| * @param searchResponseFetcher The function used to compute the value to be put in the cache, if there is no value in the cache already | ||
| * @param listener A listener to be notified of the value in the cache | ||
| */ | ||
| @FixForMultiProject(description = "The enrich cache will currently leak data between projects. We need to either disable or fix it.") |
There was a problem hiding this comment.
Seems pretty serious if it happens, I suggest we create a JIRA issue to track it for better visibility.
There was a problem hiding this comment.
I already created ES-10936 and put it on the agenda for today's weekly Data Management team meeting :)
|
@ywangd, most processors work within the scope of a single document and thus won't need a project ID. There are some exceptions, like the enrich, pipeline, and more processors. I think that in any of those cases, they'll at some point call a method that uses However, some parts are perhaps more difficult to identify (e.g. the enrich cache I mentioned), so I'll add a ticket to do a final check on all the processors that they're not leaking information between projects. I opened ES-11061. |
elasticsearchmachine
added
the
serverless-linked
Makes the execution and use of enrich policies project-aware. Note: this does not make the enrich cache project-aware. That is to be handled in a follow-up PR.
Makes the execution and use of enrich policies project-aware. Note: this does not make the enrich cache project-aware. That is to be handled in a follow-up PR.
3 participants
Makes the execution and use of enrich policies project-aware.
Note: this does not make the enrich cache project-aware. That is to be handled in a follow-up PR.