o365: fix error propagation within cel program #15445
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
efd6
added
Integration:o365
The logic for handling error propagation from failed subscriptions
requests was incorrect. The downstream logic was expecting to find the
error in the root of state, but the subscription request block was
leaving it in the work object.
Later error handling was incorrectly leaving any error state in the root
of state, when it should have been placed under events.
In the case that a subscription has failed, there may not be an entry in
the last_for list for the subscription, resulting in a look-up failure
during assessment for completion of the hour-window steps.
Finally, the postamble logic was brittle to failed requests, assuming
that the todo_type and todo_content list would be present, which is not
necessarily the case when a subscription has failed.
This has been manually tested against the mock committed in the deploy
directory (with removal of authentication for convenience) with the
following outcomes:
only one, invalid, subscription:
"""
-- data.json --
{
"url": "http://localhost:9001",
"want_more": false,
"base": {
"tenant_id": "test-cel-tenant-id",
"list_contents_start_time": "12h",
"batch_interval": "1h",
"maximum_age": "167h55m",
"content_types": "Audit.TypeRequiringAdditionalPermissions"
}
}
-- out.json --
{
"base": {
"batch_interval": "1h",
"content_types": "Audit.TypeRequiringAdditionalPermissions",
"list_contents_start_time": "12h",
"maximum_age": "167h55m",
"tenant_id": "test-cel-tenant-id"
},
"events": {
"error": {
"code": "401",
"id": "401 Unauthorized",
"message": "POST /activity/feed/subscriptions/start?contentType=Audit.TypeRequiringAdditionalPermissions: {\"error\":{\"code\":\"AF10001\",\"message\":\"The permission set (...) sent in the request does not include the expected permission.\"}}"
}
},
"url": "http://localhost:9001",
"want_more": false,
"work": {
"curr_type": "Audit.TypeRequiringAdditionalPermissions",
"todo_type": []
}
}
"""
start with an invalid subscription:
"""
-- data.json --
{
"url": "http://localhost:9001",
"want_more": false,
"base": {
"tenant_id": "test-cel-tenant-id",
"list_contents_start_time": "12h",
"batch_interval": "1h",
"maximum_age": "167h55m",
"content_types": "Audit.TypeRequiringAdditionalPermissions, Audit.SharePoint"
}
}
-- out.json --
{
"base": {
"batch_interval": "1h",
"content_types": "Audit.TypeRequiringAdditionalPermissions, Audit.SharePoint",
"list_contents_start_time": "12h",
"maximum_age": "167h55m",
"tenant_id": "test-cel-tenant-id"
},
"events": {
"error": {
"code": "401",
"id": "401 Unauthorized",
"message": "POST /activity/feed/subscriptions/start?contentType=Audit.TypeRequiringAdditionalPermissions: {\"error\":{\"code\":\"AF10001\",\"message\":\"The permission set (...) sent in the request does not include the expected permission.\"}}"
}
},
"url": "http://localhost:9001",
"want_more": true,
"work": {
"curr_type": "Audit.TypeRequiringAdditionalPermissions",
"todo_type": [
"Audit.SharePoint"
]
}
}
{
"base": {
"batch_interval": "1h",
"content_types": "Audit.TypeRequiringAdditionalPermissions, Audit.SharePoint",
"list_contents_start_time": "12h",
"maximum_age": "167h55m",
"tenant_id": "test-cel-tenant-id"
},
"cursor": {
"last_for": {
"audit.sharepoint": "2025-09-23T13:07:36.915764483Z"
}
},
"events": [
{
… more …
"""
end with an invalid subscription:
"""
-- data.json --
{
"url": "http://localhost:9001",
"want_more": false,
"base": {
"tenant_id": "test-cel-tenant-id",
"list_contents_start_time": "12h",
"batch_interval": "1h",
"maximum_age": "167h55m",
"content_types": "Audit.SharePoint, Audit.TypeRequiringAdditionalPermissions"
}
}
-- out.json --
… more before …
"enabled": true,
"type": "Audit.SharePoint"
},
"todo_content": [],
"todo_type": [
"Audit.TypeRequiringAdditionalPermissions"
]
}
}
{
"base": {
"batch_interval": "1h",
"content_types": "Audit.SharePoint, Audit.TypeRequiringAdditionalPermissions",
"list_contents_start_time": "12h",
"maximum_age": "167h55m",
"tenant_id": "test-cel-tenant-id"
},
"cursor": {
"last_for": {
"audit.sharepoint": "2025-09-24T00:10:30.808672557Z"
}
},
"events": {
"error": {
"code": "401",
"id": "401 Unauthorized",
"message": "POST /activity/feed/subscriptions/start?contentType=Audit.TypeRequiringAdditionalPermissions: {\"error\":{\"code\":\"AF10001\",\"message\":\"The permission set (...) sent in the request does not include the expected permission.\"}}"
}
},
"url": "http://localhost:9001",
"want_more": false,
"work": {
"curr_type": "Audit.TypeRequiringAdditionalPermissions",
"todo_type": []
}
}
"""
efd6
force-pushed
the
s6431-o365-error_propagation
branch
from
8d726de to
0e26446
Compare
🚀 Benchmarks reportTo see the full report comment with |
💚 Build Succeeded
cc @efd6 |
|
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
kcreddy
approved these changes
Sep 24, 2025
Contributor
kcreddy
left a comment
kcreddy
left a comment
There was a problem hiding this comment.
LGTM. Minor clarification
Comment on lines
+469
to
+473
| state.work.curr_type.to_lower().as(curr_type, curr_type in last_for ? | ||
| timestamp(last_for[curr_type]) < now - duration("1h") | ||
| : | ||
| false | ||
| ) |
Contributor
There was a problem hiding this comment.
Suggested change
| state.work.curr_type.to_lower().as(curr_type, curr_type in last_for ? | |
| timestamp(last_for[curr_type]) < now - duration("1h") | |
| : | |
| false | |
| ) | |
| state.work.curr_type.to_lower().as(curr_type, curr_type in last_for && timestamp(last_for[curr_type]) < now - duration("1h")) |
Can we do this?
Contributor
Author
There was a problem hiding this comment.
I prefer the clarity that is in the current version (not really significant here, but the current version is also more efficient since of !curr_type in last_for then the timestamp parsing is not done, while it is in the proposed version).
|
Package o365 - 2.29.2 containing this change is available at https://epr.elastic.co/package/o365/2.29.2/ |
5 tasks
tehbooom
pushed a commit
to tehbooom/integrations
that referenced
this pull request
Nov 19, 2025
The logic for handling error propagation from failed subscriptions
requests was incorrect. The downstream logic was expecting to find the
error in the root of state, but the subscription request block was
leaving it in the work object.
Later error handling was incorrectly leaving any error state in the root
of state, when it should have been placed under events.
In the case that a subscription has failed, there may not be an entry in
the last_for list for the subscription, resulting in a look-up failure
during assessment for completion of the hour-window steps.
Finally, the postamble logic was brittle to failed requests, assuming
that the todo_type and todo_content list would be present, which is not
necessarily the case when a subscription has failed.
This has been manually tested against the mock committed in the deploy
directory (with removal of authentication for convenience) with the
following outcomes:
only one, invalid, subscription:
"""
-- data.json --
{
"url": "http://localhost:9001",
"want_more": false,
"base": {
"tenant_id": "test-cel-tenant-id",
"list_contents_start_time": "12h",
"batch_interval": "1h",
"maximum_age": "167h55m",
"content_types": "Audit.TypeRequiringAdditionalPermissions"
}
}
-- out.json --
{
"base": {
"batch_interval": "1h",
"content_types": "Audit.TypeRequiringAdditionalPermissions",
"list_contents_start_time": "12h",
"maximum_age": "167h55m",
"tenant_id": "test-cel-tenant-id"
},
"events": {
"error": {
"code": "401",
"id": "401 Unauthorized",
"message": "POST /activity/feed/subscriptions/start?contentType=Audit.TypeRequiringAdditionalPermissions: {\"error\":{\"code\":\"AF10001\",\"message\":\"The permission set (...) sent in the request does not include the expected permission.\"}}"
}
},
"url": "http://localhost:9001",
"want_more": false,
"work": {
"curr_type": "Audit.TypeRequiringAdditionalPermissions",
"todo_type": []
}
}
"""
start with an invalid subscription:
"""
-- data.json --
{
"url": "http://localhost:9001",
"want_more": false,
"base": {
"tenant_id": "test-cel-tenant-id",
"list_contents_start_time": "12h",
"batch_interval": "1h",
"maximum_age": "167h55m",
"content_types": "Audit.TypeRequiringAdditionalPermissions, Audit.SharePoint"
}
}
-- out.json --
{
"base": {
"batch_interval": "1h",
"content_types": "Audit.TypeRequiringAdditionalPermissions, Audit.SharePoint",
"list_contents_start_time": "12h",
"maximum_age": "167h55m",
"tenant_id": "test-cel-tenant-id"
},
"events": {
"error": {
"code": "401",
"id": "401 Unauthorized",
"message": "POST /activity/feed/subscriptions/start?contentType=Audit.TypeRequiringAdditionalPermissions: {\"error\":{\"code\":\"AF10001\",\"message\":\"The permission set (...) sent in the request does not include the expected permission.\"}}"
}
},
"url": "http://localhost:9001",
"want_more": true,
"work": {
"curr_type": "Audit.TypeRequiringAdditionalPermissions",
"todo_type": [
"Audit.SharePoint"
]
}
}
{
"base": {
"batch_interval": "1h",
"content_types": "Audit.TypeRequiringAdditionalPermissions, Audit.SharePoint",
"list_contents_start_time": "12h",
"maximum_age": "167h55m",
"tenant_id": "test-cel-tenant-id"
},
"cursor": {
"last_for": {
"audit.sharepoint": "2025-09-23T13:07:36.915764483Z"
}
},
"events": [
{
… more …
"""
end with an invalid subscription:
"""
-- data.json --
{
"url": "http://localhost:9001",
"want_more": false,
"base": {
"tenant_id": "test-cel-tenant-id",
"list_contents_start_time": "12h",
"batch_interval": "1h",
"maximum_age": "167h55m",
"content_types": "Audit.SharePoint, Audit.TypeRequiringAdditionalPermissions"
}
}
-- out.json --
… more before …
"enabled": true,
"type": "Audit.SharePoint"
},
"todo_content": [],
"todo_type": [
"Audit.TypeRequiringAdditionalPermissions"
]
}
}
{
"base": {
"batch_interval": "1h",
"content_types": "Audit.SharePoint, Audit.TypeRequiringAdditionalPermissions",
"list_contents_start_time": "12h",
"maximum_age": "167h55m",
"tenant_id": "test-cel-tenant-id"
},
"cursor": {
"last_for": {
"audit.sharepoint": "2025-09-24T00:10:30.808672557Z"
}
},
"events": {
"error": {
"code": "401",
"id": "401 Unauthorized",
"message": "POST /activity/feed/subscriptions/start?contentType=Audit.TypeRequiringAdditionalPermissions: {\"error\":{\"code\":\"AF10001\",\"message\":\"The permission set (...) sent in the request does not include the expected permission.\"}}"
}
},
"url": "http://localhost:9001",
"want_more": false,
"work": {
"curr_type": "Audit.TypeRequiringAdditionalPermissions",
"todo_type": []
}
}
"""
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed commit message
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Related issues
Screenshots