Defer the Okta/notification fan-out to a post-response BackgroundTask#512
Open
somethingnew2-0 wants to merge 1 commit into
Open
Defer the Okta/notification fan-out to a post-response BackgroundTask#512somethingnew2-0 wants to merge 1 commit into
somethingnew2-0 wants to merge 1 commit into
Conversation
8cc5e37 to
889c813
Compare
83a8fd7 to
45d2594
Compare
…Task The multi-Okta-call operations (ModifyGroupUsers, ModifyRoleGroups, DeleteGroup, and their composers ApproveAccessRequest / CreateApp / ApproveRoleRequest) committed local DB state and then awaited a fan-out of Okta API calls plus notification dispatch before returning — that tail dominates request latency. It now runs in a FastAPI BackgroundTask after the response, so the response returns as soon as the local DB state commits. Operations hand their fan-out batch to a request-scoped collector via `defer_or_drain_fan_out`; the `defer_fan_out` dependency on the six mutating routers drains it post-response (and drains inline on the endpoint-error path, where FastAPI's error response won't carry the background task). Outside an HTTP request (CLI, syncer, MCP, direct `execute()`) there is no collector and the batch drains inline exactly as before — so a route missing the dependency degrades to correct-but-not-deferred, never incorrect. Notification payloads are expunged from the session before deferral: the drain runs after the router's `db.expire_all()` and session teardown, and the async notification hooks read ORM attributes synchronously, so without detaching they would raise MissingGreenlet/DetachedInstanceError on the dead session and send_notification would silently swallow the drop. Expunge only runs when the fan-out is actually being deferred, so the inline path keeps its objects session-attached for callers that reuse them. Fan-out failures (Okta stragglers, notification-hook errors) log at ERROR, not WARNING: Access surfaces the errors its plugins and Okta calls raise rather than swallowing them as warnings — matching the async plugin-hook dispatch. The failure still never propagates (the DB change is committed and the syncer reconciles Okta drift), and a component that expects noisy failures should catch them itself rather than rely on the drain downgrading them. Builds on the async plugin hooks (#511, now merged), which make notification dispatch awaitable — the prerequisite for moving it off the request path — and on the fan-out error-logging drain (#481, now merged). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
45d2594 to
363875c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Post-migration TODO item 10. The multi-Okta-call operations —
ModifyGroupUsers,ModifyRoleGroups,DeleteGroup, and their composersApproveAccessRequest/CreateApp/ApproveRoleRequest— committed the local DB state and then awaited a fan-out of Okta API calls plus notification dispatch before returning. That tail dominates the latency of those endpoints. It now runs in a FastAPIBackgroundTaskafter the response, so the HTTP response returns as soon as the local DB state commits.How it works
Operations hand their fan-out batch to a request-scoped collector via
defer_or_drain_fan_out. Adefer_fan_outdependency on the six mutating routers registers the drain as a post-responseBackgroundTask(and drains inline on the endpoint-error path, since the error response FastAPI builds won't carry the background task). Outside an HTTP request — CLI, syncer, MCP, directexecute()— there's no collector and the batch drains inline exactly as before, so a route that lacks the dependency degrades to correct-but-not-deferred, never incorrect.Things worth a reviewer's attention
db.expire_all()and (in prod) session teardown, and notification hooks read ORM attributes synchronously — so a deferred hook holding live ORM objects would raiseMissingGreenlet/DetachedInstanceErrorandsend_notificationwould silently drop the notification. Expunging detaches the payload with its loaded state intact, and only runs when the fan-out is actually being deferred (the inline path keeps objects attached for callers that reuse them). A dedicated test fails without it.asyncio.wait(notasyncio.gather) so a cancelled request/drain doesn't tear down the in-flight Okta calls it trails.ASGITransportthe deferred drain completes within the awaited request, so existing Okta call-count assertions keep working unchanged.Previously stacked on #481 and #511; both have merged, so this is now a single commit on
main.🤖 Generated with Claude Code