Implement graceful shutdown with CancellationToken #159
Conversation
Reviewer's GuideIntroduces a coordinated graceful shutdown mechanism using Tokio’s CancellationToken and TaskTracker, refactors worker tasks and their lifecycle management, adjusts tests to the new pattern, and updates dependencies and documentation accordingly. Sequence diagram for graceful shutdown using CancellationToken and TaskTrackersequenceDiagram
participant MainServer
participant TaskTracker
participant CancellationToken
participant WorkerTask
MainServer->>TaskTracker: spawn(worker_task(..., token, tracker))
loop For each worker
TaskTracker->>WorkerTask: Start worker_task
end
MainServer->>CancellationToken: Wait for shutdown signal
CancellationToken-->>WorkerTask: Signal cancellation
WorkerTask-->>TaskTracker: Complete and notify
MainServer->>TaskTracker: tracker.wait()
TaskTracker-->>MainServer: All tasks complete
MainServer->>CancellationToken: cancel()
MainServer->>TaskTracker: close()
MainServer->>TaskTracker: tracker.wait().await
TaskTracker-->>MainServer: Confirm shutdown complete
Class diagram for updated server and worker task shutdown logicclassDiagram
class Server {
+listener
+factory
+on_preamble_success
+on_preamble_failure
+workers
+run(shutdown)
}
class TaskTracker {
+spawn(task)
+wait()
+close()
}
class CancellationToken {
+cancel()
+cancelled()
}
class WorkerTask {
+worker_task(listener, factory, on_success, on_failure, shutdown, tracker)
}
Server --> TaskTracker : uses
Server --> CancellationToken : uses
TaskTracker --> WorkerTask : spawns
WorkerTask --> CancellationToken : checks
WorkerTask --> TaskTracker : notifies
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Warning Rate limit exceeded@leynos has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 9 minutes and 54 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (2)
Summary by CodeRabbit
WalkthroughThe shutdown and task management mechanism was refactored from using a Tokio broadcast channel and manual join handle tracking to utilising Changes
Sequence Diagram(s)sequenceDiagram
participant Main as Main Server Task
participant Tracker as TaskTracker
participant Token as CancellationToken
participant Worker as Worker Task
Main->>Token: Create CancellationToken
Main->>Tracker: Create TaskTracker
loop For each worker
Main->>Tracker: Spawn worker_task with Token, Tracker
Tracker->>Worker: Run worker_task
end
Main->>Token: Await shutdown signal
Token-->>Worker: Signal cancellation
Worker->>Tracker: Complete task on shutdown
Main->>Tracker: Wait for all tasks to finish
Possibly related PRs
Poem
✨ Finishing Touches
🧪 Generate Unit Tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![codescene-delta-analysis[bot]](https://avatars.githubusercontent.com/in/53921?s=60&v=4){kind=small}
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @leynos - I've reviewed your changes - here's some feedback:
- In
run, remove the redundantshutdown_token.cancel()aftertracker.close()and consider moving the single cancellation call to beforetracker.close()for clearer shutdown ordering. - Rename ambiguous local variables like
tto more descriptive names to improve readability.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `run`, remove the redundant `shutdown_token.cancel()` after `tracker.close()` and consider moving the single cancellation call to before `tracker.close()` for clearer shutdown ordering.
- Rename ambiguous local variables like `t` to more descriptive names to improve readability.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (4)
Cargo.toml(1 hunks)docs/asynchronous-outbound-messaging-roadmap.md(1 hunks)src/server.rs(6 hunks)tests/connection_actor.rs(2 hunks)
🧰 Additional context used
📓 Path-based instructions (10)
`docs/**/*.md`: Documentation must use en-GB-oxendict spelling and grammar (with the exception of "license" which is to be left unchanged for community consistency).
docs/**/*.md: Documentation must use en-GB-oxendict spelling and grammar (with the exception of "license" which is to be left unchanged for community consistency).
📄 Source: CodeRabbit Inference Engine (AGENTS.md)
List of files the instruction was applied to:
docs/asynchronous-outbound-messaging-roadmap.md
`**/*.md`: Validate Markdown files using `markdownlint *.md **/*.md`. Run `mdfor...
**/*.md: Validate Markdown files usingmarkdownlint *.md **/*.md.
Runmdformat-allafter any documentation changes to format all Markdown files and fix table markup.
Validate Markdown Mermaid diagrams using thenixieCLI. The tool is already installed; runnixie *.md **/*.mddirectly instead of usingnpx.
Markdown paragraphs and bullet points must be wrapped at 80 columns.
Code blocks must be wrapped at 120 columns.
Tables and headings must not be wrapped.
📄 Source: CodeRabbit Inference Engine (AGENTS.md)
List of files the instruction was applied to:
docs/asynchronous-outbound-messaging-roadmap.md
`docs/**/*.md`: Provide user guides and examples demonstrating server-initiated messaging.
docs/**/*.md: Provide user guides and examples demonstrating server-initiated messaging.
📄 Source: CodeRabbit Inference Engine (docs/asynchronous-outbound-messaging-roadmap.md)
List of files the instruction was applied to:
docs/asynchronous-outbound-messaging-roadmap.md
`docs/**/*.md`: Conventions for writing project documentation should follow the rules outlined in the documentation style guide.
docs/**/*.md: Conventions for writing project documentation should follow the rules outlined in the documentation style guide.
📄 Source: CodeRabbit Inference Engine (docs/contents.md)
List of files the instruction was applied to:
docs/asynchronous-outbound-messaging-roadmap.md
`docs/**/*.md`: Use British English based on the Oxford English Dictionary (en-o...
docs/**/*.md: Use British English based on the Oxford English Dictionary (en-oxendict) for documentation.
The word "outwith" is acceptable in documentation.
Keep US spelling when used in an API, for examplecolor.
Use the Oxford comma in documentation.
Company names are treated as collective nouns (e.g., "Lille Industries are expanding").
Write headings in sentence case in documentation.
Use Markdown headings (#,##,###, etc.) in order without skipping levels.
Follow markdownlint recommendations for Markdown files.
Provide code blocks and lists using standard Markdown syntax.
Always use fenced code blocks with a language identifier; useplaintextfor non-code text.
Use-as the first level bullet and renumber lists when items change.
Prefer inline links using[text](url)or angle brackets around the URL in Markdown.
Expand any uncommon acronym on first use, for example, Continuous Integration (CI).
Wrap paragraphs at 80 columns in documentation.
Wrap code at 120 columns in documentation.
Do not wrap tables in documentation.
Use footnotes referenced with[^label]in documentation.
Include Mermaid diagrams in documentation where it adds clarity.
When embedding figures in documentation, useand provide concise alt text describing the content.
Add a short description before each Mermaid diagram in documentation so screen readers can understand it.
📄 Source: CodeRabbit Inference Engine (docs/documentation-style-guide.md)
List of files the instruction was applied to:
docs/asynchronous-outbound-messaging-roadmap.md
`docs/**/*.md`: Write the official documentation for the new features. Create se...
docs/**/*.md: Write the official documentation for the new features. Create separate guides for "Duplex Messaging & Pushes", "Streaming Responses", and "Message Fragmentation". Each guide must include runnable examples and explain the relevant concepts and APIs.
📄 Source: CodeRabbit Inference Engine (docs/wireframe-1-0-detailed-development-roadmap.md)
List of files the instruction was applied to:
docs/asynchronous-outbound-messaging-roadmap.md
`**/*.md`: * Avoid 2nd person or 1st person pronouns ("I", "you", "we") * Use en...
**/*.md: * Avoid 2nd person or 1st person pronouns ("I", "you", "we")
- Use en-oxendic spelling and grammar.
- Paragraphs and bullets must be wrapped to 80 columns, except where a long URL would prevent this (in which case, silence MD013 for that line)
- Code blocks should be wrapped to 120 columns.
- Headings must not be wrapped.
⚙️ Source: CodeRabbit Configuration File
List of files the instruction was applied to:
docs/asynchronous-outbound-messaging-roadmap.md
`Cargo.toml`: Use explicit version ranges in `Cargo.toml` and keep dependencies up-to-date.
Cargo.toml: Use explicit version ranges inCargo.tomland keep dependencies up-to-date.
📄 Source: CodeRabbit Inference Engine (AGENTS.md)
List of files the instruction was applied to:
Cargo.toml
`**/*.rs`: Comment why, not what. Explain assumptions, edge cases, trade-offs, o...
**/*.rs: Comment why, not what. Explain assumptions, edge cases, trade-offs, or complexity. Don't echo the obvious.
Comments must use en-GB-oxendict spelling and grammar.
Function documentation must include clear examples.
Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Place function attributes after doc comments.
Do not usereturnin single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Preferexpectoverallow.
Prefer.expect()over.unwrap().
Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider usingArcto reduce the amount of data returned.
Write unit and behavioural tests for new functionality. Run both before and after making any change.
Prefer immutable data and avoid unnecessarymutbindings.
Handle errors with theResulttype instead of panicking where feasible.
Avoidunsafecode unless absolutely necessary and document any usage clearly.
📄 Source: CodeRabbit Inference Engine (AGENTS.md)
List of files the instruction was applied to:
tests/connection_actor.rssrc/server.rs
`**/*.rs`: * Seek to keep the cyclomatic complexity of functions no more than 12...
**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.
Adhere to single responsibility and CQRS
Place function attributes after doc comments.
Do not use
returnin single-line functions.Move conditionals with >2 branches into a predicate function.
Avoid
unsafeunless absolutely necessary.Every module must begin with a
//!doc comment that explains the module's purpose and utility.Comments must use en-GB-oxendict spelling and grammar.
Lints must not be silenced except as a last resort.
#[allow]is forbidden.- Only narrowly scoped
#[expect(lint, reason = "...")]is allowed.- No lint groups, no blanket or file-wide suppression.
- Include
FIXME:with link if a fix is expected.Use
rstestfixtures for shared setup and to avoid repetition between tests.Replace duplicated tests with
#[rstest(...)]parameterised cases.Prefer
mockallfor mocks/stubs.Prefer
.expect()over.unwrap()Ensure that any API or behavioural changes are reflected in the documentation in
docs/Ensure that any completed roadmap steps are recorded in the appropriate roadmap in
docs/
⚙️ Source: CodeRabbit Configuration File
List of files the instruction was applied to:
tests/connection_actor.rssrc/server.rs
🧠 Learnings (1)
tests/connection_actor.rs (1)
Learnt from: CR
PR: leynos/wireframe#0
File: docs/wireframe-1-0-detailed-development-roadmap.md:0-0
Timestamp: 2025-06-29T18:04:58.493Z
Learning: Graceful shutdown should be implemented using tokio_util::sync::CancellationToken for signaling and tokio_util::task::TaskTracker to ensure all connection actors terminate cleanly.
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build-test
🔇 Additional comments (7)
Cargo.toml (1)
10-10: Appropriate feature flag addition for graceful shutdown.The addition of the
"rt"feature fortokio-utilcorrectly enables the required runtime utilities (CancellationTokenandTaskTracker) for the new graceful shutdown implementation.docs/asynchronous-outbound-messaging-roadmap.md (1)
41-41: Documentation accurately reflects implementation status.The roadmap correctly marks the graceful shutdown implementation as complete, which aligns with the code changes in
src/server.rsand the new test coverage intests/connection_actor.rs.tests/connection_actor.rs (2)
9-9: Import correctly updated for new test requirements.The addition of
TaskTrackerto the import statement supports the new graceful shutdown test.
238-264: Well-structured test for graceful shutdown behaviour.The test effectively validates that:
- Multiple
ConnectionActorinstances can be tracked usingTaskTracker- Cancellation via
CancellationTokentriggers graceful shutdown- All tracked tasks complete within the expected timeout
The 500ms timeout is reasonable for verifying task completion.
src/server.rs (3)
33-33: Import statement correctly adds required utilities.The addition of
CancellationTokenandTaskTrackerfromtokio_utilsupports the refactored graceful shutdown implementation.
352-372: Worker task correctly implements graceful shutdown pattern.The refactored implementation properly:
- Uses biased
select!to prioritise shutdown signals- Tracks spawned tasks via
TaskTrackerfor coordinated shutdown- Maintains the existing error handling and backoff logic
809-826: Test correctly validates the new shutdown mechanism.The updated test properly exercises the
CancellationTokenandTaskTrackerbased shutdown, ensuring the worker task responds to cancellation signals within the expected timeout.
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
leynos
deleted the
codex/implement-graceful-shutdown-with-cancellationtoken
branch
1 participant
Summary
rtfeatureTaskTrackerandCancellationTokenfor server shutdownTesting
make lintmake testhttps://chatgpt.com/codex/tasks/task_e_68670889d3b08322b5a7c930edc42286
Summary by Sourcery
Enable graceful shutdown by replacing the broadcast channel mechanism with CancellationToken and TaskTracker, refactoring task spawning and shutdown logic in the server and worker tasks, updating tests and docs accordingly, and enabling the required tokio-util feature in Cargo.toml
New Features:
Enhancements:
Build:
Documentation:
Tests: