Feat: implement a signature processor for DMQ #2477

jpraynaud · 2025-05-14T08:17:14Z

Content

This PR includes the implementation of a signature processor for the DMQ in the aggregator:

Added a SignatureProcessor trait for continuously consuming signatures and registering them with the certifier service
Added a SequentialSignatureProcessor implementation of the SignatureProcessor trait
Added a SignatureConsumer trait for receiving the signatures
Added a SignatureConsumerNoop implementation of the SignatureConsumer trait (until the DMQ consumer is available)
Added constructioon of the SignatureProcessor in the dependency injection
Launched the SignatureProcessor in a separate thread of the serve command

Pre-submit checklist

Branch
- Tests are provided (if possible)
- Crates versions are updated (if relevant)
- CHANGELOG file is updated (if relevant)
- Commit sequence broadly makes sense
- Key commits have useful messages
PR
- All check jobs of the CI have succeeded
- Self-reviewed the diff
- Useful pull request description
- Reviewer requested
Documentation
- No new TODOs introduced

Issue(s)

Closes #2470

…onsumer' trait

…natureProcessor' trait

…es names

mithril-aggregator/src/dependency_injection/builder/enablers/misc.rs

github-actions · 2025-05-14T08:26:02Z

Test Results

3 files ±0 57 suites ±0 11m 50s ⏱️ +18s
1 908 tests +3 1 908 ✅ +3 0 💤 ±0 0 ❌ ±0
2 382 runs +3 2 382 ✅ +3 0 💤 ±0 0 ❌ ±0

Results for commit 1097177. ± Comparison against base commit 9df5a56.

This pull request removes 9 and adds 12 tests. Note that renamed tests count towards both.

mithril-signer ‑ services::signature_publisher::signature_publisher_delayer::tests::should_call_both_publishers_when_first_succeeds
mithril-signer ‑ services::signature_publisher::signature_publisher_delayer::tests::should_call_second_publisher_even_if_first_fails_and_log_error
mithril-signer ‑ services::signature_publisher::signature_publisher_delayer::tests::should_return_and_log_error_if_second_publisher_fails
mithril-signer ‑ services::signature_publisher::signature_publisher_delayer::tests::should_wait_before_calling_second_publisher
mithril-signer ‑ services::signature_publisher::signature_publisher_retrier::tests::should_call_publish_once_when_no_retry_policy
mithril-signer ‑ services::signature_publisher::signature_publisher_retrier::tests::should_not_retry_when_publish_fails_and_retry_policy_is_never
mithril-signer ‑ services::signature_publisher::signature_publisher_retrier::tests::should_retry_and_return_error_after_max_attempts
mithril-signer ‑ services::signature_publisher::signature_publisher_retrier::tests::should_retry_once_and_succeed_on_second_attempt
mithril-signer ‑ services::signature_publisher::signature_publisher_retrier::tests::should_wait_between_retries_according_to_policy

mithril-aggregator ‑ services::signature_consumer::noop::tests::signature_consumer_noop_never_returns
mithril-aggregator ‑ services::signature_processor::tests::processor_process_signatures_succeeds
mithril-aggregator ‑ services::signature_processor::tests::processor_run_succeeds
mithril-signer ‑ services::signature_publisher::delayer::tests::should_call_both_publishers_when_first_succeeds
mithril-signer ‑ services::signature_publisher::delayer::tests::should_call_second_publisher_even_if_first_fails_and_log_error
mithril-signer ‑ services::signature_publisher::delayer::tests::should_return_and_log_error_if_second_publisher_fails
mithril-signer ‑ services::signature_publisher::delayer::tests::should_wait_before_calling_second_publisher
mithril-signer ‑ services::signature_publisher::retrier::tests::should_call_publish_once_when_no_retry_policy
mithril-signer ‑ services::signature_publisher::retrier::tests::should_not_retry_when_publish_fails_and_retry_policy_is_never
mithril-signer ‑ services::signature_publisher::retrier::tests::should_retry_and_return_error_after_max_attempts
…

♻️ This comment has been updated with latest results.

Copilot

Pull Request Overview

This PR implements a new signature processor for the DMQ in the aggregator, introducing new traits and implementations to continuously consume signatures and register them with the certifier service.

Added the SignatureProcessor and SequentialSignatureProcessor to process signatures sequentially.
Added a no-op SignatureConsumer implementation and integrated both components into dependency injection, launching the processor in a separate thread during serve.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
mithril-signer/src/services/signature_publisher/mod.rs	Renamed and re-exported signature publisher modules for clarity.
mithril-aggregator/src/services/signature_processor.rs	Added a new trait and implementation for processing signatures sequentially, including tests.
mithril-aggregator/src/services/signature_consumer/noop.rs	Introduced a no-op signature consumer that blocks indefinitely.
mithril-aggregator/src/services/signature_consumer/interface.rs	Defined the signature consumer interface.
mithril-aggregator/src/services/mod.rs	Updated module exports to include new signature consumer and processor.
mithril-aggregator/src/dependency_injection/builder/enablers/misc.rs	Added dependency builder functions for signature consumer and processor.
mithril-aggregator/src/commands/serve_command.rs	Integrated the signature processor into the serve command, running it in a separate thread and stopping it during shutdown.

Comments suppressed due to low confidence (1)

mithril-aggregator/src/services/signature_processor.rs:18

The run method continuously loops by calling process_signatures even after the stop flag is set. Consider modifying the loop to exit (e.g., using a break statement) when the stop condition is true.

async fn run(&self) -> StdResult<()> { loop { self.process_signatures().await?; } }

Copilot · 2025-05-14T08:45:36Z

mithril-aggregator/src/services/signature_processor.rs

+impl SignatureProcessor for SequentialSignatureProcessor {
+    async fn process_signatures(&self) -> StdResult<()> {
+        if *self.stop.lock().await {
+            warn!(self.logger, "Stoped signature processor");


There is a spelling mistake in the log message: "Stoped" should be corrected to "Stopped".

Suggested change

warn!(self.logger, "Stoped signature processor");

warn!(self.logger, "Stopped signature processor");

dlachaume

LGTM 👍

dlachaume · 2025-05-14T16:30:10Z

mithril-aggregator/src/services/signature_consumer/interface.rs

+    StdResult,
+};
+
+/// A signature consumer which blocks until a messages are available.


Suggested change

/// A signature consumer which blocks until a messages are available.

/// A signature consumer which blocks until a message is available.

jpraynaud added 7 commits May 13, 2025 18:34

feat(aggregator): add a 'SignatureConsumer' trait

5ec7422

feat(aggregator): implement a 'SignatureConsumerNoop' for 'SignatureC…

3eb9fce

…onsumer' trait

feat(aggregator): add a 'SignatureProcessor' trait

36d410b

feat(aggregator): implement a 'SequentialSignatureProcessor' for 'Sig…

d5f4069

…natureProcessor' trait

feat(aggregator): add build 'SequentialSignatureProcessor' in DI

abf5efc

feat(aggregator): wire 'SequentialSignatureProcessor' in serve command

70bbddd

refactor(signer): avoid redundancies in signature publisher sub modul…

a59552a

…es names

jpraynaud self-assigned this May 14, 2025

github-advanced-security bot found potential problems May 14, 2025

View reviewed changes

mithril-aggregator/src/dependency_injection/builder/enablers/misc.rs Fixed Show fixed Hide fixed

mithril-aggregator/src/dependency_injection/builder/enablers/misc.rs Fixed Show fixed Hide fixed

jpraynaud requested a review from Copilot May 14, 2025 08:44

Copilot AI reviewed May 14, 2025

View reviewed changes

chore(aggregator): fix clippy warnings

1097177

jpraynaud force-pushed the jpraynaud/2470-signature-processor-dmq branch from 62bcf4f to 1097177 Compare May 14, 2025 08:47

jpraynaud requested review from dlachaume, turmelclem and Alenar May 14, 2025 08:47

jpraynaud temporarily deployed to testing-preview May 14, 2025 08:57 — with GitHub Actions Inactive

dlachaume approved these changes May 14, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: implement a signature processor for DMQ #2477

Feat: implement a signature processor for DMQ #2477

jpraynaud commented May 14, 2025 •

edited

Loading

github-actions bot commented May 14, 2025 •

edited

Loading

Copilot AI left a comment

Copilot AI May 14, 2025

dlachaume left a comment

dlachaume May 14, 2025

	warn!(self.logger, "Stoped signature processor");
	warn!(self.logger, "Stopped signature processor");

	/// A signature consumer which blocks until a messages are available.
	/// A signature consumer which blocks until a message is available.

Feat: implement a signature processor for DMQ #2477

Are you sure you want to change the base?

Feat: implement a signature processor for DMQ #2477

Conversation

jpraynaud commented May 14, 2025 • edited Loading

Content

Pre-submit checklist

Issue(s)

github-actions bot commented May 14, 2025 • edited Loading

Test Results

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Copilot AI May 14, 2025

Choose a reason for hiding this comment

dlachaume left a comment

Choose a reason for hiding this comment

dlachaume May 14, 2025

Choose a reason for hiding this comment

jpraynaud commented May 14, 2025 •

edited

Loading

github-actions bot commented May 14, 2025 •

edited

Loading