Skip to content

Feat: implement a signature processor for DMQ #2477

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
May 15, 2025

Conversation

jpraynaud
Copy link
Member

@jpraynaud jpraynaud commented May 14, 2025

Content

This PR includes the implementation of a signature processor for the DMQ in the aggregator:

  • Added a SignatureProcessor trait for continuously consuming signatures and registering them with the certifier service
  • Added a SequentialSignatureProcessor implementation of the SignatureProcessor trait
  • Added a SignatureConsumer trait for receiving the signatures
  • Added a SignatureConsumerNoop implementation of the SignatureConsumer trait (until the DMQ consumer is available)
  • Added constructioon of the SignatureProcessor in the dependency injection
  • Launched the SignatureProcessor in a separate thread of the serve command
  • Use unique stop signal for:
    • HTTP server, Metrics server and Signature processor in aggregator
    • Metrics server in signer

Pre-submit checklist

  • Branch
    • Tests are provided (if possible)
    • Crates versions are updated (if relevant)
    • CHANGELOG file is updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • All check jobs of the CI have succeeded
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested
  • Documentation
    • No new TODOs introduced

Issue(s)

Closes #2470

@jpraynaud jpraynaud self-assigned this May 14, 2025
Copy link

github-actions bot commented May 14, 2025

Test Results

    3 files  ±0     57 suites  ±0   11m 20s ⏱️ -38s
1 912 tests +4  1 912 ✅ +4  0 💤 ±0  0 ❌ ±0 
2 392 runs  +4  2 392 ✅ +4  0 💤 ±0  0 ❌ ±0 

Results for commit d138a2b. ± Comparison against base commit 7dd379c.

♻️ This comment has been updated with latest results.

@jpraynaud jpraynaud requested a review from Copilot May 14, 2025 08:44
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a new signature processor for the DMQ in the aggregator, introducing new traits and implementations to continuously consume signatures and register them with the certifier service.

  • Added the SignatureProcessor and SequentialSignatureProcessor to process signatures sequentially.
  • Added a no-op SignatureConsumer implementation and integrated both components into dependency injection, launching the processor in a separate thread during serve.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
mithril-signer/src/services/signature_publisher/mod.rs Renamed and re-exported signature publisher modules for clarity.
mithril-aggregator/src/services/signature_processor.rs Added a new trait and implementation for processing signatures sequentially, including tests.
mithril-aggregator/src/services/signature_consumer/noop.rs Introduced a no-op signature consumer that blocks indefinitely.
mithril-aggregator/src/services/signature_consumer/interface.rs Defined the signature consumer interface.
mithril-aggregator/src/services/mod.rs Updated module exports to include new signature consumer and processor.
mithril-aggregator/src/dependency_injection/builder/enablers/misc.rs Added dependency builder functions for signature consumer and processor.
mithril-aggregator/src/commands/serve_command.rs Integrated the signature processor into the serve command, running it in a separate thread and stopping it during shutdown.
Comments suppressed due to low confidence (1)

mithril-aggregator/src/services/signature_processor.rs:18

  • The run method continuously loops by calling process_signatures even after the stop flag is set. Consider modifying the loop to exit (e.g., using a break statement) when the stop condition is true.
async fn run(&self) -> StdResult<()> { loop { self.process_signatures().await?; } }

@jpraynaud jpraynaud force-pushed the jpraynaud/2470-signature-processor-dmq branch from 62bcf4f to 1097177 Compare May 14, 2025 08:47
@jpraynaud jpraynaud temporarily deployed to testing-preview May 14, 2025 08:57 — with GitHub Actions Inactive
Copy link
Collaborator

@dlachaume dlachaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Copy link
Collaborator

@turmelclem turmelclem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jpraynaud jpraynaud force-pushed the jpraynaud/2470-signature-processor-dmq branch from 1097177 to a1fadc3 Compare May 15, 2025 09:47
@jpraynaud jpraynaud force-pushed the jpraynaud/2470-signature-processor-dmq branch from a1fadc3 to 1f5055c Compare May 15, 2025 10:05
@jpraynaud jpraynaud requested a review from Copilot May 15, 2025 10:05
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a signature processor for the DMQ in the aggregator, introducing new traits and implementations to continuously consume and process signatures.

  • Introduces the SignatureProcessor trait and its sequential implementation in the aggregator.
  • Adds the SignatureConsumer trait with a noop implementation until a DMQ consumer is available.
  • Updates dependency injection and server shutdown signal handling to run the signature processor in a separate thread.

Reviewed Changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
mithril-signer/src/services/signature_publisher/mod.rs Refactored module names and re-exports for signature publishing.
mithril-signer/src/main.rs Replaced oneshot channels with watch channels for shutdown signaling.
mithril-signer/Cargo.toml Bumped version to reflect minor changes.
mithril-aggregator/src/services/signature_processor.rs Added new signature processor trait and its sequential implementation.
mithril-aggregator/src/services/signature_consumer/noop.rs Added a no-op consumer that never returns signatures.
mithril-aggregator/src/services/signature_consumer/fake.rs Added a fake consumer implementation, along with tests for expected behavior.
Dependency Injection Modules Integrated stop signal channel and wired up the new signature processor in the DI builder.
mithril-aggregator/src/commands/serve_command.rs Updated server startup/shutdown to include signature processing and stop signals.
internal/mithril-metric/src/server.rs Updated shutdown signal handling for metrics server using watch channels.
Cargo.toml and CHANGELOG.md Updated versions and changelog to support the feature changes.

@jpraynaud jpraynaud force-pushed the jpraynaud/2470-signature-processor-dmq branch from 1f5055c to d138a2b Compare May 15, 2025 10:12
@jpraynaud jpraynaud temporarily deployed to testing-preview May 15, 2025 10:21 — with GitHub Actions Inactive
Copy link
Collaborator

@Alenar Alenar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jpraynaud jpraynaud merged commit 4d382b9 into main May 15, 2025
38 checks passed
@jpraynaud jpraynaud deleted the jpraynaud/2470-signature-processor-dmq branch May 15, 2025 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement a signature processor for DMQ node in aggregator
4 participants