Create a more rigid overseer builder pattern that fails at compile ti…

…me (#4753)

Introduces `Missing<Field>` and `Init<Field>` states, that are used in place of builder generics, and make this possible.

bridges/bin/rialto/node/src/overseer.rs

@@ -33,7 +33,7 @@ use polkadot_node_core_dispute_coordinator::Config as DisputeCoordinatorConfig;
 use polkadot_node_core_provisioner::ProvisionerConfig;
 use polkadot_node_network_protocol::request_response::{v1 as request_v1, IncomingRequestReceiver};
 use polkadot_overseer::{
-	metrics::Metrics as OverseerMetrics, BlockInfo, MetricsTrait, Overseer, OverseerBuilder,
+	metrics::Metrics as OverseerMetrics, BlockInfo, MetricsTrait, Overseer, InitializedOverseerBuilder,
 	OverseerConnector, OverseerHandle,
 };
 use polkadot_primitives::v1::ParachainHost;
@@ -139,7 +139,7 @@ pub fn prepared_overseer_builder<Spawner, RuntimeClient>(
 		disputes_enabled,
 	}: OverseerGenArgs<'_, Spawner, RuntimeClient>,
 ) -> Result<
-	OverseerBuilder<
+	InitializedOverseerBuilder<
 		Spawner,
 		Arc<RuntimeClient>,
 		CandidateValidationSubsystem,

node/overseer/overseer-gen/README.md

@@ -0,0 +1,78 @@
+# overseer pattern
+
+The overseer pattern is a partial actor pattern
+
+## proc-macro
+
+The proc macro provides a convenience generator with a builder pattern,
+where at it's core it creates and spawns a set of subsystems, which are purely
+declarative.
+
+```rust
+    #[overlord(signal=SigSigSig, event=Event, gen=AllMessages, error=OverseerError)]
+    pub struct Overseer {
+        #[subsystem(MsgA)]
+        sub_a: AwesomeSubSysA,
+
+        #[subsystem(MsgB)]
+        sub_b: AwesomeSubSysB,
+    }
+```
+
+* Each subsystem is annotated with `#[subsystem(_)]` where `MsgA` respectively `MsgB` are the messages
+being consumed by that particular subsystem. Each of those subsystems is required to implement the subsystem
+trait.
+* `error=` tells the overseer to use the user provided
+error type, if not provided a builtin one is used. Note that this is the one error type used throughout all calls, so make sure it does impl `From<E>` for all other error types `E` that are relevant to your application.
+* `event=` declares an external event type, that injects certain events
+into the overseer, without participating in the subsystem pattern.
+* `signal=` defines a signal type to be used for the overseer. This is a shared "clock" for all subsystems.
+* `gen=` defines a wrapping `enum` type that is used to wrap all messages that can be consumed by _any_ subsystem.
+
+```rust
+    /// Execution context, always requred.
+    pub struct DummyCtx;
+
+    /// Task spawner, always required.
+    pub struct DummySpawner;
+
+    fn main() {
+        let _overseer = Overseer::builder()
+            .sub_a(AwesomeSubSysA::default())
+            .sub_b(AwesomeSubSysB::default())
+            .spawner(DummySpawner)
+            .build();
+    }
+```
+
+In the shown `main`, the overseer is created by means of a generated, compile time erroring
+builder pattern.
+
+The builder requires all subsystems, baggage fields (additional struct data) and spawner to be
+set via the according setter method before `build` method could even be called. Failure to do
+such an initialization will lead to a compile error. This is implemented by encoding each
+builder field in a set of so called `state generics`, meaning that each field can be either
+`Init<T>` or `Missing<T>`, so each setter translates a state from `Missing` to `Init` state
+for the specific struct field. Therefore, if you see a compile time error that blames about
+`Missing` where `Init` is expected it usually means that some subsystems or baggage fields were
+not set prior to the `build` call.
+
+To exclude subsystems from such a check, one can set `wip` attribute on some subsystem that
+is not ready to be included in the Overseer:
+
+```rust
+    #[overlord(signal=SigSigSig, event=Event, gen=AllMessages, error=OverseerError)]
+    pub struct Overseer {
+        #[subsystem(MsgA)]
+        sub_a: AwesomeSubSysA,
+
+        #[subsystem(MsgB), wip]
+        sub_b: AwesomeSubSysB, // This subsystem will not be required nor allowed to be set
+    }
+```
+
+Baggage fields can be initialized more than one time, however, it is not true for subsystems:
+subsystems must be initialized only once (another compile time check) or be _replaced_ by
+a special setter like method `replace_<subsystem>`.
+
+A task spawner and subsystem context are required to be defined with `SpawnNamed` and respectively `SubsystemContext` implemented.

node/overseer/overseer-gen/examples/dummy.rs

@@ -2,6 +2,7 @@
 
 use polkadot_node_network_protocol::WrongVariant;
 use polkadot_overseer_gen::*;
+use std::collections::HashMap;
 
 /// Concrete subsystem implementation for `MsgStrukt` msg type.
 #[derive(Default)]
@@ -88,14 +89,16 @@ impl NetworkMsg {
 }
 
 #[overlord(signal=SigSigSig, event=EvX, error=Yikes, network=NetworkMsg, gen=AllMessages)]
-struct Xxx {
+struct Xxx<T> {
 	#[subsystem(MsgStrukt)]
 	sub0: AwesomeSubSys,
 
 	#[subsystem(no_dispatch, blocking, Plinko)]
 	plinkos: GoblinTower,
 
 	i_like_pi: f64,
+	i_like_generic: T,
+	i_like_hash: HashMap<f64, f64>,
 }
 
 #[derive(Debug, Clone)]
@@ -125,12 +128,16 @@ impl SpawnNamed for DummySpawner {
 struct DummyCtx;
 
 fn main() {
-	let (overseer, _handle): (Xxx<_>, _) = Xxx::builder()
+	let (overseer, _handle): (Xxx<_, f64>, _) = Xxx::builder()
 		.sub0(AwesomeSubSys::default())
 		.plinkos(GoblinTower::default())
 		.i_like_pi(::std::f64::consts::PI)
+		.i_like_generic(42.0)
+		.i_like_hash(HashMap::new())
 		.spawner(DummySpawner)
 		.build()
 		.unwrap();
 	assert_eq!(overseer.i_like_pi.floor() as i8, 3);
+	assert_eq!(overseer.i_like_generic.floor() as i8, 42);
+	assert_eq!(overseer.i_like_hash.len() as i8, 0);
 }