Idea: A facility for tracking Integration

[gavofyork]

At present, Accumulation is the highest level service compute phase which is explicitly acknowledged by JAM. However for Cross-service Work-Item Entanglement, it is important that any changes to the entangled services are synchronised and premised on expected changes being enacted in all entangled services.

If Accumulation is infallible and conditionless (i.e. will always make the same, expected, change regardless of what other WPs might have been Accumulated before), then it is enough to place all entangled Work Items in a single WP.

However, most Accumulation code (not least the CoreChains and CorePlay services) will not be conditionless and the repercussions of entanglement must only be enacted once all services have ascertained that all other entangled services are each committed to enact them.

An on-chain facility to pub/sub for the integration of Work Items would help achieve this without excessive use of transfer and peek.

[sourabhniyogi]

From your comment on #129 "a WP might imply a block is valid and finalisable yet when it gets to Accumulate, some other WP has since been accumulated which has finalized a block which is mutually exclusive with that of the first WP" motivating "Integration" -- where I believe you envision the CoreChains JAM service to not commit finalized blocks in accumulate until integration happens, right?

In my idea of a parallel ETH L1/L2 validation JAM service for ORUs, following @rphmeier's suggestion to aggressively validate ORUs state transitions on all forks (both L1 and L2) in refine, I am envisioning an integration-less accumulate writing validated blocks to JAM State whether these validated ORU blocks are finalized or not. Then ANOTHER "finalizer" service can bring in work items from
(a) ETH L1's finality checkpoints (every 13 mins)
and
(b) ZK proofs of finality from prover networks optimized for ORUs that provides real finality beyond (a):

• Succinct's SP1
• Risc Zeros Kailua
• probably others ...

and basically does the "integration" work:

• establishing chainhood
• doing the bookkeeping for the finalized chain only
• forgetting blocks only on non-canonical chains

The true finality of ORUs is not based on (a) but really a social time period (of 7 days). But with (b), it can be reduced to under an hour ($50MM/yr for 50 ETH ORUs from OP Succinct, plausibly). JAM's beefy root for finalization would not be based on (a) alone (since its not really finalized) but (b). We would not delay writes to JAM State for validating blocks just because (b) is not there though ... because doing it based on (a) is more practical for rollups without the hefty ZK prover budget. I don't see the need to block the results of WP being written to JAM State waiting for (a) [13 mins] or (b) [1 hour] by requiring integration.

Are non-Polkadot rollups basically not needing the integration you envisioned because we want to see the validation on all possible forks for ORUs? What did I miss?