pvf-precheck: Integrate PVF pre-checking into paras module

[pepyakin]

Closes #4009

This is the most of the runtime-side change needed for #3211.

Here is how it works.

                                           ┌──────────┐
                       supermajority       │          │
                   ┌────────for───────────▶│ accepted │
       vote────┐   │                       │          │
        │      │   │                       └──────────┘
        │      │   │
        │  ┌───────┐
        │  │       │
        └─▶│ init  │────supermajority      ┌──────────┐
           │       │       against         │          │
           └───────┘           └──────────▶│ rejected │
            ▲  │                           │          │
            │  │ session                   └──────────┘
            │  └──change
            │     │
            │     ▼
            ┌─────┐
start──────▶│reset│
            └─────┘

The PVF pre-checking can be triggered either by an upgrade or by
onboarding (i.e. calling schedule_para_initialize). The PVF
pre-checking process is identified by the PVF code hash that is being
voted on. If there is already PVF pre-checking process running, then no
new PVF pre-checking process will be started. Instead, we just subscribe
to the existing one.

If there is no PVF pre-checking process running but the PVF code hash
was already saved in the storage, that necessarily means (I invite the
reviewers to double-check this invariant) that the PVF already passed
pre-checking. This is equivalent to instant approving of the PVF.

The pre-checking process can be concluded either by obtaining a
supermajority or if it expires.

Each validator checks the list of PVFs available for voting. The vote is
binary, i.e. accept or reject a given PVF. As soon as the supermajority
of votes are collected for one of the sides of the vote, the voting is
concluded in that direction and the effects of the voting are enacted.

Only validators from the active set can participate in the vote. The set
of active validators can change each session. That's why we reset the
votes each session. A voting that observed a certain number of sessions
will be rejected.

The effects of the PVF accepting depend on the operations requested it:

1. All onboardings subscribed to the approved PVF pre-checking process will
   get scheduled and after passing 2 session boundaries they will be onboarded.
2. All upgrades subscribed to the approved PVF pre-checking process will
   get scheduled very similarly to the existing process. Upgrades with
   pre-checking are really the same process that is just delayed by the
   time required for pre-checking voting. In case of instant approval the
   mechanism is exactly the same. This is important from parachains
   compatibility standpoint since following the delayed upgrade requires
   the parachain to implement
   Look at the upgrade go-ahead and restriction signals cumulus#517.

In case, PVF pre-checking process was concluded with rejection, then all
the requesting operations get cancelled. For onboarding it means it gets
without movement: the lifecycle of such parachain is terminated on the
Onboarding state and after rejection the lifecycle is none. That in
turn means that the caller can attempt registering the parachain once
more. For upgrading it means that the upgrade process is aborted: that
flashes go-ahead signal with Abort flag.

Rejection leads to removing the allegedly bad validation code from the
chain storage. Among other things, this implies that the operation can
be re-requested. That allows for retrying an operation in case there was
some bug. At the same time it does not look as a DoS vector due to the
caching performed by the nodes.

PVF pre-checking can be enabled and disabled. Initially, according to
the changes in #4420, this mechanism is disabled. Triggering the PVF
pre-checking when it is disabled just means that we insta approve the
requesting operation. This should lead to the behavior being unchanged.

TODO:

• if UpgradeGoAheadSignal is set in schedule_code_upgrade it will be immediately reset in note_new_head
• run benchmarks

Follow-ups:

• expose runtime APIs
• implementers' guide changes
• ensure this module can be controlled by governance (e.g. via force_* functions)
• deposit events for PVF pre-checking related events
• weights for include_pvf_check_statement

skip check-dependent-cumulus

[pepyakin]

Current dependencies on/for this PR:

• master
  • PR pvf-precheck: Integrate PVF pre-checking into paras module #4457 (this PR)
    • PR paras: add log target #4478
    • PR pvf-precheck: update rustdoc for paras module #4459

This comment was auto-generated by Graphite.

[pepyakin]

Oh sh~, TIL: "votings" is not a thing.

[drahnr]

You mean as in a timeout?

[pepyakin]

I meant to be vague that the pre-checking should finish first and only then a new upgrade will be allowed. It just so happens that the pre-checking timeout is typically takes the most time. However, it is also possible that the last decisive vote comes in on the last block of the session and thus ends the PVF voting, so it is not always timeout.

[drahnr]

Exhaustiveness is imho better than trying to avoid it. Helps reviewing the code and on-boarding new people.

[drahnr]

Generally looks good, I have to review a few sections more carefully:

• the "hack"
• ref counting consistency

All my comments are just nits, code looks pretty nice (minus the existence of "hack")

[drahnr]

nit: Is there a particular reason we don't try_mutate this in place? It feels a bit error prone having the vote tracking the vote at the very end in an else case.

[pepyakin]

yeah it was also bothering me, but I think in this case cure is the same or worse than the disease.

In the case of achieving the quorum, we should remove the vote from the list and the map. Probably we can switch the entire storage value to be OptionQuery, but then we will have to deal with this everywhere as well. Then, I am trying to put mutations of the map closer to the list, but in this case the mutation will happen inside of try_mutate but the list mutation will be left alone.

We could perhaps refactor the code so that we return an enum which then says what to do with the vote: update or remove, but that's additional code only to make sure we do not accidentally miss this update, so it feels to me not worth it

[drahnr]

Uh, also: could you include the ascii graph in the comment :)

[pepyakin]

ah, right, I can add it in the next PR in the stack: #4459