Disable logging of rustls to get rid off the following log lines:
```
Sending fatal alert BadCertificate
```

Upstream also removed them: rustls/rustls#1278


Closes: #3252

Closes #1961

The fact that this takes two sessions to come into effect is not
obvious. Just added some docs to explain that.

Also tidied up uses of "broker chain" -> "coretime chain"

A part of #3326 

Removes all #[pallet::getter] usage from the contracts mock network
pallet. As the storage values were pub(super), read-only visibility was
lost external to the crate upon the removal of the macros. I have
implemented custom getters as a replacement, keeping the api the same.

If we care very much about consistency of the
storagevalue::<T>::get() syntax, the other option would be to set
the storage values to pub. Though I find preserving data authority
better myself.

@muraca

As per #3326, removes pallet::getter usage from the pallet
authority-discovery. The syntax `StorageItem::<T, I>::get()` should be
used instead.

cc @muraca

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

…kNumber=u32` (#4434)

Closes: #4428

The block header is required to derive inherents for a relay chain next
block, this is useful in testing environments.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

…4302)

#### Problem
`take()` consumes only 1 read worth of weight in
`single-block-migrations` example, while `take()`
[is](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/support/src/storage/unhashed.rs#L63)
`get() + kill()`, i.e should be 1 read + 1 write. I think this could
mislead developers who follow this example to write their migrations

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Library `substrate-frame-cli` seems unused.

Last non-dependabot update to the folder was over [two years
ago](https://github.com/paritytech/polkadot-sdk/commits/master/substrate/utils/frame/frame-utilities-cli).

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

The state-trie migration is completed on Rococo Asset-Hub as
double-checked
[here](#4174 (comment)).
Undeploying now.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Extracting the logic for generating and verifying ancestry proofs from
this PR: #1903 with small
adjustments

@Lederstrumpf I added you as a co-author to each commit. Please let me
know if you want me to also associate an e-mail address with your name

…4326)

Currently the `CheckWeight` `SignedExtension` was tracking the size of
the proof and the extrinsic length separately. But in reality we need
one more check that ensures we don't hit the PoV limit with both
combined.

The rest of the logic remains unchanged. One scenario where the changes
make a difference is when we enter this branch:

https://github.com/paritytech/polkadot-sdk/blob/f34d8e3cf033e2a22a41b505c437972a5dc83d78/substrate/frame/system/src/extensions/check_weight.rs#L185-L198

This was previously allowing to some extrinsics that is exceeding the
block limit but are withing the reserved area of `BlockWeights`. This
will now be caught by the later check I introduced. I think the new
behaviour makes sense, since the proof size dimension is designed for
parachains and they don't want to go over the limit and get rejected.


In the long run we should maybe get rid of `RuntimeBlockLength`
alltogether, however that would require a deprecation process and can
come at a later point.

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

PR adds forklift settings and forklift to test-github-actions

cc paritytech/ci_cd#939

Reworks prospective-parachains so that we allow a number of unconnected
candidates (for which we don't know the parent candidate yet). Needed
for elastic scaling:
#3541. Without this,
candidate B will not be validated and backed until candidate A (its
parent) is validated and a backing statement reaches the validator.

Due to the high complexity of the subsystem, I rewrote parts of it so
that we don't concern ourselves with candidates which form cycles or
which form parachain forks. We now have "Fragment chains" instead of
"Fragment trees". This greatly simplifies some of the code and is a
compromise we can make. We just need to make sure that cycle-producing
parachains don't brick the relay chain and that fork-producing
parachains can still make some progress (on one core at least). The only
forks that are allowed are those on the relay chain, obviously.

Unconnected candidates are kept in the `CandidateStorage` and whenever a
new candidate is introduced, we try to repopulate the chain with as many
candidates as we can.

Also fixes #3219

Guide changes will be done as part of:
#3699

TODOs:

- [x] see if we can replace the `Cow` over the candidate commitments
with an `Arc` over the entire `ProspectiveCandidate`. It's only being
overwritten in unit tests. We can work around that.
- [x] finish fragment_chain unit tests
- [x] add more prospective-parachains subsystem tests
- [x] test with zombienet what happens if a parachain is creating cycles
(it should not brick the relay chain).
- [x] test with zombienet a parachain that is creating forks. it should
keep producing blocks from time to time (one bad collator should not DOS
the parachain, even if throughput decreases)
- [x] add some more logs and metrics
- [x] add prdoc and remove the "silent" label

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Andrei Sandu <andrei-mihail@parity.io>

- Moved check-runtime-migration from gitlab to github
- Moved `test-syscalls` and `cargo-check-all-benches` from gitlab to
github

This PR bumps `proc-macro-crate` to the latest version.

In order to test a runtime from
https://github.com/polkadot-fellows/runtimes/ with the latest version of
polkadot-sdk one needs to use `cargo vendor` to extract all runtime
dependencies, patch them by hand and then build the runtime.

However at the moment 'vendored' builds fail due to
bkchr/proc-macro-crate#48. To fix this
`proc-macro-crate` should be updated to version `3.0.1` or higher.

---------

Co-authored-by: command-bot <>

…ing (#4442)

Support async backing in `--dev` mode

This PR improve the relay mock `MockValidationDataInherentDataProvider`
to mach expectations of async backing runtimes.

* Add para_head in the mock relay proof
* Add relay slot in the mock relay proof 

fix #4437

`*-host` and `*-port` are obsolete and we'll hopefully remove them in
the future (already WIP for Rococo <> Westend relayer)

Github currently [doesn't support creating
subfolders](https://github.com/orgs/community/discussions/18055) in
`.github/workflows` folder. PR unifies naming in this folder. The idea
is that all files ideally should have naming
`<stage>-<short_descriptive_name>.yml`. The stage names are taken from
gitlab and needed only for logical division.
Changes in actions:
 - `check-licences` runs on self-hosted runners
- `check-workspace` and `check-markdown` are moved to `checks-quick.yml`
 - `test-linux-stable-int` moved to a new file `tests-linux-stable.yml`
- `fmt-check.yml` file is removed, formatting is checked in
`checks-quick.yml`

related to
paritytech/parity-bridges-common#2962
Usage example:
```sh
RUST_LOG=runtime=trace,rpc=trace,bridge=trace \
        ./target/release/substrate-relay relay-messages-range bridge-hub-rococo-to-bridge-hub-westend \
        --source-host localhost \
        --source-port 8943 \
        --source-version-mode Auto \
        --source-signer //Eve \
        --source-transactions-mortality 4 \
        --target-host localhost \
        --target-port 8945 \
        --target-version-mode Auto \
        --target-signer //Eve \
        --target-transactions-mortality 4 \
        --lane 00000002 \
        --at-source-block 34 \
        --messages-start 1 \
        --messages-end 1
INFO bridge Connecting to BridgeHubRococo node at ws://localhost:8943
INFO bridge Connecting to BridgeHubWestend node at ws://localhost:8945
TRACE bridge Refined weight of BridgeHubRococo->BridgeHubWestend message [0, 0, 0, 2]/1: at-source: Weight(ref_time: 0, proof_size: 0), at-target: Weight(ref_time: 452953993, proof_size: 0)
TRACE bridge Sent transaction to BridgeHubWestend node: 0x38552f4db6bc78baecb52ebd2f7d103b1c919c16b83129dc083bf01b7281955b
TRACE bridge BridgeHubWestend transaction 0x38552f4db6bc78baecb52ebd2f7d103b1c919c16b83129dc083bf01b7281955b has been included in block: (0x29a20bdca8726df0b32af9067290b7fc0a886908da3a30f3db60a6ea52be4604, 0)
TRACE bridge BridgeHubWestend transaction 0x38552f4db6bc78baecb52ebd2f7d103b1c919c16b83129dc083bf01b7281955b has been finalized at block: 0x29a20bdca8726df0b32af9067290b7fc0a886908da3a30f3db60a6ea52be4604
```

Bumps [nix](https://github.com/nix-rust/nix) from 0.27.1 to 0.28.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/nix-rust/nix/blob/master/CHANGELOG.md">nix's
changelog</a>.</em></p>
<blockquote>
<h2>[0.28.0] - 2024-02-24</h2>
<h3>Added</h3>
<ul>
<li>
<p>Added <code>mkdtemp</code> wrapper (<a
href="https://redirect.github.com/nix-rust/nix/pull/1297">#1297</a>)</p>
</li>
<li>
<p>Add associated constants <code>UTIME_OMIT</code>
<code>UTIME_NOW</code> for <code>TimeSpec</code>
(<a
href="https://redirect.github.com/nix-rust/nix/pull/1879">#1879</a>)</p>
</li>
<li>
<p>Added <code>EventFd</code> type. (<a
href="https://redirect.github.com/nix-rust/nix/pull/1945">#1945</a>)</p>
</li>
<li>
<ul>
<li>Added <code>impl From&lt;Signal&gt; for SigSet</code>.</li>
<li>Added <code>impl std::ops::BitOr for SigSet</code>.</li>
<li>Added <code>impl std::ops::BitOr for Signal</code>.</li>
<li>Added <code>impl std::ops::BitOr&lt;Signal&gt; for
SigSet</code></li>
</ul>
<p>(<a
href="https://redirect.github.com/nix-rust/nix/pull/1959">#1959</a>)</p>
</li>
<li>
<p>Added <code>TlsGetRecordType</code> control message type and
corresponding enum for
linux (<a
href="https://redirect.github.com/nix-rust/nix/pull/2065">#2065</a>)</p>
</li>
<li>
<p>Added <code>Ipv6HopLimit</code> to
<code>::nix::sys::socket::ControlMessage</code> for Linux,
MacOS, FreeBSD, DragonflyBSD, Android, iOS and Haiku.
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2074">#2074</a>)</p>
</li>
<li>
<p>Added <code>Icmp</code> and <code>IcmpV6</code> to
<code>SockProtocol</code>
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2103">#2103</a>)</p>
</li>
<li>
<p>Added rfork support for FreeBSD in <code>unistd</code>
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2121">#2121</a>)</p>
</li>
<li>
<p>Added <code>MapFlags::map_hugetlb_with_size_log2</code> method for
Linux targets
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2125">#2125</a>)</p>
</li>
<li>
<p>Added <code>mmap_anonymous</code> function
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2127">#2127</a>)</p>
</li>
<li>
<p>Added <code>mips32r6</code> and <code>mips64r6</code> support for
signal, ioctl and ptrace
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2138">#2138</a>)</p>
</li>
<li>
<p>Added <code>F_GETPATH</code> FcntlFlags entry on
Apple/NetBSD/DragonflyBSD for
<code>::nix::fcntl</code>. (<a
href="https://redirect.github.com/nix-rust/nix/pull/2142">#2142</a>)</p>
</li>
<li>
<p>Added <code>F_KINFO</code> FcntlFlags entry on FreeBSD for
<code>::nix::fcntl</code>.
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2152">#2152</a>)</p>
</li>
<li>
<p>Added <code>F_GETPATH_NOFIRMLINK</code> and
<code>F_BARRIERFSYNC</code> FcntlFlags entry
on Apple for <code>::nix::fcntl</code>.
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2155">#2155</a>)</p>
</li>
<li>
<p>Added newtype <code>Flock</code> to automatically unlock a held flock
upon drop.
Added <code>Flockable</code> trait to represent valid types for
<code>Flock</code>.
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2170">#2170</a>)</p>
</li>
<li>
<p>Added <code>SetSockOpt</code> impls to enable Linux Kernel TLS on a
TCP socket and to
import TLS parameters. (<a
href="https://redirect.github.com/nix-rust/nix/pull/2175">#2175</a>)</p>
</li>
<li>
<ul>
<li>Added the <code>::nix::sys::socket::SocketTimestamp</code> enum for
configuring the
<code>TsClock</code> (a.k.a <code>SO_TS_CLOCK</code>) sockopt</li>
<li>Added FreeBSD's <code>ScmRealtime</code> and
<code>ScmMonotonic</code> as new options in
<code>::nix::sys::socket::ControlMessageOwned</code></li>
</ul>
<p>(<a
href="https://redirect.github.com/nix-rust/nix/pull/2187">#2187</a>)</p>
</li>
<li>
<p>Added new fanotify API: wrappers for <code>fanotify_init</code> and
<code>fanotify_mark</code>
(<a
href="https://redirect.github.com/nix-rust/nix/pull/2194">#2194</a>)</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/nix-rust/nix/commit/21ab06ef23de214174ddb039be5b5f08750d19e6"><code>21ab06e</code></a>
chore: release 0.8.0</li>
<li><a
href="https://github.com/nix-rust/nix/commit/595c6eb2515723eae1f1db1b9103c95f339380ec"><code>595c6eb</code></a>
chore: changelog for 0.28.0 (<a
href="https://redirect.github.com/nix-rust/nix/issues/2303">#2303</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/79b86a7dff071a2a59858d1c98e202fdf38ebd06"><code>79b86a7</code></a>
mmsg revert signature simplification (<a
href="https://redirect.github.com/nix-rust/nix/issues/2227">#2227</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/08e05a5f66d14d4f06288c9a725e87f473eba41a"><code>08e05a5</code></a>
refactor: remove redundant imports to fix CI (<a
href="https://redirect.github.com/nix-rust/nix/issues/2320">#2320</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/197f55b3ccbce3273bf6ce119d1a8541b5df5d66"><code>197f55b</code></a>
refactor: update nix (<a
href="https://redirect.github.com/nix-rust/nix/issues/2311">#2311</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/f34d00d353a8ebcb16eb8810b00d613e4a294925"><code>f34d00d</code></a>
chore(deps): try libc 0.2.153 (<a
href="https://redirect.github.com/nix-rust/nix/issues/2309">#2309</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/062f6c0f0306743a401138fc2612d59e818d5768"><code>062f6c0</code></a>
test: clean clippy needless_borrow false positive issue (<a
href="https://redirect.github.com/nix-rust/nix/issues/2310">#2310</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/ca173ff30cc42e0fc7a7ba2cf7c1fcbe61dccfa5"><code>ca173ff</code></a>
Fix the mount tests (<a
href="https://redirect.github.com/nix-rust/nix/issues/2269">#2269</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/0dfcf3228489a5060a53e69783a1eea6f43b11a9"><code>0dfcf32</code></a>
test: tweak test_fanotify to monitor ONLY 1 file (<a
href="https://redirect.github.com/nix-rust/nix/issues/2294">#2294</a>)</li>
<li><a
href="https://github.com/nix-rust/nix/commit/0fb346031f6ddc87a57377101a6063133a25e443"><code>0fb3460</code></a>
Temporarily disable CI on GNU hurd (<a
href="https://redirect.github.com/nix-rust/nix/issues/2307">#2307</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/nix-rust/nix/compare/v0.27.1...v0.28.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=nix&package-manager=cargo&previous-version=0.27.1&new-version=0.28.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

A small step towards
https://forum.polkadot.network/t/polkadot-parachain-omni-node-gathering-ideas-and-feedback/7823
and #4352.

Many parachains use `camelCase` and/or `PascalCase`ing for their chain
spec extension. Sometimes the only reason polkadot-parachain cannot sync
them is because it cannot parse the chain spec extension. This PR
relaxes the requirement for the extension to be camel case.

---------

Co-authored-by: Branislav Kontur <bkontur@gmail.com>

This PR changes the fork calculation and pruning algorithm to enable
future block header pruning. It's required because the previous
algorithm relied on the block header persistence. It follows the
[related
discussion](#1570)

The previous code contained this comment describing the situation:
```
	/// Note a block height finalized, displacing all leaves with number less than the finalized
	/// block's.
	///
	/// Although it would be more technically correct to also prune out leaves at the
	/// same number as the finalized block, but with different hashes, the current behavior
	/// is simpler and our assumptions about how finalization works means that those leaves
	/// will be pruned soon afterwards anyway.
	pub fn finalize_height(&mut self, number: N) -> FinalizationOutcome<H, N> {
```

The previous algorithm relied on the existing block headers to prune
forks later and to enable block header pruning we need to clear all
obsolete forks right after the block finalization to not depend on the
related block headers in the future.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

... to get rid of the macro debug logs like this from build output
```
[/home/alexggh/.cargo/registry/src/index.crates.io-6f17d22bba15001f/fatality-proc-macro-0.1.0/src/types.rs:171:23] input = TokenStream [
    Literal {
        kind: Str,
        symbol: "Error while accessing runtime information",
        suffix: None,
        span: #0 bytes(6943..6986),
    },
]
```

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>

…estend Coretime (#3936)

Please add RadiumBlock bootnodes to the spec files for Westend Coretime
and Westend People

---------

Co-authored-by: Dónal Murray <donal.murray@parity.io>

Add small fixes, reorg debug definition and add more verbose debug logs
in k8s logs reading.
Thx!

Link was dead

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

This is the second PR in preparation for
#454.

## Also see
- **Precursor** #3889.
- **Follow up** #3905.

Overall changes are documented here (lot more visual 😍):
https://hackmd.io/@ak0n/454-np-governance

## Changes
### Delegation Interface
Provides delegation primitives for staking. 

Introduces two new roles:
- Agent: These are accounts who receive delegation from other accounts
(delegators) and stakes on behalf of them. The funds are held in
delegator accounts.
- Delegator: Accounts who delegate their funds to an agent authorising
them to use it for staking.

Supports
- A way for delegators to add or withdraw delegation to an agent.
- A way for an agent to slash a delegator during a slashing event.

### Pallet Delegated Staking
- Implements `DelegationInterface`.
- Lazy slashing: Any slashes to an Agent is posted in a ledger but not
immediately slashed. The agent can call
`DelegationInterface::delegator_slash` to slash the member and clear the
corresponding slash from its ledger.
- Consumes `StakingInterface` to provide `CoreStaking` features. In
reality, this will be `pallet-staking`.
- Ensures bookkeeping for agent and delegator are correct but leaves the
management of reward and slash logic upto the consumer of this pallet.
- While it does not expose any calls yet, it is written with the intent
of exposing these primitives via extrinsics.

## TODO
- [x] Improve unit tests in the pallet.
- [x] Separate slash reward perbill for rewarding the slash reporters?
- [x] Review if we should add more events.

---------

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>

This is for adding onfinality polkadot bootnode. Please correct me if
this is not the right place for adding a new bootnode

The storage item ExtrinsicIndex is already taken before the `finalize()`
in `note_finished_extrinsics()`, rendering it's always 0 in the log.
This commit fixes it by using the proper API for extrinsics count.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

#3166 made private
functions used in `spawn_tasks()` public but forgot to add them in
exported functions of the crate.

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

This bump of versions:

https://github.com/paritytech/polkadot-sdk/pull/4409/files#diff-13ee4b2252c9e516a0547f2891aa2105c3ca71c6d7a1e682c69be97998dfc87eR11936

reintroduced a dependency to proc-macro-crate 2.0.0 which is suffering
from: bkchr/proc-macro-crate#42 this, so bump
parity-scale-codec to a newer version to eliminate the bad
proc-macro-crate 2.0.0 dependency.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: command-bot <>

`ubuntu-latest` are free for public repo so it's better (and cheaper) to
use them for small jobs

Add RUNTIME_METADATA_HASH variable to
[`cache.extraEnv`](https://github.com/paritytech/forklift/tree/0.12.4?tab=readme-ov-file#cache)
in config.toml (setting available from forklift 0.12.4)

…ers (#4276)

Dear team, hello, 

Please consider this request to add bootnode endpoints to the chainspecs
of `coretime-westend` and `coretime-kusama`.

These bootnodes are operated by the Infrastructure Builders Programme as
part of the commitments of their members.

All these endpoints have been tested according to the procedure
indicated
[here](https://wiki.ibp.network/docs/consumers/bootnodes#testing-a-bootnode).

Additionally, please also consider some maintenance changes to
Gatotech's bootnode endpoint in several other chains.

Many thanks!!

Best regrds!

**_Milos_**

---------

Co-authored-by: hitchhooker <tommi@niemi.lol>
Co-authored-by: Bastian Köcher <info@kchr.de>

# Context

XCM docs are currently an md book hosted with github pages:
https://paritytech.github.io/xcm-docs/.
While that's fine, it's not in line with the work being done in the
polkadot-sdk docs.

# Main addition

This PR aims to fix that by bringing the docs back to this repo.
This does not have all the information currently present in the mdbook
xcm-docs but aims to be a good chunk of it and fully replace it over
time.

I also added the sections `guides` and `cookbook` which will be very
useful for users wanting to get into XCM.
For now I only added one example to the cookbook, but have ideas for
guides and more examples.
Having this docs be in rust docs is very useful for the cookbook.

# TODO

- [x] Use `FungibleAdapter`
- [x] Improve and relocate mock message queue
- [x] Fix license issue. Why does docs/sdk/ not have this problem? (Just
added the licenses)

# Next steps

- More examples in the cookbook
- End-to-end XCM guide with zombienet testing

---------

Co-authored-by: command-bot <>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

Fix after #4449

`cumulus-pallet-dmp-queue` is not needed anymore since
#1246.

The only logic that remains in the pallet is a lazy migration in the
[`on_idle`](https://github.com/paritytech/polkadot-sdk/blob/8d62c13b2541920c37fb9d9ca733fcce91e96573/cumulus/pallets/dmp-queue/src/lib.rs#L158)
hook.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

…ets (#4392)

As per #3326, removes pallet::getter usage from the bounties and
child-bounties pallets. The syntax `StorageItem::<T, I>::get()` should
be used instead.

Changes to one pallet involved changes in the other, so I figured it'd
be best to combine these two.

cc @muraca

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Re-applying #2302 after increasing the `MaxPageSize`.  

Remove `without_storage_info` from the XCMP queue pallet. Part of
#323

Changes:
- Limit the number of messages and signals a HRMP channel can have at
most.
- Limit the number of HRML channels.

A No-OP migration is put in place to ensure that all `BoundedVec`s still
decode and not truncate after upgrade. The storage version is thereby
bumped to 5 to have our tooling remind us to deploy that migration.

## Integration

If you see this error in your try-runtime-cli:  
```pre
Max message size for channel is too large. This means that the V5 migration can be front-run and an
attacker could place a large message just right before the migration to make other messages un-decodable.
Please either increase `MaxPageSize` or decrease the `max_message_size` for this channel. Channel max:
102400, MaxPageSize: 65535
```

Then increase the `MaxPageSize` of the `cumulus_pallet_xcmp_queue` to
something like this:
```rust
type MaxPageSize = ConstU32<{ 103 * 1024 }>;
```

There is currently no easy way for on-chain governance to adjust the
HRMP max message size of all channels, but it could be done:
#3145.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>

…rotocol (#3833)

This PR adds to the DHT only the peers that support the genesis/fork/kad
protocol.
Before this PR, any peer that supported the legacy `/kad/[id]` protocol
was added to the DHT.

This is the first step in removing the support for the legacy kad
protocols.

While I have adjusted unit tests to validate the appropriate behavior,
this still needs proper testing in our stack.

Part of #504.

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

The bridge relay is **not** using `tokio`, while `jsonrpsee` does. To
make it work together, we are spawning a separate tokio task for every
jsonrpsee subscription, which holds a subscription reference. It looks
like we are not stopping those tasks when we no longer need it and when
there are more than `1024` active subscriptions, `jsonrpsee` stops
opening new subscriptions. This PR adds an `cancel` signal that is sent
to the background task when we no longer need a subscription.

Demote `Ignored block announcement because all validation slots for this
peer are occupied.` message to debug level.

This is mostly an indicator of somebody spamming the node or (more
likely) some node actively keeping up with the network but not
recognizing it's in a major sync mode, so sending zillions of block
announcements (have seen this on Versi).

This warning shouldn't be considered an error by the end user, so let's
make it debug.

Ref. #1929.

As per #3326, removes usage of the pallet::getter macro from the
democracy pallet. The syntax `StorageItem::<T, I>::get()` should be used
instead.

cc @muraca

…c committee in next store period (#4478)

While syncing Ethereum consensus updates to the Snowbridge Ethereum
light client, the syncing process stalled due to error
`InvalidSyncCommitteeUpdate` when importing the next sync committee for
period `1087`.

This bug manifested specifically because our light client checkpoint is
a few weeks old (submitted to governance weeks ago) and had to catchup
until a recent block. Since then, we have done thorough testing of the
catchup sync process.

### Symptoms
- Import next sync committee for period `1086` (essentially period
`1087`). Light client store period = `1086`.
- Import header in period `1087`. Light client store period = `1087`.
The current and next sync committee is not updated, and is now in an
outdated state. (current sync committee = `1086` and current sync
committee = `1087`, where it should be current sync committee = `1087`
and current sync committee = `None`)
- Import next sync committee for period `1087` (essentially period
`1088`) fails because the expected next sync committee's roots don't
match.

### Bug
The bug here is that the current and next sync committee's didn't
handover when an update in the next period was received.

### Fix
There are two possible fixes here:
1. Correctly handover sync committees when a header in the next period
is received.
2. Reject updates in the next period until the next sync committee
period is known.

We opted for solution 2, which is more conservative and requires less
changes.

### Polkadot-sdk versions
This fix should be backported in polkadot-sdk versions 1.7 and up.

Snowfork PR: Snowfork#145

---------

Co-authored-by: Vincent Geddes <117534+vgeddes@users.noreply.github.com>

resolves #3315

---------

Co-authored-by: doordashcon <jessechejieh@doordashcon.local>
Co-authored-by: command-bot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

…s refactors

Using Dynamic Parameters for contracts seems like a bad idea for now.

Given that we have benchmarks for each host function (in addition to our
extrinsics), parameter storage reads will be counted multiple times. We
will work on updates to the benchmarking framework to mitigate this
issue in future iterations.

---------

Co-authored-by: command-bot <>

Before relayer crates have been moved + merged, the `MetricsParams` type
has been created from a `substrate-relay` crate (binary) and hence it
has been setting the `substrate_relay_build_info` metic value properly -
to the binary version. Now it is created from the
`substrate-relay-helper` crate, which has the fixed (it isn't published)
version `0.1.0`, so our relay provides incorrect metric value. This
'breaks' our monitoring tools - we see that all relayers have that
incorrect version, which is not cool.

The idea is to have a global static variable (shame on me) that is
initialized by the binary during initialization like we do with the
logger initialization already. Was considering some alternative options:
- adding a separate argument to every relayer subcommand and propagating
it to the `MetricsParams::new()` causes a lot of changes and introduces
even more noise to the binary code, which is supposed to be as small as
possible in the new design. But I could do that if team thinks it is
better;
- adding a `structopt(skip) pub relayer_version: RelayerVersion`
argument to all subcommand params won't work, because it will be
initialized by default and `RelayerVersion` needs to reside in some util
crate (not the binary), so it'll have the wrong value again.

…4490)

Co-authored-by: command-bot <>

Changes the Ethereum client storage scope to public, so it can be set in
a migration.

When merged, we should backport to the all other release branches:

- [ ] release-crates-io-v1.7.0 - patch release the fellows BridgeHubs
runtimes #4504
- [ ] release-crates-io-v1.8.0 -
#4505
- [ ] release-crates-io-v1.9.0 -
#4506
- [ ] release-crates-io-v1.10.0 -
#4507
- [ ] release-crates-io-v1.11.0 -
#4508
- [ ] release-crates-io-v1.12.0 (commit soon)

…ce on the pool account (#4503)

addresses #4440 (will
close once we have this in prod runtimes).
related: #2037.

An extra consumer reference is preventing pools to be destroyed. When a
pool is ready to be destroyed, we
can safely clear the consumer references if any. Notably, I only check
for one extra consumer reference since that is a known bug. Anything
more indicates possibly another issue and we probably don't want to
silently absorb those errors as well.

After this change, pools with extra consumer reference should be able to
destroy normally.

Co-authored-by: jimwfs <169986508+jimwfs@users.noreply.github.com>

…4512)

Replace usage of deprecated
`substrate_rpc_client::ChildStateApi::storage_keys` with
`substrate_rpc_client::ChildStateApi::storage_keys_paged`.

Required for successful scraping of Aleph Zero state.

)

As per #3326, removes pallet::getter macro usage from
pallet-fast-unstake. The syntax `StorageItem::<T, I>::get()` should be
used instead.

cc @muraca

---------

Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>

Implements #4429

Collators only need to maintain the implicit view for the paraid they
are collating on.
In this case, bypass prospective-parachains entirely. It's still useful
to use the GetMinimumRelayParents message from prospective-parachains
for validators, because the data is already present there.

This enables us to entirely remove the subsystem from collators, which
consumed resources needlessly

Aims to resolve #4167 

TODO:
- [x] fix unit tests

)

Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>

)

closes paritytech/parity-bridges-common#3000

Recently we've changed our bridge configuration for Rococo <> Westend
and our new relayer has started to submit transactions every ~ `30`
seconds. Eventually, it switches itself into limbo state, where it can't
submit more transactions - all `author_submitAndWatchExtrinsic` calls
are failing with the following error: `ERROR bridge Failed to send
transaction to BridgeHubRococo node: Call(ErrorObject { code:
ServerError(-32006), message: "Too many subscriptions on the
connection", data: Some(RawValue("Exceeded max limit of 1024")) })`.

Some links for those who want to explore:
- server side (node) has a strict limit on a number of active
subscriptions. It fails to open a new subscription if this limit is hit:
https://github.com/paritytech/jsonrpsee/blob/a4533966b997e83632509ad97eea010fc7c3efc0/server/src/middleware/rpc/layer/rpc_service.rs#L122-L132.
The limit is set to `1024` by default;
- internally this limit is a semaphore with `limit` permits:
https://github.com/paritytech/jsonrpsee/blob/a4533966b997e83632509ad97eea010fc7c3efc0/core/src/server/subscription.rs#L461-L485;
- semaphore permit is acquired in the first link;
- the permit is "returned" when the `SubscriptionSink` is dropped:
https://github.com/paritytech/jsonrpsee/blob/a4533966b997e83632509ad97eea010fc7c3efc0/core/src/server/subscription.rs#L310-L325;
- the `SubscriptionSink` is dropped when [this `polkadot-sdk`
function](https://github.com/paritytech/polkadot-sdk/blob/278486f9bf7db06c174203f098eec2f91839757a/substrate/client/rpc/src/utils.rs#L58-L94)
returns. In other words - when the connection is closed, the stream is
finished or internal subscription buffer limit is hit;
- the subscription has the internal buffer, so sending an item contains
of two steps: [reading an item from the underlying
stream](https://github.com/paritytech/polkadot-sdk/blob/278486f9bf7db06c174203f098eec2f91839757a/substrate/client/rpc/src/utils.rs#L125-L141)
and [sending it over the
connection](https://github.com/paritytech/polkadot-sdk/blob/278486f9bf7db06c174203f098eec2f91839757a/substrate/client/rpc/src/utils.rs#L111-L116);
- when the underlying stream is finished, the `inner_pipe_from_stream`
wants to ensure that all items are sent to the subscriber. So it: [waits
until the current send operation
completes](https://github.com/paritytech/polkadot-sdk/blob/278486f9bf7db06c174203f098eec2f91839757a/substrate/client/rpc/src/utils.rs#L146-L148)
and then [send all remaining items from the internal
buffer](https://github.com/paritytech/polkadot-sdk/blob/278486f9bf7db06c174203f098eec2f91839757a/substrate/client/rpc/src/utils.rs#L150-L155).
Once it is done, the function returns, the `SubscriptionSink` is
dropped, semaphore permit is dropped and we are ready to accept new
subscriptions;
- unfortunately, the code just calls the `pending_fut.await.is_err()` to
ensure that [the current send operation
completes](https://github.com/paritytech/polkadot-sdk/blob/278486f9bf7db06c174203f098eec2f91839757a/substrate/client/rpc/src/utils.rs#L146-L148).
But if there are no current send operation (which is normal), then the
`pending_fut` is set to terminated future and the `await` never
completes. Hence, no return from the function, no drop of
`SubscriptionSink`, no drop of semaphore permit, no new subscriptions
allowed (once number of susbcriptions hits the limit.

I've illustrated the issue with small test - you may ensure that if e.g.
the stream is initially empty, the
`subscription_is_dropped_when_stream_is_empty` will hang because
`pipe_from_stream` never exits.

…4465)

closes paritytech/parity-bridges-common#2963

See issue above for rationale
I've been thinking about adding similar calls to other pallets, but:
- for parachains pallet I haven't been able to think of a case when we
will need that given how long referendum takes. I.e. if storage proof
format changes and we want to unstuck the bridge, it'll take a large a
few weeks to sync a single parachain header, then another weeks for
another and etc.
- for messages pallet I've made the similar call initially, but it just
changes a storage key (`OutboundLanes` and/or `InboundLanes`), so
there's no any logic here and it may be simply done using
`system.set_storage`.

---------

Co-authored-by: command-bot <>

)

This PR introduces custom types / substrate wrappers for `Multiaddr`,
`multiaddr::Protocol`, `Multihash`, `ed25519::*` and supplementary types
like errors and iterators.

This is needed to unblock `libp2p` upgrade PR
#1631 after
#2944 was merged.
`libp2p` and `litep2p` currently depend on different versions of
`multiaddr` crate, and introduction of this "common ground" types is
needed to support independent version upgrades of `multiaddr` and
dependent crates in `libp2p` & `litep2p`.

While being just convenient to not tie versions of `libp2p` & `litep2p`
dependencies together, it's currently not even possible to keep `libp2p`
& `litep2p` dependencies updated to the same versions as `multiaddr` in
`libp2p` depends on `libp2p-identity` that we can't include as a
dependency of `litep2p`, which has it's own `PeerId` type. In the
future, to keep things updated on `litep2p` side, we will likely need to
fork `multiaddr` and make it use `litep2p` `PeerId` as a payload of
`/p2p/...` protocol.

With these changes, common code in substrate uses these custom types,
and `litep2p` & `libp2p` backends use corresponding libraries types.

This version includes the latest release of pjs/api
(https://github.com/polkadot-js/api/releases/tag/v11.1.1).
Thx!

Closes #4366

This PR backports version bumps and reorganisation of the prdoc files
from the `1.12.0` release branch back to `master`

We are actually using an older version, where the log line is in a
different file.

This implements the `CheckMetadataHash` extension as described in
[RFC78](https://polkadot-fellows.github.io/RFCs/approved/0078-merkleized-metadata.html).

Besides the signed extension, the `substrate-wasm-builder` is extended
to support generating the metadata-hash.

Closes: #291

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

..and other high level docs.


# Polling 

Please vote in the reactions of this PR 

- 👍 I agree to replace the website of this repo to
https://paritytech.github.io/polkadot-sdk/master/polkadot_sdk_docs/index.html
- 👎 Keep it as polkadot.network
- 🤷 Different opinion

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

Third and final PR in the set, closes
#454.

Original PR: #2680

## Precursors:
- #3889.
- #3904.

## Follow up issues/improvements
- #4404

Overall changes are documented here (lot more visual 😍):
https://hackmd.io/@ak0n/454-np-governance

## Summary of various roles 🤯
### Pallet Staking
**Nominator**: An account that directly stakes on `pallet-staking` and
nominates a set of validators.
**Stakers**: Common term for nominators and validators.
Virtual Stakers: Same as stakers, but they are keyless accounts and
their locks are managed by a pallet external to `pallet-staking`.

### Pallet Delegated Staking
**Agent**: An account that receives delegation from other accounts
(delegators) and stakes on their behalf. They are also Virtual Stakers
in `pallet-staking` where `pallet-delegated-staking` manages its locks.
**Delegator**: An account that delegates some funds to an agent.

### Pallet Nomination Pools
**Pool account**: Keyless account of a pool where funds are pooled.
Members pledge their funds towards the pools. These are going to become
`Agent` accounts in `pallet-delegated-staking`.
**Pool Members**: They are individual members of the pool who
contributed funds to it. They are also `Delegator` in
`pallet-delegated-staking`.

## Changes
### Multiple Stake strategies

**TransferStake**: The current nomination pool logic can be considered a
staking strategy where delegators transfer funds to pool and stake. In
this scenario, funds are locked in pool account, and users lose the
control of their funds.

**DelegateStake**: With this PR, we introduce a new staking strategy
where individual delegators delegate fund to pool. `Delegate` implies
funds are locked in delegator account itself. Important thing to note
is, pool does not have funds of its own, but it has authorization from
its members to use these funds for staking.

We extract out all the interaction of pool with staking interface into a
new trait `StakeStrategy`. This is the logic that varies between the
above two staking strategies. We use the trait `StakeStrategy` to
implement above two strategies: `TransferStake` and `DelegateStake`.

### NominationPool
Consumes an implementation of `StakeStrategy` instead of
`StakingInterface`. I have renamed it from `Staking` to `StakeAdapter`
to clarify the difference from the earlier used trait.

To enable delegation based staking in pool, Nomination pool can be
configured as:
```
type StakeAdapter = pallet_nomination_pools::adapter::DelegateStake<Self, DelegatedStaking>;
```

Note that with the following configuration, the changes in the PR are
no-op.
```
type StakeAdapter = pallet_nomination_pools::adapter::TransferStake<Self, Staking>;
```

## Deployment roadmap
Plan to enable this only in Westend. In production runtimes, we can keep
pool to use `TransferStake` which will be no functional change.

Once we have a full audit, we can enable this in Kusama followed by
Polkadot.

## TODO
- [x] Runtime level (Westend) migration for existing nomination pools.
- [x] Permissionless call/ pallet::tasks for claiming delegator funds.
- [x] Add/update benches.
- [x] Migration tests.
- [x] Storage flag to mark `DelegateStake` migration and integrity
checks to not allow `TransferStake` for migrated runtimes.

---------

Signed-off-by: Matteo Muraca <mmuraca247@gmail.com>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>
Signed-off-by: Adrian Catangiu <adrian@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: divdeploy <chenguangxue@outlook.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: hongkuang <liurenhong@outlook.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: gemini132 <164285545+gemini132@users.noreply.github.com>
Co-authored-by: Matteo Muraca <56828990+muraca@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Alexandru Gheorghe <49718502+alexggh@users.noreply.github.com>
Co-authored-by: Alessandro Siniscalchi <asiniscalchi@gmail.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
Co-authored-by: Ross Bulat <ross@parity.io>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: Sam Johnson <sam@durosoft.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com>
Co-authored-by: Clara van Staden <claravanstaden64@gmail.com>
Co-authored-by: Ron <yrong1997@gmail.com>
Co-authored-by: Vincent Geddes <vincent@snowfork.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Dino Pačandi <3002868+Dinonard@users.noreply.github.com>
Co-authored-by: Andrei Eres <eresav@me.com>
Co-authored-by: Alin Dima <alin@parity.io>
Co-authored-by: Andrei Sandu <andrei-mihail@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Lulu <morgan@parity.io>
Co-authored-by: Juan Girini <juangirini@gmail.com>
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com>
Co-authored-by: Dónal Murray <donal.murray@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kutsal Kaan Bilgin <kutsalbilgin@gmail.com>
Co-authored-by: Ermal Kaleci <ermalkaleci@gmail.com>
Co-authored-by: ordian <write@reusable.software>
Co-authored-by: divdeploy <166095818+divdeploy@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sergej Sakac <73715684+Szegoo@users.noreply.github.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: HongKuang <166261675+HongKuang@users.noreply.github.com>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io>
Co-authored-by: Egor_P <egor@parity.io>
Co-authored-by: Aaro Altonen <48052676+altonen@users.noreply.github.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Léa Narzis <78718413+lean-apple@users.noreply.github.com>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: georgepisaltu <52418509+georgepisaltu@users.noreply.github.com>
Co-authored-by: command-bot <>
Co-authored-by: PG Herveou <pgherveou@gmail.com>
Co-authored-by: jimwfs <wqq1479787@163.com>
Co-authored-by: jimwfs <169986508+jimwfs@users.noreply.github.com>
Co-authored-by: polka.dom <polkadotdom@gmail.com>

This one also works and it is easier.

Cherry-picked fix from upcoming
#4494

---------

Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>

fix #4163

This PR does the following:
Update to pallet-contracts-proc-macro: 
- Parse #[cfg] so we can add a dummy noop host function for benchmark.
- Generate BenchEnv::<host_fn> so we can call host functions directly in
the benchmark.
- Add the weight of the noop host function before calling the host
function itself

Update benchmarks:
- Update all host function benchmark, a host function benchmark now
simply call the host function, instead of invoking the function n times
from within a contract.
- Refactor RuntimeCosts & Schedule, for most host functions, we can now
use the generated weight function directly instead of computing the diff
with the cost! macro

```rust
// Before
#[benchmark(pov_mode = Measured)]
fn seal_input(r: Linear<0, API_BENCHMARK_RUNS>) {
    let code = WasmModule::<T>::from(ModuleDefinition {
        memory: Some(ImportedMemory::max::<T>()),
        imported_functions: vec![ImportedFunction {
            module: "seal0",
            name: "seal_input",
            params: vec![ValueType::I32, ValueType::I32],
            return_type: None,
        }],
        data_segments: vec![DataSegment { offset: 0, value: 0u32.to_le_bytes().to_vec() }],
        call_body: Some(body::repeated(
            r,
            &[
                Instruction::I32Const(4), // ptr where to store output
                Instruction::I32Const(0), // ptr to length
                Instruction::Call(0),
            ],
        )),
        ..Default::default()
    });

    call_builder!(func, code);

    let res;
    #[block]
    {
        res = func.call();
    }
    assert_eq!(res.did_revert(), false);
}
```

```rust
// After
fn seal_input(n: Linear<0, { code::max_pages::<T>() * 64 * 1024 - 4 }>) {
    let mut setup = CallSetup::<T>::default();
    let (mut ext, _) = setup.ext();
    let mut runtime = crate::wasm::Runtime::new(&mut ext, vec![42u8; n as usize]);
    let mut memory = memory!(n.to_le_bytes(), vec![0u8; n as usize],);
    let result;
    #[block]
    {
        result = BenchEnv::seal0_input(&mut runtime, &mut memory, 4, 0)
    }
    assert_ok!(result);
    assert_eq!(&memory[4..], &vec![42u8; n as usize]);
}
``` 

[Weights
compare](https://weights.tasty.limo/compare?unit=weight&ignore_errors=true&threshold=10&method=asymptotic&repo=polkadot-sdk&old=master&new=pg%2Frework-host-benchs&path_pattern=substrate%2Fframe%2Fcontracts%2Fsrc%2Fweights.rs%2Cpolkadot%2Fruntime%2F*%2Fsrc%2Fweights%2F**%2F*.rs%2Cpolkadot%2Fbridges%2Fmodules%2F*%2Fsrc%2Fweights.rs%2Ccumulus%2F**%2Fweights%2F*.rs%2Ccumulus%2F**%2Fweights%2Fxcm%2F*.rs%2Ccumulus%2F**%2Fsrc%2Fweights.rs)

---------

Co-authored-by: command-bot <>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>

Define `OpaqueValue` and use it instead of
`grandpa::OpaqueKeyOwnershipProof` and `beefy:OpaqueKeyOwnershipProof`

Related to
#4522 (comment)

We'll need to introduce a runtime API method that calls the
`report_fork_voting_unsigned()` extrinsic. This method will need to
receive the ancestry proof as a paramater. I'm still not sure, but there
is a chance that we'll send the ancestry proof as an opaque type.

So let's introduce this `OpaqueValue`. We can already use it to replace
`grandpa::OpaqueKeyOwnershipProof` and `beefy:OpaqueKeyOwnershipProof`
and maybe we'll need it for the ancestry proof as well.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

The XCM docs were pretty much moved to the new rust docs format in
#2633, with the addition
of the XCM cookbook, which I plan to add more examples to shortly.

These docs were not mentioned in the polkadot-sdk rust docs, this PR
just mentions them there, so people can actually find them.

We need to use the `polkadot-ckb-merkle-mountain-range` dependency
published on `crates.io` in order to unblock the release of the
`sp-mmr-primitives` crate

…{MockBlock, MockBlockU32, MockBlockU128}` (#4543)

While doing some migration/rebase I came in to the situation, where I
needed to change `mocking::MockBlock` to `mocking::MockBlockU32`:
```
#[derive_impl(frame_system::config_preludes::TestDefaultConfig)]
impl frame_system::Config for TestRuntime {
	type Block = frame_system::mocking::MockBlockU32<TestRuntime>;
	type AccountData = pallet_balances::AccountData<ThisChainBalance>;
}
```
But actual `TestDefaultConfig` for `frame_system` is using `ConstU64`
for `type BlockHashCount = frame_support::traits::ConstU64<10>;`
[here](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/system/src/lib.rs#L303).
Because of this, it force me to specify and add override for `type
BlockHashCount = ConstU32<10>`.

This PR tries to fix this with `TestBlockHashCount` implementation for
`TestDefaultConfig` which supports `u32`, `u64` and `u128` as a
`BlockNumber`.

### How to simulate error
Just by removing `type BlockHashCount = ConstU32<250>;`
[here](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/multisig/src/tests.rs#L44)
```
:~/parity/olkadot-sdk$ cargo test -p pallet-multisig
   Compiling pallet-multisig v28.0.0 (/home/bparity/parity/aaa/polkadot-sdk/substrate/frame/multisig)
error[E0277]: the trait bound `ConstU64<10>: frame_support::traits::Get<u32>` is not satisfied
   --> substrate/frame/multisig/src/tests.rs:41:1
    |
41  | #[derive_impl(frame_system::config_preludes::TestDefaultConfig)]
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `frame_support::traits::Get<u32>` is not implemented for `ConstU64<10>`
    |
    = help: the following other types implement trait `frame_support::traits::Get<T>`:
              <ConstU64<T> as frame_support::traits::Get<u64>>
              <ConstU64<T> as frame_support::traits::Get<std::option::Option<u64>>>
note: required by a bound in `frame_system::Config::BlockHashCount`
   --> /home/bparity/parity/aaa/polkadot-sdk/substrate/frame/system/src/lib.rs:535:24
    |
535 |         type BlockHashCount: Get<BlockNumberFor<Self>>;
    |                              ^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `Config::BlockHashCount`
    = note: this error originates in the attribute macro `derive_impl` which comes from the expansion of the macro `frame_support::macro_magic::forward_tokens_verbatim` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0277`.
error: could not compile `pallet-multisig` (lib test) due to 1 previous error 
```




## For reviewers:

(If there is a better solution, please let me know!)

The first commit contains actual attempt to fix the problem:
3c5499e.
The second commit is just removal of `BlockHashCount` from all other
places where not needed by default.

Closes: #1657

---------

Co-authored-by: Bastian Köcher <git@kchr.de>

@serban300 could you please do the same for the MMR crate? Am not sure
what commit was released since there are no release tags in the repo.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

# Umbrella Crate

The Polkadot-SDK "umbrella" is a crate that re-exports all other
published crates. This makes it
possible to have a very small `Cargo.toml` file that only has one
dependency, the umbrella
crate. This helps with selecting the right combination of crate
versions, since otherwise 3rd
party tools are needed to select a compatible set of versions.

## Features

The umbrella crate supports no-std builds and can therefore be used in
the runtime and node.
There are two main features: `runtime` and `node`. The `runtime` feature
enables all `no-std`
crates, while the `node` feature enables all `std` crates. It should be
used like any other
crate in the repo, with `default-features = false`.

For more fine-grained control, additionally, each crate can be enabled
selectively. The umbrella
exposes one feature per dependency. For example, if you only want to use
the `frame-support`
crate, you can enable the `frame-support` feature.

The umbrella exposes a few more general features:
- `tuples-96`: Needs to be enabled for runtimes that have more than 64
pallets.
- `serde`: Specifically enable `serde` en/decoding support.
- `experimental`: Experimental enable experimental features - should not
yet used in production.
- `with-tracing`: Enable tracing support.
- `try-runtime`, `runtime-benchmarks` and `std`: These follow the
standard conventions.
- `runtime`: As described above, enable all `no-std` crates.
- `node`: As described above, enable all `std` crates.
- There does *not* exist a dedicated docs feature. To generate docs,
enable the `runtime` and
`node` feature. For docs.rs the manifest contains specific configuration
to make it show up
  all re-exports.

There is a specific `zepter` check in place to ensure that the features
of the umbrella are
correctly configured. This check is run in CI and locally when running
`zepter`.

## Generation

The umbrella crate needs to be updated every time when a new crate is
added or removed from the
workspace. It is checked in CI by calling its generation script. The
generation script is
located in `./scripts/generate-umbrella.py` and needs dependency
`cargo_workspace`.

Example: `python3 scripts/generate-umbrella.py --sdk . --version 1.9.0`

## Usage

> Note: You can see a live example in the `staging-node-cli` and
`kitchensink-runtime` crates.

The umbrella crate can be added to your runtime crate like this:

`polkadot-sdk = { path = "../../../../umbrella", features = ["runtime"],
default-features =
false}`

or for a node:

`polkadot-sdk = { path = "../../../../umbrella", features = ["node"],
default-features = false
}`

In the code, it is then possible to bring all dependencies into scope
via:

`use polkadot_sdk::*;`

### Known Issues

The only known issue so far is the fact that the `use` statement brings
the dependencies only
into the outer module scope - not the global crate scope. For example,
the following code would
need to be adjusted:

```rust
use polkadot_sdk::*;

mod foo {
   // This does sadly not compile:
   frame_support::parameter_types! { }

   // Instead, we need to do this (or add an equivalent `use` statement):
   polkadot_sdk::frame_support::parameter_types! { }
}
```

Apart from this, no issues are known. There could be some bugs with how
macros locate their own
re-exports. Please compile issues that arise from using this crate.

## Dependencies

The umbrella crate re-exports all published crates, with a few
exceptions:
- Runtime crates like `rococo-runtime` etc are not exported. This
otherwise leads to very weird
  compile errors and should not be needed anyway.
- Example and fuzzing crates are not exported. This is currently
detected by checking the name
of the crate for these magic words. In the future, it will utilize
custom metadata, as it is
  done in the `rococo-runtime` crate.
- The umbrella crate itself. Should be obvious :)

## Follow Ups
- [ ] Re-writing the generator in Rust - the python script is at its
limit.
- [ ] Using custom metadata to exclude some crates instead of filtering
by names.
- [ ] Finding a way to setting the version properly. Currently its
locked in the CI script.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

… and 4MB for Kusama + testnets (#4399)

Doing this change ensures that we minimize the CPU usage we spend in
reed-solomon by only doing the re-encoding into chunks if PoV size is
less than 4MB (which means all PoVs right now)
 
Based on susbystem benchmark results we concluded that it is safe to
bump this number higher. At worst case scenario the network pressure for
a backing group of 5 is around 25% of the network bandwidth in hw specs.

Assuming 6s block times (max_candidate_depth 3) and needed_approvals 30
the amount of bandwidth usage of a backing group used would hover above
`30 * 4 * 3 = 360MB` per relay chain block. Given a backing group of 5
that gives 72MB per block per validator -> 12 MB/s.

<details>
<summary>Reality check on Kusama PoV sizes (click for chart)</summary>
<br>
<img width="697" alt="Screenshot 2024-05-07 at 14 30 38"
src="https://github.com/paritytech/polkadot-sdk/assets/54316454/bfed32d4-8623-48b0-9ec0-8b95dd2a9d8c">
</details>

---------

Signed-off-by: Andrei Sandu <andrei-mihail@parity.io>