feat: ⏫ upgrade to Polkadot SDK stable2407

[TDemeco]

v1.14.0 to stable2407 - Polkadot SDK Upgrade

• Changelog for Node Dev:
  • #4848 - Optimized logic for gossiping assignments, reducing cpu usage by 15% to 20%
  • #4922 - Optimized finalization performance with a quicker, more efficient, less complex version.
  • #4755 - Send PeerViewChange message with high priority, updates orchestra to 0.4.0. This improves propagation times of gossiped messages.
  • #4937 - prospective-parachains rework, again
    • Note: the change of removing support for parachain forks was reverted, since there were valid scenarios where a parachain wanted to fork.
  • #5169 & #5367 - Bump serde_json from 1.0.120 to 1.0.121
    • Note: bumped in StorageHub as well.
  • #5393 - Add feature (full-pov-size) to allow Aura collators to use the full PoV size instead of half of it
    • Note: no changes needed from our side in the code, but we could experiment enabling this.
• Changelog for Runtime Dev:
  • #5113 - Make the candidate relay parent progression check more strict for elastic scaling.
    • Note: already backported to v1.14.0 so it’s included in the previous update.
  • #4863 - Make tracing::log work in the runtime
  • #4777 - XCM builder pattern allows clear_origin before buy_execution.
    • Note: I did not use the builder pattern in tests since it wasn’t allowed to do this (which by all XCM standards was supossed to be allowed), so XCM’s from sh-xcm-simulator are built from vectors. I don’t think it’s worth to refactor them since it’s logically equivalent.
  • #5581 - Clear the message queue before executing a dry-run of an XCM, so the only messages left are the ones the users of the API actually care about.
    • Note: implemented for sh-xcm-simulator.
  • #5644 - Improve weight annotations, preventing allocations when calculating the weights.
  • #5466 - frame-omni-bencher patch:
    • Note: main thing for us is the addition of a README with rust docs for usage.
  • #5580 - Improve error message for pallet macro generated code, for calls, to ensure they return DispatchResult or DispatchResultWithPostInfo
• Changelog for Node Operator:
  • #4097 - Introduce basic slot-based collator for elastic scaling. This is a new experimental collator variant that should only be used for testing and exploration.
    • Note: the differences with the lookahead collator are:
      • The new variant is slot-based (it waits for the next slot of the parachain to start authoring)
      • It uses the current best relay parent as anchor
      • It tends to be anchored to one relay parent earlier. While lookahead generally waits for a new relay block to arrive before it attemps to build a block, this new slot-based one starts authoring directly on the slot boundary, where a new relay chain block has probably not yet arrived.
  • #5040 - Update libp2p-websocket to v0.42.2.
    • Note: This was backported to v1.14.0 so it was already included in the previous update.
  • #4663 - Add elastic scaling MVP guide, available here
    • Note: since this is quite experimental it doesn’t make sense to implement it yet, but it’s nice to have a reference to the guide we will probably use in the future.

Addition

This PR also includes a TON of fixes for the integration tests. It seems everything is working correctly now, BUT a few errors here and there are expected because of the nature of the tests and the limitations that arise with using GitHub workers.

[snowmead]

Looks good, just a couple of comments

[ffarall]

Amazing job!