Update to Polkadot v0.9.29 #1041

NunoAlexandre · 2022-10-18T08:55:59Z

Description

Upgrading centrifuge-chain to the latest Polkadot version, v0.9.29

Fixes #976

Changes and Descriptions

This change also includes the upgrade to v0.9.28: https://github.com/paritytech/polkadot/releases/tag/v0.9.28

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
[] Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

cargo build --release
cargo test --workspace --release --features test-benchmarks,try-runtime,runtime-benchmarks

Checklist:

I have added Rust doc comments to structs, enums, traits and functions
I have made corresponding changes to the documentation
I have performed a self-review of my code
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged and published in downstream modules
I rebased on the latest parachain branch

runtime/altair/src/lib.rs

NunoAlexandre · 2022-10-18T13:38:19Z

I'm pulling my hair out for this getrandom issue 🤯 all I find online suggests adding an explicit dependency on that crate, like so:

I've tried that + adding default-features = false and add that crate to the /std list.

When I do that, then the error becomes (in short):

error: failed to run custom build command for `centrifuge-runtime v0.10.14 (/Users/n/centrifuge/centrifuge-chain/runtime/centrifuge)`

Caused by:
  process didn't exit successfully: `/Users/n/centrifuge/centrifuge-chain/target/debug/build/centrifuge-runtime-60c64d492688fc79/build-script-build` (exit status: 1)
  --- stdout
  Information that should be included in a bug report.
  Executing build command: "/Users/n/.rustup/toolchains/nightly-2022-05-09-aarch64-apple-darwin/bin/cargo" "rustc" "--target=wasm32-unknown-unknown" "--manifest-path=/Users/n/centrifuge/centrifuge-chain/target/debug/wbuild/centrifuge-runtime/Cargo.toml" "--color=always" "--profile" "release"
  Using rustc version: rustc 1.62.0-nightly (cb1219871 2022-05-08)


  --- stderr
     Compiling secp256k1-sys v0.6.1
  The following warnings were emitted during compilation:

  warning: error: unable to create target: 'No available targets are compatible with triple "wasm32-unknown-unknown"'
  warning: 1 error generated.
  warning: error: unable to create target: 'No available targets are compatible with triple "wasm32-unknown-unknown"'
  warning: 1 error generated.

  error: failed to run custom build command for `secp256k1-sys v0.6.1`

I was trying following the same logic and playing with the default-features and /std for secp256k1 but just spinning my wheels at this point.

@mikiquantum @branan do you maybe have some cue on how to tackle this one?

Edit: I have updated our internal repos that also depend on the parity repos with cargo update to ensure that all the repos are pointing to the same revisions of substrate/polkadot/cumulus but unfortunately that didn't cut it at this time.

Edit 2: interesting lead: interlay/interbtc#504 (comment)

pallets/collator-allowlist/Cargo.toml

pallets/crowdloan-claim/Cargo.toml

NunoAlexandre · 2022-10-19T19:00:09Z

Got it!

So get around the getrandom issue, I've added it as an explicit dependency with default-features = false and `features = ["js"]' to the root Cargo, and to each of the runtime's Cargo files. I may be wrong but when I just tried adding it to the root Cargo it didn't' seem to work but we should double check.

Past that point I bumped into the secp256k1-syserror reported here; On my machine, an Apple M1, I was able to get around that by installing zig (brew install zig) and retried building with CC="zig cc" cargo build --release. I then faced some substrate-relaed compilation errors, symtoms of some default-features not being set properly (it failed to compile some substrate crates because it didn't know vec![] etc).

At this point I built each runtime separately and realised that altair compiled fine but centrifuge and development failed. So I cherry picked which crates those two runtimes depended on that altair did not and found out that our pallet-claims had some substrate dependencies with default-features = true; setting those to false did the trick, and now we are back to getting the normal and expected update compilation errors 🎉

cc @gruberb

NunoAlexandre · 2022-10-20T07:35:05Z

@gruberb I've just cleaned up the changes I added for the getrandom fix. Let me know if I can help any further, otherwise leaving the rest up to you 👍

gruberb · 2022-10-20T07:47:06Z

runtime/centrifuge/src/lib.rs

@@ -1252,8 +1252,8 @@ impl_runtime_apis! {
 			let weight = Executive::try_runtime_upgrade().unwrap();
 			(weight, RuntimeBlockWeights::get().max_block)
 		}
-		fn execute_block_no_check(block: Block) -> Weight {
-			Executive::execute_block_no_check(block)
+		fn execute_block(block: Block, state_root_check: bool, try_state: TryStateSelect) -> Weight;


TODO: Have to figure out how this is actually used vs. and if we used execute_block_no_check in the past.

ASAIR the no-check prevents digest checks for block building. Do not think we can anything about the change and hopefully parity has this correctly ^^

gruberb · 2022-10-24T11:21:05Z

This PR is finally ready for review! I am currently trying to figure out why one loans benchmark test is failing, but the logic etc. can be reviewed!

mustermeiszer

Some questions and some feature flags that must be set true in dev. Furthermore, there are a few lines that need to be changed in the integrations-tests for instanciating a timestamp.

But damn, this was a lot of work! Thanks, @gruberb!!

libs/proofs/Cargo.toml

libs/traits/Cargo.toml

pallets/claims/Cargo.toml

pallets/collator-allowlist/Cargo.toml

pallets/crowdloan-reward/Cargo.toml

runtime/integration-tests/src/pools/utils/env.rs

scripts/frame-weight-template.hbs

scripts/runtime-weight-template.hbs

src/service.rs

wip

Let's commit this and see if the CI gets the same build error.

This moves the issue down to failed to run custom build command for `secp256k1-sys v0.6.1`. This GH issue suggests it may be an issue with M1 support: tomusdrw/rust-web3#543

Having it on the root Cargo.toml is enough :)

I just realized that we probably only needed this because we were importing dependencies from pallet-claims with the default-features enabled, which caused getrandom from being pulled with enabled std WHILE trying to build it for wasm, which caused the failing compilation error. Now that those dependencies are imported correctly with default-features = false, we in fact don't need this getrandom workaround anymore :)

* loans benchmarks: ensure generated rates match our rules * integration tests: Do not create duplicate NFT class I have no idea how either of thesee was working before ¯\_(ツ)_/¯

gruberb · 2022-10-25T10:18:23Z

All green now, and comments from @mustermeiszer resolved! Should we try to merge this soon? @branan @NunoAlexandre

mustermeiszer

@gruberb Just a small change needed. Then I will make this green ^^

runtime/integration-tests/src/pools/utils/env.rs

mustermeiszer

LGTM! Thanks for adapting.

NunoAlexandre force-pushed the polkadot-v0.9.29 branch from 151f160 to a74f58e Compare October 18, 2022 09:00

lemunozm reviewed Oct 18, 2022

View reviewed changes

runtime/altair/src/lib.rs Show resolved Hide resolved

NunoAlexandre mentioned this pull request Oct 18, 2022

Polkadot v0.9.29 #977

Closed

13 tasks

NunoAlexandre force-pushed the polkadot-v0.9.29 branch from a74f58e to 4449b14 Compare October 18, 2022 09:20

NunoAlexandre assigned gruberb and NunoAlexandre Oct 18, 2022

NunoAlexandre commented Oct 19, 2022

View reviewed changes

pallets/collator-allowlist/Cargo.toml Outdated Show resolved Hide resolved

NunoAlexandre commented Oct 19, 2022

View reviewed changes

pallets/crowdloan-claim/Cargo.toml Outdated Show resolved Hide resolved

NunoAlexandre force-pushed the polkadot-v0.9.29 branch from e74d77b to b5fcafb Compare October 19, 2022 17:52

gruberb reviewed Oct 20, 2022

View reviewed changes

gruberb force-pushed the polkadot-v0.9.29 branch from 1fd7b10 to f59d6e9 Compare October 21, 2022 11:36

gruberb marked this pull request as ready for review October 24, 2022 11:20

mustermeiszer requested changes Oct 24, 2022

View reviewed changes

gruberb and others added 13 commits October 25, 2022 09:13

Update to polkadot v0.9.29 (#1040)

6890c58

wip

pallet-pools: Enable default-features in dev-deps

69c611f

Align all serde depds to v1.0.119

a53f51a

cargo: update to fetch latest internal revisions

53fa863

Set all dev.dependencies' default-features=true

58315fa

try: getrandom = { version = "0.2", features = ["js"] }

8433aba

Let's commit this and see if the CI gets the same build error.

sort std

52d08a8

try: Add getrandom with default-features = false

c9b2675

This moves the issue down to failed to run custom build command for `secp256k1-sys v0.6.1`. This GH issue suggests it may be an issue with M1 support: tomusdrw/rust-web3#543

Fix wrong 'default-features = true'

eb51944

Sort benchmarks

4c70f54

Sorting

194276e

Fix weights in dev runtime

8d16698

Fix command.rs

d753fb1

NunoAlexandre and others added 22 commits October 25, 2022 09:13

Clean up explicit dependency on getrandom

5d2f0b8

Having it on the root Cargo.toml is enough :)

fmt

720cdbe

Adjusting mocks and runtimes

6e27019

Adjust to new execute_block function and replace Weights in mocks

135e0cc

flake.nix

63a8918

Trying to adjust to new try_execute_block

2508bde

Adding try-runtime to std

740da2a

Update Cargo.lock

62bae90

flake.nix

4168e2d

Remove logs

06ccb1e

Adding try-runtime

fef0f49

Adjust try-runtimes

9c23d43

Add try-runtimes

314b876

Use latest chainbridge

fe495f7

Update integration tests

f156d80

flake.nix

70061c6

Adjusting interest-accrual to new Weight type

077e180

Fix typo

b9a2645

Small fixes

ec18a66

* loans benchmarks: ensure generated rates match our rules * integration tests: Do not create duplicate NFT class I have no idea how either of thesee was working before ¯\_(ツ)_/¯

set dev-defaults to true, update fudge timers

1caad8d

cargo fmt

db83730

gruberb force-pushed the polkadot-v0.9.29 branch from 395f53c to db83730 Compare October 25, 2022 07:31

Update weight scripts

19ddafe

mustermeiszer requested changes Oct 25, 2022

View reviewed changes

runtime/integration-tests/src/pools/utils/env.rs Outdated Show resolved Hide resolved

runtime/integration-tests/src/pools/utils/env.rs Outdated Show resolved Hide resolved

Remove

9ee905d

mustermeiszer approved these changes Oct 25, 2022

View reviewed changes

gruberb merged commit 25fed2e into main Oct 25, 2022

NunoAlexandre deleted the polkadot-v0.9.29 branch November 14, 2022 09:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to Polkadot v0.9.29 #1041

Update to Polkadot v0.9.29 #1041

NunoAlexandre commented Oct 18, 2022 •

edited by gruberb

Loading

NunoAlexandre commented Oct 18, 2022 •

edited

Loading

NunoAlexandre commented Oct 19, 2022

NunoAlexandre commented Oct 20, 2022

gruberb Oct 20, 2022

mustermeiszer Oct 20, 2022

gruberb commented Oct 24, 2022

mustermeiszer left a comment

gruberb commented Oct 25, 2022

mustermeiszer left a comment •

edited

Loading

mustermeiszer left a comment

Update to Polkadot v0.9.29 #1041

Update to Polkadot v0.9.29 #1041

Conversation

NunoAlexandre commented Oct 18, 2022 • edited by gruberb Loading

Description

Changes and Descriptions

Type of change

How Has This Been Tested?

Checklist:

NunoAlexandre commented Oct 18, 2022 • edited Loading

NunoAlexandre commented Oct 19, 2022

NunoAlexandre commented Oct 20, 2022

gruberb Oct 20, 2022

Choose a reason for hiding this comment

mustermeiszer Oct 20, 2022

Choose a reason for hiding this comment

gruberb commented Oct 24, 2022

mustermeiszer left a comment

Choose a reason for hiding this comment

gruberb commented Oct 25, 2022

mustermeiszer left a comment • edited Loading

Choose a reason for hiding this comment

mustermeiszer left a comment

Choose a reason for hiding this comment

NunoAlexandre commented Oct 18, 2022 •

edited by gruberb

Loading

NunoAlexandre commented Oct 18, 2022 •

edited

Loading

mustermeiszer left a comment •

edited

Loading