This repository has been archived by the owner on Nov 15, 2023. It is now read-only.
Backing and collator protocol traces including para-id #2620
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rphmeier
added
A0-please_review
Co-authored-by: Andronik Ordian <write@reusable.software>
|
bot merge |
|
Waiting for commit status. |
ghost
merged commit
rphmeier
added a commit
that referenced
this pull request
Mar 14, 2021
* improve backing/provisioner spans * span for collation requests * add para_id to unbacked candidate spans * differentiate validation-construction and find-assignment in selection * better find-assignment spans * organize unbacked-candidate spans directly under job root * Update node/core/provisioner/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Andronik Ordian <write@reusable.software>
This was referenced Mar 15, 2021
ordian
added a commit
that referenced
this pull request
Mar 19, 2021
* improve backing/provisioner spans * span for collation requests * add para_id to unbacked candidate spans * differentiate validation-construction and find-assignment in selection * better find-assignment spans * organize unbacked-candidate spans directly under job root * Update node/core/provisioner/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Andronik Ordian <write@reusable.software>
ordian
added a commit
that referenced
this pull request
Mar 19, 2021
* master: Don't accept incoming connections for collators (#2644) Improve the logging (#2645) Update for the new substrate client API (#2570) integrate faster erasure code (#2608) Companion for #8372 (Replace 'Module' with 'Pallet' in construct_runtime macro) (#2629) Request based collation fetching (#2621) Companion for Substrate#8386 (#2634) Polkadot companion for Substrate PR #7640 (Store multiple Justifications per block) (#2358) yet another set of logging improvements (#2638) Reduce number of active leaves at startup (#2631) re benchmark (#2630) Fix wrong deposit amount in council voters. (#2562) Add /data symlink to Docker containers (#2627) Companion for sub/8176 (#2622) Remove TODO from substrate#2986 (#2628) update ring to 0.16.20 (#2626) New slots/auctions architecture (#2294) add tracing when no assignment in candidate selection (#2623) Backing and collator protocol traces including para-id (#2620) more diagnostic logs for approval-voting (#2618)
ordian
added a commit
that referenced
this pull request
Mar 24, 2021
* ci: initial fuzzer job * erasure-coding: update fuzzer Cargo.lock * syntax fix * try this first * install honggfuzz deps * try not * try if else * try SIGINT * ignore hfuzz dirs * ??? * bash is growing on me * decouple builds from running * fix a typo * try copying dirs * fix indentation * try using absolute paths * another try * caching is not worth it * remove outdated needs * cleanup and add futher TODOs * Update .github/workflows/honggfuzz.yml * more diagnostic logs for approval-voting (#2618) * Backing and collator protocol traces including para-id (#2620) * improve backing/provisioner spans * span for collation requests * add para_id to unbacked candidate spans * differentiate validation-construction and find-assignment in selection * better find-assignment spans * organize unbacked-candidate spans directly under job root * Update node/core/provisioner/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Andronik Ordian <write@reusable.software> * add tracing when no assignment in candidate selection (#2623) * New slots/auctions architecture (#2294) * TODOs * Add auctions.rs, comment on changes needed. * Remove cruft from slots * Remove more from auctions.rs * More logic drafting in slots. * More logic in slots.rs * patch some errors * more fixes * last nit * Cleanups in slots.rs * Cleanups in slots.rs * patches * make build * crowdloan to new api * auction compile * Use ParaId instead of FundIndex in Crowdloan (#2303) * use paraid instead of fundindex * Update crowdloan.rs * check caller is manager * Auction tests and fix build warnings. * Configurable origin for initiating auctions * Remove on_finalize * #2303 (manual merge) * Tests for Slots * some registrar tests * Apply suggestions from code review Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Update runtime/common/src/slots.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * Slots uses Registrar for CurrentChains * swap works test * on swap impl * traitify parachain cleanup * explicit lifecycle tracking for paras * initial implementation of lifecycles and upgrades * clean up a bit * Update runtime/common/src/slots.rs Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> * fix doc comment * more rigid lifecycle checks * include paras which are transitioning, and lifecycle query * format guide * update api * update guide * explicit outgoing state, fix genesis * handle outgoing with transitioning paras * Revert "explicit lifecycle tracking for paras" This reverts commit 4177af7. * remove lifecycle tracking from registrar * do not include transitioning paras in identifier * Update paras_registrar.rs * final patches to registrar * Fix test * use noop in test * clean up pending swap on deregistration * finish registrar tests * Update roadmap/implementers-guide/src/runtime/paras.md * Update roadmap/implementers-guide/src/runtime/paras.md * Update roadmap/implementers-guide/src/runtime/paras.md * Apply suggestions from code review * Use matches macro * Correct terms * Apply suggestions from code review * Remove direct need for Slots and Registrar from Crowdloan * Rejig things slightly * actions queue * Revert "actions queue" This reverts commit b2e9011. * Traitify Auction interface. * Mockups and initial code for Crowdloan testing * One test... * collapse onboarding state * fix some crowdloan tests * one more * start benchmarks for auctions * benchmark bid * fix more crowdloan tests * onboard and begin retirement no longer exist * Revert "onboard and begin retirement no longer exist" This reverts commit 2e100fd. * Simplify crowdloan and make it work. * Fixes * fix some * finish merge fixes * fix refund bug in auctions * Add traits to Registrar for tests and benchmarks * fix more auction benchmarks * Fix TestAuctioneer * finish crowdloan benchmarks * start setting up full integration tests * expand integration tests * finish basic integration test * add more integration tests * begin slots benchmarks * start paras registrar benchmarks * fix merge * fix tests * clean up paras registrar * remove println * remove outdated cleanup config * update benchmarks * Add WeightInfo * enable runtime-benchmarks feature flag * complete swap benchmark * add parachains and onboarding into westend * add benchmarks and genesis * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * fix benchmark execution * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * Use `new_raise_len` in crowdloan on_initialize * Update paras_registrar.rs * fix westend merge * impl on_swap for crowdloan * Check fund exists before create * update for crowdloan sig * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * slots on_initialize * use integration tests environment for benchmarks * fix hrmp event * auction on_initialize * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * fix storage name in auctions * add auction_index to winning data * winning data takes into account current auction index * remove println * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * Revert "add auction_index to winning data" * PastRandomness. * Fixes * Use new randomness * fix use of randomness in auctions and runtime config * expose consts * fix auction test * add deposit per byte for para registration * basic swap integration test * make swap test more comprehensive * Add WinningVec for easier retrieval in the front-end. * clean up `WinningVec` at the end * Add event for when a new best bid comes in * Fix propagation of winners in ending period * fix benchmarks, refund weight in dissolve * fix unused * remove some TODOs * setup opaque keys for paras in westend * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * remove unused * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * back to regular runtime config * use saturating math where user input can be * better first slot check * Update runtime/common/src/claims.rs * update westend onswap impl Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> * update ring to 0.16.20 (#2626) * Remove TODO from substrate#2986 (#2628) * Companion for sub/8176 (#2622) * Merge * Fixes * Fix build * remove dep. * undo dep. * upadte substrate * cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/ * Fix lock * revert lock; cargo update -p sp-io * from_rational_approx -> from_rational * Silence more warnings Co-authored-by: Gav Wood <gavin@parity.io> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> * Add /data symlink to Docker containers (#2627) * add /data symlink to Docker * fix comments * Fix wrong deposit amount in council voters. (#2562) * Fix wrong deposit amount in council voters. * Fix some build * make it all compile.. so far. * Fix * break build * Okay fix it again * re benchmark (#2630) * Change something * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/ * cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/ Co-authored-by: Parity Benchmarking Bot <admin@parity.io> * Reduce number of active leaves at startup (#2631) Currently we will take all leaves and give that to the overseer on startup, but this is a bad idea when the finality is lagging for example. There can be many of unfinalized leaves, we don't even need to look at anymore. To solve this, the pr adds a maximum of 4 leaves we forward to the overseer and the pr also checks that we only pass uncles of the best block. * yet another set of logging improvements (#2638) * Polkadot companion for Substrate PR #7640 (Store multiple Justifications per block) (#2358) * service: update for substrate PR #7640 * update substrate * Add Pallet Babe to Integration Tests Runtime Co-authored-by: André Silva <andrerfosilva@gmail.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> * Companion for Substrate#8386 (#2634) * Companion for Substrate#8386 paritytech/substrate#8386 * "Update Substrate" Co-authored-by: parity-processbot <> * Request based collation fetching (#2621) * Introduce collation fetching protocol also move to mod.rs * Allow `PeerId`s in requests to network bridge. * Fix availability distribution tests. * Move CompressedPoV to primitives. * Request based collator protocol: validator side - Missing: tests - Collator side - don't connect, if not connected * Fixes. * Basic request based collator side. * Minor fix on collator side. * Don't connect in requests in collation protocol. Also some cleanup. * Fix PoV distribution * Bump substrate * Add back metrics + whitespace fixes. * Add back missing spans. * More cleanup. * Guide update. * Fix tests * Handle results in tests. * Fix weird compilation issue. * Add missing ) * Get rid of dead code. * Get rid of redundant import. * Fix runtime build. * Cleanup. * Fix wasm build. * Format fixes. Thanks @andronik ! * Companion for #8372 (Replace 'Module' with 'Pallet' in construct_runtime macro) (#2629) * Replace 'Module' with 'Pallet'. * "Update Substrate" * fix babe usage * fix benchmark Co-authored-by: parity-processbot <> Co-authored-by: thiolliere <gui.thiolliere@gmail.com> * integrate faster erasure code (#2608) Breaks compatibility for distributing PoV and PersistentValidationData between validators. Ref #2442 * Update for the new substrate client API (#2570) * Update for the new substrate client API * Code review suggestions * Update substrate * Improve the logging (#2645) * Don't accept incoming connections for collators (#2644) * Don't accept incoming connections for collators on the `Collation` peer set. * Better docs. * fix reconstruct fuzzer name * make script more robust * REVERTME: test run * REVERTME: test run II * Revert "REVERTME: test run II" This reverts commit 58375df. * Revert "REVERTME: test run" This reverts commit 9759cb6. Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Gavin Wood <gavin@parity.io> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com> Co-authored-by: Parity Benchmarking Bot <admin@parity.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Martin Pugh <pugh@s3kr.it> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com> Co-authored-by: André Silva <andrerfosilva@gmail.com> Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com> Co-authored-by: Shaun Wang <spxwang@gmail.com> Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
This pull request was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is meant to help narrow down at what point in the code the 2-minute backing stalls are occurring. With this, we'll at least be able to see whether validators request collations, validate, and back candidates. It could be a scheduler issue, an issue with networking (discovery), or a runtime (inclusion) issue.