Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Malus: add disputed block percentage #6100

Merged
merged 52 commits into from
Oct 13, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
ed18805
Malus: add disputed block percentage
bredamatt Oct 4, 2022
1eb3844
Cleanup tests
bredamatt Oct 4, 2022
f46e97c
* Replace unwrap with expect and meaningful error message
bredamatt Oct 4, 2022
99bc912
* Remove Inner
bredamatt Oct 4, 2022
a259272
* Rename sampled variable
bredamatt Oct 6, 2022
f371e5e
* Add percentage option to dispute_ancestor
bredamatt Oct 6, 2022
f3dca67
* Support static probability for `ReplaceValidationResult` proxy
bredamatt Oct 7, 2022
d08d405
* Add `--percentage` to `back-garbage-candidate` variant
bredamatt Oct 7, 2022
e9a1b3e
* Add probabilistic behavior to `dispute-ancestor` variant
bredamatt Oct 7, 2022
4fff00b
* More descriptive comments
bredamatt Oct 7, 2022
bfe0aa7
* cargo +nightly fmt --all
bredamatt Oct 7, 2022
4704e98
* Move Bernoulli distributrion to ReplaceValidationResult constructor
bredamatt Oct 7, 2022
0ec9bc1
* Remove dangling comment
bredamatt Oct 7, 2022
563fa17
* Consistent log
bredamatt Oct 7, 2022
f789958
* Add logs based on sampled value
bredamatt Oct 7, 2022
decb9e2
* Cargo +nightly fmt --all
bredamatt Oct 7, 2022
3ad5cec
* Remove unused percentage attributed after moving Bernoulli to const…
bredamatt Oct 10, 2022
5001fa5
Squashed commit of the following:
bredamatt Oct 10, 2022
b6e446e
Revert "Squashed commit of the following:"
bredamatt Oct 10, 2022
0a4d2b9
Companion for BEEFY: Simplify hashing for pallet-beefy-mmr (#6098)
serban300 Oct 4, 2022
dffb976
Keep sessions in window for the full unfinalized chain (#6054)
sandreim Oct 4, 2022
89d1f31
Bump lru from 0.7.8 to 0.8.0 (#6060)
dependabot[bot] Oct 4, 2022
b012597
Batch vote import in dispute-distribution (#5894)
eskimor Oct 4, 2022
2fd74d9
Add unknown words (#6105)
eskimor Oct 4, 2022
7e89bab
Buffered connection management for collator-protocol (#6022)
slumber Oct 5, 2022
c169279
Properly migrate weights to v2 (#6091)
KiChjang Oct 5, 2022
233c158
Pass through `runtime-benchmark` feature (#6110)
athei Oct 5, 2022
1655839
Companion for #11649: Bound uses of `Call` (#5729)
gavofyork Oct 5, 2022
e2a977a
update kvdb & co (#6111)
ordian Oct 5, 2022
4a6cf48
Skip `unexpected metric type`
bkontur Oct 6, 2022
78b8294
service: use MmrRootProvider as custom BEEFY payload provider (compan…
acatangiu Oct 6, 2022
82f7ad5
Maximum value for `MultiplierUpdate` (#6021)
Szegoo Oct 6, 2022
978d87f
Companion for upgrading pin-project (#6118)
bkchr Oct 7, 2022
0717246
Companion for 12109 (#5929)
Lezek123 Oct 9, 2022
1e96dfd
Add event to asset claim (#6029)
girazoki Oct 10, 2022
857e635
Fix flaky test (#6131)
slumber Oct 10, 2022
96185c3
ci/guide: install mdbook-graphviz (#6119)
ordian Oct 10, 2022
612302c
Revert "Squashed commit of the following:"
bredamatt Oct 10, 2022
35af7ad
Merge branch 'master' of github.com:paritytech/polkadot into bredamat…
bredamatt Oct 10, 2022
cf3f453
* Remove unused imports
bredamatt Oct 10, 2022
f1440e7
* cargo +nightly fmt --all
bredamatt Oct 10, 2022
6f54587
Make tweaks based on PR comments
bredamatt Oct 11, 2022
9842ee9
unit test related to gum formatting
bredamatt Oct 11, 2022
56347c6
cargo +nightly fmt --all
bredamatt Oct 11, 2022
4a7dd91
Merge branch 'master' of github.com:paritytech/polkadot into bredamat…
bredamatt Oct 12, 2022
72591a9
Resolve merge conflicts
bredamatt Oct 12, 2022
23d9498
cargo +nightly fmt --all
bredamatt Oct 12, 2022
be1f5a9
Fix tests so they use cli rather than cmd
bredamatt Oct 12, 2022
cdc68c4
CI unused import check fix
bredamatt Oct 12, 2022
dd96c47
Move info! log to startup
bredamatt Oct 12, 2022
5c20d8c
make info log more comprehensible
bredamatt Oct 12, 2022
7529e94
Merge branch 'master' of github.com:paritytech/polkadot into bredamat…
bredamatt Oct 13, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Bump lru from 0.7.8 to 0.8.0 (#6060)
* Bump lru from 0.7.8 to 0.8.0

Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.7.8 to 0.8.0.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](jeromefroe/lru-rs@0.7.8...0.8.0)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Change `LruCache` paramerter to `NonZeroUsize`

* Change type of `session_cache_lru_size` to `NonZeroUsize`

* Add expects instead of unwrap

Co-authored-by: Bastian Köcher <info@kchr.de>

* Use match to get rid of expects

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
  • Loading branch information
3 people authored and bredamatt committed Oct 10, 2022
commit 89d1f3164d2fd74b2daa76822f6347be2c1f358a
18 changes: 9 additions & 9 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion node/core/approval-voting/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ futures-timer = "3.0.2"
parity-scale-codec = { version = "3.1.5", default-features = false, features = ["bit-vec", "derive"] }
gum = { package = "tracing-gum", path = "../../gum" }
bitvec = { version = "1.0.0", default-features = false, features = ["alloc"] }
lru = "0.7"
lru = "0.8"
merlin = "2.0"
schnorrkel = "0.9.1"
kvdb = "0.11.0"
Expand Down
7 changes: 6 additions & 1 deletion node/core/approval-voting/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ use std::{
collections::{
btree_map::Entry as BTMEntry, hash_map::Entry as HMEntry, BTreeMap, HashMap, HashSet,
},
num::NonZeroUsize,
sync::Arc,
time::Duration,
};
Expand Down Expand Up @@ -104,7 +105,11 @@ const APPROVAL_CHECKING_TIMEOUT: Duration = Duration::from_secs(120);
/// Value rather arbitrarily: Should not be hit in practice, it exists to more easily diagnose dead
/// lock issues for example.
const WAIT_FOR_SIGS_TIMEOUT: Duration = Duration::from_millis(500);
const APPROVAL_CACHE_SIZE: usize = 1024;
const APPROVAL_CACHE_SIZE: NonZeroUsize = match NonZeroUsize::new(1024) {
Some(cap) => cap,
None => panic!("Approval cache size must be non-zero."),
};

const TICK_TOO_FAR_IN_FUTURE: Tick = 20; // 10 seconds.
const APPROVAL_DELAY: Tick = 2;
const LOG_TARGET: &str = "parachain::approval-voting";
Expand Down
2 changes: 1 addition & 1 deletion node/core/dispute-coordinator/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ gum = { package = "tracing-gum", path = "../../gum" }
parity-scale-codec = "3.1.5"
kvdb = "0.11.0"
thiserror = "1.0.31"
lru = "0.7.7"
lru = "0.8.0"
fatality = "0.0.6"

polkadot-primitives = { path = "../../../primitives" }
Expand Down
10 changes: 8 additions & 2 deletions node/core/dispute-coordinator/src/scraping/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,10 @@
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.

use std::collections::{BTreeMap, HashSet};
use std::{
collections::{BTreeMap, HashSet},
num::NonZeroUsize,
};

use futures::channel::oneshot;
use lru::LruCache;
Expand Down Expand Up @@ -44,7 +47,10 @@ mod tests;
/// `last_observed_blocks` LRU. This means, this value should the very least be as large as the
/// number of expected forks for keeping chain scraping efficient. Making the LRU much larger than
/// that has very limited use.
const LRU_OBSERVED_BLOCKS_CAPACITY: usize = 20;
const LRU_OBSERVED_BLOCKS_CAPACITY: NonZeroUsize = match NonZeroUsize::new(20) {
Some(cap) => cap,
None => panic!("Observed blocks cache size must be non-zero"),
};

/// Chain scraper
///
Expand Down
2 changes: 1 addition & 1 deletion node/network/availability-distribution/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ sp-keystore = { git = "https://github.com/paritytech/substrate", branch = "maste
thiserror = "1.0.31"
rand = "0.8.5"
derive_more = "0.99.17"
lru = "0.7.7"
lru = "0.8.0"
fatality = "0.0.6"

[dev-dependencies]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.

use std::collections::HashSet;
use std::{collections::HashSet, num::NonZeroUsize};

use lru::LruCache;
use rand::{seq::SliceRandom, thread_rng};
Expand Down Expand Up @@ -85,7 +85,7 @@ impl SessionCache {
pub fn new() -> Self {
SessionCache {
// We need to cache the current and the last session the most:
session_info_cache: LruCache::new(2),
session_info_cache: LruCache::new(NonZeroUsize::new(2).unwrap()),
}
}

Expand Down
2 changes: 1 addition & 1 deletion node/network/availability-recovery/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ edition = "2021"

[dependencies]
futures = "0.3.21"
lru = "0.7.7"
lru = "0.8.0"
rand = "0.8.5"
fatality = "0.0.6"
thiserror = "1.0.31"
Expand Down
6 changes: 5 additions & 1 deletion node/network/availability-recovery/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

use std::{
collections::{HashMap, VecDeque},
num::NonZeroUsize,
pin::Pin,
time::Duration,
};
Expand Down Expand Up @@ -77,7 +78,10 @@ const LOG_TARGET: &str = "parachain::availability-recovery";
const N_PARALLEL: usize = 50;

// Size of the LRU cache where we keep recovered data.
const LRU_SIZE: usize = 16;
const LRU_SIZE: NonZeroUsize = match NonZeroUsize::new(16) {
Some(cap) => cap,
None => panic!("Availability-recovery cache size must be non-zero."),
};

const COST_INVALID_REQUEST: Rep = Rep::CostMajor("Peer sent unparsable request");

Expand Down
2 changes: 1 addition & 1 deletion node/network/dispute-distribution/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ sp-application-crypto = { git = "https://github.com/paritytech/substrate", branc
sp-keystore = { git = "https://github.com/paritytech/substrate", branch = "master" }
thiserror = "1.0.31"
fatality = "0.0.6"
lru = "0.7.7"
lru = "0.8.0"

[dev-dependencies]
async-trait = "0.1.57"
Expand Down
5 changes: 4 additions & 1 deletion node/network/dispute-distribution/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
//! The sender is responsible for getting our vote out, see [`sender`]. The receiver handles
//! incoming [`DisputeRequest`]s and offers spam protection, see [`receiver`].

use std::num::NonZeroUsize;

use futures::{channel::mpsc, FutureExt, StreamExt, TryFutureExt};

use polkadot_node_network_protocol::authority_discovery::AuthorityDiscovery;
Expand Down Expand Up @@ -145,7 +147,8 @@ where
) -> Self {
let runtime = RuntimeInfo::new_with_config(runtime::Config {
keystore: Some(keystore),
session_cache_lru_size: DISPUTE_WINDOW.get() as usize,
session_cache_lru_size: NonZeroUsize::new(DISPUTE_WINDOW.get() as usize)
.expect("Dispute window can not be 0; qed"),
});
let (tx, sender_rx) = mpsc::channel(1);
let disputes_sender = DisputeSender::new(tx, metrics.clone());
Expand Down
13 changes: 10 additions & 3 deletions node/network/dispute-distribution/src/receiver/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

use std::{
collections::HashSet,
num::NonZeroUsize,
pin::Pin,
task::{Context, Poll},
};
Expand Down Expand Up @@ -61,6 +62,11 @@ const COST_NOT_A_VALIDATOR: Rep = Rep::CostMajor("Reporting peer was not a valid
/// How many statement imports we want to issue in parallel:
pub const MAX_PARALLEL_IMPORTS: usize = 10;

const BANNED_PEERS_CACHE_SIZE: NonZeroUsize = match NonZeroUsize::new(MAX_PARALLEL_IMPORTS) {
Some(cap) => cap,
None => panic!("Banned peers cache size should not be 0."),
};

/// State for handling incoming `DisputeRequest` messages.
///
/// This is supposed to run as its own task in order to easily impose back pressure on the incoming
Expand Down Expand Up @@ -146,7 +152,8 @@ where
) -> Self {
let runtime = RuntimeInfo::new_with_config(runtime::Config {
keystore: None,
session_cache_lru_size: DISPUTE_WINDOW.get() as usize,
session_cache_lru_size: NonZeroUsize::new(DISPUTE_WINDOW.get() as usize)
.expect("Dispute window can not be 0; qed"),
});
Self {
runtime,
Expand All @@ -156,7 +163,7 @@ where
pending_imports: PendingImports::new(),
// Size of MAX_PARALLEL_IMPORTS ensures we are going to immediately get rid of any
// malicious requests still pending in the incoming queue.
banned_peers: LruCache::new(MAX_PARALLEL_IMPORTS),
banned_peers: LruCache::new(BANNED_PEERS_CACHE_SIZE),
metrics,
}
}
Expand Down Expand Up @@ -222,7 +229,7 @@ where
}

// Wait for a free slot:
if self.pending_imports.len() >= MAX_PARALLEL_IMPORTS as usize {
if self.pending_imports.len() >= MAX_PARALLEL_IMPORTS {
// Wait for one to finish:
let r = self.pending_imports.next().await;
self.ban_bad_peer(r.expect("pending_imports.len() is greater 0. qed."))?;
Expand Down
2 changes: 1 addition & 1 deletion node/overseer/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ polkadot-node-metrics = { path = "../metrics" }
polkadot-primitives = { path = "../../primitives" }
orchestra = "0.0.2"
gum = { package = "tracing-gum", path = "../gum" }
lru = "0.7"
lru = "0.8"
parity-util-mem = { version = "0.11.0", default-features = false }
sp-core = { git = "https://github.com/paritytech/substrate", branch = "master" }
async-trait = "0.1.57"
Expand Down
6 changes: 5 additions & 1 deletion node/overseer/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@
use std::{
collections::{hash_map, HashMap},
fmt::{self, Debug},
num::NonZeroUsize,
pin::Pin,
sync::Arc,
time::Duration,
Expand Down Expand Up @@ -112,7 +113,10 @@ pub use orchestra::{

/// Store 2 days worth of blocks, not accounting for forks,
/// in the LRU cache. Assumes a 6-second block time.
pub const KNOWN_LEAVES_CACHE_SIZE: usize = 2 * 24 * 3600 / 6;
pub const KNOWN_LEAVES_CACHE_SIZE: NonZeroUsize = match NonZeroUsize::new(2 * 24 * 3600 / 6) {
Some(cap) => cap,
None => panic!("Known leaves cache size must be non-zero"),
};

#[cfg(test)]
mod tests;
Expand Down
2 changes: 1 addition & 1 deletion node/service/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ kvdb = "0.11.0"
kvdb-rocksdb = { version = "0.15.2", optional = true }
parity-db = { version = "0.3.16", optional = true }
async-trait = "0.1.57"
lru = "0.7"
lru = "0.8"

# Polkadot
polkadot-node-core-parachains-inherent = { path = "../core/parachains-inherent" }
Expand Down
2 changes: 1 addition & 1 deletion node/subsystem-util/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ thiserror = "1.0.31"
fatality = "0.0.6"
gum = { package = "tracing-gum", path = "../gum" }
derive_more = "0.99.17"
lru = "0.7.7"
lru = "0.8.0"

polkadot-node-subsystem = {path = "../subsystem" }
polkadot-node-jaeger = { path = "../jaeger" }
Expand Down
11 changes: 7 additions & 4 deletions node/subsystem-util/src/runtime/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

//! Convenient interface to runtime information.

use std::cmp::max;
use std::num::NonZeroUsize;

use lru::LruCache;

Expand Down Expand Up @@ -52,7 +52,7 @@ pub struct Config {
pub keystore: Option<SyncCryptoStorePtr>,

/// How many sessions should we keep in the cache?
pub session_cache_lru_size: usize,
pub session_cache_lru_size: NonZeroUsize,
}

/// Caching of session info.
Expand Down Expand Up @@ -95,7 +95,7 @@ impl Default for Config {
Self {
keystore: None,
// Usually we need to cache the current and the last session.
session_cache_lru_size: 2,
session_cache_lru_size: NonZeroUsize::new(2).expect("2 is larger than 0; qed"),
}
}
}
Expand All @@ -109,7 +109,10 @@ impl RuntimeInfo {
/// Create with more elaborate configuration options.
pub fn new_with_config(cfg: Config) -> Self {
Self {
session_index_cache: LruCache::new(max(10, cfg.session_cache_lru_size)),
session_index_cache: LruCache::new(
cfg.session_cache_lru_size
.max(NonZeroUsize::new(10).expect("10 is larger than 0; qed")),
),
session_info_cache: LruCache::new(cfg.session_cache_lru_size),
keystore: cfg.keystore,
}
Expand Down