Deterministic genesis hash for `--dev` runtime

[ggwpez]

When I start a substrate node in --dev node, it sometimes has different genesis hashes.
It does not happen on every start but sometimes after building? I could not really track to what it is related.

It would be nice to have deterministic genesis hashes such that import/export-block can be used for testing and in CI.
Either by fixing it or using a --seed flag.

@athei said it could be because of randomized staking nominations.

[athei]

I think it happens on every start. The chain spec is built at runtime. This is the code I was talking about:

substrate/bin/node/cli/src/chain_spec.rs

Lines 274 to 291 in 41567c3

	// stakers: all validators and nominators.
	let mut rng = rand::thread_rng();
	let stakers = initial_authorities
	.iter()
	.map(|x| (x.0.clone(), x.1.clone(), STASH, StakerStatus::Validator))
	.chain(initial_nominators.iter().map(|x| {
	use rand::{seq::SliceRandom, Rng};
	let limit = (MaxNominations::get() as usize).min(initial_authorities.len());
	let count = rng.gen::<usize>() % limit;
	let nominations = initial_authorities
	.as_slice()
	.choose_multiple(&mut rng, count)
	.into_iter()
	.map(|choice| choice.0.clone())
	.collect::<Vec<_>>();
	(x.clone(), x.clone(), STASH, StakerStatus::Nominator(nominations))
	}))
	.collect::<Vec<_>>();

[shawntabrizi]

@kianenigma @emostov can you use some seeded randomness or something else here that is deterministic?

[athei]

contracts-ui relies the fact that every re-start of a --dev chain yields a different genesis hash in order to determine whether this is a new instance of a chain.

@statictype

[bkchr]

contracts-ui relies the fact that every re-start of a --dev chain yields a different genesis hash in order to determine whether this is a new instance of a chain.

@statictype

This doesn't sounds like a good idea ;)

[kianenigma]

Both development and staging testnets are actually passing vec![] to initial_nominators, so I am not sure if this is the issue. Nonetheless, I added this for my election-testing shenanigens, and I have been doing those in a separate repo now for sometime, so @ggwpez if this is an annoyance, feel free to remove the whole randomness.

@@ -272,20 +272,12 @@ pub fn testnet_genesis(
                });

        // stakers: all validators and nominators.
-       let mut rng = rand::thread_rng();
        let stakers = initial_authorities
                .iter()
                .map(|x| (x.0.clone(), x.1.clone(), STASH, StakerStatus::Validator))
                .chain(initial_nominators.iter().map(|x| {
-                       use rand::{seq::SliceRandom, Rng};
-                       let limit = (MaxNominations::get() as usize).min(initial_authorities.len());
-                       let count = rng.gen::<usize>() % limit;
-                       let nominations = initial_authorities
-                               .as_slice()
-                               .choose_multiple(&mut rng, count)
-                               .into_iter()
-                               .map(|choice| choice.0.clone())
-                               .collect::<Vec<_>>();
+                       let nominations = initial_authorities.iter().take(MaxNominations::get() as usize).cloned().collect::<Vec<_>>();
                        (x.clone(), x.clone(), STASH, StakerStatus::Nominator(nominations))
                }))
                .collect::<Vec<_>>();

[statictype]

Contracts UI stores information about uploaded contracts in a local db, so when the chain is restarted we want to purge the db. We were using the genesis hash to check if there was a restart. Is there any value we can use that is unique to the chain instance? Worst case scenario we let the users deal with deleting contracts that are not on-chain anymore, but doing it automatically is much nicer UX

[xlc]

just use hash of block 1 instead of block 0?

[statictype]

just use hash of block 1 instead of block 0?

we were doing that before substrate-contracts-node started authoring blocks on demand. now blocks are created only when a transaction is submitted

[ggwpez]

Is there any value we can use that is unique to the chain instance?

We can keep the old behaviour with something like --seed random.

[statictype]

that could be interesting for other reasons, but there is no guarantee people will use it and the UI needs to handle all cases

[athei]

contracts-ui relies the fact that every re-start of a --dev chain yields a different genesis hash in order to determine whether this is a new instance of a chain.
@statictype

This doesn't sounds like a good idea ;)

I think --dev chains should actually have different genesis hashes on each startup. In contrast to a named chain spec they are supposed to be restarted regularly and hence represent different instances of a chain each time. Same for every local chain.

Where I agree with you is that relying on this implementation detail (staking nomination randomization) is bad. We should make it explicit. Just include some explicit randomness in the genesis for dev and local chains.

Then we add a --deterministic-genesis flag to resolve this issue.