tx/metrics: Add metrics for the RPC v2 transactionWatch_v1_submitAndWatch

[lexnv]

This PR adds metrics for the following RPC subscription: transactionWatch_v1_submitAndWatch

Metrics are exposed in two ways:

• simple counters of how many events we've seen globally
• a histogram vector of execution times, which is labeled by initial event -> final event
  • This helps us identify how long it takes the transaction pool to advance the state of the events, and further debug issues

Part of: #8336

(outdated) PoC Dashboards

Next steps

• initial dashboards with a live node
• adjust testing

[michalkucharczyk]

Just quickly looked into this.

How to compute the latency between submit -> finalized?

This would require us to know what was transcation lifecycle? (e.g. submitted -> inblock -> retracted -> inblock -> retracted -> inblock -> finalized.

We have all the intermediate steps, but don't have a way to compute submitted -> finalized?

But I am be missing something at the moment, need to have a second look.

[lexnv]

This would require us to know what was transcation lifecycle? (e.g. submitted -> inblock -> retracted -> inblock -> retracted -> inblock -> finalized.
We have all the intermediate steps, but don't have a way to compute submitted -> finalized?

Yep exactly, and since the submitted -> finalized is an important metric, I'll add it as well thanks for the suggestion 🙏 Believe it would be difficult otherwise to create an accurate dashboard if the transaction state switches between inblock <-> retracted multiple times

[michalkucharczyk]

I think that what we need is time to event (since submission) for every event type.
It may also make sense to deduplicated inblock / retracted so only first matters.

[paritytech-workflow-stopper]

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/14712873864
Failed job name: build-rustdoc

[iulianbarbu]

LGTM!

[iulianbarbu]

nit: it would help a tiny bit mentioning the time unit in the metrics descriptions, while reading the code.

[iulianbarbu]

nit:

Suggested change

	let rpc_v2_metrics = if let Some(registry) = config.prometheus_registry() {
	Some(sc_rpc_spec_v2::transaction::TransactionMetrics::new(registry)?)
	} else {
	None
	};
	let rpc_v2_metrics = config.prometheus_registry().map(|registry| sc_rpc_spec_v2::transaction::TransactionMetrics::new(registry)).transpose()?;

[iulianbarbu]

Do we need the clone?

[michalkucharczyk]

nit: inspect maybe could be used?

[michalkucharczyk]

@olliecorbisiero mentioned you would like that some adjustment of buckets maybe required. We can wait with merge to confirm that buckets are fine.

[michalkucharczyk]

@olliecorbisiero here are new buckets for in-block.

Buckets for internal pool events (they you are using now in dashboard) were exactly the same:

polkadot-sdk/substrate/client/transaction-pool/src/fork_aware_txpool/metrics.rs

Lines 141 to 147 in 23c41b2

	in_block: register(
	Histogram::with_opts(histogram_opts!(
	"substrate_sub_txpool_timing_event_in_block",
	"Histogram of timings for reporting InBlock event",
	linear_buckets(0.0, 3.0, 20).unwrap()
	))?,
	registry,

Let us know what would work for you?