feat(payload): poll only new jobs

[chirag-bgh]

Closes #4403

[codecov]

Codecov Report

Merging #4496 (66b2fac) into main (cd7e113) will decrease coverage by 0.02%.
Report is 3 commits behind head on main.
The diff coverage is 15.68%.

Files Changed	Coverage Δ
crates/payload/builder/src/service.rs	28.46% <15.68%> (-6.18%)	⬇️

... and 10 files with indirect coverage changes

Flag	Coverage Δ
integration-tests	16.66% <15.68%> (-0.02%)	⬇️
unit-tests	63.80% <15.68%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
reth binary	29.71% <ø> (ø)
blockchain tree	83.58% <ø> (ø)
pipeline	90.56% <ø> (ø)
storage (db)	75.06% <ø> (ø)
trie	94.84% <ø> (-0.04%)	⬇️
txpool	47.50% <ø> (ø)
networking	77.42% <ø> (-0.03%)	⬇️
rpc	57.41% <ø> (-0.03%)	⬇️
consensus	63.40% <ø> (ø)
revm	31.74% <ø> (ø)
payload builder	5.71% <15.68%> (-0.64%)	⬇️
primitives	86.17% <ø> (+0.01%)	⬆️

[mattsse]

ty for taking this on,

Initially I thought we can use futuresunordered here, but this would break some features like

reth/crates/payload/builder/src/service.rs

Lines 181 to 193 in 5f329e7

	/// Returns the best payload for the given identifier that has been built so far and terminates
	/// the job if requested.
	fn resolve(&mut self, id: PayloadId) -> Option<PayloadFuture> {
	let job = self.payload_jobs.iter().position(|(_, job_id)| *job_id == id)?;
	let (fut, keep_alive) = self.payload_jobs[job].0.resolve();
	if keep_alive == KeepPayloadJobAlive::No {
	let (_, id) = self.payload_jobs.remove(job);
	trace!(%id, "terminated resolved job");
	}
	Some(Box::pin(fut))
	}

after thinking about this for a bit, I think this feat/optimization isn't actually useful since we don't expect a lot of concurrent payload jobs anyway. And we expect the future to be very cheap.

hence, I'm closing this, sorry about this.