Feat: continuous flashblock building

[julio4]

📝 Summary

Add a enable_continuous_building flag for flashblock builder enabled by default.

When continuous building is enabled, at each interval the builder keep trying building flashblock payloads corresponding to the given interval (a FlashblockCandidate). This is done in a sequential loop so at any time we have one best candidate (after building the first one).

This is implemented in the build_next_flashblock_continuous, which should provides similar behaviors as the non-continuous build_next_flashblock (where we build one flashblock and directly send it).

The logic of building a candidate is defined in refresh_best_flashblock_candidate. Because each candidate can have completely different transactions/ordering, they all need to be built from the same base state. All further mutations are done in a separate simulation_state based on the fb base state and saved within the candidate as a couple CacheState, Option<TransitionState>, which can be used to apply the state of the best candidate and move forward in the building process.

This whole process is blocking. However between each candidate building we check for block_cancel and fb_cancel (and also check for block_cancel within refresh_best_flashblock_candidate before building the final block). This replicates current behavior, however I think that this could cause some isues in specific situations when the refresh_best_flashblock_candidate takes too much time after fb_cancel is triggered and could introduce slight delays in payload distribution/start of next flashblock. This could be improved with more careful asynchronous model IMO.

It is only useful to try to build a new candidate when the mempool is updated, which is not checked currently and just naively try to build candidates. This can also be improved.

[SozinM]

i think you just want to keep bundle state in here

[julio4]

I didn't found a way to apply BundleState to a &mut State.
Yes we can create a new state with the bundle state as prestate but:

• &mut state holds ownership of the underlying db, so it's not possible to move db to new state easily
• bundle/cache prestate are not compatible, and we use the cache of the state in other part such as when simulating builder transactions

[SozinM]

let's implement compare function?

[julio4]

We can but where? Comparing flashblock payloads can means very different things depending on the context. Here it's still isolated in a closure and within context of comparing candidates for the same interval.

[SozinM]

i suggest following refactor:
Use build_next flashblock function in the loop, instead of having custom function for this
If you need you could refactor build_next_flashblock and execute_best_transaction to fit more nicely

[noot]

Suggested change

	if block_cancel.is_cancelled() {
	self.record_flashblocks_metrics(
	ctx,
	info,
	ctx.target_flashblock_count(),
	span,
	"Payload building complete, channel closed or job cancelled",
	);
	return Ok(None);
	}
	// interval end: abort worker and publish current best immediately (below)
	if ctx.cancel.is_cancelled() {
	break;
	}
	// Build one candidate
	best = self.refresh_best_flashblock_candidate(
	best,
	ctx,
	&*info,
	state,
	&state_provider,
	best_txs,
	block_cancel,
	target_gas_for_batch,
	target_da_for_batch,
	)?;
	let handle = tokio::task::spawn_blocking(|| {
	refresh_best_flashblock_candidate(
	best,
	ctx,
	info,
	state,
	&state_provider,
	best_txs,
	block_cancel,
	target_gas_for_batch,
	target_da_for_batch,
	)
	});
	tokio::select! {
	biased;
	_ = block_cancel.cancelled() => {
	break;
	}
	result = handle => {
	match result {
	Ok(Ok(Some(new_best))) => {
	best = Some(new_best);
	}
	Ok(Ok(None)) => {
	return Ok(None);
	}
	Ok(Err(e)) => {
	return Err(e);
	}
	Err(e) => {
	return Err(eyre::eyre!("flashblock candidate building task panicked: {}", e));
	}
	}
	}
	}

something like this would address the issue in your description where refresh_best_flashblock_candidate might take too long, would need to refactor refresh_best_flashblock_candidate to not be a method on self, and update type param DB to be Sync.