feat: implement IndexedGossipQueue #5803
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Performance Report✔️ no performance regression detected Full benchmark results
|
dapplion
reviewed
Jul 26, 2023
dapplion
reviewed
Jul 26, 2023
| * If not, pick the last key | ||
| */ | ||
| next(): T[] | null { | ||
| let key: string | null = this.minChunkSizeKeys.last(); |
There was a problem hiding this comment.
I don't think this approach is best, consider this case:
- a key receives minChunkSize-1 items at time T1 and then 1 extra item at time T2
- It is appended to minChunkSizeKeys at time T2 and will be picked up first, but most of its items are at time T1
Similarly if a key does not receive minChunkSize it will always be de-prioritized. This approach risks respecting FIFO for some load patterns.
The goals for a FIFO queue with indexing that intends to batch should be:
- Prioritize latest items
- Buffer items to allow batching, but limit buffering by max wait or max length
For 1. in batches, what decide the timeliness of a batch?
- First arrival
- Latest arrival
- Avg of items
There was a problem hiding this comment.
yes, this is not ideal. Initially I did like this:
- always pick the last item (like LIFO)
- process it along with
min(64, number of messages with same data/key)
the performance is far from the current approach since it can't batch much messages. I think it's not easy to pick a strategy, we need to:
- define some criterias to pick a strategy, one is: job wait time + job time + avg(forwarded messages)
- do some dynamic configurations here, configure it at runtime and compare
|
It's not easy to enforce FIFO and fairness rules having multiple list of objects that arrive at different times. We do FIFO for attestations because the most recent attestations have the most value to us. Given a list of single-bit gossip attestations (assuming all valid) their value is equal the sum of One approach that would be fair but maybe expensive is to compute this "score" everytime type MsgList = {
avgRecvTimestampMs: number;
items: GossipMsg[];
};
type GossipMsgId = string;
type GossipMsg = any;
const msgListScore = (a: MsgList): number => a.items.length / Math.max(1000, Date.now() - a.avgRecvTimestampMs);
const getKey = (_: GossipMsg): GossipMsgId => "TODO";
class Queue {
queuesMap: Map<GossipMsgId, MsgList>;
queuesList: MsgList[];
add(item: GossipMsg): void {
const key = getKey(item);
let queue = this.queuesMap.get(key);
if (!queue) {
queue = {avgRecvTimestampMs: Date.now(), items: [item]};
this.queuesMap.set(key, queue);
} else {
queue.items.push(item);
// Compute average adding 1 value: avg = (avg * N + x) / (N + 1)
queue.avgRecvTimestampMs =
(queue.items.length * queue.avgRecvTimestampMs + Date.now()) / (queue.items.length + 1);
}
}
getNext(): GossipMsg[] | undefined {
// Sort in place by score, can be optimized runing this sometimes only
this.queuesList.sort((a, b) => msgListScore(a) - msgListScore(b));
return this.queuesList.shift()?.items;
}
} |
@dapplion yes that's expensive, tested the 
|
please @tuyennhv do not post spiky graphs, post either histograms or % of values above a certain threshold, or an avg_over_time |
|
The queue must have some mechanism to pick up groups of attestation datas that do not reach the minimum count. |
|
the current mechanism works as below: with
@dapplion what do you expect, or want to improve with the above example? I can give it a try in a real node |
|
Let's go with this and have histogram metrics to measure delay in the queue since first item |
dapplion
approved these changes
Aug 13, 2023
|
🎉 This PR is included in v1.11.0 🎉 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Description
GossipQueueinterface, the current implementation isLinearGossipQueue. This does not affect the current performance of node since noone is using the new gossip queue for nowIndexedGossipQueuepart of #5416