Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement peerDAS on electra #6353

Draft
wants to merge 57 commits into
base: unstable
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
4805a2e
feat: placeholder PR for electra
g11tech Jan 24, 2024
499d93c
feat: implement peerDAS on electra
g11tech Jan 24, 2024
156ef53
fix: docker build issue for c-kzg
matthewkeil Jun 21, 2024
47eedae
feat: get various sync mechanisms working with/without sharded data
g11tech Jul 14, 2024
d423004
feat: add the modifications to work with devnet2
g11tech Jul 14, 2024
a0c5d27
fix: refactor to add and use nodeid computation and clear out nodeid …
g11tech Jul 16, 2024
c7f6341
fix the types/test
g11tech Aug 9, 2024
81aaeb5
feat: add and use metadatav3 for peer custody subnet
g11tech Aug 12, 2024
e6c613f
rename electra fork to peerdas for rebase and make csc in metadata uint8
g11tech Aug 27, 2024
a3533f8
add supernode flag to configure node custody requirement and make it …
g11tech Aug 27, 2024
54579b0
add more info for debugging
g11tech Aug 27, 2024
180f7d8
fix log
g11tech Aug 27, 2024
4b6f167
fix bug
g11tech Aug 27, 2024
ae7678e
fx
g11tech Aug 27, 2024
a33a72f
subnet count 128
g11tech Aug 27, 2024
585165e
remove banning unknown block, addmore log
g11tech Aug 28, 2024
2833ac0
make the csc encoding updates as per latest spec
g11tech Sep 5, 2024
bf08852
resolve availability when datacolumns are downloaded and matched
g11tech Sep 7, 2024
006e781
add debug log
g11tech Sep 7, 2024
aece0ab
fix add missing data availability resolutions
g11tech Sep 10, 2024
387da88
add more log
g11tech Sep 10, 2024
2bc1a0d
add cache tracking
g11tech Sep 10, 2024
5e1de6f
trying some fix
g11tech Sep 10, 2024
d7721f8
fix bug
g11tech Sep 10, 2024
bd84892
more log
g11tech Sep 10, 2024
de341b5
add send more log
g11tech Sep 10, 2024
8c21168
make pull a little less agressive
g11tech Sep 10, 2024
d35873e
further wait till cutoff for all data to be available
g11tech Sep 11, 2024
f7571f4
add some more loggig and availaibility tracking
g11tech Sep 11, 2024
74d8122
add some log for debugging inbound data columns request
g11tech Sep 11, 2024
af933fb
some fixes
g11tech Sep 11, 2024
56c8c6e
custodied column fetch debugging log
g11tech Sep 12, 2024
2b10e4d
datacolumns retrival fix
g11tech Sep 12, 2024
cdd9bae
update compute spec tests
g11tech Sep 12, 2024
c4d04ee
fix the column id compute
g11tech Sep 13, 2024
3470076
more debug log
g11tech Sep 13, 2024
4ec7aff
edge case optimization
g11tech Sep 13, 2024
a33303f
feat: refactor and unit test getDataColumnSidecars (#7072)
matthewkeil Sep 16, 2024
bd4f7f9
feat: update ckzg to final DAS version (#7050)
matthewkeil Sep 16, 2024
b1940ee
fix: remove ckzg build script (#7089)
matthewkeil Sep 17, 2024
20ef4c6
feat: validate data column sidecars (#7073)
matthewkeil Sep 17, 2024
fee7c08
validate inclusion proof
g11tech Sep 17, 2024
574837a
use sample subnets for data availability
g11tech Sep 21, 2024
6a77828
add debug console log
g11tech Sep 21, 2024
cec27d6
handle edge case
g11tech Sep 21, 2024
a3de70f
turn persisting network identity to default true
g11tech Sep 26, 2024
cce193b
improve logging for debugging
g11tech Oct 1, 2024
2736b8c
add enhance datacolumn serving logs
g11tech Oct 1, 2024
a0e0087
more log
g11tech Oct 1, 2024
b04aaef
migrate datacolumns to finalized
g11tech Oct 2, 2024
7c9a01c
add debug and fix datacolumns migration and improve log
g11tech Oct 2, 2024
1c08ab3
some fixing of beacon params
g11tech Oct 2, 2024
513bccc
fix
g11tech Oct 2, 2024
fccf9a2
add prevdownload tracker
g11tech Oct 8, 2024
8689c76
feat: check for no commitments on block or column in sidecar validati…
matthewkeil Oct 22, 2024
c8075d0
feat: log peer disconnect info (#7231)
matthewkeil Nov 25, 2024
e8bc729
refactor: peerdas types (#7243)
matthewkeil Dec 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: log peer disconnect info (#7231)
* feat: log disconnect reason

* feat: log peerScore update

* fix: pretty print peerId

* fix: use prettyPrintPeerId
  • Loading branch information
matthewkeil authored Nov 25, 2024
commit c8075d003e59655f8dba4493d767c97ea66799e4
2 changes: 1 addition & 1 deletion packages/beacon-node/src/network/core/networkCore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ export class NetworkCore implements INetworkCore {

const metrics = metricsRegistry ? createNetworkCoreMetrics(metricsRegistry) : null;
const peersData = new PeersData();
const peerRpcScores = new PeerRpcScoreStore(opts, metrics);
const peerRpcScores = new PeerRpcScoreStore(opts, metrics, logger);
const statusCache = new LocalStatusCache(initialStatus);

// Bind discv5's ENR to local metadata
Expand Down
1 change: 1 addition & 0 deletions packages/beacon-node/src/network/peers/peerManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -736,6 +736,7 @@ export class PeerManager {
try {
const reason = GOODBYE_KNOWN_CODES[goodbye.toString()] || "";
this.metrics?.peerGoodbyeSent.inc({reason});
this.logger.debug("disconnected peer", {reason, peerId: prettyPrintPeerId(peer)});

const conn = getConnection(this.libp2p, peer.toString());
if (conn && Date.now() - conn.timeline.open > LONG_PEER_CONNECTION_MS) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ export interface IPeerRpcScoreStore {
export interface IPeerScore {
getScore(): number;
getGossipScore(): number;
add(scoreDelta: number): void;
add(scoreDelta: number): number;
update(): number;
updateGossipsubScore(newScore: number, ignore: boolean): void;
getStat(): PeerScoreStat;
Expand Down
7 changes: 5 additions & 2 deletions packages/beacon-node/src/network/peers/score/score.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,13 @@ export class RealScore implements IPeerScore {
return this.gossipScore;
}

add(scoreDelta: number): void {
add(scoreDelta: number): number {
let newScore = this.lodestarScore + scoreDelta;
if (newScore > MAX_SCORE) newScore = MAX_SCORE;
if (newScore < MIN_SCORE) newScore = MIN_SCORE;

this.setLodestarScore(newScore);
return newScore;
}

/**
Expand Down Expand Up @@ -139,7 +140,9 @@ export class MaxScore implements IPeerScore {
return DEFAULT_SCORE;
}

add(): void {}
add(): number {
return DEFAULT_SCORE;
}

update(): number {
return MAX_SCORE;
Expand Down
11 changes: 8 additions & 3 deletions packages/beacon-node/src/network/peers/score/store.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import {PeerId} from "@libp2p/interface";
import {MapDef, pruneSetToMax} from "@lodestar/utils";
import {Logger, MapDef, pruneSetToMax} from "@lodestar/utils";
import {NetworkCoreMetrics} from "../../core/metrics.js";
import {prettyPrintPeerId} from "../../util.js";
import {DEFAULT_SCORE, MAX_ENTRIES, MAX_SCORE, MIN_SCORE, SCORE_THRESHOLD} from "./constants.js";
import {
IPeerRpcScoreStore,
Expand Down Expand Up @@ -29,11 +30,13 @@ const peerActionScore: Record<PeerAction, number> = {
export class PeerRpcScoreStore implements IPeerRpcScoreStore {
private readonly scores: MapDef<PeerIdStr, IPeerScore>;
private readonly metrics: NetworkCoreMetrics | null;
private readonly logger: Logger | null;

// TODO: Persist scores, at least BANNED status to disk

constructor(opts: PeerRpcScoreOpts = {}, metrics: NetworkCoreMetrics | null = null) {
constructor(opts: PeerRpcScoreOpts = {}, metrics: NetworkCoreMetrics | null = null, logger: Logger | null = null) {
this.metrics = metrics;
this.logger = logger;
this.scores = opts.disablePeerScoring ? new MapDef(() => new MaxScore()) : new MapDef(() => new RealScore());
}

Expand All @@ -55,8 +58,10 @@ export class PeerRpcScoreStore implements IPeerRpcScoreStore {

applyAction(peer: PeerId, action: PeerAction, actionName: string): void {
const peerScore = this.scores.getOrDefault(peer.toString());
peerScore.add(peerActionScore[action]);
const scoreChange = peerActionScore[action];
const newScore = peerScore.add(scoreChange);

this.logger?.debug("peer score adjusted", {scoreChange, newScore, peerId: prettyPrintPeerId(peer), actionName});
this.metrics?.peersReportPeerCount.inc({reason: actionName});
}

Expand Down