Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: electra fork #6986

Merged
merged 76 commits into from
Aug 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
d87e637
feat: placeholder PR for electra
g11tech Jan 24, 2024
7f3ede7
feat: implement EIP-6110 (#6042)
ensi321 Feb 19, 2024
3eeb006
chore: fix CI failure due to recent merge from `unstable` (#6646)
ensi321 Apr 12, 2024
4b707a9
feat: implement execution layer exits eip 7002 (#6651)
g11tech Apr 22, 2024
a656c69
chore: update spec test version for electra fork (#6717)
ensi321 May 1, 2024
bef3eff
feat: add presets and ssz types for EIP-7549 (#6715)
ensi321 May 4, 2024
8a4c657
chore: fix the rebase build (#6735)
g11tech May 4, 2024
8db4ada
feat: upgrade 7002 exits to withdrawal request (#6736)
g11tech May 5, 2024
244dc88
feat: implement maxEB EIP-7251 (#6539)
ensi321 May 7, 2024
a145d2d
feat: beacon node process electra attestations EIP-7549 (#6738)
ensi321 May 7, 2024
8e15350
feat: handle the EL payload sending data in deposit requests instead …
g11tech May 8, 2024
84cf396
feat: implement EIP-7549 (#6689)
ensi321 May 8, 2024
09e7510
fix: attestation pool for electra (#6744)
twoeths May 8, 2024
0893276
feat: update engineapi endpoints to v4 (#6747)
g11tech May 8, 2024
2b0ec12
feat: rename deposit receipt to deposit request for Pectra (#6748)
ensi321 May 8, 2024
3cf76cb
test: enable spec tests related to eip-7549 (#6741)
nazarhussain May 8, 2024
a0fff8f
fix: fix e2e test in electra-fork (#6751)
ensi321 May 8, 2024
15b7cdf
feat: get the basic integration working with the ethereumjs electra b…
g11tech May 8, 2024
9f599f3
feat: apply some fixes and hacks to get the single node devnet workin…
g11tech May 9, 2024
c34eac7
fix: get aggregate and proofs signature sets (#6757)
twoeths May 10, 2024
81e7682
test(spec): fix attestors slashing specs for electra fork (#6758)
nazarhussain May 10, 2024
f2be317
chore: fix types and lint (#6750)
g11tech May 13, 2024
6393d0a
fix: fix electra genesis spec test (#6764)
ensi321 May 14, 2024
55567db
feat: support missing electra spec test (#6765)
ensi321 May 14, 2024
8108016
test: fix ssz types in fork_choice spec tests (#6767)
nflaig May 14, 2024
52be86e
chore: update EffectiveBalanceIncrements type (#6763)
jeluard May 14, 2024
9bd982c
Fix ssz_static
ensi321 May 14, 2024
dc2a197
fix: inline sourcemaps to help with debugging (#6768)
matthewkeil May 14, 2024
3e6990a
fix: additional epoch calculation logic for consolidation churn (#6770)
nazarhussain May 14, 2024
bd1bb6c
fix: electra fork transition spec tests (#6769)
twoeths May 14, 2024
f0eab38
test: fix ssz_static spec tests for all forks (#6771)
nflaig May 14, 2024
65b6827
chore(spec): remove the skip specs for electra (#6772)
nazarhussain May 14, 2024
7f2a77a
fix: use mutable validator object (#6774)
ensi321 May 14, 2024
486a766
test: fix balance spec tests (#6777)
matthewkeil May 14, 2024
f82b355
fix: effective balance cache is not in sync with validator effective …
ensi321 May 14, 2024
1d57ac7
fix: make electra-fork passes lint and check-types (#6785)
ensi321 May 16, 2024
edc396e
fix: update data format of WithdrawalRequestV1 (#6789)
nflaig May 16, 2024
943c913
fix: publish attestations with non-zero committee index (#6790)
ensi321 May 16, 2024
7a50b6f
fix: validator monitor summaries should not render during epoch 0 (#6…
ensi321 May 16, 2024
d490b41
fix: attestation duty validation (#6792)
ensi321 May 16, 2024
296ce19
fix: align BeaconBlockBody and BlindedBeaconBlockBody (#6782)
wemeetagain May 16, 2024
7b76bfc
test: improve ssz tests consistency (#6776)
jeluard May 16, 2024
dbd6a49
fix: batch validation for electra attestations (#6788)
twoeths May 16, 2024
9db912a
fix: update withdrawal request container to match consensus spec (#6797)
nflaig May 16, 2024
23868c6
fix: get seen AttData key from SignedAggregateAndProof electra (#6802)
twoeths May 17, 2024
bd5c8b7
test: only skip ssz_static tests associated to missing type (#6798)
nflaig May 23, 2024
3d19cb4
chore: types and lint fixes (#6819)
g11tech May 25, 2024
e75d906
feat: add engine_getPayloadBodiesByHash and ByRange V2 (#6852)
ensi321 Jun 8, 2024
d87de02
fix: align WithdrawalRequestV1 field names with latest spec (#6877)
nflaig Jun 12, 2024
44e0041
feat: move attestation committee at the end of attestation (#6883)
ensi321 Jun 14, 2024
43fe8de
rebase fixes
g11tech Jun 19, 2024
4dfe326
feat: handle exited/exiting validators during top up (#6880)
ensi321 Jun 21, 2024
39edbfd
feat: add EL triggered consolidation and remove `ExecutionLayer` pref…
ensi321 Jun 21, 2024
12d7689
feat: support electra devnet-1 (#6892)
ensi321 Jun 25, 2024
1d9c3a3
fix: electra rebase fixes on the new generic typing model
g11tech Jun 25, 2024
5e7bf4c
feat: add and parse consolidation requests from engine api
g11tech Jul 1, 2024
08445fd
feat: make produce block/signed block contents api multifork
g11tech Jul 1, 2024
3a98b2f
fix: electra rebase fixes on the allforks type refactor
g11tech Jul 1, 2024
371e962
chore: rebase fixes
g11tech Jul 30, 2024
5f9f223
feat: add Electra attestation V2 endpoints (#6951)
ensi321 Aug 5, 2024
e5cda9c
Update packages/beacon-node/src/chain/validation/aggregateAndProof.ts
ensi321 Aug 6, 2024
da0f40c
fix: fix the devnet2 bug of invalid stateroot calc post consolidation
g11tech Aug 7, 2024
dc4fa9e
chore: further rebase fixes to unstable
g11tech Aug 9, 2024
ce6a0c7
fix: cached balances in epoch transition (#7018)
twoeths Aug 13, 2024
4b2f34b
chore: review electra branch - part 1 (#7015)
nflaig Aug 13, 2024
9c86957
chore: review electra branch - part 2 (#7019)
nflaig Aug 13, 2024
a157b55
feat: support post-electra attestation and attester slashing events (…
nflaig Aug 15, 2024
7d900c5
feat: pre-electra support from attestation pool (#6998)
ensi321 Aug 16, 2024
3918aae
fix: use attestation v1 endpoints pre-electra (#7024)
nflaig Aug 17, 2024
5fd2d09
fix: use correct partial withdrawal index to evaluate processing them
g11tech Aug 20, 2024
23fb9fb
fix: ssz v0.17.1 (#7035)
twoeths Aug 20, 2024
cb7878b
chore: rebase fixes
g11tech Aug 23, 2024
8ccbed7
chore: fix api handler used during testing (#7047)
nflaig Aug 23, 2024
ed18ff0
fix: improve processEffectiveBalanceUpdates (#7043)
twoeths Aug 27, 2024
7fe30b4
fix: perf test relative import from state-transition (#7055)
matthewkeil Aug 28, 2024
03f7396
fix: use next epoch for pending balance/consolidations processing (#7…
g11tech Aug 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
},
"dependencies": {
"@chainsafe/persistent-merkle-tree": "^0.8.0",
"@chainsafe/ssz": "^0.17.0",
"@chainsafe/ssz": "^0.17.1",
"@lodestar/config": "^1.21.0",
"@lodestar/params": "^1.21.0",
"@lodestar/types": "^1.21.0",
Expand Down
2 changes: 1 addition & 1 deletion packages/api/src/beacon/client/events.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export type ApiClient = ApiClientMethods<Endpoints>;
*/
export function getClient(config: ChainForkConfig, baseUrl: string): ApiClient {
const definitions = getDefinitions(config);
const eventSerdes = getEventSerdes();
const eventSerdes = getEventSerdes(config);

return {
eventstream: async ({topics, signal, onEvent, onError, onClose}) => {
Expand Down
25 changes: 23 additions & 2 deletions packages/api/src/beacon/routes/beacon/block.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
SignedBlockContents,
sszTypesFor,
} from "@lodestar/types";
import {ForkName, ForkPreExecution, isForkBlobs, isForkExecution} from "@lodestar/params";
import {ForkName, ForkPreElectra, ForkPreExecution, isForkBlobs, isForkExecution} from "@lodestar/params";
import {Endpoint, RequestCodec, RouteDefinitions, Schema} from "../../../utils/index.js";
import {EmptyMeta, EmptyResponseCodec, EmptyResponseData, WithVersion} from "../../../utils/codecs.js";
import {
Expand Down Expand Up @@ -101,10 +101,22 @@ export type Endpoints = {
"GET",
BlockArgs,
{params: {block_id: string}},
BeaconBlockBody["attestations"],
BeaconBlockBody<ForkPreElectra>["attestations"],
ExecutionOptimisticAndFinalizedMeta
>;

/**
* Get block attestations
* Retrieves attestation included in requested block.
*/
getBlockAttestationsV2: Endpoint<
"GET",
BlockArgs,
{params: {block_id: string}},
BeaconBlockBody["attestations"],
ExecutionOptimisticFinalizedAndVersionMeta
>;

/**
* Get block header
* Retrieves block header for given block id.
Expand Down Expand Up @@ -251,6 +263,15 @@ export function getDefinitions(config: ChainForkConfig): RouteDefinitions<Endpoi
meta: ExecutionOptimisticAndFinalizedCodec,
},
},
getBlockAttestationsV2: {
url: "/eth/v2/beacon/blocks/{block_id}/attestations",
method: "GET",
req: blockIdOnlyReq,
resp: {
data: WithVersion((fork) => ssz[fork].BeaconBlockBody.fields.attestations),
meta: ExecutionOptimisticFinalizedAndVersionCodec,
},
},
getBlockHeader: {
url: "/eth/v1/beacon/headers/{block_id}",
method: "GET",
Expand Down
210 changes: 195 additions & 15 deletions packages/api/src/beacon/routes/beacon/pool.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
/* eslint-disable @typescript-eslint/naming-convention */
import {ValueOf} from "@chainsafe/ssz";
import {ChainForkConfig} from "@lodestar/config";
import {phase0, capella, CommitteeIndex, Slot, ssz} from "@lodestar/types";
import {isForkPostElectra} from "@lodestar/params";
import {phase0, capella, CommitteeIndex, Slot, ssz, electra, AttesterSlashing} from "@lodestar/types";
import {Schema, Endpoint, RouteDefinitions} from "../../../utils/index.js";
import {
ArrayOf,
Expand All @@ -12,19 +13,31 @@ import {
EmptyRequest,
EmptyResponseCodec,
EmptyResponseData,
WithVersion,
} from "../../../utils/codecs.js";
import {MetaHeader, VersionCodec, VersionMeta} from "../../../utils/metadata.js";
import {toForkName} from "../../../utils/fork.js";
import {fromHeaders} from "../../../utils/headers.js";

// See /packages/api/src/routes/index.ts for reasoning and instructions to add new routes

const AttestationListType = ArrayOf(ssz.phase0.Attestation);
const AttesterSlashingListType = ArrayOf(ssz.phase0.AttesterSlashing);
const AttestationListTypePhase0 = ArrayOf(ssz.phase0.Attestation);
const AttestationListTypeElectra = ArrayOf(ssz.electra.Attestation);
const AttesterSlashingListTypePhase0 = ArrayOf(ssz.phase0.AttesterSlashing);
const AttesterSlashingListTypeElectra = ArrayOf(ssz.electra.AttesterSlashing);
const ProposerSlashingListType = ArrayOf(ssz.phase0.ProposerSlashing);
const SignedVoluntaryExitListType = ArrayOf(ssz.phase0.SignedVoluntaryExit);
const SignedBLSToExecutionChangeListType = ArrayOf(ssz.capella.SignedBLSToExecutionChange);
const SyncCommitteeMessageListType = ArrayOf(ssz.altair.SyncCommitteeMessage);

type AttestationList = ValueOf<typeof AttestationListType>;
type AttesterSlashingList = ValueOf<typeof AttesterSlashingListType>;
type AttestationListPhase0 = ValueOf<typeof AttestationListTypePhase0>;
type AttestationListElectra = ValueOf<typeof AttestationListTypeElectra>;
type AttestationList = AttestationListPhase0 | AttestationListElectra;

type AttesterSlashingListPhase0 = ValueOf<typeof AttesterSlashingListTypePhase0>;
type AttesterSlashingListElectra = ValueOf<typeof AttesterSlashingListTypeElectra>;
type AttesterSlashingList = AttesterSlashingListPhase0 | AttesterSlashingListElectra;

type ProposerSlashingList = ValueOf<typeof ProposerSlashingListType>;
type SignedVoluntaryExitList = ValueOf<typeof SignedVoluntaryExitListType>;
type SignedBLSToExecutionChangeList = ValueOf<typeof SignedBLSToExecutionChangeListType>;
Expand All @@ -39,10 +52,22 @@ export type Endpoints = {
"GET",
{slot?: Slot; committeeIndex?: CommitteeIndex},
{query: {slot?: number; committee_index?: number}},
AttestationList,
AttestationListPhase0,
EmptyMeta
>;

/**
* Get Attestations from operations pool
* Retrieves attestations known by the node but not necessarily incorporated into any block
*/
getPoolAttestationsV2: Endpoint<
"GET",
{slot?: Slot; committeeIndex?: CommitteeIndex},
{query: {slot?: number; committee_index?: number}},
AttestationList,
VersionMeta
>;

/**
* Get AttesterSlashings from operations pool
* Retrieves attester slashings known by the node but not necessarily incorporated into any block
Expand All @@ -52,10 +77,23 @@ export type Endpoints = {
"GET",
EmptyArgs,
EmptyRequest,
AttesterSlashingList,
AttesterSlashingListPhase0,
EmptyMeta
>;

/**
* Get AttesterSlashings from operations pool
* Retrieves attester slashings known by the node but not necessarily incorporated into any block
*/
getPoolAttesterSlashingsV2: Endpoint<
// ⏎
"GET",
EmptyArgs,
EmptyRequest,
AttesterSlashingList,
VersionMeta
>;

/**
* Get ProposerSlashings from operations pool
* Retrieves proposer slashings known by the node but not necessarily incorporated into any block
Expand Down Expand Up @@ -105,12 +143,28 @@ export type Endpoints = {
*/
submitPoolAttestations: Endpoint<
"POST",
{signedAttestations: AttestationList},
{signedAttestations: AttestationListPhase0},
{body: unknown},
EmptyResponseData,
EmptyMeta
>;

/**
* Submit Attestation objects to node
* Submits Attestation objects to the node. Each attestation in the request body is processed individually.
*
* If an attestation is validated successfully the node MUST publish that attestation on the appropriate subnet.
*
* If one or more attestations fail validation the node MUST return a 400 error with details of which attestations have failed, and why.
*/
submitPoolAttestationsV2: Endpoint<
"POST",
{signedAttestations: AttestationList},
{body: unknown; headers: {[MetaHeader.Version]: string}},
EmptyResponseData,
EmptyMeta
>;

/**
* Submit AttesterSlashing object to node's pool
* Submits AttesterSlashing object to node's pool and if passes validation node MUST broadcast it to network.
Expand All @@ -123,6 +177,18 @@ export type Endpoints = {
EmptyMeta
>;

/**
* Submit AttesterSlashing object to node's pool
* Submits AttesterSlashing object to node's pool and if passes validation node MUST broadcast it to network.
*/
submitPoolAttesterSlashingsV2: Endpoint<
"POST",
{attesterSlashing: AttesterSlashing},
{body: unknown; headers: {[MetaHeader.Version]: string}},
EmptyResponseData,
EmptyMeta
>;

/**
* Submit ProposerSlashing object to node's pool
* Submits ProposerSlashing object to node's pool and if passes validation node MUST broadcast it to network.
Expand Down Expand Up @@ -172,7 +238,7 @@ export type Endpoints = {
>;
};

export function getDefinitions(_config: ChainForkConfig): RouteDefinitions<Endpoints> {
export function getDefinitions(config: ChainForkConfig): RouteDefinitions<Endpoints> {
return {
getPoolAttestations: {
url: "/eth/v1/beacon/pool/attestations",
Expand All @@ -183,19 +249,43 @@ export function getDefinitions(_config: ChainForkConfig): RouteDefinitions<Endpo
schema: {query: {slot: Schema.Uint, committee_index: Schema.Uint}},
},
resp: {
data: AttestationListType,
data: AttestationListTypePhase0,
meta: EmptyMetaCodec,
},
},
getPoolAttestationsV2: {
url: "/eth/v2/beacon/pool/attestations",
method: "GET",
req: {
writeReq: ({slot, committeeIndex}) => ({query: {slot, committee_index: committeeIndex}}),
parseReq: ({query}) => ({slot: query.slot, committeeIndex: query.committee_index}),
schema: {query: {slot: Schema.Uint, committee_index: Schema.Uint}},
},
resp: {
data: WithVersion((fork) => (isForkPostElectra(fork) ? AttestationListTypeElectra : AttestationListTypePhase0)),
meta: VersionCodec,
},
},
getPoolAttesterSlashings: {
url: "/eth/v1/beacon/pool/attester_slashings",
method: "GET",
req: EmptyRequestCodec,
resp: {
data: AttesterSlashingListType,
data: AttesterSlashingListTypePhase0,
meta: EmptyMetaCodec,
},
},
getPoolAttesterSlashingsV2: {
url: "/eth/v2/beacon/pool/attester_slashings",
method: "GET",
req: EmptyRequestCodec,
resp: {
data: WithVersion((fork) =>
isForkPostElectra(fork) ? AttesterSlashingListTypeElectra : AttesterSlashingListTypePhase0
),
meta: VersionCodec,
},
},
getPoolProposerSlashings: {
url: "/eth/v1/beacon/pool/proposer_slashings",
method: "GET",
Expand Down Expand Up @@ -227,16 +317,61 @@ export function getDefinitions(_config: ChainForkConfig): RouteDefinitions<Endpo
url: "/eth/v1/beacon/pool/attestations",
method: "POST",
req: {
writeReqJson: ({signedAttestations}) => ({body: AttestationListType.toJson(signedAttestations)}),
parseReqJson: ({body}) => ({signedAttestations: AttestationListType.fromJson(body)}),
writeReqSsz: ({signedAttestations}) => ({body: AttestationListType.serialize(signedAttestations)}),
parseReqSsz: ({body}) => ({signedAttestations: AttestationListType.deserialize(body)}),
writeReqJson: ({signedAttestations}) => ({body: AttestationListTypePhase0.toJson(signedAttestations)}),
parseReqJson: ({body}) => ({signedAttestations: AttestationListTypePhase0.fromJson(body)}),
writeReqSsz: ({signedAttestations}) => ({body: AttestationListTypePhase0.serialize(signedAttestations)}),
parseReqSsz: ({body}) => ({signedAttestations: AttestationListTypePhase0.deserialize(body)}),
schema: {
body: Schema.ObjectArray,
},
},
resp: EmptyResponseCodec,
},
submitPoolAttestationsV2: {
url: "/eth/v2/beacon/pool/attestations",
method: "POST",
req: {
writeReqJson: ({signedAttestations}) => {
const fork = config.getForkName(signedAttestations[0]?.data.slot ?? 0);
return {
body: isForkPostElectra(fork)
? AttestationListTypeElectra.toJson(signedAttestations as AttestationListElectra)
: AttestationListTypePhase0.toJson(signedAttestations as AttestationListPhase0),
headers: {[MetaHeader.Version]: fork},
};
},
parseReqJson: ({body, headers}) => {
const fork = toForkName(fromHeaders(headers, MetaHeader.Version));
return {
signedAttestations: isForkPostElectra(fork)
? AttestationListTypeElectra.fromJson(body)
: AttestationListTypePhase0.fromJson(body),
};
},
writeReqSsz: ({signedAttestations}) => {
const fork = config.getForkName(signedAttestations[0]?.data.slot ?? 0);
return {
body: isForkPostElectra(fork)
? AttestationListTypeElectra.serialize(signedAttestations as AttestationListElectra)
: AttestationListTypePhase0.serialize(signedAttestations as AttestationListPhase0),
headers: {[MetaHeader.Version]: fork},
};
},
parseReqSsz: ({body, headers}) => {
const fork = toForkName(fromHeaders(headers, MetaHeader.Version));
return {
signedAttestations: isForkPostElectra(fork)
? AttestationListTypeElectra.deserialize(body)
: AttestationListTypePhase0.deserialize(body),
};
},
schema: {
body: Schema.ObjectArray,
headers: {[MetaHeader.Version]: Schema.String},
},
},
resp: EmptyResponseCodec,
},
submitPoolAttesterSlashings: {
url: "/eth/v1/beacon/pool/attester_slashings",
method: "POST",
Expand All @@ -251,6 +386,51 @@ export function getDefinitions(_config: ChainForkConfig): RouteDefinitions<Endpo
},
resp: EmptyResponseCodec,
},
submitPoolAttesterSlashingsV2: {
url: "/eth/v2/beacon/pool/attester_slashings",
method: "POST",
req: {
writeReqJson: ({attesterSlashing}) => {
const fork = config.getForkName(Number(attesterSlashing.attestation1.data.slot));
return {
body: isForkPostElectra(fork)
? ssz.electra.AttesterSlashing.toJson(attesterSlashing)
: ssz.phase0.AttesterSlashing.toJson(attesterSlashing),
headers: {[MetaHeader.Version]: fork},
};
},
parseReqJson: ({body, headers}) => {
const fork = toForkName(fromHeaders(headers, MetaHeader.Version));
return {
attesterSlashing: isForkPostElectra(fork)
? ssz.electra.AttesterSlashing.fromJson(body)
: ssz.phase0.AttesterSlashing.fromJson(body),
};
},
writeReqSsz: ({attesterSlashing}) => {
const fork = config.getForkName(Number(attesterSlashing.attestation1.data.slot));
return {
body: isForkPostElectra(fork)
? ssz.electra.AttesterSlashing.serialize(attesterSlashing as electra.AttesterSlashing)
: ssz.phase0.AttesterSlashing.serialize(attesterSlashing as phase0.AttesterSlashing),
headers: {[MetaHeader.Version]: fork},
};
},
parseReqSsz: ({body, headers}) => {
const fork = toForkName(fromHeaders(headers, MetaHeader.Version));
return {
attesterSlashing: isForkPostElectra(fork)
? ssz.electra.AttesterSlashing.deserialize(body)
: ssz.phase0.AttesterSlashing.deserialize(body),
};
},
schema: {
body: Schema.Object,
headers: {[MetaHeader.Version]: Schema.String},
},
},
resp: EmptyResponseCodec,
},
submitPoolProposerSlashings: {
url: "/eth/v1/beacon/pool/proposer_slashings",
method: "POST",
Expand Down
Loading
Loading