Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lightclient routes #181

Closed
wants to merge 10 commits into from
Closed
12 changes: 12 additions & 0 deletions apis/eventstream/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ get:
- finalized_checkpoint
- chain_reorg
- contribution_and_proof
- lightclient_head_update
responses:
"200":
description: Opened SSE stream.
Expand Down Expand Up @@ -73,6 +74,17 @@ get:
value: |
event: contribution_and_proof
data: {"message": {"aggregator_index": "997", "contribution": {"slot": "168097", "beacon_block_root": "0x56f1fd4262c08fa81e27621c370e187e621a67fc80fe42340b07519f84b42ea1", "subcommittee_index": "0", "aggregation_bits": "0xffffffffffffffffffffffffffffffff", "signature": "0x85ab9018e14963026476fdf784cc674da144b3dbdb47516185438768774f077d882087b90ad642469902e782a8b43eed0cfc1b862aa9a473b54c98d860424a702297b4b648f3f30bdaae8a8b7627d10d04cb96a2cc8376af3e54a9aa0c8145e3"}, "selection_proof": "0x87c305f04bfe5db27c2b19fc23e00d7ac496ec7d3e759cbfdd1035cb8cf6caaa17a36a95a08ba78c282725e7b66a76820ca4eb333822bd399ceeb9807a0f2926c67ce67cfe06a0b0006838203b493505a8457eb79913ce1a3bcd1cc8e4ef30ed"}, "signature": "0xac118511474a94f857300b315c50585c32a713e4452e26a6bb98cdb619936370f126ed3b6bb64469259ee92e69791d9e12d324ce6fd90081680ce72f39d85d50b0ff977260a8667465e613362c6d6e6e745e1f9323ec1d6f16041c4e358839ac"}
lightclient_head_update:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a need for _update here; the event is, by definition, an update.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, tho I'm unsure if lightclient_head can also be confusing since you can get multiple events for the same head which is not the same behavior as the head event.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was called lightlcient_head_update because the data structure emitted is called a LightClientHeadUpdate somewhere.

Similar to the LightClientUpdate in the specs.

Not saying its a great name, just saying..

description: |
The node has received a new or better lightclient head update. `header` is the block header signed by `sync_aggregate`.
Updates may be collected via
- Next block's syncAggregate field
- Aggregating messages from gossip topic `sync_committee_contribution_and_proof`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this considered safe, if this information is not yet in a (voted upon) block? Is there any way of knowing the level of finality of the provided head?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this considered safe, if this information is not yet in a (voted upon) block?

Yes, adding the SyncAggregate is only useful to give on-chain incentives to sync committee members. Gathering the data from gossip topics is identical.

Is there any way of knowing the level of finality of the provided head?

Not really, the main data points lightclients can rely on are:

  • participation of this SyncAggregate
  • if the block is finalized or not

- Aggregating messages from gossip topics `sync_committee_{subnet_id}`
Nodes should throttle emitting updates unless a new update is sufficiently better, i.e. has 16 more participants.
dapplion marked this conversation as resolved.
Show resolved Hide resolved
value: |
event: lightclient_head_update
data: {"message": {"sync_aggregate": {"sync_committee_bits": "0xfffffffffbfffffffffffffffffffffffffffffffffffffffffffefffffffffffffffffffffffeffffffffffeffffffffffffffffffffdffffffffffffffffff", "sync_committee_signature": "0x8a5e838431917331c461f38ec0f6c0b2f29c6cd2c9c9b7ed970b4700a8bc66bdb4b11ca129dd9c72fc1bbb1959c4a7cc0a0d8d107267f3cc364951747ad5ddd757555358e5910cec387179996a6e698d813df723ae02a825b6034af062913b91"}, "header": {"slot": "2642381", "proposer_index": "189806", "parent_root": "0xa00016e7462de60629c9b81b10d1f79c29ae43d81972a3c0e1e92801e3599d9d", "state_root": "0xaec6a955ff0ff9c0081828bad99342b3e60c4744a65e8c7ef1fd01bb1162b4ca", "body_root": "0xeed126eddef983e1270cf72bea05f62195590576a7472a6ae1b43a7bd1aa6f23"}}}
"400":
description: "The topics supplied could not be parsed"
content:
Expand Down
41 changes: 41 additions & 0 deletions apis/lightclient/committee_updates.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
get:
operationId: getCommitteeUpdates
summary: Get committee updates in an inclusive range of sync periods
description: |
Returns an array of best updates in the requested periods within the inclusive range `from` - `to`.
Best is defined by (in order of priority):
- Is finalized update + > 2/3 participations
- Highest participation (bit count)
- Oldest update
Tracks oldest instead of newest to reduce disk writes and the likelyhood of re-orgs. Clients may choose
dapplion marked this conversation as resolved.
Show resolved Hide resolved
a different set of "best" criteria if necessary.
tags:
- Lightclient
parameters:
- name: from
in: query
required: true
description: "Sync period start of requested range"
schema:
$ref: '../../beacon-node-oapi.yaml#/components/schemas/Uint64'
- name: to
in: query
required: true
description: "Sync period end of requested range (inclusive)"
schema:
$ref: '../../beacon-node-oapi.yaml#/components/schemas/Uint64'
responses:
"200":
description: Success
content:
application/json:
schema:
title: GetCommitteeUpdatesResponse
type: object
properties:
data:
type: array
items:
$ref: "../../beacon-node-oapi.yaml#/components/schemas/LightClientUpdate"
"500":
$ref: "../../beacon-node-oapi.yaml#/components/responses/InternalError"
21 changes: 21 additions & 0 deletions apis/lightclient/head_update.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
get:
operationId: getHeadUpdate
summary: Get latest best head update
description: |
Returns the latest best head update available. Clients should use the SSE type `lightclient_head_update`
dapplion marked this conversation as resolved.
Show resolved Hide resolved
unless to get the very first head update after syncing, or if SSE are not supported by the server.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Events are part of the spec so the latter part is not a relevant comment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the latter part is not a relevant comment.

Do you refer to the part where SSE may not be supported by the server? At least Lodestar allows to disable the events namespace individually. Also some reverse proxies may not support SSE depending on their buffering configuration.

tags:
- Lightclient
responses:
"200":
description: Success
content:
application/json:
schema:
title: GetHeadUpdateResponse
type: object
properties:
data:
$ref: "../../beacon-node-oapi.yaml#/components/schemas/LightclientHeadUpdate"
"500":
$ref: "../../beacon-node-oapi.yaml#/components/responses/InternalError"
54 changes: 54 additions & 0 deletions apis/lightclient/proof.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
get:
operationId: getProof
summary: Get multiproof of `paths` for `state_id`
description: |
Returns a multiproof of `paths` at the requested `stateId`.
The requested `stateId` may not be available. Regular nodes only keep recent states in memory.
tags:
- Lightclient
parameters:
- name: state_id
in: path
required: true
$ref: '../../beacon-node-oapi.yaml#/components/parameters/StateId'
- name: paths
in: query
required: true
description: Array of JSON serialized array of paths
schema:
type: array
items:
type: string
description: |
example: "'[\"balances\", 0]'"

responses:
"200":
description: Success
content:
application/octet-stream:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a JSON response here as well. This will also give a location to explain the structure of the multiproof.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JSON representation of this format is widely inefficient. Since JSON is the default I'll argue into only offering application/octet-stream to force consumers to use the efficient path

schema:
description: "Serialized TreeOffset multiproof"
"400":
description: "Invalid state ID"
content:
application/json:
schema:
allOf:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
code: 400
message: "Invalid state ID: current"
"404":
description: "State not found"
content:
application/json:
schema:
allOf:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
code: 404
message: "State not found"
"500":
$ref: '../../beacon-node-oapi.yaml#/components/responses/InternalError'

50 changes: 50 additions & 0 deletions apis/lightclient/snapshot.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
get:
operationId: getSnapshot
summary: Fetch a snapshot with a proof to a trusted block root.
description: |
Fetch a snapshot with a proof to a trusted block root.
The trusted block root should be fetched with similar means to a weak subjectivity checkpoint.
Only block roots for checkpoints within the weak subjectivity period must be guaranteed to be available.
If the node started with a recent weak subjectivity checkpoint, the node may not backfill snapshots
for older checkpoints.
tags:
- Lightclient
parameters:
- name: block_root
in: path
required: true
$ref: '../../beacon-node-oapi.yaml#/components/parameters/StateId'
dapplion marked this conversation as resolved.
Show resolved Hide resolved
responses:
"200":
description: Success
content:
application/json:
schema:
title: GetSnapshotResponse
type: object
properties:
data:
$ref: '../../beacon-node-oapi.yaml#/components/schemas/LightclientSnapshot'
"400":
description: "Invalid block root"
content:
application/json:
schema:
allOf:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
code: 400
message: "Invalid block root: head"
"404":
description: "Snapshot not found"
content:
application/json:
schema:
allOf:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
code: 404
message: "Snapshot not found for block root"
"500":
$ref: '../../beacon-node-oapi.yaml#/components/responses/InternalError'

17 changes: 17 additions & 0 deletions beacon-node-oapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ tags:
description: Set of endpoints to debug chain and shouldn't be exposed publicly.
- name: Events
description: Set of endpoints to for event subscription.
- name: Lightclient
description: Endpoints to support server-based lightclients
- name: Node
description: Endpoints to query node related informations
- name: Validator
Expand Down Expand Up @@ -96,6 +98,15 @@ paths:
/eth/v1/debug/beacon/heads:
$ref: './apis/debug/heads.yaml'

/eth/v1/lightclient/committee_updates:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API has historically defined committee as a beacon committee, and sync_committee as a sync committee, so I believe you want the latter here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if it's scoped in the lightclient namespace is there a risk of confusion?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I support using sync_committee for clarity here.

$ref: "./apis/lightclient/committee_updates.yaml"
/eth/v1/lightclient/head_update:
$ref: "./apis/lightclient/head_update.yaml"
/eth/v1/lightclient/proof/{state_id}:
$ref: "./apis/lightclient/proof.yaml"
/eth/v1/lightclient/snapshot/{block_root}:
$ref: "./apis/lightclient/snapshot.yaml"

/eth/v1/node/identity:
$ref: "./apis/node/identity.yaml"
/eth/v1/node/peers:
Expand Down Expand Up @@ -237,6 +248,12 @@ components:
$ref: './types/altair/sync_committee.yaml#/Altair/SyncCommitteeContribution'
Altair.SyncCommittee:
$ref: './types/altair/sync_committee.yaml#/Altair/SyncCommitteeByValidatorIndices'
LightClientUpdate:
$ref: './types/altair/lightclient.yaml#/Altair/LightClientUpdate'
LightclientHeadUpdate:
$ref: './types/altair/lightclient.yaml#/Altair/LightclientHeadUpdate'
LightclientSnapshot:
$ref: './types/altair/lightclient.yaml#/Altair/LightclientSnapshot'

parameters:
StateId:
Expand Down
58 changes: 58 additions & 0 deletions types/altair/lightclient.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
Altair:
LightClientUpdate:
type: object
properties:
header:
$ref: '../block.yaml#/BeaconBlockHeader'
nextSyncCommittee:
$ref: './sync_committee.yaml#/SyncCommittee'
nextSyncCommitteeBranch:
$ref: '#/Altair/SyncCommitteeBranch'
finalityHeader:
$ref: '../block.yaml#/BeaconBlockHeader'
finalityBranch:
$ref: '#/Altair/FinalityBranch'
syncCommitteeBits:
$ref: './sync_aggregate.yaml#/Altair/SyncCommitteeBits'
syncCommitteeSignature:
allOf:
- $ref: '../primitive.yaml#/Signature'
- description: 'Signature by the validator(s) over the block root of `slot`'
forkVersion:
$ref: '../primitive.yaml#/ForkVersion'

FinalityBranch:
type: array
description: "Branch to state.finalized_checkpoint.root"
items:
allOf:
- $ref: '../primitive.yaml#/Root'
minItems: 6
maxItems: 6

SyncCommitteeBranch:
type: array
description: "Branch to state.next_sync_committee or state.current_sync_committee"
items:
allOf:
- $ref: '../primitive.yaml#/Root'
minItems: 5
maxItems: 5

LightclientHeadUpdate:
type: object
properties:
header:
$ref: '../block.yaml#/BeaconBlockHeader'
syncAggregate:
$ref: './sync_aggregate.yaml#/SyncAggregate'

LightclientSnapshot:
type: object
properties:
header:
$ref: '../block.yaml#/BeaconBlockHeader'
currentSyncCommittee:
$ref: './sync_committee.yaml#/SyncCommittee'
currentSyncCommitteeBranch:
$ref: '#/Altair/SyncCommitteeBranch'
10 changes: 6 additions & 4 deletions types/altair/sync_aggregate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ Altair:
description: "The [`SyncAggregate`](https://github.com/ethereum/eth2.0-specs/blob/v1.1.0-alpha.1/specs/altair/beacon-chain.md#syncaggregate) object from the Eth2.0 Altair spec."
properties:
sync_committee_bits:
type: string
pattern: "^0x[a-fA-F0-9]+$"
example: "0x01"
description: "Aggregation bits of sync"
$ref: '#/Altair/SyncCommitteeBits'
sync_committee_signature:
$ref: '../primitive.yaml#/Signature'
SyncCommitteeBits:
allOf:
- description: 'A bit is set if a signature from the validator at the corresponding index in the subcommittee is present in the aggregate `signature`.'
- $ref: "../primitive.yaml#/Hex"
- example: "0x01"
6 changes: 1 addition & 5 deletions types/altair/sync_committee.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,10 +73,7 @@ Altair:
- $ref: '../primitive.yaml#/Uint64'
- description: 'The index of the subcommittee that the contribution pertains to.'
aggregation_bits:
allOf:
- description: 'A bit is set if a signature from the validator at the corresponding index in the subcommittee is present in the aggregate `signature`.'
- $ref: "../primitive.yaml#/Hex"
- example: "0x01"
$ref: './sync_aggregate.yaml#/Altair/SyncCommitteeBits'
signature:
allOf:
- $ref: '../primitive.yaml#/Signature'
Expand All @@ -99,4 +96,3 @@ Altair:
allOf:
- $ref: '#/Altair/ValidatorsByIndex'
- description: 'Subcommittee slices of the current sync committee'