Skip to content

Commit f957884

Browse files
authored
docs: update indexer docs for Horizon-only operation (indexer-rs v2.0.0) (#1073)
* docs: update indexer docs for Horizon-only operation (indexer-rs v2.0.0) Reflect that indexer-service-rs and indexer-tap-agent v2.0.0 require Graph Horizon and no longer support legacy V1 TAP receipts. Simplify the GraphTally configuration guide to show only required Horizon fields, add the /healthz endpoint and receipt validation documentation, document the v2.0.0 upgrade path, and update the indexing overview to reference the Rust indexer components instead of the deprecated TypeScript stack. * docs: apply inline code formatting for indexer-rs component names
1 parent 7238c25 commit f957884

File tree

4 files changed

+141
-202
lines changed

4 files changed

+141
-202
lines changed

website/src/pages/en/graph-horizon/overview.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ The **`SubgraphService`** is the first and currently only data service implement
4141

4242
### Integrated Payments Protocol
4343

44-
While the current protocol uses TAP (Timeline Aggregation Protocol) for payments, Horizon brings the payments infrastructure directly into the core protocol. The payments protocol in Horizon was designed as a generalized version of TAP v1, allowing integration with TAPv2 (GraphTally) but also any other arbitrary payment collection system.
44+
Horizon brings the payments infrastructure directly into the core protocol. The payments protocol in Horizon was designed as a generalized framework, with [GraphTally](/indexing/tap) (formerly TAP) as the integrated payment collection system. This architecture also supports other arbitrary payment collection systems in the future.
4545

4646
### GraphPayments and PaymentsEscrow
4747

website/src/pages/en/graph-horizon/what-changes.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,10 @@ Horizon does not change the way indexing rewards work, the rewards formula remai
4242

4343
### Serving queries and the gateway behavior
4444

45-
The indexer stack will support both the current and Horizon gateway infrastructure, which means with the same indexer stack version indexers will be able to serve queries before and after the Horizon upgrade. It’s worth clarifying however the gateway behavior:
45+
The gateway now exclusively uses TAPv2 (GraphTally) receipts. Indexers must run `indexer-service-rs` and `indexer-tap-agent` v2.0.0 or later, which require Graph Horizon to be enabled.
4646

47-
- Before Horizon, the gateway will serve queries using TAPv1 receipts.
48-
- After Horizon, the gateway will _only_ serve queries using TAPv2 (GraphTally) receipts. This means that indexers that have not upgraded their stack to Horizon will stop receiving queries. The gateway infrastructure will continue to accept old TAPv1 receipts and vouchers to ensure those can be redeemed and no fees are lost.
47+
- The gateway only serves queries using TAPv2 (GraphTally) receipts. Indexers that have not upgraded their stack to Horizon will not receive queries.
48+
- Starting with `indexer-service-rs` and `indexer-tap-agent` v2.0.0, legacy TAPv1 receipt support has been removed entirely. All receipt processing uses Horizon exclusively.
4949

5050
## For Delegators
5151

website/src/pages/en/indexing/overview.mdx

Lines changed: 34 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ The minimum stake for an Indexer is currently set to 100K GRT.
1717

1818
### What are the revenue streams for an Indexer?
1919

20-
**Query fee rebates** - Payments for serving queries on the network. These payments are mediated via state channels between an Indexer and a gateway. Each query request from a gateway contains a payment and the corresponding response a proof of query result validity.
20+
**Query fee rebates** - Payments for serving queries on the network. These payments are mediated via [GraphTally](/indexing/tap) between an Indexer and a gateway. Each query request from a gateway contains a signed receipt and the corresponding response a proof of query result validity.
2121

2222
**Indexing rewards** - Generated via a 3% annual protocol wide inflation, the indexing rewards are distributed to Indexers who are indexing Subgraph deployments for the network.
2323

@@ -127,13 +127,15 @@ Indexers may differentiate themselves by applying advanced techniques for making
127127

128128
At the center of an Indexer's infrastructure is the Graph Node which monitors the indexed networks, extracts and loads data per a Subgraph definition and serves it as a [GraphQL API](/about/#how-the-graph-works). The Graph Node needs to be connected to an endpoint exposing data from each indexed network; an IPFS node for sourcing data; a PostgreSQL database for its store; and Indexer components which facilitate its interactions with the network.
129129

130-
- **PostgreSQL database** - The main store for the Graph Node, this is where Subgraph data is stored. The Indexer service and agent also use the database to store state channel data, cost models, indexing rules, and allocation actions.
130+
- **PostgreSQL database** - The main store for the Graph Node, this is where Subgraph data is stored. The Indexer service, TAP agent, and Indexer agent also use the database to store receipt data, cost models, indexing rules, and allocation actions.
131131

132132
- **Data endpoint** - For EVM-compatible networks, Graph Node needs to be connected to an endpoint that exposes an EVM-compatible JSON-RPC API. This may take the form of a single client or it could be a more complex setup that load balances across multiple. It's important to be aware that certain Subgraphs will require particular client capabilities such as archive mode and/or the parity tracing API.
133133

134134
- **IPFS node (version less than 5)** - Subgraph deployment metadata is stored on the IPFS network. The Graph Node primarily accesses the IPFS node during Subgraph deployment to fetch the Subgraph manifest and all linked files. Network Indexers do not need to host their own IPFS node, an IPFS node for the network is hosted at https://ipfs.thegraph.com.
135135

136-
- **Indexer service** - Handles all required external communications with the network. Shares cost models and indexing statuses, passes query requests from gateways on to a Graph Node, and manages the query payments via state channels with the gateway.
136+
- **Indexer service ([`indexer-service-rs`](https://github.com/graphprotocol/indexer-rs))** - Handles all required external communications with the network. Shares cost models and indexing statuses, passes query requests from gateways on to a Graph Node, and manages query payments via [GraphTally](/indexing/tap).
137+
138+
- **TAP agent ([`indexer-tap-agent`](https://github.com/graphprotocol/indexer-rs))** - Aggregates GraphTally receipts into RAVs (Receipt Aggregate Vouchers) and reconciles redemptions. Exactly one instance should run per indexer.
137139

138140
- **Indexer agent** - Facilitates the Indexers interactions onchain including registering on the network, managing Subgraph deployments to its Graph Node/s, and managing allocations.
139141

@@ -155,12 +157,12 @@ Note: To support agile scaling, it is recommended that query and indexing concer
155157
| 8030 | Subgraph indexing status API | /graphql | --index-node-port | - |
156158
| 8040 | Prometheus metrics | /metrics | --metrics-port | - |
157159

158-
#### Indexer Service
160+
#### Indexer Service (`indexer-service-rs`)
159161

160162
| Port | Purpose | Routes | CLI Argument | Environment Variable |
161163
| --- | --- | --- | --- | --- |
162-
| 7600 | GraphQL HTTP server<br />(for paid Subgraph queries) | /subgraphs/id/...<br />/status<br />/channel-messages-inbox | --port | `INDEXER_SERVICE_PORT` |
163-
| 7300 | Prometheus metrics | /metrics | --metrics-port | - |
164+
| 7600 | GraphQL HTTP server<br />(for paid Subgraph queries) | /subgraphs/id/...<br />/status<br />/healthz<br />/cost<br />/info | --config | - |
165+
| 7300 | Prometheus metrics | /metrics | - | - |
164166

165167
#### Indexer Agent
166168

@@ -363,31 +365,29 @@ docker-compose up
363365

364366
### Indexer components
365367

366-
To successfully participate in the network requires almost constant monitoring and interaction, so we've built a suite of Typescript applications for facilitating an Indexers network participation. There are three Indexer components:
368+
To successfully participate in the network requires almost constant monitoring and interaction, so we've built a suite of applications for facilitating an Indexers network participation. There are four Indexer components:
367369

368370
- **Indexer agent** - The agent monitors the network and the Indexer's own infrastructure and manages which Subgraph deployments are indexed and allocated towards onchain and how much is allocated towards each.
369371

370-
- **Indexer service** - The only component that needs to be exposed externally, the service passes on Subgraph queries to the graph node, manages state channels for query payments, shares important decision making information to clients like the gateways.
372+
- **[`indexer-service-rs`](https://github.com/graphprotocol/indexer-rs)** - The only component that needs to be exposed externally, the service passes on Subgraph queries to the graph node, validates [GraphTally](/indexing/tap) receipts, and shares important decision making information to clients like the gateways.
373+
374+
- **[`indexer-tap-agent`](https://github.com/graphprotocol/indexer-rs)** - Aggregates GraphTally receipts into RAVs and manages receipt-to-payment lifecycle. Run exactly one instance per indexer.
371375

372376
- **Indexer CLI** - The command line interface for managing the Indexer agent. It allows Indexers to manage cost models, manual allocations, actions queue, and indexing rules.
373377

374378
#### Getting started
375379

376-
The Indexer agent and Indexer service should be co-located with your Graph Node infrastructure. There are many ways to set up virtual execution environments for your Indexer components; here we'll explain how to run them on baremetal using NPM packages or source, or via kubernetes and docker on the Google Cloud Kubernetes Engine. If these setup examples do not translate well to your infrastructure there will likely be a community guide to reference, come say hi on [Discord](https://discord.gg/graphprotocol)! Remember to [stake in the protocol](/indexing/overview/#stake-in-the-protocol) before starting up your Indexer components!
380+
The Indexer agent, `indexer-service-rs`, and `indexer-tap-agent` should be co-located with your Graph Node infrastructure. There are many ways to set up virtual execution environments for your Indexer components; here we'll explain how to run them using docker or via kubernetes on the Google Cloud Kubernetes Engine. If these setup examples do not translate well to your infrastructure there will likely be a community guide to reference, come say hi on [Discord](https://discord.gg/graphprotocol)! Remember to [stake in the protocol](/indexing/overview/#stake-in-the-protocol) before starting up your Indexer components!
377381

378-
#### From NPM packages
382+
#### Indexer agent (from NPM)
379383

380384
```sh
381-
npm install -g @graphprotocol/indexer-service
382385
npm install -g @graphprotocol/indexer-agent
383386

384387
# Indexer CLI is a plugin for Graph CLI, so both need to be installed:
385388
npm install -g @graphprotocol/graph-cli
386389
npm install -g @graphprotocol/indexer-cli
387390

388-
# Indexer service
389-
graph-indexer-service start ...
390-
391391
# Indexer agent
392392
graph-indexer-agent start ...
393393

@@ -398,58 +398,28 @@ graph indexer connect http://localhost:18000/
398398
graph indexer ...
399399
```
400400

401-
#### From source
402-
403-
```sh
404-
# From Repo root directory
405-
yarn
406-
407-
# Indexer Service
408-
cd packages/indexer-service
409-
./bin/graph-indexer-service start ...
401+
#### `indexer-service-rs` and `indexer-tap-agent` (Docker)
410402

411-
# Indexer agent
412-
cd packages/indexer-agent
413-
./bin/graph-indexer-service start ...
414-
415-
# Indexer CLI
416-
cd packages/indexer-cli
417-
./bin/graph-indexer-cli indexer connect http://localhost:18000/
418-
./bin/graph-indexer-cli indexer ...
419-
```
420-
421-
#### Using docker
422-
423-
- Pull images from the registry
403+
The `indexer-service-rs` and `indexer-tap-agent` are Rust applications distributed as Docker images. See the [GraphTally guide](/indexing/tap) for detailed configuration.
424404

425405
```sh
426-
docker pull ghcr.io/graphprotocol/indexer-service:latest
427-
docker pull ghcr.io/graphprotocol/indexer-agent:latest
406+
docker pull ghcr.io/graphprotocol/indexer-service-rs:latest
407+
docker pull ghcr.io/graphprotocol/indexer-tap-agent:latest
428408
```
429409

430-
Or build images locally from source
410+
Run both services with a shared TOML configuration file:
431411

432412
```sh
433-
# Indexer service
434-
docker build \
435-
--build-arg NPM_TOKEN=<npm-token> \
436-
-f Dockerfile.indexer-service \
437-
-t indexer-service:latest \
438-
# Indexer agent
439-
docker build \
440-
--build-arg NPM_TOKEN=<npm-token> \
441-
-f Dockerfile.indexer-agent \
442-
-t indexer-agent:latest \
443-
```
444-
445-
- Run the components
413+
# indexer-service-rs (horizontally scalable)
414+
docker run -p 7600:7600 -v ./config.toml:/config.toml \
415+
ghcr.io/graphprotocol/indexer-service-rs:latest --config /config.toml
446416

447-
```sh
448-
docker run -p 7600:7600 -it indexer-service:latest ...
449-
docker run -p 18000:8000 -it indexer-agent:latest ...
417+
# indexer-tap-agent (run exactly one instance)
418+
docker run -v ./config.toml:/config.toml \
419+
ghcr.io/graphprotocol/indexer-tap-agent:latest --config /config.toml
450420
```
451421

452-
**NOTE**: After starting the containers, the Indexer service should be accessible at [http://localhost:7600](http://localhost:7600) and the Indexer agent should be exposing the Indexer management API at [http://localhost:18000/](http://localhost:18000/).
422+
**NOTE**: After starting the containers, the indexer service should be accessible at [http://localhost:7600](http://localhost:7600) and the Indexer agent should be exposing the Indexer management API at [http://localhost:18000/](http://localhost:18000/).
453423

454424
#### Using K8s and Terraform
455425

@@ -488,30 +458,16 @@ graph-indexer-agent start \
488458
| pino-pretty
489459
```
490460

491-
#### Indexer service
461+
#### `indexer-service-rs` and `indexer-tap-agent`
462+
463+
Both `indexer-service-rs` and `indexer-tap-agent` are configured using a shared TOML file. See the [GraphTally guide](/indexing/tap) for detailed configuration and deployment instructions.
492464

493465
```sh
494-
SERVER_HOST=localhost \
495-
SERVER_PORT=5432 \
496-
SERVER_DB_NAME=is_staging \
497-
SERVER_DB_USER=<DB_USERNAME> \
498-
SERVER_DB_PASSWORD=<DB_PASSWORD> \
499-
graph-indexer-service start \
500-
--ethereum <MAINNET_ETH_ENDPOINT> \
501-
--ethereum-network mainnet \
502-
--mnemonic <MNEMONIC> \
503-
--indexer-address <INDEXER_ADDRESS> \
504-
--port 7600 \
505-
--metrics-port 7300 \
506-
--graph-node-query-endpoint http://localhost:8000/ \
507-
--graph-node-status-endpoint http://localhost:8030/graphql \
508-
--postgres-host localhost \
509-
--postgres-port 5432 \
510-
--postgres-username <DB_USERNAME> \
511-
--postgres-password <DB_PASSWORD> \
512-
--postgres-database is_staging \
513-
--network-subgraph-endpoint http://query-node-0:8000/subgraphs/id/QmUzRg2HHMpbgf6Q4VHKNDbtBEJnyp5JWCh2gUX9AV6jXv \
514-
| pino-pretty
466+
# Run indexer-service-rs
467+
indexer-service-rs --config /path/to/config.toml
468+
469+
# Run indexer-tap-agent (exactly one instance)
470+
indexer-tap-agent --config /path/to/config.toml
515471
```
516472

517473
#### Indexer CLI

0 commit comments

Comments
 (0)