feat(iota-graphql-rpc): Subscriptions support #9365
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Description of change This PR is the first iteration attempt to provide streaming support for the `iota-indexer` as a library componente. Currently it supports streaming of `StoredEvents` and `StoredTransactions`. It also supports filters. ## Links to any relevant issues fixes #8315 ## How the change has been tested created a separated binary which used the `iota-indexer` as a library, used `testnet` network as source of checkpoints for synchronization, subscribed to incoming events, applied different filter to assert that they were respected. - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [x] Patch-specific tests (correctness, functionality coverage) ### Infrastructure QA (only required for crates that are maintained by @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [x] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [x] Verification of API backward compatibility. ### Release Notes - [x] Indexer: Add streaming support for `events` and `transactions`.
…8698) # Description of change This PR adds the feature to add subscription support for `events` and `transactions` for GraphQL. For filtering capabilities it leverages the [`@oneOf`](https://graphql.org/blog/2025-09-04-multioption-inputs-with-oneof/) directive which does not allow more than one filter at a time for the input filter object. ## Links to any relevant issues fixes #8561 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - ran graphql tests ```shell cargo nextest run -p iota-graphql-rpc --features pg_integration --no-fail-fast --test-threads 1 ``` - used the iota binary to spin up an indexer and graphql instance and test the subscription queries ``` cargo r --features indexer -- start --force-regenesis --with-indexer --with-graphql ``` - used the integrated graphiQL ui to issue subscription queries - used a third party graphql client (Postman) to issue the same queries > [!NOTE] > At approximately 1k subscribers, the local machine reached its operational limit. The sheer intensity and concurrent demand of managing the network, indexer, GraphQL server, and a high volume of WebSocket connections prevented reliable testing. In future iteration we could add **Prometheus** metrics to track the total amount of active subscribers, how many subscribers experience lags, ect... ### Infrastructure QA (only required for crates that are maintained by @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [ ] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [ ] Verification of API backward compatibility. > [!NOTE] > The following patch does not affect the normal operation of the indexer, thus tests were skipped. ### Release Notes - [x] GraphQL: Add `Subscription` support for `Event` and `TransactionBlock`.
…r` into separate crate (#8949) # Description of change This PR isolates the `stream` module in `iota-indexer` into a separate crate `iota-indexer-streaming`. This will allow us to evolve and maintain it independently. ## Links to any relevant issues fixes #8839 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - started a local network with graphql server using the `iota` cli tool to assert that the streaming was working as expected. ### Infrastructure QA (only required for crates that are maintained by @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [ ] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [ ] Verification of API backward compatibility. > [!NOTE] > The patch does not affect the indexer normal operations, thus no tests were conducted. ### Release Notes - [x] Indexer: introduce to monorepo the `iota-indexer-streaming` crate which holds logic related to `iota-indexer` streaming capabilities.
# Description of change This PR adds Prometheus metrics to the `iota-indexer-streaming`. It allows to monitor active subscribers, latency of internal process and to have a clear picture how it behaves under load. ## Links to any relevant issues fixes #8711 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - Used the `iota` cli, to quickly spin up a local network with a indexer & graphql server, since the latter uses the `iota-indexer-streming` we were able to see the metrics since it already exposes a Prometheus server. ### Infrastructure QA (only required for crates that are maintained by @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [ ] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [ ] Verification of API backward compatibility. > [!NOTE] > This patch does not affect the indexer normal operations, thus no further tests were conducted. ### Release Notes - [x] Indexer: `iota-indexer-streaming` add Prometheus metrics.
…et` & `mainnet` networks (#9271) # Description of change This PR restricts the usage of the GraphQL Subscription feature only for `testnet` & `mainnet` networks. ## Links to any relevant issues fixes #9208 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) ### Infrastructure QA (only required for crates that are maintained by @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [ ] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [ ] Verification of API backward compatibility. > [!NOTE] > The path does not affect the normal operation of the indexer, thus no tests were conducted. ### Release Notes - [x] GraphQL: Usage of the Subscription feature is only supported on `devnet` and local netwroks, as the feature is still in alpha.
…es (#9294) # Description of change This PR modifies the log level from `INFO` to `DEBUG` for Subscription Responses in the Logger extension to avoid log pollution. Since every item received from the Subscription stream is treated as a GraphQL Response. With multiple subscribers at the same time, the logs become too polluted and hard to read. ## Links to any relevant issues fixes #9114 ## How the change has been tested - [x] Basic tests (linting, compilation, formatting, unit/integration tests) - [ ] Patch-specific tests (correctness, functionality coverage) - Ran the GraphQL server by using the iota cli ```shell RUST_LOG=off,iota_graphql_rpc=info cargo r --features indexer -- start --force-regenesis --with-indexer --with-graphql ``` When making Subscription requests, the logs were no longer present. If enabling `RUST_LOG=off,iota_graphql_rpc=debug`, the logs were visible. ```shell 2025-11-14T14:58:14.201495Z INFO iota_graphql_rpc::extensions::logger: [Validation] query_id=9eef57e6-edf0-42cd-829b-6e4b469b8919 session_id=127.0.0.1:52594 complexity=3 depth=2 2025-11-14T14:58:14.456023Z DEBUG iota_graphql_rpc::extensions::logger: [Subscription] {transactions: {__typename: "TransactionBlock", digest: "4aQ17xRBjo9bCgQLcKUdqatUjm7Yck2kUtbJTfwDfj4c"}} query_id=9eef57e6-edf0-42cd-829b-6e4b469b8919 session_id=127.0.0.1:52594 ``` ### Infrastructure QA (only required for crates that are maintained by @iotaledger/infrastructure) - [ ] Synchronization of the indexer from genesis for a network including migration objects. - [ ] Restart of indexer synchronization locally without resetting the database. - [ ] Restart of indexer synchronization on a production-like database. - [ ] Deployment of services using Docker. - [ ] Verification of API backward compatibility. > [!NOTE] > This patch does not affect the normal operations of the indexer; thus, no tests were conducted.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 6 Skipped Deployments
|
github-actions
bot
added
ts-sdk
iota-ci
added
infrastructure
muXxer
approved these changes
Nov 20, 2025
kodemartin
approved these changes
Nov 20, 2025
marc2332
approved these changes
Nov 20, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
This PR adds the GraphQL subscription support for
EventsandTrnsactions. Internally it leverages theInMemorybroker from the new introducediota-indexer-streamingcrate in the monorepo.Currently this feature is only available for
devnet& local networks, as the feature is still in alpha.Links to any relevant issues
fixes #8784
How the change has been tested
used the integrated graphiQL ui to issue subscription queries
used a third party graphql client (Postman) to issue the same queries
Basic tests (linting, compilation, formatting, unit/integration tests)
Patch-specific tests (correctness, functionality coverage)
Infrastructure QA (only required for crates that are maintained by @iotaledger/infrastructure)
Note
The following patch does not affect the normal operation of the indexer, thus tests were skipped.
Release Notes
iota-indexer-streamingcrate which holds logic related toiota-indexerstreaming capabilities for streamingeventsandtransactions.Subscriptionsupport forEventandTransactionBlock. Currently this feature is only available fordevnet& local networks, as the feature is still in alpha.