[GraphQL/Docs] Documenting Connections, Cursors, Pagination etc.

.gitignore

@@ -70,4 +70,4 @@ yarn-error.log*
 .env
 checkpoints_dir/*
 light_client.yaml
-docs/content/references/sui-api/graphql/*
+docs/content/references/sui-graphql/*

crates/sui-graphql-rpc/src/main.rs

@@ -34,8 +34,8 @@ async fn main() {
         Command::GenerateDocsExamples => {
             let mut buf: PathBuf = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
             // we are looking to put examples content in
-            // sui/docs/content/references/sui-api/graphql/examples.mdx
-            let filename = "docs/content/references/sui-api/graphql/examples.mdx";
+            // sui/docs/content/references/sui-graphql/examples.mdx
+            let filename = "docs/content/references/sui-graphql/examples.mdx";
             buf.pop();
             buf.pop();
             buf.push(filename);

docs/content/concepts/graphql-rpc/pagination.mdx

@@ -4,97 +4,92 @@ title: Pagination
 
 {@include: ../../snippets/info-graphql-release.mdx}
 
-
-## Multiple data in one query
-
-The previous RPC service required multiple queries for more complex data retrieval. Now, with GraphQL, you can specify the data you need in one single query.
-
-For example, the following query retrieves the first 20 transaction blocks (along with the digest, the sender's address, the gas object used to pay for the transaction, the gas price, and the gas budget) after a specific transaction block at epoch `97`. In the previous RPC, this would have required multiple API calls.
+GraphQL supports queries that fetch multiple kinds of data, potentially nested. For example, the following query retrieves the first 20 transaction blocks (along with the digest, the sender's address, the gas object used to pay for the transaction, the gas price, and the gas budget) after a specific transaction block at epoch `97`.
 
 ```graphql
-# Fetch the first 10 transactions for epoch 97
 query {
   epoch(id: 97) {
     transactionBlocks(first: 10) {
       pageInfo {
         hasNextPage
         endCursor
       }
-      edges {
-        cursor
-        node {
-          digest
-          sender {
-            address
-          }
-          effects {
-            gasEffects {
-              gasObject {
-                address
-              }
+      nodes {
+        digest
+        sender {
+          address
+        }
+        effects {
+          gasEffects {
+            gasObject {
+              address
             }
           }
-          gasInput {
-            gasPrice
-            gasBudget
-          }
+        }
+        gasInput {
+          gasPrice
+          gasBudget
         }
       }
     }
   }
 }
 ```
 
-:::info
-By default, the number of results is limited to 50 items. To learn more about how GraphQL works with pagination, check out the official [documentation](https://graphql.org/learn/pagination/) and our following section on pagination.
-:::
+But what happens if there are too many transactions to return in a single response? The service applies a [limit](./limits) on the maximum page size for variable size responses (like the `transactionBlock` query) and further results need to be fetched via [pagination](https://graphql.org/learn/pagination/).
 
-## Pagination
+## Connections
+Fields that return a paginated response accept at least the following optional parameters:
 
-Sui GraphQL RPC limits the number of items that are returned in a request. The default page limit size is set to 50 items. If there are more items requested than the max default page limit, use `cursors` to navigate between pages. A cursor refers to a specific position in the dataset. To access the start and end cursors of a page, use the `pageInfo` field  that exposes the cursor data and two additional fields indicating if there is a previous or next page. For example, if we want to get the `checkpoints` data:
+- `first`, a limit on page size that is met by dropping excess results from the end.
+- `after`, a cursor that bounds the results from below, exclusively.
+- `last`, a limit on page size that is met by dropping excess results from the start.
+- `before`, a cursor that bounds the results from above, exclusively.
 
-```graphql
-query {
-  checkpoints {
-    pageInfo {
-      hasPreviousPage
-      hasNextPage
-      startCursor
-      endCursor
-    }
-  }
-}
-```
+They also return a type that conforms to the [GraphQL Cursor Connections Specification](https://relay.dev/graphql/connections.htm), meaning its name ends in `Connection`, and it contains at least the following fields:
 
-The query's result is:
+- `pageInfo`, of type [PageInfo](../../references/sui-graphql/reference/objects/page-info.mdx), which indicates whethere there are further pages before or after the page returned.
+- `nodes`, the content of the paginated response, as a list of the type being paginated (`TransactionBlock` in the previous example).
+- `edges`, similar to `nodes` but associating each node with its [cursor](#cursors).
 
-```json
-{
-  "data": {
-    "checkpoints": {
-      "pageInfo": {
-        "hasPreviousPage": false,
-        "hasNextPage": true,
-        "startCursor": "MA",
-        "endCursor": "NA"
-      }
-    }
-  }
-}
-```
+## Cursors
+
+Cursors are opaque identifiers for paginated results. The only valid source for a cursor parameter (like `after` and `before`) is a cursor field from a previous paginated response (like `PageInfo.startCursor`, `PageInfo.endCursor`, or `Edge.cursor`). The underlying format of the cursor is an implementation detail, and is not guaranteed to remain fixed across versions of the GraphQL service, so it should not be relied on -- generating cursors out of thin air is not expected or supported.
+
+Cursors are used to bound results from below (with `after`) and above (with `before`). In both cases, the bound is exclusive, meaning it does not include the result that the cursor points to in the bounded region.
+
+### Consistency
+
+Cursors also guarantee **consistent** pagination. If the first paginated query reads the state of the network at checkpoint `X`, then a future call to fetch the next page of results using the cursors returned by the first query will continue to read from the network at checkpoint `X`, even if data for future checkpoints is now available.
+
+This property requires that cursors that are used together (for example when supplying an `after` and `before` bound) are fixed on the same checkpoint, otherwise the query will produce an error.
+
+### Available Range
 
-The `pageInfo.startCursor` and `pageInfo.endCursor` indicate the index of the first and last item in the response. You can use `pageInfo.hasNextPage` to determine if there is a next page, and then use the `endCursor`'s value and pass it to the `after` filter in the connection to traverse the next set of elements:
+The GraphQL service does not support consistent pagination for arbitrarily old cursors. A cursor can grow stale, if the checkpoint it is from is no longer in the **available range**. The upper- and lower-bounds of that range can be queried as follows:
 
 ```graphql
-query {
-  checkpoints(after: "MA") {
-    nodes {
-      digest
-    }
+{
+  availableRange {
+    first { sequenceNumber }
+    last { sequenceNumber }
   }
 }
 ```
 
+The results are the first and last checkpoint for which pagination will continue to work and produce a consistent result. At the time of writing the available range offers a 5 to 15 minute buffer period to finish pagination in.
+
+## Page Limits
+
+After results are bounded using cursors, a page size limit is applied using the `first` and `last` parameters. These parameters are required to be less than or equal to the max page size [limit](./limits), and if neither are provided, a default is selected. In addition to setting a limit, `first` and `last` control where excess elements are discarded from. For example, if there are `10` potential results -- `R0`, `R1`, ..., `R9` -- after cursor bounds have been applied, then
+
+- a limit of `first: 3` would select `R0`, `R1`, `R2`, and
+- a limit of `last: 3` would select `R7`, `R8`, `R9`.
+
 :::info
-When using pagination filters, you can only specify `first` or `last`, but not both.
+It is an error to apply both a `first` and a `last` limit.
 :::
+
+## Examples
+
+To see these principles put into practice, consult the examples for [paginating forwards](../../guides/developer/getting-started/graphql-rpc#page-forward) and [paginating backwards](../../guides/developer/getting-started/graphql-rpc#page-back) in the getting started guide.

docs/content/guides/developer/advanced/graphql-migration.mdx

@@ -9,7 +9,7 @@ import TabItem from '@theme/TabItem';
 
 This guide compares JSON-RPC queries to their equivalent GraphQL counterpart. While it is possible to systematically rewrite JSON-RPC queries (for example, `sui_getTotalTransactionBlocks`) to their GraphQL counterparts using this guide, it is recommended that you revisit your application's query patterns to take full advantage of the flexibility that GraphQL offers in serving queries that touch multiple potentially nested endpoints (for example transactions, balances, coins), and use the following examples to get a flavor of how the two APIs express similar concepts.
 
-For a comprehensive list of all available GraphQL APIs, consult the [reference](../../../references/sui-api/graphql).
+For a comprehensive list of all available GraphQL APIs, consult the [reference](../../../references/sui-graphql).
 
 ### Example 1: Get total transaction blocks
 

docs/content/guides/developer/getting-started/graphql-rpc.mdx

@@ -14,7 +14,7 @@ The online IDE is available for [mainnet](https://sui-mainnet.mystenlabs.com/gra
 - Both [mainnet](https://sui-mainnet.mystenlabs.com/graphql) and [testnet](https://sui-testnet.mystenlabs.com/graphql) services are rate-limited to keep network throughput optimized.
 :::
 
-For more details about some concepts used in the examples below, please see the [GraphQL concepts](concepts/graphql-rpc.mdx) page, and consult the [reference](../../../references/sui-api/graphql) for full documentation on the supported schema.
+For more details about some concepts used in the examples below, please see the [GraphQL concepts](concepts/graphql-rpc.mdx) page, and consult the [reference](../../../references/sui-graphql) for full documentation on the supported schema.
 
 
 ## Discovering the schema
@@ -316,6 +316,52 @@ fragment DynamicFieldValueSelection on DynamicFieldValue {
 }
 ```
 
+## Paginating checkpoints forward, five at a time {#page-forward}
+
+```graphql
+query ($after: String) {
+  checkpoints(first: 5, after: $after) {
+    pageInfo {
+      hasNextPage
+      endCursor
+    }
+    nodes {
+      digest
+      timestamp
+    }
+  }
+}
+```
+
+Sets up a paginated query, starting at the genesis checkpoint, reading five checkpoints at a time, in increasing order of sequence number. The value of `pageInfo.hasNextPage` determines whether there is another page to be read, and the value of `pageInfo.endCursor` is fed back in as the cursor to read `$after`.
+
+:::info
+This example uses GraphQL [variables](/concepts/graphql-rpc/variables) and [pagination](/concepts/graphql-rpc/pagination).
+:::
+
+## Paginating checkpoints backwards, five at a time {#page-back}
+
+```graphql
+query ($before: String) {
+  checkpoints(last: 5, before: $before) {
+    pageInfo {
+      hasPreviousPage
+      startCursor
+    }
+    nodes {
+      digest
+      timestamp
+    }
+  }
+}
+```
+
+Sets up a paginated query, starting at the latest indexed checkpoint, reading five checkpoints at a time, in decreasing order of sequence number. The value of `pageInfo.hasPreviousPage` determines whether there is another page to be read, and the value of `pageInfo.startCursor` is fed back in as the cursor to read `$before`.
+
+:::info
+This example uses GraphQL [variables](/concepts/graphql-rpc/variables) and [pagination](/concepts/graphql-rpc/pagination).
+:::
+
 ## Executing a transaction
 
 Transaction execution takes in two arguments, `txBytes` and `signatures`. `txBytes` is the serialized unsigned transaction data, which can be generated when using the Sui CLI's `client call` [command](/references/cli/client), to call a Move function by passing the `--serialize-unsigned-transaction` flag. The `signatures` can be generated using Sui CLI's [keytool](/references/cli/keytool) command `sui keytool sign`. More information on Sui CLI can be found [here](/references/cli).

docs/content/references.mdx

@@ -15,7 +15,7 @@ Reference the Sui framework documentation to understand the Move code that power
 <Card title="Sui Framework" href="https://github.com/MystenLabs/sui/tree/main/crates/sui-framework/docs">
 Sui framework reference for the Move packages that power the blockchain.
 </Card>
-<Card title="GraphQL RPC" href="references/sui-api/graphql"/>
+<Card title="GraphQL RPC" href="references/sui-graphql"/>
 <Card title="Sui API reference" href="references/sui-api">
 API reference for the SUI API in JSON RPC format.
 </Card>

docs/content/references/sui-api/graphql.mdx → docs/content/references/sui-graphql.mdx

@@ -1,6 +1,6 @@
 ---
-title: Sui GraphQL RPC Overview
-description: The Sui GraphQL RPC is a public service that enables interacting with the Sui network. 
+title: Sui GraphQL API
+description: The Sui GraphQL RPC is a public service that enables interacting with the Sui network.
 ---
 
 The Sui GraphQL RPC is a public service that enables interacting with the Sui [network](https://sui.io/networkinfo).
@@ -9,4 +9,4 @@ To get started with the GraphQL RPC, check out the [Getting Started](/guides/dev
 
 Check out the `Queries` reference page on the left sidebar for the root types related to querying the service (e.g., doing a `dryRun`). If you'd like to execute a transaction on the network, check out the `Mutations` page in the left sidebar.
 
-As Sui uses an object-centric model, it might be of interest to explore the [Object](/references/sui-api/graphql/reference/objects/object.mdx) and [Address](/references/sui-api/graphql/reference/objects/address.mdx) types too. For other types and reference API, consult the pages under GraphQL RPC API shown in the left sidebar.
+As Sui uses an object-centric model, it might be of interest to explore the [Object](/references/sui-graphql/reference/objects/object.mdx) and [Address](/references/sui-graphql/reference/objects/address.mdx) types too. For other types and reference API, consult the pages under GraphQL RPC API shown in the left sidebar.

docs/content/sidebars/references.js

@@ -14,26 +14,12 @@ const references = [
 	},
 	{
 		type: 'category',
-		label: 'Sui API',
+		label: 'Sui JSON-RPC API',
 		link: {
 			type: 'doc',
 			id: 'references/sui-api',
 		},
 		items: [
-			{
-				type: 'category',
-				label: 'GraphQL RPC API',
-				link: {
-					type: 'doc',
-					id: 'references/sui-api/graphql'
-				},
-				items: [
-					{
-						type: 'autogenerated',
-						dirName: 'references/sui-api/graphql/reference',
-					},
-				],
-			},
 			{
 				type: 'link',
 				label: 'API Reference',
@@ -42,6 +28,20 @@ const references = [
 			'references/sui-api/rpc-best-practices',
 		],
 	},
+	{
+		type: 'category',
+		label: 'Sui GraphQL API',
+		link: {
+			type: 'doc',
+			id: 'references/sui-graphql',
+		},
+		items: [
+			{
+				type: 'autogenerated',
+				dirName: 'references/sui-graphql/reference',
+			},
+		],
+	},
 	{
 		type: 'category',
 		label: 'Sui CLI',

docs/site/docusaurus.config.js

@@ -44,7 +44,7 @@ const config = {
         schema:
           "../../crates/sui-graphql-rpc/schema/current_progress_schema.graphql",
         rootPath: "../content", // docs will be generated under rootPath/baseURL
-        baseURL: "references/sui-api/graphql/reference",
+        baseURL: "references/sui-graphql/reference",
         loaders: {
           GraphQLFileLoader: "@graphql-tools/graphql-file-loader",
         },