Skip to content

Commit

Permalink
change all checkpoints.mainnet|testnet.sui.io references to gcp endpo…
Browse files Browse the repository at this point in the history
  • Loading branch information
johnjmartin authored Dec 16, 2024
1 parent e6414a9 commit e1eb239
Show file tree
Hide file tree
Showing 6 changed files with 19 additions and 15 deletions.
2 changes: 1 addition & 1 deletion crates/sui-bridge-indexer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ bridge-indexer --config-path config.yaml

```yaml
---
remote_store_url: https://checkpoints.mainnet.sui.io
remote_store_url: https://storage.googleapis/mysten-mainnet-checkpoints
eth_rpc_url: {eth rpc url}
sui_rpc_url: {sui rpc url}

Expand Down
8 changes: 4 additions & 4 deletions docs/content/guides/developer/advanced/custom-indexer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ Data ingestion for your indexer supports several checkpoint stream sources.

The most straightforward stream source is to subscribe to a remote store of checkpoint contents. Mysten Labs provides the following buckets:

- Testnet: `https://checkpoints.testnet.sui.io`
- Mainnet: `https://checkpoints.mainnet.sui.io`
- Testnet: `https://storage.googleapis.com/mysten-testnet-checkpoints`
- Mainnet: `https://storage.googleapis.com/mysten-mainnet-checkpoints`

The checkpoint files are stored in the following format: `https://checkpoints.testnet.sui.io/<checkpoint_id>.chk`. You can download the checkpoint file by sending an HTTP GET request to the relevant URL. Try it yourself for checkpoint 1 at [https://checkpoints.testnet.sui.io/1.chk](https://checkpoints.testnet.sui.io/1.chk).
The checkpoint files are stored in the following format: `https://storage.googleapis.com/mysten-testnet-checkpoints/<checkpoint_id>.chk`. You can download the checkpoint file by sending an HTTP GET request to the relevant URL. Try it yourself for checkpoint 1 at [https://storage.googleapis/mysten-testnet-checkpoints/1.chk](https://storage.googleapis/mysten-testnet-checkpoints/1.chk).

```mermaid
flowchart LR
Expand Down Expand Up @@ -112,7 +112,7 @@ Specify both a local and remote store as a fallback to ensure constant data flow
```rust
executor.run(
PathBuf::from("./chk".to_string()), // path to a local directory
Some("https://checkpoints.testnet.sui.io".to_string()), // Remote Checkpoint Store
Some("https://storage.googleapis/mysten-testnet-checkpoints".to_string()), // Remote Checkpoint Store
vec![], // optional remote store access options
ReaderOptions::default(),
exit_receiver,
Expand Down
18 changes: 11 additions & 7 deletions examples/custom-indexer/rust/remote_reader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@

use anyhow::Result;
use async_trait::async_trait;
use sui_data_ingestion_core::{setup_single_workflow, Worker};
use sui_types::full_checkpoint_content::CheckpointData;
use sui_data_ingestion_core::{Worker, setup_single_workflow};

struct CustomWorker;

Expand All @@ -14,20 +14,24 @@ impl Worker for CustomWorker {
async fn process_checkpoint(&self, checkpoint: &CheckpointData) -> Result<()> {
// custom processing logic
// print out the checkpoint number
println!("Processing checkpoint: {}", checkpoint.checkpoint_summary.to_string());
println!(
"Processing checkpoint: {}",
checkpoint.checkpoint_summary.to_string()
);
Ok(())
}
}

#[tokio::main]
async fn main() -> Result<()> {
let (executor, term_sender) = setup_single_workflow(
CustomWorker,
"https://checkpoints.testnet.sui.io".to_string(),
0, /* initial checkpoint number */
5, /* concurrency */
"https://storage.googleapis.com/mysten-testnet-checkpoints".to_string(),
0, /* initial checkpoint number */
5, /* concurrency */
None, /* extra reader options */
).await?;
)
.await?;
executor.await?;
Ok(())
}
2 changes: 1 addition & 1 deletion nre/helm/indexer/values.reader.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

args: ["--db-url", "$(DB_URL)", "--rpc-client-url", "https://public-rpc.mainnet.sui.io", "--remote-store-url", "https://checkpoints.mainnet.sui.io", "--rpc-server-worker"]
args: ["--db-url", "$(DB_URL)", "--rpc-client-url", "https://public-rpc.mainnet.sui.io", "--remote-store-url", "https://storage.googleapis/mysten-mainnet-checkpoints", "--rpc-server-worker"]

podAnnotations:
app: sui-indexer-reader
Expand Down
2 changes: 1 addition & 1 deletion nre/helm/indexer/values.writer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

args: ["--db-url", "$(DB_URL)", "--rpc-client-url", "https://public-rpc.mainnet.sui.io", "--remote-store-url", "https://checkpoints.mainnet.sui.io", "--fullnode-sync-worker"]
args: ["--db-url", "$(DB_URL)", "--rpc-client-url", "https://public-rpc.mainnet.sui.io", "--remote-store-url", "https://storage.googleapis/mysten-mainnet-checkpoints", "--fullnode-sync-worker"]

# only 1 writer needed
replicaCount: 1
Expand Down
2 changes: 1 addition & 1 deletion nre/helm/indexer/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ env:
RPC_CLIENT:
value: "https://public-rpc.mainnet.sui.io"
REMOTE_STORE:
value: "https://checkpoints.mainnet.sui.io"
value: "https://storage.googleapis/mysten-mainnet-checkpoints"
DB_URL:
valueFrom:
secretKeyRef:
Expand Down

0 comments on commit e1eb239

Please sign in to comment.