The canonical reference for everything outside the binary: the gRPC wire surface, the command-line interface, and the published Rust crates. This chapter is descriptive rather than tutorial — for behavior and rationale see Client API and Usage, Consensus Integration, and The Allocator. For deployment shapes and operational tuning see Operations and Deployment.
The wire source of truth lives in the tsoracle-proto and tsoracle-standalone .proto files; the CLI source of truth lives in crates/tsoracle-bin/src/cli.rs; the Rust crate reference lives on docs.rs. Anything stated here that contradicts those sources is a documentation bug — please file an issue.
| Surface | Audience | Source of truth | Reference |
|---|---|---|---|
gRPC over HTTP/2 — tsoracle.v1.TsoService |
Applications using tsoracle-client or any gRPC client |
crates/tsoracle-proto/proto/tsoracle/v1/tso.proto |
§ gRPC wire reference |
gRPC over HTTP/2 — tsoracle.admin.v1.MembershipAdmin |
Cluster operators (openraft only) | crates/tsoracle-standalone/proto/admin.proto |
§ Admin gRPC reference |
CLI — tsoracle binary |
Operators running the standalone server | crates/tsoracle-bin/src/cli.rs |
§ CLI reference |
| Rust crates on crates.io | Embedders and driver authors | the per-crate lib.rs |
§ Rust crate API and docs.rs/tsoracle |
Seven RPCs — GetTs (timestamps), GetSeq / GetSeqBatch (gapless sequences), GetCurrentMaxSafe (legacy safe-point reads), AcquireLease, RenewLease, ReleaseLease, and GetSafeFrontier (lease-aware safe-frontier reads). Default port is 50551 (loopback by default; configurable via --listen). The server speaks gRPC over HTTP/2 in either plaintext (h2c) or TLS, controlled by --tls-cert/--tls-key/--tls-client-ca.
Allocate count consecutive timestamps. Only the current leader serves this RPC.
Request — GetTsRequest:
| Field | Type | Constraint | Meaning |
|---|---|---|---|
count |
uint32 (1) |
>= 1; upper bound is the 18-bit logical capacity, LOGICAL_MAX + 1 = 262_144 |
Number of consecutive timestamps to allocate. |
Response — GetTsResponse (on success):
| Field | Type | Meaning |
|---|---|---|
physical_ms |
uint64 (1) |
46-bit milliseconds-since-Unix-epoch component of the grant. |
logical_start |
uint32 (2) |
18-bit logical counter. The response covers logicals [logical_start, logical_start + count - 1] at this physical_ms. |
count |
uint32 (3) |
Equals the request's count. The server never returns a partial grant. |
epoch_hi |
uint64 (4) |
Upper 64 bits of the 128-bit leader epoch that issued this grant. |
epoch_lo |
uint64 (5) |
Lower 64 bits of the leader epoch. Lexicographic (epoch_hi, epoch_lo) order equals numeric order; per-node strictly monotone across leader terms. |
Success contract:
- Each issued timestamp packs as
physical_ms << 18 | logical, wherelogical ∈ [logical_start, logical_start + count - 1]. - Across all successful
GetTsresponses from any node since cluster genesis, the packed integer values are strictly increasing and unique — no duplicates and no regression, even across leader transitions. - See Monotonicity proof for the argument.
Error statuses returned from GetTs:
tonic::Code |
When | Trailer |
|---|---|---|
FAILED_PRECONDITION |
This node is not currently the leader. | tsoracle-leader-hint-bin |
INVALID_ARGUMENT |
count == 0 or count > LOGICAL_MAX + 1 (i.e. count > 262_144). |
— |
OUT_OF_RANGE |
The requested grant would exceed the 46-bit physical_ms field or the 18-bit logical field. |
— |
UNAVAILABLE |
The leader's persist path returned a transient driver error (the window advance did not commit). Clients may retry; tsoracle-client does. |
— |
INTERNAL |
A bug-class error: window-exhausted with no extension available, persist hardware failure, or a violated allocator invariant. Clients should not retry without inspection. | — |
tsoracle-client filters the tsoracle-leader-hint-bin trailer out of FAILED_PRECONDITION errors before surfacing the Status to caller code (it is consumed as a routing hint, not propagated). Other gRPC clients see the trailer as standard binary metadata.
Read the current safe-point — the highest physical_ms at which any timestamp issued at or before that millisecond is durably backed by the leader's persisted window. External coordinators poll this to fence reads/writes that depend on a durable timestamp watermark.
Request — GetCurrentMaxSafeRequest: empty.
Response — GetCurrentMaxSafeResponse:
| Field | Type | Meaning |
|---|---|---|
max_safe_physical_ms |
uint64 (1) |
Safe-point in physical-millisecond units (not the 64-bit packed form). Equal to persisted_high_water - window_ahead_ms - 1 on the leader when a window has been admitted; zero otherwise. |
epoch_hi |
uint64 (2) |
Upper 64 bits of the leader epoch attached to this value. |
epoch_lo |
uint64 (3) |
Lower 64 bits of the leader epoch. |
Response contract:
- Leaders return the authoritative current value.
- Followers return
max_safe_physical_ms = 0and the zero epoch. Target the leader for an authoritative value; followers do not redirect this RPC. - A leader that has not yet admitted its first window (cold start before the failover fence completes) also returns
0. - Long-lived pollers should fence stale-leader values by tracking
(epoch_hi, epoch_lo)and discarding values from a smaller epoch after a larger one.
This RPC does not return FAILED_PRECONDITION, and the response carries no trailer.
Allocate a contiguous block of count dense ordinals for a named key — a gapless, never-reused run. Only the current leader serves this RPC. Dense support is driver-specific (see Driver Comparison); a driver without it returns UNIMPLEMENTED. This RPC is non-idempotent — see the error table.
Request — GetSeqRequest:
| Field | Type | Constraint | Meaning |
|---|---|---|---|
key |
string (1) |
non-empty UTF-8, <= MAX_SEQ_KEY_LEN (128 bytes) |
Names the independent counter the block is drawn from. |
count |
uint32 (2) |
>= 1; upper bound is the server's configured max_seq_count (default 65_536) |
Size of the contiguous block to reserve. |
Response — GetSeqResponse (on success):
| Field | Type | Meaning |
|---|---|---|
key |
string (1) |
Echoes the request key. |
start |
uint64 (2) |
First ordinal of the block — the durably-committed pre-advance counter value. The block is [start, start + count). |
count |
uint32 (3) |
Equals the request's count. The server never returns a partial grant. |
epoch |
EpochWire (4) |
The leader epoch that issued the block, bundled as a nested message (always present on success). Same EpochWire shape as the leader-hint trailer. |
Success contract:
- The block is
[start, start + count)— every ordinal present, none skipped. - Across all successful
GetSeqresponses for a givenkey, the issued ordinals are dense and never reused: each grant begins exactly where the previous one ended, even across leader transitions and restarts. A fresh key starts at0. - Each
keyis an independent counter; different keys advance separately.
Error statuses returned from GetSeq:
tonic::Code |
When | Trailer | Pre-commit-certain? |
|---|---|---|---|
FAILED_PRECONDITION |
This node is not the leader (carries the hint), or the dense counter would overflow u64 (no hint). |
tsoracle-leader-hint-bin on the not-leader case only |
Yes |
INVALID_ARGUMENT |
Empty/oversized key, count == 0, or count over the server's max_seq_count. |
— | Yes |
RESOURCE_EXHAUSTED |
The number of distinct keys reached the driver's cardinality cap. | — | Yes |
UNIMPLEMENTED |
This consensus driver has no dense-sequence support. | — | Yes (nothing advanced) |
UNAVAILABLE |
A transient driver fault (storage hiccup, momentary quorum loss); the advance did not commit. | — | Yes |
INTERNAL |
A permanent driver fault or violated invariant reached after the request was admitted; the durable advance may have committed. | — | No — ambiguous |
Because the dense path is non-idempotent, tsoracle-client treats any post-send failure that is not provably pre-commit-certain (a transport timeout after the request was sent, or INTERNAL) as ClientError::SeqUncertain and does not retry it — the advance may already have committed. Every pre-commit-certain status above is handled or retried normally. See GetSeq — dense, gapless sequences for the caller's reconciliation contract.
Acquire or idempotently re-acquire a stamping lease for an opaque holder group. Only the current leader serves this RPC.
Request — AcquireLeaseRequest:
| Field | Type | Constraint | Meaning |
|---|---|---|---|
holder |
bytes (1) |
1..=128 bytes | Opaque holder-group key; compared only for byte equality. |
holder_epoch |
uint64 (2) |
caller-defined monotone value within holder |
Equal epoch is idempotent, higher epoch supersedes, lower epoch rejects. |
ttl_ms |
uint64 (3) |
server floor/ceiling, defaults 5s/300s | Requested lease TTL in milliseconds. |
Response — AcquireLeaseResponse:
| Field | Type | Meaning |
|---|---|---|
lease_id |
uint64 (1) |
Stable identity for renew/release. Equal to the acquire-time committed high-water value. |
ts_upper_bound |
uint64 (2) |
Highest physical millisecond the holder may stamp. |
expires_at_ms |
uint64 (3) |
Server-clock expiry in unix milliseconds. Expiry is now_ms >= expires_at_ms. |
epoch |
EpochWire (4) |
Leader epoch that granted the lease. |
Error statuses returned from AcquireLease:
tonic::Code |
When | Trailer |
|---|---|---|
FAILED_PRECONDITION |
This node is not the leader, or the requested holder epoch is stale. | Leader hint on not-leader only |
INVALID_ARGUMENT |
Empty/oversized holder or TTL outside server bounds. | — |
UNIMPLEMENTED |
The consensus driver does not support durable leases. | — |
UNAVAILABLE |
A transient high-water or lease-set persist failure occurred before the grant committed in memory. | — |
INTERNAL |
Permanent driver fault or allocator invariant violation. | — |
Advance a live lease's bound and expiry, re-arming its acquire-time TTL.
Request — RenewLeaseRequest: lease_id (uint64, field 1).
Response — RenewLeaseResponse: ts_upper_bound (uint64, field 1), expires_at_ms (uint64, field 2), and epoch (EpochWire, field 3).
Error statuses returned from RenewLease:
tonic::Code |
When | Trailer |
|---|---|---|
FAILED_PRECONDITION |
Not leader, expired lease, or superseded lease. | Leader hint on not-leader only |
NOT_FOUND |
Unknown lease id. | — |
UNIMPLEMENTED |
The consensus driver does not support durable leases. | — |
UNAVAILABLE |
Transient persist failure before the renewal committed in memory. | — |
INTERNAL |
Permanent driver fault or invariant violation. | — |
Surrender a lease immediately. Release is idempotent: unknown, expired, or already-released ids succeed without a durable write.
Request — ReleaseLeaseRequest: lease_id (uint64, field 1).
Response — ReleaseLeaseResponse: empty.
Error statuses returned from ReleaseLease:
tonic::Code |
When | Trailer |
|---|---|---|
FAILED_PRECONDITION |
This node is not the leader. | Leader hint |
UNIMPLEMENTED |
The consensus driver does not support durable leases. | — |
UNAVAILABLE |
Transient lease-set persist failure. | — |
INTERNAL |
Permanent driver fault or invariant violation. | — |
Read the lease-aware safe frontier: min(unexpired lease ts_upper_bound values, committed high-water), in physical-millisecond units.
Request — GetSafeFrontierRequest: empty.
Response — GetSafeFrontierResponse:
| Field | Type | Meaning |
|---|---|---|
frontier_physical_ms |
uint64 (1) |
Lease-aware frontier; zero on followers or before a leader admits a window. |
epoch_hi |
uint64 (2) |
Upper 64 bits of the leader epoch. |
epoch_lo |
uint64 (3) |
Lower 64 bits of the leader epoch. |
Followers return frontier_physical_ms = 0 and zero epoch, matching GetCurrentMaxSafe; this RPC does not redirect.
See § Admin gRPC reference below.
When GetTs returns FAILED_PRECONDITION, the server attaches a binary metadata trailer named tsoracle-leader-hint-bin whose body is the postcard encoding of a LeaderHint message.
LeaderHint field |
Type | Semantics |
|---|---|---|
leader_endpoint |
optional string (1) |
Scheme-less host:port of the leader's TsoService endpoint as known to this node. Absent on cold start or right after a leadership change. The string must not carry a http:// or https:// scheme — the client rejects a scheme-bearing hint to defend against TLS downgrade. |
leader_epoch |
optional EpochWire (2) |
The leader's 128-bit epoch, bundled as a nested message so a half-populated epoch is unrepresentable on the wire. Either both halves are present (epoch.hi, epoch.lo) or neither is. |
EpochWire {
uint64 hi = 1; // more significant half
uint64 lo = 2; // less significant half
}
Contract:
- The hint is advisory, not authoritative. A stale hint costs an extra client round-trip; it does not affect correctness.
- A hint whose
leader_epochis strictly smaller than the client's cached leader epoch is dropped (the client treats it as evidence of a delayed response from an old term). - The metadata key is suffixed
-binper gRPC convention; consumers must request binary metadata and base64-decode if their gRPC stack does not handle that automatically.
See The leader-hint trailer for the consumption protocol on the client side.
The membership-admin surface is bound to a separate listen address (--admin-listen), separate from the public TsoService port, and is implemented only by the openraft driver. The file and paxos drivers return UNSUPPORTED for every mutating RPC; ListMembers works on all drivers and reports a fixed view for paxos/file.
All mutating RPCs are idempotent on the target id: re-issuing a successful change is a no-op. The server serializes them on an internal lock — two reconfigurations cannot race. Mutating RPCs are leader-only: a non-leader returns ok=false, error=NOT_LEADER, and (when known) leader_admin_endpoint set to the leader's admin port. The tsoracle admin CLI uses this for one-step re-routing.
| RPC | Request | Response | Leader-only |
|---|---|---|---|
ListMembers |
ListMembersRequest{} |
MembershipView |
No |
AddLearner |
AddLearnerRequest{ id, raft_addr, service_endpoint, admin_endpoint } |
ChangeResponse |
Yes |
Promote |
PromoteRequest{ id } |
ChangeResponse |
Yes |
RemoveNode |
RemoveNodeRequest{ id } |
ChangeResponse |
Yes |
ActivateFormat |
ActivateFormatRequest{ target } |
ChangeResponse |
Yes |
| Field | Type | Meaning |
|---|---|---|
members |
repeated MemberEntry (1) |
The membership as this node currently sees it. |
has_leader |
bool (2) |
True iff leader is meaningful. |
leader |
uint64 (3) |
Cluster-current leader's node id. Meaningful only when has_leader == true — consumers must not interpret leader == 0 as "no leader" (proto3 has no optional uint64, so this pair encodes Option<u64>). |
| Field | Type | Meaning |
|---|---|---|
id |
uint64 (1) |
Cluster-unique node id, stable across address changes. |
role |
MemberRole (2) |
VOTER (counted in quorum) or LEARNER (replicates, not in quorum). |
raft_addr |
string (3) |
host:port of the node's openraft peer transport. Consensus traffic. |
service_endpoint |
string (4) |
Scheme-less host:port of the node's public TsoService endpoint. What clients use; what leader hints carry. |
admin_endpoint |
string (5) |
host:port of the node's admin gRPC endpoint. What NOT_LEADER responses on the admin port redirect to. |
The three address fields are required at AddLearner time and stored verbatim in the membership log; the server does not parse or canonicalize them.
| Field | Type | Meaning |
|---|---|---|
ok |
bool (1) |
Authoritative success flag. Always check this before reading error. |
error |
AdminErrorKind (2) |
Typed error kind. Meaningful only when ok == false. |
leader_admin_endpoint |
string (3) |
Set when error == NOT_LEADER and a leader admin endpoint is known. Empty string means no hint available (cold start or no leader yet). |
message |
string (4) |
Human-readable detail — for example, the driver's error string when error == DRIVER. Empty on success. Suitable for operator display; programmatic dispatch should be on error, not on this string. |
| Value | When |
|---|---|
ADMIN_ERROR_KIND_UNSPECIFIED (0) |
proto3 default. The server never emits this with ok == false; the wire never carries error=UNSPECIFIED paired with ok=false. |
NOT_LEADER (1) |
Mutating RPC reached a follower. leader_admin_endpoint carries the leader's admin port when known. Note that ActivateFormat's NOT_LEADER always leaves leader_admin_endpoint empty (the underlying error is a unit variant). |
UNSUPPORTED (2) |
This driver does not implement runtime membership changes. Returned by paxos and file for every mutating RPC. |
NOT_MEMBER (3) |
Promote against an id not currently in the membership. |
NOT_CAUGHT_UP (4) |
Reserved. The current openraft impl folds learner-not-caught-up into DRIVER with the upstream error text; this variant is reserved for a future explicit catch-up check. |
WOULD_LOSE_QUORUM (5) |
RemoveNode against the last remaining voter. Other quorum-loss shapes (e.g. removing two of three) are the operator's responsibility. |
TIMEOUT (6) |
Reserved. Not currently produced — the openraft impl folds timeouts into DRIVER. Reserved for future explicit-deadline plumbing. |
DRIVER (7) |
Catch-all for an upstream driver error not captured by a typed variant. message carries the driver's error string. |
MEMBERS_BELOW_TARGET (8) |
ActivateFormat rejected by the all-members gate: at least one member's max_readable_version is below the requested target. |
TARGET_OUT_OF_RANGE (9) |
ActivateFormat apply-arm defense-in-depth: target is outside the local binary's readable range. |
MEMBERSHIP_CHANGED (10) |
ActivateFormat apply-keyed no-op: the gated set drifted out of the membership by the entry's own log position. Operator re-gates and re-issues. |
Adding new variants is backwards compatible: clients that don't recognize a value can fall back to the message field.
AddLearner— Idempotent onid: a node already in the membership is a no-op even if the addresses differ. Blocks until the learner is registered. Errors:NOT_LEADER,UNSUPPORTED,DRIVER.Promote— Idempotent onid(already-voter is a no-op). Errors:NOT_LEADER,NOT_MEMBER,UNSUPPORTED,DRIVER(which today subsumes the upstream "learner not caught up" case).RemoveNode— Idempotent onid(unknown ids returnok). Refuses to remove the last remaining voter (WOULD_LOSE_QUORUM). Removing the current leader is allowed; openraft commits the removal while still leader, then steps down. Errors:NOT_LEADER,WOULD_LOSE_QUORUM,UNSUPPORTED,DRIVER.ActivateFormat— Initiate a format-version activation totarget. Runs the all-members capability gate, proposes the bump via raft, and reports the apply-keyed outcome. Errors:NOT_LEADER,MEMBERS_BELOW_TARGET,TARGET_OUT_OF_RANGE,MEMBERSHIP_CHANGED,UNSUPPORTED,DRIVER. See Format-migration upgrade for the broader story.
Source of truth: crates/tsoracle-bin/src/cli.rs. The crate is published as tsoracle and produces a single binary, also named tsoracle.
tsoracle [SERVE-FILE-FLAGS]
tsoracle serve file [SERVE-FILE-FLAGS]
tsoracle serve openraft [SERVE-COMMON-FLAGS] [OPENRAFT-FLAGS]
tsoracle serve paxos [SERVE-COMMON-FLAGS] [PAXOS-FLAGS]
tsoracle init --seed-physical-ms <N> [--state-dir <PATH>]
tsoracle admin <SUBCOMMAND> ... # openraft feature only
Bare tsoracle (no subcommand) is exactly equivalent to tsoracle serve file with all defaults. tsoracle --help and tsoracle <SUBCOMMAND> --help print clap-generated usage; this section is the long-form reference.
These come from CommonServeArgs. Every serve mode accepts them with the same defaults.
| Flag | Type | Default | Meaning |
|---|---|---|---|
--listen |
SocketAddr |
127.0.0.1:50551 |
Client-facing gRPC listen address (the TsoService port). |
--window-ahead |
duration (humantime) | 3s |
How far ahead of wall-clock the leader advances its persisted high-water. Higher values reduce extension frequency at the cost of a larger time-to-live for a stale leader's window. See Sizing window_ahead. |
--failover-advance |
duration | 1s |
How far past the recovered high-water the failover fence advances on leadership gain. Must exceed the worst-case wall-clock skew between leaders. See Sizing failover_advance. |
--lease-ttl-floor |
duration | 5s |
Lower bound on requested lease TTLs. |
--lease-ttl-ceiling |
duration | 300s |
Upper bound on requested lease TTLs. |
--log |
string | info |
tracing env-filter directive (info, debug, tsoracle_server=debug,info, etc.). |
--tls-cert |
path | — | PEM server certificate chain for the client gRPC API. Enables TLS on --listen. |
--tls-key |
path | — | PEM private key paired with --tls-cert. |
--tls-client-ca |
path | — | PEM CA used to verify client certificates. Enables client mTLS on the API port. |
--tls-cert and --tls-key must be supplied together; supplying --tls-client-ca without them is an error.
Single-node, fsync-durable. The default mode. No replication — losing the node loses availability.
Common flags (above), plus:
| Flag | Type | Default | Meaning |
|---|---|---|---|
--state-dir |
path | ./tsoracle-data |
Where to persist window state. Created on first start; first run against an empty directory begins at high-water 0. |
HA via openraft. Three or more nodes. Compiled in when the binary is built with the openraft feature (enabled by default on crates.io releases).
Common flags (above), plus:
| Flag | Type | Default | Meaning |
|---|---|---|---|
--id |
u64 |
required | This node's numeric raft id. Must be unique across the cluster. |
--raft-addr |
SocketAddr |
required | Bind address for the raft peer transport (consensus traffic). Plaintext on loopback; routable bind requires the --peer-tls-* triple or --allow-insecure-peer. See Peer-port trust boundary. |
--raft-dir |
path | required | Directory for raft log + state-machine data. Persisted across restarts. |
--bootstrap |
bool | false |
Initialize the cluster on this node, first boot only. Used exactly once per cluster lifetime. |
--members |
string | — | Initial membership, only valid with --bootstrap. Format: id=raft_host:port/service_host:port/admin_host:port,.... |
--heartbeat-ms |
u64 |
250 |
Raft heartbeat interval. |
--election-min-ms |
u64 |
1000 |
Lower bound on the randomized election timeout. |
--election-max-ms |
u64 |
2000 |
Upper bound on the randomized election timeout. |
--admin-listen |
SocketAddr |
— | Bind address for the membership-admin gRPC server. Omitting the flag serves no admin surface. Plaintext on loopback; non-loopback bind requires the --admin-tls-* triple. |
--admin-tls-cert |
path | — | PEM server certificate for the admin gRPC server. Enables admin mTLS — needs all three --admin-tls-* flags. |
--admin-tls-key |
path | — | PEM private key paired with --admin-tls-cert. |
--admin-tls-ca |
path | — | PEM CA used to verify connecting admin clients. Operator-dedicated; not the peer CA. |
--peer-tls-cert |
path | — | PEM node certificate for the peer transport. Enables peer mTLS — needs all three --peer-tls-* flags. |
--peer-tls-key |
path | — | PEM private key paired with --peer-tls-cert. |
--peer-tls-ca |
path | — | PEM CA used to verify connecting peers. Cluster-dedicated. |
--allow-insecure-peer |
bool | false |
Opt out of the peer-listener secure-by-default guard. Allows routable bind without --peer-tls-*. Matches the helm chart's tls.allowInsecurePeer. Intended for single-host dev or service-mesh-terminated mTLS only. |
HA via OmniPaxos. Three or more nodes. Compiled in when the binary is built with the paxos feature (enabled by default on crates.io releases).
Common flags (above), plus:
| Flag | Type | Default | Meaning |
|---|---|---|---|
--node-id |
u64 |
required | This node's OmniPaxos pid. Must be unique across the cluster. |
--peer-listen |
SocketAddr |
required | Bind address for the paxos peer transport. Plaintext on loopback; routable bind requires the --peer-tls-* triple or --allow-insecure-peer. See Peer-port trust boundary. |
--peers |
string | required | Comma-separated id=host:port paxos peer addresses. Required at every start, not just bootstrap. |
--tso-peers |
string | required | Comma-separated id=host:port tsoracle service addresses (per-node TsoService ports), used to populate LeaderHint for follower redirects. |
--data-dir |
path | required | Directory for the paxos log + meta. |
--tick-interval |
duration | 20ms |
OmniPaxos tick interval. |
--peer-tls-cert |
path | — | PEM node certificate for the peer transport. Enables peer mTLS — needs all three --peer-tls-* flags. |
--peer-tls-key |
path | — | PEM private key paired with --peer-tls-cert. |
--peer-tls-ca |
path | — | PEM CA used to verify connecting peers. |
--allow-insecure-peer |
bool | false |
Opt out of the peer-listener secure-by-default guard. Allows routable bind without --peer-tls-*. Matches the helm chart's tls.allowInsecurePeer. Intended for single-host dev or service-mesh-terminated mTLS only. |
Initialize a fresh file-driver state directory at a seeded high-water. Useful when migrating from a prior timestamp source — see Migration.
| Flag | Type | Default | Meaning |
|---|---|---|---|
--state-dir |
path | ./tsoracle-data |
Directory to initialize. Must not already contain file-driver state. |
--seed-physical-ms |
u64 |
required | High-water to seed. Every subsequent issued timestamp's physical_ms will be strictly greater than this value. |
Administer cluster membership over the admin gRPC port. Available only when the binary is built with the openraft feature. Backed by the tsoracle.admin.v1.MembershipAdmin RPCs above.
Each subcommand accepts --endpoint (any node's admin endpoint — e.g. https://127.0.0.1:50561) plus the optional --client-tls-* triple for connecting to an mTLS-enabled admin server.
Common admin-client flags (AdminClientTlsArgs):
| Flag | Type | Meaning |
|---|---|---|
--client-tls-cert |
path | PEM client certificate to present to the admin gRPC server. |
--client-tls-key |
path | PEM private key for --client-tls-cert. |
--client-tls-ca |
path | PEM CA used to verify the admin server's certificate. |
List current members. Answerable by any node.
| Flag | Type | Meaning |
|---|---|---|
--endpoint |
string | Admin endpoint of any node. |
--client-tls-* |
path | See above. |
Add a non-voting learner.
| Flag | Type | Meaning |
|---|---|---|
--endpoint |
string | Admin endpoint to address. The CLI follows a NOT_LEADER redirect to leader_admin_endpoint. |
--id |
u64 |
New node's raft id. Cluster-unique. |
--raft-addr |
string | host:port of the new node's raft peer transport. |
--service-endpoint |
string | Scheme-less host:port of the new node's public TsoService endpoint. |
--admin-endpoint |
string | host:port of the new node's admin gRPC endpoint. |
--client-tls-* |
path | See above. |
Promote a learner to voter.
| Flag | Type | Meaning |
|---|---|---|
--endpoint |
string | Admin endpoint to address. Follows NOT_LEADER redirect. |
--id |
u64 |
Id of the learner to promote. |
--client-tls-* |
path | See above. |
Remove a node by id (voter or learner).
| Flag | Type | Meaning |
|---|---|---|
--endpoint |
string | Admin endpoint to address. Follows NOT_LEADER redirect. |
--id |
u64 |
Id of the node to remove. |
--client-tls-* |
path | See above. |
Initiate a format-version activation. Runs the all-members capability gate, then proposes the bump via raft.
| Flag | Type | Meaning |
|---|---|---|
--endpoint |
string | Admin endpoint of any cluster member. The CLI does not auto-redirect on NOT_LEADER for this RPC — the underlying error variant carries no endpoint hint, so re-issue against a known leader. |
--target |
u8 |
Target format version. Must be within the local binary's readable range and supported by every cluster member. |
--client-tls-* |
path | See above. |
Exit codes specific to activate-format:
| Code | Meaning |
|---|---|
0 |
Success. |
2 |
Gate rejected the activation — at least one member's max_readable_version is below --target (MEMBERS_BELOW_TARGET). |
3 |
Receiving node is a follower (NOT_LEADER). |
4 |
Receiving node's local readable range does not include --target (TARGET_OUT_OF_RANGE). |
1 |
Any other failure. |
Other admin subcommands use only 0 (success) and 1 (failure).
Published on crates.io. Generated reference: https://docs.rs/tsoracle (and analogously for each crate below). Where prose chapters complement the generated docs, the third column points at them.
| Crate | Role | Companion chapter |
|---|---|---|
tsoracle |
The CLI binary (this section above). Depends on the crates below. | § CLI reference |
tsoracle-client |
Async Rust client. The recommended way to call TsoService from Rust code. |
Client API and Usage, The Client Driver |
tsoracle-server |
The server library. Embed Server::builder()...build()?.serve_with_shutdown(...) to host the gRPC service in your own binary. |
Architecture Deep Dive, Key Subsystems |
tsoracle-core |
Sync algorithm core: the window allocator. No I/O, no async. | The Allocator |
tsoracle-consensus |
The ConsensusDriver trait. Implement this to back tsoracle with your own replicated log. |
Consensus Integration |
tsoracle-driver-file |
Single-node fsync-durable driver. The default. | Driver Comparison |
tsoracle-driver-openraft |
Driver wrapping an openraft cluster. | Driver Comparison |
tsoracle-driver-paxos |
Driver wrapping an OmniPaxos cluster. | Driver Comparison |
tsoracle-standalone |
Per-driver bootstrap helpers + peer transport. Used by the CLI binary; also embeddable. | — |
tsoracle-proto |
Generated protobuf types for tsoracle.v1.TsoService. Use directly only when bypassing tsoracle-client. |
§ gRPC wire reference |
tsoracle-codec |
Versioned postcard codec for on-disk and in-meta records. | Format-migration upgrade |
tsoracle-openraft-toolkit |
Lower-level openraft glue, including LogStoreCodec<C>. For driver authors and embedders who already host their own openraft. |
Consensus Integration |
tsoracle-paxos-toolkit |
Lower-level OmniPaxos glue. | Consensus Integration |
tsoracle-failpoint, tsoracle-yieldpoint |
Test-only fault-injection points. Compiled in only when the failpoints / yieldpoints feature is enabled. |
Failpoint Testing, Yield-point Testing |
- The gRPC public service is versioned by package:
tsoracle.v1.TsoService. Field numbers are stable; new optional fields may be added withinv1. A future breaking wire change would land astsoracle.v2. - The admin gRPC service is
tsoracle.admin.v1.MembershipAdmin. NewAdminErrorKindvariants may be added withinv1— consumers must fall back to themessagefield when they encounter an unrecognized enum value, exactly as required by the proto contract. - The Rust crates follow Cargo semver. Each crate is versioned independently; consult the crate's
CHANGELOG.md(or the release-plz entries on the GitHub Releases page) for migration notes. - The CLI surface is stable across patch and minor releases of the same crates.io major version of the
tsoraclebinary crate. Flag removals or rename without alias are major-version events. - On-disk format compatibility is governed by the per-record
format_versionbyte (currently4). The binary supports a documented[MIN_READABLE_VERSION, MAX_READABLE_VERSION]range so a rolling upgrade can convert in place. See Format-migration upgrade for the full story, including thetsoracle admin activate-formatflow that flips the cluster-wide active write version.