Open
Conversation
martonp
force-pushed
the
oracleQuotas
branch
from
66112b1 to
d34230b
Compare
Move source definitions into oracle/sources/ with provider implementations in oracle/sources/providers/ and shared utilities in oracle/sources/utils/. This separates the source abstraction from the oracle core logic. Also adds quota tracking for each of the sources.
Updates the algorithm to determine when to perform the next fetch for each source to be based on the quotas of the entire mesh. The sustainable fetch rate of the entire network is calculated, and the entire network can deterministically determine which node should perform the next fetch. Additionally, an Oracle Snapshot function is added which will allow the admin tools to view the state of the oracle.
Add quota handshake and heartbeat protocols for sharing quota information between tatanka nodes.
Replace the CLI-based tatankactl with an interactive bubbletea TUI. Add oracle source status view with per-source detail screens showing schedule, fetch order, quotas, and price/fee rate contributions. Add aggregated data view with drill-down into individual tickers.
martonp
force-pushed
the
oracleQuotas
branch
from
d34230b to
a4d4006
Compare
Contributor
|
refactor and quota additions look good, running local tests soon. I think Update: local tests look good too. We might have to look at the mempool source again, it's failing all fetch attempts. |
dnldd
reviewed
Feb 12, 2026
Comment on lines
+54
to
+64
| func (cfg *QuotaTrackerConfig) verify() { | ||
| if cfg == nil { | ||
| panic("quota tracker config is nil") | ||
| } | ||
| if cfg.FetchQuota == nil { | ||
| panic("fetch quota function is required") | ||
| } | ||
| if cfg.Log == nil { | ||
| panic("logger is required") | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
why not consolidate config errors (errors.Join) and return it instead of panicking?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds network-coordinated fetch scheduling for oracle sources, so nodes in the mesh share rate-limited API quotas intelligently instead of each node fetching independently.
Source interface and quota tracking
Oracle source implementations refactored into a subpackage structure (
oracle/sources/,oracle/sources/providers/,oracle/sources/utils/).A
Sourceinterface is created that must be implemented by each source, and it includes aQuotaStatus()function, which returns the source's remaining API quota (fetches remaining, fetch limit, and reset time).Sources fall into two categories:
QuotaStatus()returnsmath.MaxInt64for remaining fetches.QuotaTrackerthat periodically reconciles against the provider's quota API. Multiple sources can share a single tracker when they share an API key, and each declares itsCreditsPerRequestso the tracker can convert raw API credits into fetch counts.Network-coordinated fetch scheduling
When multiple nodes run the same oracle sources, they should coordinate so they don't all fetch from the same rate-limited API simultaneously. The quota manager computes a
networkSchedulefor each source using quota data from all active peers in the mesh.The algorithm has three steps:
Sustainable rate: Each peer's quota yields a rate (fetches/sec) computed as
remainingFetches / timeUntilReset, with a 10% safety margin. The network rate is the sum of all peer rates, and the sustainable period is its reciprocal, clamped between the source'sMinPeriodand 1 hour.Deterministic ordering: Peers are ranked by
SHA256(timeWindow, nodeID) / node_rate. The time window rotates everyMinPeriodseconds so the ordering reshuffles periodically, while dividing by the node's sustainable rate biases toward nodes with higher quotas. This ordering is deterministic. Every node computes the same result from the same inputs without any leader election.Per-node timing: The first node in the ordering fetches at
networkNextFetchTime. Each subsequent node waits an additional 3-second propagation delay, giving the previous node time to fetch and gossip the result before the next node attempts the same fetch.Quota sharing protocol
Nodes share their quota information through three mechanisms:
/tatanka/quota-handshake/1.0.0stream protocol.Tatankactl UI
The tatankactl CLI is replaced with an interactive bubbletea UI. In addition to the connection information that was already available, all details of the oracle's functioning can be monitored through the tool.