Indexing performance metrics #545
Conversation
WalkthroughThe changes introduce enhanced Prometheus metric tracking for source manager states and source height, add new Prometheus counter modules, and extend the Prometheus metric interface. Additionally, the Changes
Sequence Diagram(s)sequenceDiagram
participant SourceManager
participant Prometheus
SourceManager->>Prometheus: trackNewStatus(Idle/WaitingForNewBlock/Querieng)
Note right of SourceManager: On status change, record elapsed time to Prometheus counters
SourceManager->>Prometheus: incrementMany(counter, labels, value)
Note right of Prometheus: Counter for time spent in each state is updated
GlobalState->>Prometheus: set(SourceHeight, {chainId, source}, blockNumber)
Note right of Prometheus: Updates metric if new highest block is detected
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (1)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
codegenerator/cli/npm/envio/src/sources/SourceManager.res (1)
3-4: Typo in enum variant "Querieng".There appears to be a spelling error in the sourceManagerStatus enum variant - "Querieng" should likely be "Querying".
-type sourceManagerStatus = Idle | WaitingForNewBlock | Querieng +type sourceManagerStatus = Idle | WaitingForNewBlock | QueryingEnsure this change is also applied to all references of this variant throughout the code.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
codegenerator/cli/npm/envio/src/Prometheus.res(3 hunks)codegenerator/cli/npm/envio/src/sources/SourceManager.res(5 hunks)codegenerator/cli/src/config_parsing/chain_helpers.rs(4 hunks)codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
codegenerator/cli/src/config_parsing/chain_helpers.rs (1)
codegenerator/cli/src/cli_args/interactive_init/evm_prompts.rs (1)
HypersyncNetwork(224-232)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: build_and_test
🔇 Additional comments (11)
codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res (1)
346-355: Great addition of source height metrics to Prometheus.This change enhances observability by tracking the current block height from data sources in Prometheus metrics. The metric is properly labeled with chain ID and source name for easy correlation.
codegenerator/cli/npm/envio/src/Prometheus.res (2)
126-131: Nice addition ofincrementManymethod for metrics.The new method appropriately extends the metric interface to support incrementing counters by arbitrary values, not just by one. This is essential for tracking elapsed time measurements.
367-389: Good addition of indexing performance metrics.These new counter modules will provide valuable insights into where time is spent during indexing:
- IndexingIdleTime: tracks idle time (potential bottleneck indicator)
- IndexingSourceWaitingTime: tracks time waiting for new blocks
- IndexingQueryTime: tracks time spent querying chain data-sources
The metrics use consistent naming conventions and include helpful descriptions.
codegenerator/cli/src/config_parsing/chain_helpers.rs (4)
253-253: Correctly updated Moonbeam network configuration.Adding the
HypersyncNetworksubenum to Moonbeam ensures it's properly categorized for hypersync-related functionality.
343-344: Good addition of Swell network.The new Swell network has been properly added with a unique ID (1923) and correctly tagged with the HypersyncNetwork subenum.
528-529: Properly updated block confirmation threshold.The Swell network has been added to the appropriate match arm to use the default confirmed block threshold, consistent with similar networks like Sonic.
589-589: Correctly updated network tier classification.Both Swell and Moonbeam have been properly added to the Stone tier in the HypersyncNetwork's get_tier method.
codegenerator/cli/npm/envio/src/sources/SourceManager.res (4)
12-13: Good addition of status tracking fields.These new fields provide the foundation for tracking the operational state of the source manager and measuring time spent in each state.
73-75: Proper initialization of new fields.The status is correctly initialized to Idle with a new timer at creation time.
78-90: Well-implemented status tracking function.The
trackNewStatusfunction properly:
- Selects the appropriate Prometheus counter based on the new status
- Records the elapsed time in the previous status
- Resets the timer for the new status
- Updates the status field
119-120: Good placement of status tracking calls.Status transitions are tracked at key points in the fetchNext function:
- When entering WaitingForNewBlock state
- When returning to Idle after waiting for a new block
- When starting query execution
- When all queries complete
This provides comprehensive coverage of the source manager's lifecycle.
Also applies to: 124-125, 140-141, 151-153
Summary by CodeRabbit