v2.31 #789
v2.31 #789
Conversation
WalkthroughLarge, cross-cutting refactor introducing checkpoint-based multichain batching and reorg tracking, replacing legacy end-of-block scan/state with Checkpoints and ReorgDetection, updating persistence/PG schema and Hasura support, adjusting in-memory store and generated templates, adding metrics, and sweeping test/template updates. Changes
Sequence Diagram(s)%% High-level checkpoint-aware batch processing and reorg flow
sequenceDiagram
participant CM as ChainManager
participant B as Batch
participant EP as EventProcessing
participant GS as GlobalState
participant CF as ChainFetcher
participant RD as ReorgDetection
participant SCT as SafeCheckpointTracking
participant DB as Persistence/PG
CM->>B: Batch.make(~checkpointIdBeforeBatch, ~chainsBeforeBatch)
B-->>CM: Batch.t { items, checkpointIds, progressedChainsById, totalBatchSize }
CM->>EP: processEventBatch(~batch)
EP->>EP: preloadBatchOrThrow(~batch, checkpointId)
EP->>EP: runBatchHandlersOrThrow(~batch, checkpointId)
EP->>GS: update in-memory store with checkpointId
EP->>DB: executeBatch(~batch, inMemoryStore)
CF->>RD: registerReorgGuard(reorgGuard, currentBlockHeight)
alt reorg detected
RD-->>CF: ReorgDetected
CF->>GS: submit ReorgDetected
GS->>GS: FindReorgDepth -> FoundReorgDepth -> RollbackReady
GS->>DB: query rollbackTargetCheckpointId
GS->>EP: prepareRollbackDiff(rollbackTargetCheckpointId)
else no reorg
RD-->>CF: NoReorg
end
note right of SCT: update safe checkpoints on committed batches
EP->>SCT: updateOnNewBatch(sourceBlockNumber, chainId, batchCheckpointIds)
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Large, heterogeneous architectural changes across batching, persistence, rollback, templates, metrics, and tests; careful per-module review and cross-checking of DB migrations, public signatures, and template generation required. Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning, 2 inconclusive)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used🧬 Code graph analysis (1)codegenerator/cli/src/config_parsing/chain_helpers.rs (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
🔇 Additional comments (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 16
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (6)
codegenerator/cli/templates/dynamic/contract_import_templates/typescript/src/EventHandlers.ts.hbs (1)
20-26: Fix ReScript template to use correct event key casing.The ReScript EventHandlers template still has the casing bug. While TypeScript and JavaScript templates were corrected to use
event.params.{{param.event_key.original}}, the ReScript template atcodegenerator/cli/templates/dynamic/contract_import_templates/rescript/src/EventHandlers.res.hbsline 11 still incorrectly usesevent.params.{{param.event_key.uncapitalized}}and must be updated to use.originalinstead.scenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.res (1)
97-107: Comments contradict asserted ordering and priority.
- The arrays are in earliest→latest order, but comments say “reverse order (latest to earliest)”.
- Comments claim “block items have higher priority than events at same block”, but expected arrays place the event before the block item (e.g., (5, 0) precedes (5, 16777216)).
Please align comments (or expectations) to avoid confusion. Also, replace the magic 16777216 (1 << 24) with a named constant in the test for readability.
- // Expected in reverse order (latest to earliest): block items have logIndex=16777216, event has logIndex=0 + // Expected in chronological order (earliest to latest): block items use BLOCK_ITEM_LOG_BASE, event has logIndex=0 +let BLOCK_ITEM_LOG_BASE = 1 lsl 24 ... - (0, 16777216), + (0, BLOCK_ITEM_LOG_BASE),Also applies to: 144-155, 192-206, 244-263, 299-301
codegenerator/cli/npm/envio/src/Persistence.res (1)
195-223: Critical issues remain: clamping logic not applied, and underlying storage layer not updated for new parameterThe review comment's concerns are valid. Verification found:
Clamping logic missing (Persistence.res:218–219): The calculation still lacks the proposed
if next < 0 { 0 } else { next }guard, allowing negative cache counts.Storage layer incomplete (PgStorage.res): The underlying
setEffectCacheOrThrowimplementation signature is missing the~invalidationsCountparameter—it only has~effect,~items, and~initialize.Test infrastructure outdated:
- Mock.res: Test mock lacks
~invalidationsCountparameter- TestHelpers_MockDb.res.hbs: Template mock lacks
~invalidationsCountparameterOnly the call site (IO.res:301) correctly passes the new parameter; the definition chain is incomplete.
codegenerator/cli/templates/static/codegen/src/Config.res (1)
40-103: Fix pgSchema parameter to use function argument instead of hardcoded storagePgSchema in Hasura calls.The bug is confirmed:
makeStorageaccepts a~pgSchemaparameter but ignores it in bothHasura.trackDatabaseandHasura.trackTablescalls, usingstoragePgSchemainstead. This breaks support for non-default schemas.Apply the proposed diff changes at both callsites:
- Line ~58:
Hasura.trackDatabase(..., ~pgSchema=pgSchema, ...)- Line ~75:
Hasura.trackTables(..., ~pgSchema=pgSchema, ...)codegenerator/cli/templates/static/codegen/src/IO.res (1)
285-307: Fix type signature mismatch inPersistence.setEffectCacheOrThrow.The type definition at line 73 declares the function should accept
~initialize: bool, but the actual implementation at line 195 accepts~invalidationsCount. The function body uses~invalidationsCountto compute the cache count (line 219), so the type signature must be updated to reflect this. Either update line 76 from~initialize: boolto~invalidationsCount: int, or refactor the implementation to use the~initializeparameter instead.codegenerator/cli/npm/envio/src/db/InternalTable.res (1)
292-295: Unify ID field usage in progress update query.makeProgressFieldsUpdateQuery still uses a literal "id" in WHERE while other queries use the dynamic field. Align to avoid future breakage.
-WHERE "id" = $1;` +WHERE "${(#id: field :> string)}" = $1;`
🧹 Nitpick comments (41)
.github/workflows/build_and_test.yml (1)
40-52: Move hardcoded credentials to GitHub Secrets.The Hasura service configuration contains credentials visible in the workflow file:
postgres:testingin the database URL (line 43) andtestingas the admin secret (line 47). While these are ephemeral test credentials for CI, they should be stored as GitHub repository secrets rather than hardcoded in the workflow.Replace the hardcoded credentials with GitHub secrets:
- hasura: - image: hasura/graphql-engine:v2.43.0 - env: - HASURA_GRAPHQL_DATABASE_URL: postgres://postgres:testing@postgres:5432/envio-dev - HASURA_GRAPHQL_ENABLE_CONSOLE: "true" - HASURA_GRAPHQL_ENABLED_LOG_TYPES: startup, http-log, webhook-log, websocket-log, query-log - HASURA_GRAPHQL_NO_OF_RETRIES: 10 - HASURA_GRAPHQL_ADMIN_SECRET: testing - HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES: "true" - HASURA_GRAPHQL_UNAUTHORIZED_ROLE: public - PORT: 8080 + hasura: + image: hasura/graphql-engine:v2.43.0 + env: + HASURA_GRAPHQL_DATABASE_URL: ${{ secrets.HASURA_DB_URL }} + HASURA_GRAPHQL_ENABLE_CONSOLE: "true" + HASURA_GRAPHQL_ENABLED_LOG_TYPES: startup, http-log, webhook-log, websocket-log, query-log + HASURA_GRAPHQL_NO_OF_RETRIES: 10 + HASURA_GRAPHQL_ADMIN_SECRET: ${{ secrets.HASURA_ADMIN_SECRET }} + HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES: "true" + HASURA_GRAPHQL_UNAUTHORIZED_ROLE: public + PORT: 8080(Configure these secrets in your GitHub repository settings: Settings → Secrets and variables → Actions)
codegenerator/cli/templates/dynamic/codegen/src/RegisterHandlers.res.hbs (1)
56-56: Remove unused localchainThe binding isn’t used; it adds noise and may trigger warnings.
- let chain = ChainMap.Chain.makeUnsafe(~chainId={{chain_config.network_config.id}})scenarios/test_codegen/test/WriteRead_test.res (1)
42-43: Track TODO: Array of booleans support.The array of booleans is commented out with a TODO. Please ensure this limitation is tracked as a known issue or future enhancement.
Would you like me to search for existing issues tracking boolean array support, or should a new issue be created?
scenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.res (1)
10-13: Unused helper.getBlockData is defined but unused in this test. Consider removing to reduce noise.
scenarios/test_codegen/test/lib_tests/Persistence_test.res (1)
81-83: Initial state shape update looks correct; reduce duplication.Adding reorgCheckpoints and checkpointId matches the updated Persistence.initialState. Consider a small factory to avoid duplicating the literals across tests.
+let makeInitialState = (~cleanRun, ~chains=[], ~cache=Js.Dict.empty()) => { + cleanRun, chains, cache, reorgCheckpoints: [], checkpointId: 0 +} ... - let initialState: Persistence.initialState = { - cleanRun: true, - chains: [], - cache: Js.Dict.empty(), - reorgCheckpoints: [], - checkpointId: 0, - } + let initialState: Persistence.initialState = makeInitialState(~cleanRun=true) ... - let initialState: Persistence.initialState = { - cleanRun: false, - chains: [], - cache: Js.Dict.empty(), - reorgCheckpoints: [], - checkpointId: 0, - } + let initialState: Persistence.initialState = makeInitialState(~cleanRun=false)Also applies to: 147-149
codegenerator/cli/templates/static/codegen/src/globalState/GlobalStateManager.res (1)
49-61: Promise error piping can be simplified.You can pipe the promise rejection directly to handleFatalError without the extra resolve wrapper.
- S.taskReducer(self.state, task, ~dispatchAction=action => - dispatchAction(~stateId, self, action) - ) - ->Promise.catch(e => { - e->handleFatalError - Promise.resolve() - }) - ->ignore + S.taskReducer(self.state, task, ~dispatchAction=action => + dispatchAction(~stateId, self, action) + ) + ->Promise.catch(handleFatalError) + ->ignorecodegenerator/cli/npm/envio/src/Batch.res (2)
95-177: Key consistency for per‑chain dictsYou write with
setByIntbut read viadangerouslyGetNonOption(Int.toString ...). It works in JS, but using the int accessor consistently (dangerouslyGetByIntNonOption) avoids string conversions and reduces edge cases.- progressBlockNumberPerChain->Utils.Dict.dangerouslyGetNonOption( - fetchState.chainId->Int.toString, - ) + progressBlockNumberPerChain->Utils.Dict.dangerouslyGetByIntNonOption( + fetchState.chainId, + )(Apply similarly for
batchSizePerChain.)
179-214: Avoid param shadowing for readability in addReorgCheckpoints
let prevCheckpointId = ref(prevCheckpointId)shadows the param; rename the ref to prevent confusion.- let prevCheckpointId = ref(prevCheckpointId) + let prevCheckpointIdRef = ref(prevCheckpointId) ... - let checkpointId = prevCheckpointId.contents + 1 - prevCheckpointId := checkpointId + let checkpointId = prevCheckpointIdRef.contents + 1 + prevCheckpointIdRef := checkpointId ... - prevCheckpointId.contents + prevCheckpointIdRef.contentscodegenerator/cli/templates/dynamic/codegen/src/TestHelpers_MockDb.res.hbs (2)
121-126: Helper writes use checkpointId=0; consider threading a real checkpointId when needed.Currently both delete/set write payloads set checkpointId: 0. It’s fine with shouldSaveHistory=false, but if history gets enabled later, this will produce misleading history rows. Consider accepting an optional ~checkpointId for these helpers to support tests that assert on history.
Confirm no tests rely on history produced by these helpers.
Also applies to: 138-143
524-528: Nit: comment typo.“contatin” → “contain”.
codegenerator/cli/npm/envio/src/Prometheus.res (1)
485-489: Naming nit.Parameter name “rollbackedProcessedEvents” reads awkwardly. Consider “rolledBackProcessedEvents” or “eventsRolledBack”.
scenarios/test_codegen/test/E2E_test.res (1)
9-9: Typo in test name.“Currectly” → “Correctly”.
scenarios/test_codegen/test/ChainManager_test.res (2)
111-117: ReorgDetection config alignment with chain config.Hardcoding ~maxReorgDepth=200 while chain configs vary can produce inconsistent rollback thresholds in tests. Prefer sourcing from chainConfig.maxReorgDepth for each fetcher to reflect real behavior.
- reorgDetection: ReorgDetection.make( - ~chainReorgCheckpoints=[], - ~maxReorgDepth=200, - ~shouldRollbackOnReorg=false, - ), + reorgDetection: ReorgDetection.make( + ~chainReorgCheckpoints=[], + ~maxReorgDepth=chainConfig.maxReorgDepth, + ~shouldRollbackOnReorg=false, + ),
160-176: Avoid deep recursion in long runs.The recursive testThatCreatedEventsAreOrderedCorrectly(...) can grow deep with large random inputs. Convert to a loop to prevent stack growth in CI.
- let rec testThatCreatedEventsAreOrderedCorrectly = (chainManager, lastEvent) => { + let testThatCreatedEventsAreOrderedCorrectly = (initialChainManager, lastEvent) => { + let rec loop = chainManager => { let {items, totalBatchSize, progressedChainsById} = ChainManager.createBatch( chainManager, ~batchSizeTarget=10000, ) // ensure that the events are ordered correctly - if totalBatchSize === 0 { - chainManager - } else { + if totalBatchSize === 0 { + chainManager + } else { items->Array.forEach( item => { allEventsRead->Js.Array2.push(item)->ignore }, ) numberOfMockEventsReadFromQueues := numberOfMockEventsReadFromQueues.contents + totalBatchSize let firstEventInBlock = items[0]->Option.getExn Assert.equal( firstEventInBlock->EventUtils.getOrderedBatchItemComparator > lastEvent->EventUtils.getOrderedBatchItemComparator, true, ~message="Check that first event in this block group is AFTER the last event before this block group", ) let nextChainFetchers = chainManager.chainFetchers->ChainMap.mapWithKey( (chain, fetcher) => { let fetchState = switch progressedChainsById->Utils.Dict.dangerouslyGetByIntNonOption( chain->ChainMap.Chain.toChainId, ) { | Some(chainAfterBatch) => chainAfterBatch.fetchState | None => fetcher.fetchState } { ...fetcher, fetchState, } }, ) let nextChainManager: ChainManager.t = { ...chainManager, chainFetchers: nextChainFetchers, } - testThatCreatedEventsAreOrderedCorrectly(nextChainManager, lastEvent) - } - } + loop(nextChainManager) + } + } + loop(initialChainManager) }codegenerator/cli/templates/static/codegen/src/EventProcessing.res (3)
29-48: Safer JSON conversion without %raw and with BigInt array support.Avoid %raw
{}and handle arrays of bigint if they ever appear. Use Js.Json.object_ and a small recursive toStringBigInts.-let convertFieldsToJson = (fields: option<dict<unknown>>) => { +let convertFieldsToJson = (fields: option<dict<unknown>>) => { switch fields { - | None => %raw(`{}`) + | None => Js.Dict.empty()->Js.Json.object_ | Some(fields) => { - let keys = fields->Js.Dict.keys - let new = Js.Dict.empty() - for i in 0 to keys->Js.Array2.length - 1 { - let key = keys->Js.Array2.unsafe_get(i) - let value = fields->Js.Dict.unsafeGet(key) - // Skip `undefined` values and convert bigint fields to string - // There are not fields with nested bigints, so this is safe - new->Js.Dict.set( - key, - Js.typeof(value) === "bigint" ? value->Utils.magic->BigInt.toString->Utils.magic : value, - ) - } - new->(Utils.magic: dict<unknown> => Js.Json.t) + let rec toJsonSafe = v => { + switch Js.typeof(v) { + | "bigint" => v->Utils.magic->BigInt.toString->Utils.magic + | "object" => + if Js.Array.isArray(v) { + v->(Utils.magic: array<unknown>) + ->Js.Array2.map(toJsonSafe) + ->(Utils.magic: unknown) + } else { + v + } + | _ => v + } + } + let keys = fields->Js.Dict.keys + let out = Js.Dict.empty() + for i in 0 to keys->Js.Array2.length - 1 { + let key = keys->Js.Array2.unsafe_get(i) + let value = fields->Js.Dict.unsafeGet(key) + if Js.Undefined.testAny(value) == false { + out->Js.Dict.set(key, toJsonSafe(value)) + } + } + out->Js.Json.object_ } } }
223-303: Preload: error swallowing is intentional; add a minimal trace on sync throws.try/catch currently no‑ops on sync throws. Add a debug trace to help users diagnose preload failures without affecting flow.
- } catch { - | _ => () - } + } catch { + | exn => Logging.debug(`preload error (sync path): ${exn->Utils.prettifyExn}`) + }
364-373: Short‑circuit empty batches to skip DB path.If batch.items is empty, we can return early and avoid IO.executeBatch() overhead.
let processEventBatch = async ( ~batch: Batch.t, @@ ) => { let totalBatchSize = batch.totalBatchSize // Compute chains state for this batch let chains: Internal.chains = chainFetchers->computeChainsState @@ - try { + try { + if !batch.items->Utils.Array.notEmpty { + // Nothing to do; still register zero-cost metrics for visibility + registerProcessEventBatchMetrics(~logger=Logging.getLogger(), ~loadDuration=0, ~handlerDuration=0, ~dbWriteDuration=0) + return Ok() + } let timeRef = Hrtime.makeTimer() - if batch.items->Utils.Array.notEmpty { + if batch.items->Utils.Array.notEmpty { await batch->preloadBatchOrThrow(Also applies to: 391-399, 402-411, 416-423
scenarios/test_codegen/test/lib_tests/PgStorage_test.res (1)
746-757: Confirmed: typo in function namemakeCommitedCheckpointIdQuery—should bemakeCommittedCheckpointIdQuery.The misspelled function appears in 3 locations requiring updates:
- Definition:
codegenerator/cli/npm/envio/src/db/InternalTable.res:440- Usage:
codegenerator/cli/npm/envio/src/PgStorage.res:897- Test:
scenarios/test_codegen/test/lib_tests/PgStorage_test.res:746, 750Rename the function and update all call sites for correctness and consistency. A temporary alias is optional but not necessary given the internal scope.
codegenerator/cli/npm/envio/src/Internal.res (3)
148-159: Name clarity: consider spelling outdcs.If
dcsmeans "dynamic contracts," consider a clearer alias or a brief doc-comment, since it’s part of the public surface and appears onitemvia accessors. This reduces cognitive load for readers.
305-309: Effect cache schema: unsafe cast and JSON constraints.
cacheOutputSchemausesS.json(~validate=false)+Utils.magicto coerce toeffectOutput. This bypasses validation and can hide shape drift. Consider validating with the effect’s declaredoutputschema on load (at least in dev) or store plainJs.Json.tand re-decode at read boundaries.Example (dev-only validation):
-let cacheOutputSchema = S.json(~validate=false)->(Utils.magic: S.t<Js.Json.t> => S.t<effectOutput>) +let cacheOutputSchema = + if Env.isProd { + S.json(~validate=false)->(Utils.magic: S.t<Js.Json.t> => S.t<effectOutput>) + } else { + // decode as Json then validate using the effect's output schema when reading + S.json(~validate=true)->(Utils.magic: S.t<Js.Json.t> => S.t<effectOutput>) + }Also, if
effectOutputcan contain BigInt(s), confirm serialization strategy (e.g., stringify BigInts) to avoid JSON loss; see issue #788 context.Also applies to: 313-317
337-348: Tiny doc typo.Stray “b” in the comment (“...present in the diff) b”). Clean it up for clarity.
codegenerator/cli/npm/envio/src/SafeCheckpointTracking.res (4)
12-35: Assume or enforce monotonic ordering of checkpoints.
getSafeCheckpointIdrelies oncheckpointBlockNumbersbeing ascending. If upstream doesn’t guarantee order, sort once inmakeor assert ordering to avoid wrong picks.Possible guard:
let make = ( ~maxReorgDepth, ~shouldRollbackOnReorg, ~chainReorgCheckpoints: array<Internal.reorgCheckpoint>, ) => { if maxReorgDepth > 0 && shouldRollbackOnReorg { - let checkpointIds = Belt.Array.makeUninitializedUnsafe(chainReorgCheckpoints->Array.length) - let checkpointBlockNumbers = Belt.Array.makeUninitializedUnsafe( - chainReorgCheckpoints->Array.length, - ) + let sorted = chainReorgCheckpoints->Belt.Array.stableSortBy(c => c.blockNumber) + let checkpointIds = Belt.Array.makeUninitializedUnsafe(sorted->Array.length) + let checkpointBlockNumbers = + Belt.Array.makeUninitializedUnsafe(sorted->Array.length) - chainReorgCheckpoints->Js.Array2.forEachi((checkpoint, idx) => { + sorted->Js.Array2.forEachi((checkpoint, idx) => { checkpointIds->Belt.Array.setUnsafe(idx, checkpoint.checkpointId) checkpointBlockNumbers->Belt.Array.setUnsafe(idx, checkpoint.blockNumber) })
36-67: Edge-case: negativesafeBlockNumber.When
sourceBlockNumber < maxReorgDepth,safeBlockNumberbecomes negative; current logic returns 0 if first checkpoint is above it, which is fine. Document this behavior to avoid confusion.
69-108: Parallel arrays andgetUnsafe: add invariant checks.
batchCheckpoint*arrays must have equal length; otherwisegetUnsafecan crash or pushundefined. Add a quick length equality check (assert/log) before the loop.let updateOnNewBatch = ( safeCheckpointTracking: t, ~sourceBlockNumber: int, ~chainId: int, ~batchCheckpointIds: array<int>, ~batchCheckpointBlockNumbers: array<int>, ~batchCheckpointChainIds: array<int>, ) => { + Js.Array2.length(batchCheckpointIds) === Js.Array2.length(batchCheckpointBlockNumbers) + && Js.Array2.length(batchCheckpointIds) === Js.Array2.length(batchCheckpointChainIds) + || Js.Exn.raiseError("Mismatched checkpoint arrays in updateOnNewBatch");
110-131: Rollback across chains: clarify intent.
rollbackfilters bytargetBlockNumberwithout a chain id, affecting all chains. If intended (keeping a global safe frontier), add a brief comment; otherwise consider a chain-aware variant.codegenerator/cli/templates/static/codegen/src/eventFetching/ChainManager.res (2)
4-4: Typo in public field:commitedCheckpointId→committedCheckpointId.This leaks into generated projects’ public surface. Consider renaming now and (if needed) keeping a transitional alias to avoid churn.
-type t = { commitedCheckpointId: int, ... +type t = { committedCheckpointId: int, ... ... - commitedCheckpointId: 0, + committedCheckpointId: 0, ... - commitedCheckpointId: initialState.checkpointId, + committedCheckpointId: initialState.checkpointId,Also applies to: 27-27, 84-85
139-166: Avoid%raw(Infinity)sentinel; prefer option-based min.Using JS
Infinitycast tointis brittle. Use an option accumulator to track min safely.-let infinity = (%raw(`Infinity`): int) -let result = ref(infinity) +let minSafe: ref<option<int>> = ref(None) ... - for idx in 0 to chainFetchers->Array.length - 1 { + for idx in 0 to chainFetchers->Array.length - 1 { let chainFetcher = chainFetchers->Array.getUnsafe(idx) switch chainFetcher.safeCheckpointTracking { | None => () | Some(sct) => { let safeCheckpointId = sct->SafeCheckpointTracking.getSafeCheckpointId( ~sourceBlockNumber=chainFetcher.currentBlockHeight, ) - if safeCheckpointId < result.contents { - result := safeCheckpointId - } + switch minSafe.contents { + | None when safeCheckpointId > 0 => minSafe := Some(safeCheckpointId) + | Some(curr) when safeCheckpointId > 0 && safeCheckpointId < curr => + minSafe := Some(safeCheckpointId) + | _ => () + } } } } - if result.contents === infinity || result.contents === 0 { - None - } else { - Some(result.contents) - } + minSafe.contentsscenarios/test_codegen/test/lib_tests/FetchState_test.res (2)
2131-2131: Remove stray debug log from tests.
Js.log(updatedFetchState)is noisy in CI output and not asserted. Please drop it.- Js.log(updatedFetchState) + /* no-op */
140-140: Avoid%rawonce ReScript v12 is baseline.The
%raw(\undefined`)` workaround is fine for v11, but please gate/plan removal when v12 is required to avoid raw escapes in tests.codegenerator/cli/npm/envio/src/PgStorage.res (1)
35-39: Quote custom type names to avoid edge cases.
Custom(name)currently renders as"${pgSchema}".${name}. If a custom type contains capitals or special chars, lack of quotes can break. Quote when safe (no parentheses), otherwise keep as-is.- | Custom(name) if !(name->Js.String2.startsWith("NUMERIC(")) => `"${pgSchema}".${name}` + | Custom(name) if !(name->Js.String2.includes("(")) => `"${pgSchema}"."${name}"` + | Custom(name) => name /* e.g., parameterized like NUMERIC(38,0) handled above */codegenerator/cli/npm/envio/src/ReorgDetection.res (1)
93-109: Avoid relying on JS object key ordering for correctness.Comments assume numeric object keys are ordered ASC. While generally true for integer-like keys, prefer explicit sort for robustness.
- // Js engine automatically orders numeric object keys - let ascBlockNumberKeys = dataByBlockNumber->Js.Dict.keys + let ascBlockNumberKeys = + dataByBlockNumber + ->Js.Dict.keys + ->Js.Array2.map(Int.fromStringUnsafe) /* or safe parse with filter */ + ->Js.Array2.sortInPlaceWith((a,b)=>a-b) + ->Js.Array2.map(Int.toString)codegenerator/cli/npm/envio/src/FetchState.res (2)
1229-1234: Deterministic tie‑break for Block items (avoid random).
Js.Math.random_intintroduces nondeterminism in production ordering and tests. Use a stable field (e.g.,chainId) to break ties.- | (Some(Block(_)), _) - | (_, Some(Block(_))) => - // Currently block items don't have a timestamp, - // so we sort chains with them in a random order - Js.Math.random_int(-1, 1) + | (Some(Block(_)), _) + | (_, Some(Block(_))) => + a.chainId - b.chainId
1166-1179: Rollback: don’t advance onBlock pointer beyond prior value.Setting
latestOnBlockBlockNumber = targetBlockNumbermay jump forward if it was behind. Safer to cap at min(previous, target).- { - ...fetchState, - latestOnBlockBlockNumber: targetBlockNumber, // TODO: This is not tested. I assume there might be a possible issue of it skipping some blocks - }->updateInternal( + { + ...fetchState, + latestOnBlockBlockNumber: + Pervasives.min(fetchState.latestOnBlockBlockNumber, targetBlockNumber), + }->updateInternal(codegenerator/cli/templates/static/codegen/src/IO.res (2)
181-196: Race in error capture can overwrite the first failure.Multiple entity operations run concurrently;
specificErrormay be overwritten by later handlers. Capture the first error only.- | `invalid byte sequence for encoding "UTF8": 0x00` => + | `invalid byte sequence for encoding "UTF8": 0x00` => ... - specificError.contents = Some(PgStorage.PgEncodingError({table: entityConfig.table})) - | _ => specificError.contents = Some(exn->Utils.prettifyExn) + if specificError.contents == None { + specificError.contents = Some(PgStorage.PgEncodingError({table: entityConfig.table})) + } + | _ => + if specificError.contents == None { + specificError.contents = Some(exn->Utils.prettifyExn) + }
63-142: History batching is solid; minor polish.Good backfill-before-updates sequencing and UTF‑8 cleanup. Consider moving
shouldRemoveInvalidUtf8detection out of the inner loop and keep per‑entity to avoid repeated checks.codegenerator/cli/npm/envio/src/db/InternalTable.res (1)
441-443: Typo: “Commited” → “Committed”.Rename makeCommitedCheckpointIdQuery for clarity and grep-ability.
codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res (2)
554-578: Reorg threshold entry gate is sound; minor nit: reuse helper.This duplicates readiness logic found in FinishWaitingForNewBlock. Consider extracting a shared predicate to avoid drift.
195-233: Metrics updates tied to batch are good; ensure findFirstEventBlockNumber is O(1).If Batch.findFirstEventBlockNumber scans, consider caching per chain in Batch.t.
codegenerator/cli/templates/static/codegen/src/eventFetching/ChainFetcher.res (1)
412-415: Threshold block computation comment nit.Doc has “higehest”. Consider a quick typo fix when touching next.
codegenerator/cli/npm/envio/src/db/EntityHistory.res (1)
192-212: Pruning query is well reasoned; consider adding supporting index.For large tables, add (id, checkpoint_id) index on history tables to accelerate anchors and delete USING joins.
scenarios/test_codegen/test/rollback/Rollback_test.res (1)
166-177: Consider removing commented-out assertions.Multiple blocks of commented test assertions for
UpdateEndOfBlockRangeScannedDataexpectations appear throughout the tests. If these expectations no longer apply to the new architecture, consider removing them entirely rather than leaving them commented.Apply cleanup to remove obsolete assertions:
- // Assert.deepEqual( - // tasks.contents->Js.Array2.unsafe_get(0), - // UpdateEndOfBlockRangeScannedData({ - // blockNumberThreshold: -198, - // chain: MockConfig.chain1337, - // nextEndOfBlockRangeScannedData: { - // blockHash: block2.blockHash, - // blockNumber: block2.blockNumber, - // chainId: 1337, - // }, - // }), - // )(Apply to all similar commented blocks throughout the file.)
Also applies to: 224-235, 319-330, 352-363
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
scenarios/fuel_test/pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlscenarios/test_codegen/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (75)
.cursor/rules/navigation.mdc(1 hunks).github/workflows/build_and_test.yml(2 hunks)codegenerator/cli/npm/envio/src/Batch.res(4 hunks)codegenerator/cli/npm/envio/src/Envio.res(1 hunks)codegenerator/cli/npm/envio/src/EventRegister.res(4 hunks)codegenerator/cli/npm/envio/src/EventRegister.resi(1 hunks)codegenerator/cli/npm/envio/src/FetchState.res(14 hunks)codegenerator/cli/npm/envio/src/Internal.res(5 hunks)codegenerator/cli/npm/envio/src/InternalConfig.res(1 hunks)codegenerator/cli/npm/envio/src/Persistence.res(4 hunks)codegenerator/cli/npm/envio/src/PgStorage.res(16 hunks)codegenerator/cli/npm/envio/src/Prometheus.res(3 hunks)codegenerator/cli/npm/envio/src/ReorgDetection.res(2 hunks)codegenerator/cli/npm/envio/src/SafeCheckpointTracking.res(1 hunks)codegenerator/cli/npm/envio/src/Utils.res(2 hunks)codegenerator/cli/npm/envio/src/db/EntityHistory.res(3 hunks)codegenerator/cli/npm/envio/src/db/InternalTable.gen.ts(1 hunks)codegenerator/cli/npm/envio/src/db/InternalTable.res(11 hunks)codegenerator/cli/npm/envio/src/db/Table.res(1 hunks)codegenerator/cli/npm/envio/src/sources/EventRouter.res(1 hunks)codegenerator/cli/npm/envio/src/sources/Source.res(1 hunks)codegenerator/cli/src/config_parsing/entity_parsing.rs(1 hunks)codegenerator/cli/templates/dynamic/codegen/src/RegisterHandlers.res.hbs(1 hunks)codegenerator/cli/templates/dynamic/codegen/src/TestHelpers_MockDb.res.hbs(3 hunks)codegenerator/cli/templates/dynamic/codegen/src/Types.res.hbs(0 hunks)codegenerator/cli/templates/dynamic/codegen/src/db/Entities.res.hbs(3 hunks)codegenerator/cli/templates/dynamic/contract_import_templates/javascript/src/EventHandlers.js.hbs(1 hunks)codegenerator/cli/templates/dynamic/contract_import_templates/typescript/src/EventHandlers.ts.hbs(1 hunks)codegenerator/cli/templates/static/codegen/src/Config.res(4 hunks)codegenerator/cli/templates/static/codegen/src/EventProcessing.res(13 hunks)codegenerator/cli/templates/static/codegen/src/IO.res(6 hunks)codegenerator/cli/templates/static/codegen/src/InMemoryStore.res(4 hunks)codegenerator/cli/templates/static/codegen/src/InMemoryTable.res(5 hunks)codegenerator/cli/templates/static/codegen/src/LoadLayer.res(1 hunks)codegenerator/cli/templates/static/codegen/src/UserContext.res(4 hunks)codegenerator/cli/templates/static/codegen/src/db/DbFunctions.res(0 hunks)codegenerator/cli/templates/static/codegen/src/db/DbFunctionsImplementation.js(0 hunks)codegenerator/cli/templates/static/codegen/src/eventFetching/ChainFetcher.res(14 hunks)codegenerator/cli/templates/static/codegen/src/eventFetching/ChainManager.res(6 hunks)codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res(29 hunks)codegenerator/cli/templates/static/codegen/src/globalState/GlobalStateManager.res(3 hunks)scenarios/erc20_multichain_factory/test/MockChainData.res(1 hunks)scenarios/erc20_multichain_factory/test/RollbackDynamicContract_test.res(0 hunks)scenarios/erc20_multichain_factory/test/RollbackMultichain_test.res(0 hunks)scenarios/erc20_multichain_factory/test/TestDeleteEntity.res(0 hunks)scenarios/erc20_multichain_factory/test/TestWhereQuery.res(0 hunks)scenarios/helpers/src/ChainMocking.res(1 hunks)scenarios/helpers/src/Indexer.res(0 hunks)scenarios/test_codegen/schema.graphql(1 hunks)scenarios/test_codegen/test/ChainFetcher_test.res(0 hunks)scenarios/test_codegen/test/ChainManager_test.res(5 hunks)scenarios/test_codegen/test/ChainsStateComputation_test.res(0 hunks)scenarios/test_codegen/test/E2EEthNode_test.res(1 hunks)scenarios/test_codegen/test/E2E_test.res(4 hunks)scenarios/test_codegen/test/EventHandler_test.ts(4 hunks)scenarios/test_codegen/test/EventOrigin_test.res(1 hunks)scenarios/test_codegen/test/Integration_ts_helpers.res(2 hunks)scenarios/test_codegen/test/Mock_test.res(0 hunks)scenarios/test_codegen/test/ReorgDetection_test.res(23 hunks)scenarios/test_codegen/test/SerDe_Test.res(0 hunks)scenarios/test_codegen/test/WriteRead_test.res(1 hunks)scenarios/test_codegen/test/__mocks__/DbStub.res(0 hunks)scenarios/test_codegen/test/__mocks__/MockConfig.res(1 hunks)scenarios/test_codegen/test/helpers/Mock.res(14 hunks)scenarios/test_codegen/test/lib_tests/EntityHistory_test.res(1 hunks)scenarios/test_codegen/test/lib_tests/EventRouter_test.res(1 hunks)scenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.res(1 hunks)scenarios/test_codegen/test/lib_tests/FetchState_test.res(41 hunks)scenarios/test_codegen/test/lib_tests/Persistence_test.res(2 hunks)scenarios/test_codegen/test/lib_tests/PgStorage_test.res(20 hunks)scenarios/test_codegen/test/lib_tests/SourceManager_test.res(1 hunks)scenarios/test_codegen/test/rollback/MockChainData.res(1 hunks)scenarios/test_codegen/test/rollback/Rollback_test.res(41 hunks)scenarios/test_codegen/test/schema_types/BigDecimal_test.res(1 hunks)scenarios/test_codegen/test/schema_types/Timestamp_test.res(1 hunks)
💤 Files with no reviewable changes (13)
- scenarios/erc20_multichain_factory/test/TestDeleteEntity.res
- scenarios/test_codegen/test/ChainFetcher_test.res
- scenarios/erc20_multichain_factory/test/TestWhereQuery.res
- scenarios/test_codegen/test/Mock_test.res
- scenarios/erc20_multichain_factory/test/RollbackDynamicContract_test.res
- scenarios/erc20_multichain_factory/test/RollbackMultichain_test.res
- scenarios/test_codegen/test/SerDe_Test.res
- scenarios/test_codegen/test/ChainsStateComputation_test.res
- codegenerator/cli/templates/static/codegen/src/db/DbFunctionsImplementation.js
- scenarios/test_codegen/test/mocks/DbStub.res
- scenarios/helpers/src/Indexer.res
- codegenerator/cli/templates/static/codegen/src/db/DbFunctions.res
- codegenerator/cli/templates/dynamic/codegen/src/Types.res.hbs
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{res,resi}
📄 CodeRabbit inference engine (.cursor/rules/rescript.mdc)
**/*.{res,resi}: Never use[| item |]to create an array. Use[ item ]instead.
Must always use=for setting value to a field. Use:=only for ref values created usingreffunction.
ReScript has record types which require a type definition before hand. You can access record fields by dot likefoo.myField.
It's also possible to define an inline object, it'll have quoted fields in this case.
Use records when working with structured data, and objects to conveniently pass payload data between functions.
Never use %raw to access object fields if you know the type.
Files:
codegenerator/cli/npm/envio/src/EventRegister.resicodegenerator/cli/npm/envio/src/sources/Source.resscenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.resscenarios/test_codegen/test/WriteRead_test.rescodegenerator/cli/npm/envio/src/EventRegister.resscenarios/test_codegen/test/EventOrigin_test.resscenarios/test_codegen/test/lib_tests/EventRouter_test.rescodegenerator/cli/templates/static/codegen/src/LoadLayer.rescodegenerator/cli/npm/envio/src/SafeCheckpointTracking.resscenarios/test_codegen/test/lib_tests/SourceManager_test.resscenarios/test_codegen/test/lib_tests/Persistence_test.rescodegenerator/cli/npm/envio/src/Envio.resscenarios/test_codegen/test/rollback/MockChainData.resscenarios/test_codegen/test/schema_types/BigDecimal_test.resscenarios/test_codegen/test/E2E_test.rescodegenerator/cli/npm/envio/src/db/Table.rescodegenerator/cli/npm/envio/src/InternalConfig.resscenarios/test_codegen/test/schema_types/Timestamp_test.resscenarios/test_codegen/test/E2EEthNode_test.resscenarios/erc20_multichain_factory/test/MockChainData.rescodegenerator/cli/templates/static/codegen/src/EventProcessing.rescodegenerator/cli/npm/envio/src/sources/EventRouter.rescodegenerator/cli/templates/static/codegen/src/Config.rescodegenerator/cli/templates/static/codegen/src/InMemoryTable.resscenarios/test_codegen/test/__mocks__/MockConfig.rescodegenerator/cli/templates/static/codegen/src/InMemoryStore.rescodegenerator/cli/templates/static/codegen/src/eventFetching/ChainManager.resscenarios/helpers/src/ChainMocking.resscenarios/test_codegen/test/rollback/Rollback_test.resscenarios/test_codegen/test/lib_tests/PgStorage_test.rescodegenerator/cli/npm/envio/src/PgStorage.rescodegenerator/cli/templates/static/codegen/src/UserContext.rescodegenerator/cli/templates/static/codegen/src/IO.resscenarios/test_codegen/test/ReorgDetection_test.resscenarios/test_codegen/test/Integration_ts_helpers.resscenarios/test_codegen/test/helpers/Mock.rescodegenerator/cli/npm/envio/src/Batch.rescodegenerator/cli/npm/envio/src/Persistence.resscenarios/test_codegen/test/lib_tests/EntityHistory_test.resscenarios/test_codegen/test/ChainManager_test.rescodegenerator/cli/templates/static/codegen/src/globalState/GlobalStateManager.rescodegenerator/cli/templates/static/codegen/src/eventFetching/ChainFetcher.rescodegenerator/cli/npm/envio/src/db/EntityHistory.rescodegenerator/cli/npm/envio/src/Prometheus.rescodegenerator/cli/npm/envio/src/ReorgDetection.rescodegenerator/cli/npm/envio/src/Internal.rescodegenerator/cli/templates/static/codegen/src/globalState/GlobalState.resscenarios/test_codegen/test/lib_tests/FetchState_test.rescodegenerator/cli/npm/envio/src/FetchState.rescodegenerator/cli/npm/envio/src/Utils.rescodegenerator/cli/npm/envio/src/db/InternalTable.res
**/*.res
📄 CodeRabbit inference engine (.cursor/rules/navigation.mdc)
Prefer reading ReScript .res modules directly; compiled .js artifacts can be ignored
Files:
codegenerator/cli/npm/envio/src/sources/Source.resscenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.resscenarios/test_codegen/test/WriteRead_test.rescodegenerator/cli/npm/envio/src/EventRegister.resscenarios/test_codegen/test/EventOrigin_test.resscenarios/test_codegen/test/lib_tests/EventRouter_test.rescodegenerator/cli/templates/static/codegen/src/LoadLayer.rescodegenerator/cli/npm/envio/src/SafeCheckpointTracking.resscenarios/test_codegen/test/lib_tests/SourceManager_test.resscenarios/test_codegen/test/lib_tests/Persistence_test.rescodegenerator/cli/npm/envio/src/Envio.resscenarios/test_codegen/test/rollback/MockChainData.resscenarios/test_codegen/test/schema_types/BigDecimal_test.resscenarios/test_codegen/test/E2E_test.rescodegenerator/cli/npm/envio/src/db/Table.rescodegenerator/cli/npm/envio/src/InternalConfig.resscenarios/test_codegen/test/schema_types/Timestamp_test.resscenarios/test_codegen/test/E2EEthNode_test.resscenarios/erc20_multichain_factory/test/MockChainData.rescodegenerator/cli/templates/static/codegen/src/EventProcessing.rescodegenerator/cli/npm/envio/src/sources/EventRouter.rescodegenerator/cli/templates/static/codegen/src/Config.rescodegenerator/cli/templates/static/codegen/src/InMemoryTable.resscenarios/test_codegen/test/__mocks__/MockConfig.rescodegenerator/cli/templates/static/codegen/src/InMemoryStore.rescodegenerator/cli/templates/static/codegen/src/eventFetching/ChainManager.resscenarios/helpers/src/ChainMocking.resscenarios/test_codegen/test/rollback/Rollback_test.resscenarios/test_codegen/test/lib_tests/PgStorage_test.rescodegenerator/cli/npm/envio/src/PgStorage.rescodegenerator/cli/templates/static/codegen/src/UserContext.rescodegenerator/cli/templates/static/codegen/src/IO.resscenarios/test_codegen/test/ReorgDetection_test.resscenarios/test_codegen/test/Integration_ts_helpers.resscenarios/test_codegen/test/helpers/Mock.rescodegenerator/cli/npm/envio/src/Batch.rescodegenerator/cli/npm/envio/src/Persistence.resscenarios/test_codegen/test/lib_tests/EntityHistory_test.resscenarios/test_codegen/test/ChainManager_test.rescodegenerator/cli/templates/static/codegen/src/globalState/GlobalStateManager.rescodegenerator/cli/templates/static/codegen/src/eventFetching/ChainFetcher.rescodegenerator/cli/npm/envio/src/db/EntityHistory.rescodegenerator/cli/npm/envio/src/Prometheus.rescodegenerator/cli/npm/envio/src/ReorgDetection.rescodegenerator/cli/npm/envio/src/Internal.rescodegenerator/cli/templates/static/codegen/src/globalState/GlobalState.resscenarios/test_codegen/test/lib_tests/FetchState_test.rescodegenerator/cli/npm/envio/src/FetchState.rescodegenerator/cli/npm/envio/src/Utils.rescodegenerator/cli/npm/envio/src/db/InternalTable.res
codegenerator/cli/templates/{static,dynamic}/codegen/src/**/*
📄 CodeRabbit inference engine (.cursor/rules/navigation.mdc)
Edit template versions under codegenerator/cli/templates/static/codegen/src or codegenerator/cli/templates/dynamic/codegen/src instead of editing generated/src
Files:
codegenerator/cli/templates/static/codegen/src/LoadLayer.rescodegenerator/cli/templates/dynamic/codegen/src/TestHelpers_MockDb.res.hbscodegenerator/cli/templates/static/codegen/src/EventProcessing.rescodegenerator/cli/templates/static/codegen/src/Config.rescodegenerator/cli/templates/static/codegen/src/InMemoryTable.rescodegenerator/cli/templates/static/codegen/src/InMemoryStore.rescodegenerator/cli/templates/dynamic/codegen/src/db/Entities.res.hbscodegenerator/cli/templates/static/codegen/src/eventFetching/ChainManager.rescodegenerator/cli/templates/static/codegen/src/UserContext.rescodegenerator/cli/templates/static/codegen/src/IO.rescodegenerator/cli/templates/dynamic/codegen/src/RegisterHandlers.res.hbscodegenerator/cli/templates/static/codegen/src/globalState/GlobalStateManager.rescodegenerator/cli/templates/static/codegen/src/eventFetching/ChainFetcher.rescodegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res
scenarios/test_codegen/**/*.ts
📄 CodeRabbit inference engine (scenarios/test_codegen/.cursor/rules/hyperindex.mdc)
scenarios/test_codegen/**/*.ts: After changing any TypeScript files, run: pnpm tsc --noEmit to ensure successful compilation
When updating existing entities in handlers, always use the spread operator to create updated objects before persisting
For any external call (e.g., fetch), wrap it in an Effect via experimental_createEffect and consume via context.effect
Use !context.isPreload to skip logic that should not run during preload
In TypeScript, set relationship fields using *_id properties (e.g., token_id) rather than object references
Always cast timestamps from events to BigInt (e.g., BigInt(event.block.timestamp))
When matching addresses in configuration objects within code, ensure keys are lowercase and compare using address.toLowerCase()
Use string | undefined for optional string fields instead of string | null
Always normalize token amounts to a standard decimal (e.g., 18) before addition across tokens; use helpers like normalizeAmountToUSD()
Files:
scenarios/test_codegen/test/EventHandler_test.ts
scenarios/test_codegen/**/schema.graphql
📄 CodeRabbit inference engine (scenarios/test_codegen/.cursor/rules/hyperindex.mdc)
scenarios/test_codegen/**/schema.graphql: After changing schema.graphql, run: pnpm codegen
Do not add the @entity decorator to GraphQL schema types
Avoid time-series aggregate fields (e.g., dailyVolume) in the GraphQL schema
Never use arrays of entities in the schema (e.g., [User!]!)
Model relationships using *_id fields in the schema (e.g., user_id: String!) instead of entity references
Files:
scenarios/test_codegen/schema.graphql
🧠 Learnings (11)
📚 Learning: 2025-10-13T12:26:45.351Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: .cursor/rules/subgraph-migration.mdc:0-0
Timestamp: 2025-10-13T12:26:45.351Z
Learning: Applies to src/**/*.ts : Register dynamically created contracts using `contractRegister` above the corresponding factory event handler
Applied to files:
codegenerator/cli/npm/envio/src/EventRegister.res
📚 Learning: 2025-10-13T12:27:30.803Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: codegenerator/cli/templates/static/shared/.cursor/rules/subgraph-migration.mdc:0-0
Timestamp: 2025-10-13T12:27:30.803Z
Learning: Applies to codegenerator/cli/templates/static/shared/src/**/*.ts : Register dynamic contracts using contractRegister above the creating event’s handler (e.g., context.addPair(event.params.pair))
Applied to files:
codegenerator/cli/npm/envio/src/EventRegister.res
📚 Learning: 2025-10-13T12:27:30.803Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: codegenerator/cli/templates/static/shared/.cursor/rules/subgraph-migration.mdc:0-0
Timestamp: 2025-10-13T12:27:30.803Z
Learning: Applies to codegenerator/cli/templates/static/shared/src/**/*.ts : Prefix all entity IDs with event.chainId and never hardcode chainId=1; accept chainId in helper functions; use chain-specific Bundle IDs like `${chainId}-1`
Applied to files:
codegenerator/cli/templates/dynamic/codegen/src/TestHelpers_MockDb.res.hbsscenarios/test_codegen/test/EventHandler_test.ts
📚 Learning: 2025-08-11T08:49:57.399Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: scenarios/test_codegen/.cursor/rules/hyperindex.mdc:0-0
Timestamp: 2025-08-11T08:49:57.399Z
Learning: Applies to scenarios/test_codegen/**/schema.graphql : Do not add the entity decorator to GraphQL schema types
Applied to files:
scenarios/test_codegen/schema.graphql
📚 Learning: 2025-08-11T08:49:57.399Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: scenarios/test_codegen/.cursor/rules/hyperindex.mdc:0-0
Timestamp: 2025-08-11T08:49:57.399Z
Learning: Applies to scenarios/test_codegen/**/schema.graphql : Model relationships using *_id fields in the schema (e.g., user_id: String!) instead of entity references
Applied to files:
scenarios/test_codegen/schema.graphql
📚 Learning: 2025-10-13T12:27:30.803Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: codegenerator/cli/templates/static/shared/.cursor/rules/subgraph-migration.mdc:0-0
Timestamp: 2025-10-13T12:27:30.803Z
Learning: Applies to codegenerator/cli/templates/static/shared/src/EventHandlers.ts : Remove the original src/EventHandlers.ts after moving handlers into contract-specific files
Applied to files:
codegenerator/cli/templates/dynamic/contract_import_templates/typescript/src/EventHandlers.ts.hbs
📚 Learning: 2025-10-13T12:27:30.803Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: codegenerator/cli/templates/static/shared/.cursor/rules/subgraph-migration.mdc:0-0
Timestamp: 2025-10-13T12:27:30.803Z
Learning: Applies to codegenerator/cli/templates/static/shared/**/config.yaml : Refactor to contract-specific handler files in config.yaml and remove the single EventHandlers.ts reference
Applied to files:
codegenerator/cli/templates/dynamic/contract_import_templates/typescript/src/EventHandlers.ts.hbscodegenerator/cli/templates/dynamic/codegen/src/RegisterHandlers.res.hbs
📚 Learning: 2025-10-13T12:27:30.803Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: codegenerator/cli/templates/static/shared/.cursor/rules/subgraph-migration.mdc:0-0
Timestamp: 2025-10-13T12:27:30.803Z
Learning: Applies to codegenerator/cli/templates/static/shared/src/EventHandlers.ts : Clear all boilerplate logic from src/EventHandlers.ts and replace handlers with empty skeletons before implementing business logic
Applied to files:
codegenerator/cli/templates/dynamic/contract_import_templates/typescript/src/EventHandlers.ts.hbs
📚 Learning: 2025-10-13T12:26:45.351Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: .cursor/rules/subgraph-migration.mdc:0-0
Timestamp: 2025-10-13T12:26:45.351Z
Learning: Applies to src/EventHandlers.ts : Remove the monolithic EventHandlers.ts file after moving handlers to contract-specific files
Applied to files:
codegenerator/cli/templates/dynamic/contract_import_templates/typescript/src/EventHandlers.ts.hbs
📚 Learning: 2025-08-11T08:42:57.311Z
Learnt from: CR
PR: enviodev/hyperindex#0
File: codegenerator/cli/templates/static/shared/.cursor/rules/hyperindex.mdc:0-0
Timestamp: 2025-08-11T08:42:57.311Z
Learning: Applies to codegenerator/cli/templates/static/shared/**/*.ts : When updating existing entities in handlers, always use the spread operator because returned objects are read-only
Applied to files:
codegenerator/cli/templates/dynamic/contract_import_templates/typescript/src/EventHandlers.ts.hbs
📚 Learning: 2025-05-27T17:07:12.878Z
Learnt from: JonoPrest
PR: enviodev/hyperindex#555
File: scenarios/test_codegen/test/lib_tests/EntityHistory_test.res:631-659
Timestamp: 2025-05-27T17:07:12.878Z
Learning: In isolated tests for EntityHistory rollbacks with controlled mock data setup, the ordering of getRollbackDiff results is deterministic and doesn't require sorting before comparison with deepStrictEqual.
Applied to files:
scenarios/test_codegen/test/rollback/Rollback_test.resscenarios/test_codegen/test/lib_tests/EntityHistory_test.res
🪛 Checkov (3.2.334)
.github/workflows/build_and_test.yml
[medium] 43-44: Basic Auth Credentials
(CKV_SECRET_4)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build_and_test
| let historyTableRef = `"${pgSchema}"."${historyTableName(~entityName, ~entityIndex)}"` | ||
| `WITH target_ids AS ( | ||
| SELECT UNNEST($1::${(Text: Table.fieldType :> string)}[]) AS id | ||
| ), | ||
| missing_history AS ( | ||
| SELECT e.* | ||
| FROM "${pgSchema}"."${entityName}" e | ||
| JOIN target_ids t ON e.id = t.id | ||
| LEFT JOIN ${historyTableRef} h ON h.id = e.id | ||
| WHERE h.id IS NULL | ||
| ) | ||
| INSERT INTO ${historyTableRef} | ||
| SELECT *, 0 AS ${checkpointIdFieldName}, '${(RowAction.SET :> string)}' as ${changeFieldName} | ||
| FROM missing_history;` | ||
| } |
There was a problem hiding this comment.
Backfill INSERT uses SELECT * without an explicit column list — brittle and risky.
If entity columns change or order differs, backfill will misalign. Specify columns explicitly.
-INSERT INTO ${historyTableRef}
-SELECT *, 0 AS ${checkpointIdFieldName}, '${(RowAction.SET :> string)}' as ${changeFieldName}
-FROM missing_history;`
+INSERT INTO ${historyTableRef} (
+ ${/* data fields first, then checkpoint and change */""}
+ ${table.fields
+ ->Belt.Array.map(field => field->getFieldName)
+ ->Belt.Array.keep(name => name != checkpointIdFieldName && name != changeFieldName)
+ ->Belt.Array.map(name => "\"" ++ name ++ "\"")
+ ->Js.Array2.joinWith(", ")
+ }, "${checkpointIdFieldName}", "${changeFieldName}"
+)
+SELECT
+ ${table.fields
+ ->Belt.Array.map(field => field->getFieldName)
+ ->Belt.Array.keep(name => name != checkpointIdFieldName && name != changeFieldName)
+ ->Belt.Array.map(name => "e.\"" ++ name ++ "\"")
+ ->Js.Array2.joinWith(", ")
+ },
+ 0 AS ${checkpointIdFieldName},
+ '${(RowAction.SET :> string)}' as ${changeFieldName}
+FROM missing_history e;`Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In codegenerator/cli/npm/envio/src/db/EntityHistory.res around lines 230 to 244,
the backfill INSERT currently uses SELECT * which is brittle; replace it with an
explicit column list for the history table and an explicit SELECT column list
from the entity table to guarantee correct ordering and mapping. Concretely,
construct the target column list for ${historyTableName(~entityName,
~entityIndex)} including all history columns (e.g. the entity fields in the
exact order the history table expects plus ${checkpointIdFieldName} and
${changeFieldName}), and change the SELECT to list the corresponding entity
columns (matching types/order) followed by the literal 0 for
${checkpointIdFieldName} and '${(RowAction.SET :> string)}' for
${changeFieldName}; ensure any computed or default columns are handled
explicitly and the generated SQL uses these explicit column lists instead of
SELECT *.
| let makeGetReorgCheckpointsQuery = (~pgSchema): string => { | ||
| // Use CTE to pre-filter chains and compute safe_block once per chain | ||
| // This is faster because: | ||
| // 1. Chains table is small, so filtering it first is cheap | ||
| // 2. safe_block is computed once per chain, not per checkpoint | ||
| // 3. Query planner can materialize the small CTE result before joining | ||
| `WITH reorg_chains AS ( | ||
| SELECT | ||
| "${(#id: Chains.field :> string)}" as id, | ||
| "${(#source_block: Chains.field :> string)}" - "${(#max_reorg_depth: Chains.field :> string)}" AS safe_block | ||
| FROM "${pgSchema}"."${Chains.table.tableName}" | ||
| WHERE "${(#max_reorg_depth: Chains.field :> string)}" > 0 | ||
| AND "${(#progress_block: Chains.field :> string)}" > "${(#source_block: Chains.field :> string)}" - "${(#max_reorg_depth: Chains.field :> string)}" | ||
| ) | ||
| SELECT | ||
| cp."${(#id: field :> string)}", | ||
| cp."${(#chain_id: field :> string)}", | ||
| cp."${(#block_number: field :> string)}", | ||
| cp."${(#block_hash: field :> string)}" | ||
| FROM "${pgSchema}"."${table.tableName}" cp | ||
| INNER JOIN reorg_chains rc | ||
| ON cp."${(#chain_id: field :> string)}" = rc.id | ||
| WHERE cp."${(#block_hash: field :> string)}" IS NOT NULL | ||
| AND cp."${(#block_number: field :> string)}" >= rc.safe_block;` // Include safe_block checkpoint to use it for safe checkpoint tracking | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
Add supporting indexes for checkpoint lookups.
Queries filter/join by chain_id and block_number, and order by id. Add a composite index like (chain_id, block_number, id DESC) to speed up getRollbackTargetCheckpoint and the reorg CTE join.
🏁 Script executed:
#!/bin/bash
# Check if indexes exist on envio_checkpoints
rg -nP 'CREATE\s+INDEX.*envio_checkpoints' -C2Length of output: 0
I need to explore the codebase more broadly to understand how indexes are defined and check for existing indexes on the checkpoints table.
Let me search the codebase more comprehensively for how database indexes are defined and managed.
---
🏁 Script executed:
```shell
#!/bin/bash
# Search for migration files or schema definitions
fd -e sql -e ts -e res -e js | head -20
Length of output: 891
🏁 Script executed:
#!/bin/bash
# Look for any index definitions more broadly
rg -nP '(index|INDEX)' --type=sql -C1 | head -50Length of output: 148
🏁 Script executed:
#!/bin/bash
# Search for envio_checkpoints mentions
rg -n 'envio_checkpoints|checkpoints' --type=res -C2 | head -40Length of output: 27
🏁 Script executed:
#!/bin/bash
# Find InternalTable.res file
find . -name "InternalTable.res" -type fLength of output: 54
🏁 Script executed:
#!/bin/bash
# Search for checkpoints table definition in .res files
rg -n 'checkpoints|Checkpoints' -C3 | head -60Length of output: 9892
🏁 Script executed:
#!/bin/bash
# Look for migration or schema directories
fd -t d 'migrations|schema|db' | head -20Length of output: 212
🏁 Script executed:
#!/bin/bash
# Search for CREATE statements or index patterns in any file type
rg -n 'CREATE.*INDEX|create.*index' -C2 | head -50Length of output: 5995
🏁 Script executed:
#!/bin/bash
# Read InternalTable.res to find Checkpoints table definition and index configuration
head -500 codegenerator/cli/npm/envio/src/db/InternalTable.res | tail -200Length of output: 6558
🏁 Script executed:
#!/bin/bash
# Search for Checkpoints module in InternalTable.res
rg -n 'module Checkpoints|table.*=.*Checkpoints' codegenerator/cli/npm/envio/src/db/InternalTable.res -A10Length of output: 202
🏁 Script executed:
#!/bin/bash
# Search for getRollbackTargetCheckpoint function mentioned in review
rg -n 'getRollbackTargetCheckpoint|rollback.*checkpoint' codegenerator/cli/npm/envio/src/db/InternalTable.res -B2 -A5Length of output: 168
🏁 Script executed:
#!/bin/bash
# Check what indexes are defined for tables in InternalTable.res
rg -n 'indexes\s*=|makeCreateIndexQuery' codegenerator/cli/npm/envio/src/db/InternalTable.res -B2 -A2Length of output: 0
---
🏁 Script executed:
```shell
#!/bin/bash
# Get getRollbackTargetCheckpoint implementation
rg -n 'makeGetRollbackTargetCheckpointQuery|getRollbackTargetCheckpoint' codegenerator/cli/npm/envio/src/db/InternalTable.res -A15
Length of output: 1184
🏁 Script executed:
#!/bin/bash
# Check other table definitions for index patterns
rg -n 'let table = mkTable' codegenerator/cli/npm/envio/src/db/InternalTable.res -A15 | head -80Length of output: 3820
🏁 Script executed:
#!/bin/bash
# Look for any indexes field in table definitions across the file
cat codegenerator/cli/npm/envio/src/db/InternalTable.res | grep -n 'indexes' -B5 -A5Length of output: 0
---
🏁 Script executed:
```shell
#!/bin/bash
# Search for where indexes are created for Checkpoints or internal tables
rg -n 'makeCreateIndexQuery.*[Cc]heckpoint' --type=res -C3
Length of output: 27
🏁 Script executed:
#!/bin/bash
# Search for PgStorage index creation calls
rg -n 'makeCreateTableIndicesQuery|makeCreateIndexQuery' --type=res -C2 | head -50Length of output: 27
🏁 Script executed:
#!/bin/bash
# Check PgStorage module to understand index creation
find . -name "PgStorage.res" -type fLength of output: 47
🏁 Script executed:
#!/bin/bash
# Search for index creation calls without type filter
rg -n 'makeCreateTableIndicesQuery|makeCreateIndexQuery' -C2 | head -60Length of output: 5019
🏁 Script executed:
#!/bin/bash
# Search where indexes might be created for Checkpoints or internal tables
rg -n 'Checkpoints.*index|createIndex.*Checkpoint' -C2Length of output: 0
🏁 Script executed:
#!/bin/bash
# Look at PgStorage.res to understand how indexes are created
head -200 codegenerator/cli/npm/envio/src/PgStorage.res | tail -100Length of output: 3446
🏁 Script executed:
#!/bin/bash
# Search for where internal tables get indexes created
rg -n 'InternalTable\.(Checkpoints|Chains|RawEvents)' codegenerator/cli/npm/envio/src/PgStorage.res -B2 -A5Length of output: 2976
🏁 Script executed:
#!/bin/bash
# Look for tables with indexes defined to understand the pattern
rg -n 'indexes\s*:' codegenerator/cli -B3 -A3 | head -40Length of output: 3535
🏁 Script executed:
#!/bin/bash
# Search for Table.table type definition to understand indexes field
rg -n 'type table\s*=|indexes' codegenerator/cli/npm/envio/src/Table.res -B2 -A5 | head -60Length of output: 81
---
🏁 Script executed:
```shell
#!/bin/bash
# Find Table.res file
find . -name "Table.res" -type f
Length of output: 46
🏁 Script executed:
#!/bin/bash
# Search for table type definition
rg -n 'type table\s*=' codegenerator/cli/npm/envio/src -A10Length of output: 750
🏁 Script executed:
#!/bin/bash
# Search in makeCreateTableIndicesQuery to understand how indexes are accessed
rg -n 'makeCreateTableIndicesQuery' -A20 codegenerator/cli/npm/envio/src/PgStorage.resLength of output: 1569
---
🏁 Script executed:
```shell
#!/bin/bash
# Verify the exact Checkpoints.table definition
sed -n '403,412p' codegenerator/cli/npm/envio/src/db/InternalTable.res
Length of output: 486
Add supporting indexes for checkpoint queries.
The envio_checkpoints table at lines 403-412 lacks composite indexes despite multiple queries filtering and joining on chain_id and block_number. Both makeGetRollbackTargetCheckpointQuery (lines 501-506) and makeGetReorgCheckpointsQuery (lines 414-438) perform these operations without index support, causing full table scans.
Add ~compositeIndices=[["chain_id", "block_number"]] to the mkTable call for the Checkpoints table to enable the index generation mechanism (via makeCreateTableIndicesQuery in PgStorage.res) to create the supporting indexes.
🤖 Prompt for AI Agents
In codegenerator/cli/npm/envio/src/db/InternalTable.res around lines 403 to 412
and relevant query usage at 414 to 438, the Checkpoints table definition
(mkTable call) lacks a composite index on chain_id and block_number causing
full-table scans for queries like makeGetRollbackTargetCheckpointQuery and
makeGetReorgCheckpointsQuery; fix by adding the composite index configuration to
the mkTable call for the Checkpoints table: include
~compositeIndices=[["chain_id","block_number"]] in the mkTable parameters so the
existing makeCreateTableIndicesQuery will generate the supporting composite
index.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
codegenerator/cli/src/config_parsing/chain_helpers.rs(9 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
codegenerator/cli/src/config_parsing/chain_helpers.rs (1)
codegenerator/cli/src/cli_args/interactive_init/evm_prompts.rs (1)
HypersyncNetwork(224-232)
🪛 GitHub Actions: Build & Test Scenario
codegenerator/cli/src/config_parsing/chain_helpers.rs
[error] 689-689: config_parsing::chain_helpers::test::networks_are_defined_in_alphabetical_order failed: Networks are not sorted alphabetically (see assertion at chain_helpers.rs:689).
🔇 Additional comments (4)
codegenerator/cli/src/config_parsing/chain_helpers.rs (4)
186-187: Confirm intent: removed from HypersyncNetwork.
- GaladrielDevnet now only NetworkWithExplorer.
- UnichainSepolia now only NetworkWithExplorer.
This affects HypersyncNetwork::iter() consumers (e.g., interactive selection). Confirm this removal is intentional.
Also applies to: 383-385
473-476: Threshold mappings for new networks look consistent; validate policy.New networks mapped to DEFAULT_CONFIRMED_BLOCK_THRESHOLD. If any of these have known reorg profiles requiring 0 or custom thresholds, adjust accordingly.
Also applies to: 480-480, 519-519, 526-526
618-619: Tier assignments added; looks coherent with existing scheme.Plasma → Bronze; ChainwebTestnet22/23/24, Damon, SentientTestnet → Stone. Confirm these align with product tier policy.
Also applies to: 622-630
119-126: Alphabetical ordering violation detected; verify scope and whether pre-existing or introduced by this PR.The verification script found a constraint violation: the enum is not in alphabetical order by kebab-case—"Darwinia" appears before "Damon" (darwinia > damon alphabetically). The ChainwebTestnet22-24 additions themselves are correctly ordered and have unique IDs with no duplicates.
Determine whether this ordering issue is pre-existing or introduced by these changes. If introduced, all out-of-order variants need repositioning to maintain alphabetical ordering.
Includes:
Plus:
envio_progress_batches_countSummary by CodeRabbit
New Features
Improvements
Bug Fixes
Documentation