Repository navigation
Fix sourcedFrom blob metadata divergence #647
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kriszyp
wants to merge
13
commits into
main
Choose a base branch
from
fix/sourced-from-blob-metadata-divergence
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+643
−0
Open
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
5cf71fa
test(cluster): cover competing sourced blob fills
kriszyp 6ce93b6
test(cluster): harden sourced blob race coverage
kriszyp cfa83db
test(cluster): harden sourced race observers
kriszyp a61265c
test(cluster): explain post-probe convergence check
kriszyp 50b9aae
test(cluster): bound operation retries and take the last version row
kriszyp 61e72b5
test(cluster): correct version-scan claim, surface fetch's wrapped cause
kriszyp 4904b8f
test(cluster): cover the equal-version tie and both storage engines
kriszyp d5a8e2b
test(cluster): bound bootstrap phases and harden worker discovery
kriszyp 89aae82
test(cluster): bound worker discovery and never orphan a node
kriszyp 66ba1f2
test(cluster): settle every setup task before failing the bootstrap
kriszyp 8c48c20
test(cluster): release the barrier on every giving-up path
kriszyp 762e884
test(cluster): survive a hole in the pinned-agent array during teardown
kriszyp 9f292b8
test(cluster): yield on every miss while waiting for the late fill
kriszyp File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
5 changes: 5 additions & 0 deletions
5
integrationTests/cluster/fixture-sourced-blob-pairing/config.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| rest: true | ||
| graphqlSchema: | ||
| files: '*.graphql' | ||
| jsResource: | ||
| files: resources.js |
95 changes: 95 additions & 0 deletions
95
integrationTests/cluster/fixture-sourced-blob-pairing/resources.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| import { Readable } from 'node:stream'; | ||
| import { threadId } from 'node:worker_threads'; | ||
|
|
||
| const originUrl = process.env.HARPER_TEST_ORIGIN_URL; | ||
| // Keep payloads above core's 8 KiB inline-storage threshold. | ||
| const payloadSize = 16 * 1024; | ||
|
|
||
| function payloadFor(token) { | ||
| const bytes = Buffer.alloc(payloadSize, 0x2e); | ||
| bytes.write(token); | ||
| return createBlob(Readable.from(bytes)); | ||
| } | ||
|
|
||
| tables.PairRecord.sourcedFrom({ | ||
| async get(id, context) { | ||
| const response = await fetch(`${originUrl}/resolve`, { | ||
| method: 'POST', | ||
| headers: { 'Content-Type': 'application/json' }, | ||
| body: JSON.stringify({ id, node: server.hostname, threadId }), | ||
| }); | ||
| if (!response.ok) throw new Error(`Pairing origin returned ${response.status}: ${await response.text()}`); | ||
| const resolved = await response.json(); | ||
| // When the origin reports a shared lastModified, both nodes mint the *same* record version, | ||
| // so their competing fills tie. A tie is the case cache-fill resolution cannot order by node | ||
| // identity, and the only one version-ordered LWW never reaches. | ||
| if (resolved.lastModified != null) context.lastModified = resolved.lastModified; | ||
| return { | ||
| id, | ||
| token: resolved.token, | ||
| sourceNode: server.hostname, | ||
| sourceThread: threadId, | ||
| payload: payloadFor(resolved.token), | ||
| }; | ||
| }, | ||
| }); | ||
|
|
||
| function describeRecord(record) { | ||
| if (!record?.payload) return null; | ||
| return record.payload.bytes().then((bytes) => ({ | ||
| id: record.id, | ||
| token: record.token, | ||
| sourceNode: record.sourceNode, | ||
| sourceThread: record.sourceThread, | ||
| payloadToken: bytes.subarray(0, Buffer.byteLength(record.token)).toString(), | ||
| })); | ||
| } | ||
|
|
||
| export class PairPointProbe extends tables.PairRecord { | ||
| static async get(target) { | ||
| const record = await super.get(target); | ||
| let raw; | ||
| for (const entry of tables.PairRecord.primaryStore.getRange({ | ||
| start: target.id, | ||
| versions: true, | ||
| snapshot: false, | ||
| })) { | ||
| if (entry.key === target.id) | ||
| raw = { version: entry.version, nodeId: entry.nodeId, record: await describeRecord(entry.value) }; | ||
| break; | ||
| } | ||
| return { node: server.hostname, threadId, record: await describeRecord(record), raw }; | ||
| } | ||
| } | ||
|
|
||
| export class PairScanProbe extends Resource { | ||
| static loadAsInstance = false; | ||
|
|
||
| async get(target) { | ||
| target.checkPermission = false; | ||
| for (const entry of tables.PairRecord.primaryStore.getRange({ | ||
| start: target.id, | ||
| versions: true, | ||
| snapshot: false, | ||
| })) { | ||
| if (entry.key !== target.id) break; | ||
| return { | ||
| node: server.hostname, | ||
| threadId, | ||
| version: entry.version, | ||
| nodeId: entry.nodeId, | ||
| record: await describeRecord(entry.value), | ||
| }; | ||
| } | ||
| return { node: server.hostname, threadId, record: null }; | ||
| } | ||
| } | ||
|
|
||
| export class PairWorker extends Resource { | ||
| static loadAsInstance = false; | ||
|
|
||
| get(target) { | ||
| target.checkPermission = false; | ||
| return { threadId }; | ||
| } | ||
| } | ||
7 changes: 7 additions & 0 deletions
7
integrationTests/cluster/fixture-sourced-blob-pairing/schema.graphql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| type PairRecord @table @export { | ||
| id: String @primaryKey | ||
| token: String | ||
| sourceNode: String | ||
| sourceThread: Long | ||
| payload: Blob | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.