Skip to content

Conversation

aboodman
Copy link
Contributor

@aboodman aboodman commented Sep 6, 2025

This was surprisingly difficult because StandardSchema supports async validation. This meant that SyncedQuery needed to get forked into ValidatedQuery, but fortunately that didn't spiral too far.

The other crazy unfortunate collision was that the async validation means that withValidation becomes async. But the return value withValidation is a query. And queries themselves are promise-like. This means that when code that calls withValidation uses await, it will unwrap recursively because that's what await does.

The result is that awaiting the result of withValidation in order to return the query over the http response actually tries to run the query.

The one lucky break was that handleGetQueriesRequest actually wants {query: Q}, not Q as an input. So I was able to leverage that and have withValidation return that shape not a raw query. This stops the recursive unwrapping of promises from happening. I think the previous API was actually better, but it's the best I could think of.

We probably need to remove the ability to run queries directly, as we've now added zero.run and friends. But we cannot do that until we add tx.run. That will likely have usability impacts on reading data inside transactions.

This was surprisingly difficult because StandardSchema supports
async validation. This meant that SyncedQuery needed to get forked
into ValidatedQuery, but fortunately that didn't spiral too far.

The other crazy unfortunate collision was that the async
validation means that `withValidation` becomes async. But the return
value `withValidation` is a query. And queries themselves are
promise-like. This means that when code that calls `withValidation`
uses await, it will unwrap recursively because that's what await does.

The result is that awaiting the result of `withValidation` in order to
return the query over the http response actually tries to run the
query.

The one lucky break was that `handleGetQueriesRequest` actually wants
{query: Q}, not Q as an input. So I was able to leverage that and
have `withValidation` return that shape not a raw query. This stops
the recursive unwrapping of promises from happening. I think the
previous API was actually better, but it's the best I could think
of.

We probably need to remove the ability to run queries directly, as
we've now added zero.run and friends. But we cannot do that until we
add tx.run. That will likely have usability impacts on reading data
inside transactions.
@aboodman aboodman requested a review from tantaman September 6, 2025 04:16
Copy link

vercel bot commented Sep 6, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
replicache-docs Ready Ready Preview Comment Sep 6, 2025 4:17am
zbugs Ready Ready Preview Comment Sep 6, 2025 4:17am

Copy link

github-actions bot commented Sep 6, 2025

🐰 Bencher Report

Branchaa/standard-schema
Testbedself-hosted

🚨 3 Alerts

BenchmarkMeasure
Units
ViewBenchmark Result
(Result Δ%)
Lower Boundary
(Limit %)
src/client/zero.bench.ts > basics > All 1000 rows x 10 columns (numbers)Throughput
operations / second (ops/s) x 1e3
📈 plot
🚷 threshold
🚨 alert (🔔)
2.07 ops/s x 1e3
(-20.12%)Baseline: 2.60 ops/s x 1e3
2.33 ops/s x 1e3
(112.32%)

src/client/zero.bench.ts > pk compare > pk = NThroughput
operations / second (ops/s) x 1e3
📈 plot
🚷 threshold
🚨 alert (🔔)
39.58 ops/s x 1e3
(-4.70%)Baseline: 41.53 ops/s x 1e3
39.78 ops/s x 1e3
(100.52%)

src/client/zero.bench.ts > with filter > Lower rows 500 x 10 columns (numbers)Throughput
operations / second (ops/s) x 1e3
📈 plot
🚷 threshold
🚨 alert (🔔)
3.40 ops/s x 1e3
(-14.23%)Baseline: 3.97 ops/s x 1e3
3.67 ops/s x 1e3
(107.68%)

Click to view all benchmark results
BenchmarkThroughputBenchmark Result
operations / second (ops/s) x 1e3
(Result Δ%)
Lower Boundary
operations / second (ops/s) x 1e3
(Limit %)
src/client/custom.bench.ts > big schema📈 view plot
🚷 view threshold
661.97 ops/s x 1e3
(+10.88%)Baseline: 597.00 ops/s x 1e3
417.97 ops/s x 1e3
(63.14%)
src/client/zero.bench.ts > basics > All 1000 rows x 10 columns (numbers)📈 view plot
🚷 view threshold
🚨 view alert (🔔)
2.07 ops/s x 1e3
(-20.12%)Baseline: 2.60 ops/s x 1e3
2.33 ops/s x 1e3
(112.32%)

src/client/zero.bench.ts > pk compare > pk = N📈 view plot
🚷 view threshold
🚨 view alert (🔔)
39.58 ops/s x 1e3
(-4.70%)Baseline: 41.53 ops/s x 1e3
39.78 ops/s x 1e3
(100.52%)

src/client/zero.bench.ts > with filter > Lower rows 500 x 10 columns (numbers)📈 view plot
🚷 view threshold
🚨 view alert (🔔)
3.40 ops/s x 1e3
(-14.23%)Baseline: 3.97 ops/s x 1e3
3.67 ops/s x 1e3
(107.68%)

🐰 View full continuous benchmarking report in Bencher

Copy link

github-actions bot commented Sep 6, 2025

🐰 Bencher Report

Branchaa/standard-schema
TestbedLinux
Click to view all benchmark results
BenchmarkFile SizeBenchmark Result
kilobytes (KB)
(Result Δ%)
Upper Boundary
kilobytes (KB)
(Limit %)
zero-package.tgz📈 view plot
🚷 view threshold
1,310.53 KB
(+0.09%)Baseline: 1,309.32 KB
1,335.50 KB
(98.13%)
zero.js📈 view plot
🚷 view threshold
209.95 KB
(+0.04%)Baseline: 209.86 KB
214.06 KB
(98.08%)
zero.js.br📈 view plot
🚷 view threshold
58.96 KB
(+0.03%)Baseline: 58.94 KB
60.12 KB
(98.06%)
🐰 View full continuous benchmarking report in Bencher

@tantaman
Copy link
Contributor

tantaman commented Sep 8, 2025

nit: this is a breaking change. so feat(zero)!:

Maybe complete the refactor so we do not need to do{query: first?

Related:

Will tx in custom mutators return runnable queries or will it require the user to tx.run(query)? - #4871 (comment)

@aboodman
Copy link
Contributor Author

aboodman commented Sep 9, 2025

Yeah my plan is to complete the refactor first.

@arv
Copy link
Contributor

arv commented Sep 12, 2025

@tantaman
Copy link
Contributor

turns out we must allow withValidation to be async given the user will want to run DB queries on the server side impl of their synced query.

I forgot about this when adding all the new APIs. Before there wasn't any wrapping of the synced query so the user could create any function they wanted (sync or asnyc) on the backend to implement the synced query.

@aboodman
Copy link
Contributor Author

https://standardschema.dev/#:~:text=How%20to%20only%20allow%20synchronous%20validation%3F

I actually tried this (hopefully - i hadn't seen this link) but it didn't seem to work. The schema validators I tried always seemed to return a promise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants