control-plane-api: run publication tests on catalog-tests - #3418
Open
jgraettinger wants to merge 1 commit into
Open
control-plane-api: run publication tests on catalog-tests#3418jgraettinger wants to merge 1 commit into
jgraettinger wants to merge 1 commit into
Conversation
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
2 times, most recently
from
August 30, 2026 18:08
4d689b9 to
12b25e5
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
August 31, 2026 22:08
12b25e5 to
893de73
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
August 31, 2026 22:19
893de73 to
a7bf70c
Compare
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
2 times, most recently
from
September 1, 2026 00:51
a3a147b to
e99bf9f
Compare
Publication tests now call `catalog_tests::run_tests` in-process, against the
`build::Output` the publication already produced.
That deletes most of the machinery: `data_plane()` and its `temp-data-plane`
shell-out, `wait_for_sockets`, the activate loop that stamped build labels onto
derivation specs and activated them at 3 splits, the `flowctl-go api test` job,
and the delete loop that tore it all down again. With no data plane there is
nothing to race, so the `tokio::select!` driving the two jobs goes too.
Failures now anchor precisely: each failing case becomes a `tables::Error` scoped
to the failing *step's* source URL and JSON pointer, where before a whole run
collapsed into one error at `flow://publication/test/api/test` with "View logs
for details". A case the run never reached is reported too, so a publication
never silently understates its test coverage. Logs still stream under the "test"
stream name existing consumers select on, now gated by each task's own
`shards: {logLevel}`.
`flowctl_go` is no longer threaded anywhere: dropped from `Publisher` and
`Publisher::new`, from the agent's `main`, from its integration-test harness, and
from `test_server`. The `locate-bin` crate stays — sops and flow-connector-init
still use it.
Tests: a passing and a failing derive-sqlite catalog, asserting the failing one
surfaces exactly one publication error naming the test and rendering the diff.
They need no database, since `test_catalog` never touches Postgres.
jgraettinger
force-pushed
the
johnny/v2-tests-4-publication-tests
branch
from
September 1, 2026 01:34
e99bf9f to
5727a84
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Stack 4 of 5, one commit. Publication tests now call
catalog_tests::run_testsin-process, against thebuild::Outputthe publication already produced.Workflow steps:
No change to how a user runs tests — they still publish a draft and see test results in publication logs. What changes is where those results come from and how precisely failures are reported.
Documentation links affected:
None.
Notes for reviewers:
Most of the surface is deletion:
data_plane()and itstemp-data-planeshell-out,wait_for_sockets, the activate loop that stamped build labels onto derivation specs and activated them at 3 splits, theflowctl-go api testjob, and the delete loop that tore it all down again. With no data plane there is nothing to race, so thetokio::select!driving the two jobs goes too.flowctl_gostops being threaded throughPublisher, the agent'smain, its integration-test harness, andtest_server;locate-binstays, since sops and flow-connector-init still use it.Two behavior changes worth checking:
tables::Errorscoped to the failing step's source URL and JSON pointer, where before a whole run collapsed into one error atflow://publication/test/api/testwith "View logs for details". A case the run never reached is reported as an error too, so a publication never silently understates its test coverage.teststream name existing consumers select on, but are now gated by each task's ownshards: {logLevel}— so a user who wants a derivation's debug output in their publication logs raises that task's level.Tests are a passing and a failing derive-sqlite catalog, asserting the failing one surfaces exactly one publication error naming the test and rendering the diff. They need no database, since
test_catalognever touches Postgres.Stack — each PR is based on the one above it, so review only the top commits of each.
flowctl raw test#3417 — catalog-tests: the harness, andflowctl raw test