Skip to content

feat: move TypeScript client codegen into the SDK module - #9

Merged
TomChv merged 1 commit into
mainfrom
feat/extract-client-codegen-in-typescript-sdk
Jul 29, 2026
Merged

TomChv merged 1 commit into
mainfrom
feat/extract-client-codegen-in-typescript-sdk

Conversation

@TomChv

@TomChv TomChv commented Jul 6, 2026 •

Copy link
Copy Markdown
Member

Move TypeScript client codegen into the SDK module

Moves TypeScript client generation out of dagger/dagger's engine runtime and into this .dang SDK module. The module now generates a typed client for a bound module itself — receiving the schema + module metadata from the engine as plain data and running codegen in an ordinary container, with no nested engine session.

Important

Depends on two engine PRs, both gated v1.0.0-0:

This PR requires an engine that ships both.

What's in it

helpers/codegen — the TypeScript client generator ported from dagger/dagger's cmd/codegen as a standalone, engine-free Go module (no dagger.io/dagger dependency). Reads a pre-computed introspection schema + a meta.json (module name, engine version, bound module) and emits the client bindings. The upstream live-engine query is replaced by the meta.json input. Upstream golden tests ported to guard fidelity.

helpers/config-updator — new client config writers producing a scoped npm package: package.json (@dagger.io/<module>-client name, @dagger.io/dagger pinned to the engine version, type:module, typescript), plus tsconfig.json / deno.json.

typescript-sdk.dang — the dang wiring:

  • generateClient(ws, module, path) — the workspace analogue of mod(ws, path).generate(ws).
  • generateAllClient(ws) @generate — regenerates every registered client (currentModule.asSDK.clients), resolving each bound module via client.moduleSource and generating from clientSchemaIntrospectionJSON.
  • Renamed generateAll → generateAllModule for symmetry.
  • codegenBuilder / configUpdatorBuilder container builders (compiled like the existing helpers).

e2e — generateClientCheck, generateAllClientCheck, generateClientRespectsExistingCheck, generateClientCompilesCheck, and helperTestsCheck (runs the Go helper tests inside dagger check), plus dependency-bearing fixtures (client/app → client/dep).

Client output layout

A generated client is a scoped package:

  • dagger.gen.ts — core Dagger types (named to match the Go SDK's dagger.gen.go)
  • <module>.gen.ts — one per module: the bound module (e.g. hello.gen.ts) and each dependency; each carries its own types plus the prototype augmentations it contributes to Query/Client/Binding/Env.

Key design decisions

  • Always Remote / scoped package — the client depends on the npm @dagger.io/dagger; no TypeScript library bundle is vendored here.
  • Serve the one bound module — the client schema is core + the bound module only, so the generated serveBoundModule bootstrap serves exactly that module (no dependency loop, no includeDependencies). A LOCAL_SOURCE/DIR_SOURCE module is resolved against the workspace by its workspace-root-relative path (dag.currentWorkspace().moduleSource(path)) — cwd-independent, so it works anywhere in the project tree; a GIT_SOURCE module serves from its canonical ref + pin (dag.moduleSource(ref, { refPin })), which resolves from anywhere. A local-bound client shipped away from the project tree is unservable by design (remote-only) — see hack/designs/generated-client-module-loading.md.

Full rationale in design/client-gen.md.

Testing

  • Go unit + golden tests: go test ./... in each helper (also run inside dagger check via helperTestsCheck), including a golden test for the serveBoundModule bootstrap (local path vs. git ref + pin).
  • e2e checks green on the #13646 dev engine, except generateClientCompilesCheck, which additionally requires #13663: it type-checks the generated dag.currentWorkspace().moduleSource(...) against the engine's Workspace.moduleSource, so it only passes once the engine ships that field.

Known limitations / follow-ups

  • @dagger.io/dagger version on a dev/unreleased engine isn't on npm. generateClientCompilesCheck builds it from a pinned dagger/dagger commit; the client's package.json respects an existing local ./sdk ref (see design/client-bundle.md).
  • Deno client config exists in config-updator but isn't wired into clientDirectory yet.

@TomChv
TomChv force-pushed the feat/extract-client-codegen-in-typescript-sdk branch 2 times, most recently from 71b3bbb to 8d31ef1 Compare July 15, 2026 14:44
@TomChv

This comment was marked as outdated.

@TomChv

TomChv commented Jul 17, 2026 •

Copy link
Copy Markdown
Member Author
Screenshot 2026-07-17 at 16 47 55

Once dagger/dagger#13663 is merge, the typescript client codegen will work :D
I did a manual test and that works hehehe

@TomChv
TomChv force-pushed the feat/extract-client-codegen-in-typescript-sdk branch from 5f38891 to d9835de Compare July 20, 2026 10:34
@TomChv
TomChv force-pushed the feat/extract-client-codegen-in-typescript-sdk branch from d9835de to c310703 Compare July 21, 2026 13:13
@TomChv
TomChv marked this pull request as ready for review July 21, 2026 13:15
@TomChv
TomChv force-pushed the feat/extract-client-codegen-in-typescript-sdk branch 6 times, most recently from 1e08ca1 to 5bec371 Compare July 21, 2026 18:24
@grouville
grouville self-requested a review July 21, 2026 19:32
@TomChv
TomChv force-pushed the feat/extract-client-codegen-in-typescript-sdk branch from 5bec371 to eb8c037 Compare July 21, 2026 20:10
@TomChv
TomChv force-pushed the feat/extract-client-codegen-in-typescript-sdk branch from eb8c037 to b28ced2 Compare July 29, 2026 13:34
Client generation used to run inside dagger/dagger's engine runtime, which
opened a nested engine session and coupled client bindings to engine
internals. Moving it into this SDK module makes generation ordinary codegen in
a plain container: the engine hands over the client-facing schema and the
bound module's metadata as data, with no nested session.

A generated client now binds exactly one module and ships as a self-contained,
scoped npm package. At runtime it serves that single module through
Workspace.moduleSource, so the bindings resolve from any plain client session
rather than only from a module runtime.

Requires the engine client-codegen primitives released in v1.0.0-beta.7
(dagger/dagger#13646 and #13663).

Signed-off-by: Tom Chauveau <tom@dagger.io>
Signed-off-by: Vasek - Tom C <tom@dagger.io>
@TomChv
TomChv force-pushed the feat/extract-client-codegen-in-typescript-sdk branch from b28ced2 to b8c2bfe Compare July 29, 2026 13:35
@TomChv
TomChv merged commit 22e89be into main Jul 29, 2026
21 checks passed
TomChv added a commit that referenced this pull request Aug 24, 2026
Workspace.withNewDirectory replaces the directory it writes, where the
polyfill's fork.withDirectory merged onto it. Dropping the polyfill (#33)
flipped that semantic under two call sites, so both deleted files they do
not own.

initModule wiped the destination: `dagger module init typescript hello`
over a directory holding a foo.txt removed it, along with the module config
the engine writes there before calling the SDK. It now layers the rendered
starter onto existingDir(ws, modPath), as dagger/go-sdk#30 and
dagger/python-sdk#14 did.

generateClient / generateAllClient wiped hand-written files in the client
package — the bug #9 fixed with an overlay, lost when the polyfill went
away. Clients keep the user's files and still drop the *.gen.ts bindings of
a module that has left the closure: the SDK owns that set, the user owns
the rest. go-sdk and python-sdk keep plain replace for clients; this repo
does not, given #9.

Pruning has to happen twice, because withNewDirectory is not one operation.
On a local-directory workspace it replaces what it writes; on a synthetic
one — the shape a git-loaded workspace has, and how the Cloud checks runner
loads this repo — it merges. Isolated, same engine, same call:

  local removed:     main.ts, package.json, stale-dep.gen.ts
  synthetic removed: (nothing)

So the bindings come out of the baseline, covering replace, and off the
workspace with withoutFile, covering merge. The baseline still reads from
the untouched workspace: reading it back out of the pruned one comes up
empty on a local directory, which drops the user's files. Reported as
dagger/dagger#13955.

Three checks guard this, none of which pass without the fix:

  - init:init-over-existing-check inits over a fixture module and asserts
    removedPaths is empty. Before: dagger.json, index.ts, nested/.
  - client:generate-client-respects-existing-check gains main.ts (must
    survive) and stale-dep.gen.ts (must be pruned), so it fails under
    replace and under a plain overlay alike.
  - client:generate-client-on-synthetic-workspace-check covers the other
    half of the split, asserting the resulting tree rather than the
    removals, since removals are what diverge. It builds the workspace with
    ws.directory("/").asWorkspace, so the CI shape is reachable locally
    with no git pin.

A config-file fixture cannot catch any of it: config-updator merges those
files, so they survive a replace and the check passes anyway.

Verified with `dagger check 'e-2-e*'` (31/31) on a v1.0.0-beta.10 engine
and 60/60 in CI, plus an engine-driven `dagger module init typescript hello
-y` over a directory holding an unrelated file, which now keeps it.

Signed-off-by: Tom Chauveau <tom@dagger.io>
TomChv added a commit that referenced this pull request Aug 24, 2026
Workspace.withNewDirectory replaces the directory it writes, where the
polyfill's fork.withDirectory merged onto it. Dropping the polyfill (#33)
flipped that semantic under two call sites, so both deleted files they do
not own.

initModule wiped the destination: `dagger module init typescript hello`
over a directory holding a foo.txt removed it, along with the module config
the engine writes there before calling the SDK. It now layers the rendered
starter onto existingDir(ws, modPath), as dagger/go-sdk#30 and
dagger/python-sdk#14 did.

generateClient / generateAllClient wiped hand-written files in the client
package — the bug #9 fixed with an overlay, lost when the polyfill went
away. Clients keep the user's files and still drop the *.gen.ts bindings of
a module that has left the closure: the SDK owns that set, the user owns
the rest. go-sdk and python-sdk keep plain replace for clients; this repo
does not, given #9.

Pruning has to happen twice, because withNewDirectory is not one operation.
On a local-directory workspace it replaces what it writes; on a synthetic
one — the shape a git-loaded workspace has, and how the Cloud checks runner
loads this repo — it merges. Isolated, same engine, same call:

  local removed:     main.ts, package.json, stale-dep.gen.ts
  synthetic removed: (nothing)

So the bindings come out of the baseline, covering replace, and off the
workspace with withoutFile, covering merge. The baseline still reads from
the untouched workspace: reading it back out of the pruned one comes up
empty on a local directory, which drops the user's files. Reported as
dagger/dagger#13955.

Three checks guard this, none of which pass without the fix:

  - init:init-over-existing-check inits over a fixture module and asserts
    removedPaths is empty. Before: dagger.json, index.ts, nested/.
  - client:generate-client-respects-existing-check gains main.ts (must
    survive) and stale-dep.gen.ts (must be pruned), so it fails under
    replace and under a plain overlay alike.
  - client:generate-client-on-synthetic-workspace-check covers the other
    half of the split, asserting the resulting tree rather than the
    removals, since removals are what diverge. It builds the workspace with
    ws.directory("/").asWorkspace, so the CI shape is reachable locally
    with no git pin.

A config-file fixture cannot catch any of it: config-updator merges those
files, so they survive a replace and the check passes anyway.

Verified with `dagger check 'e-2-e*'` (31/31) on a v1.0.0-beta.10 engine
and 60/60 in CI, plus an engine-driven `dagger module init typescript hello
-y` over a directory holding an unrelated file, which now keeps it.

Signed-off-by: Tom Chauveau <tom@dagger.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants