feat(@typegpu/three): fromTSL() with textures and samplers - #2852
feat(@typegpu/three): fromTSL() with textures and samplers#2852iwoplaza wants to merge 2 commits into
Conversation
|
pkg.pr.new packages benchmark commit |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds support and documentation for importing Three.js textures into TypeGPU shaders as typed “handle” resources, enabling textureLoad / textureSample usage via fromTSL.
Changes:
- Extend
fromTSL/TSLAccessorto treat WGSL resource types (textures/samplers) as handle outputs when building TSL nodes. - Add Vitest coverage for texture load/sample using Three.js
DataTexture+ TSL sampler. - Add a new docs example and update ecosystem docs to demonstrate texture handle access.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/typegpu-three/src/typegpu-node.ts | Adds resource-output detection and routes accessor builds through handle outputs for textures/samplers. |
| packages/typegpu-three/tests/typegpu-node.test.ts | Adds mocks/helpers and tests ensuring generated fragment code/uniforms include texture access. |
| apps/typegpu-docs/src/examples/threejs/texture-access/meta.json | Registers a new interactive docs example. |
| apps/typegpu-docs/src/examples/threejs/texture-access/index.ts | Implements the interactive Three.js WebGPU texture access demo using fromTSL handles. |
| apps/typegpu-docs/src/examples/threejs/texture-access/index.html | Adds the example canvas container. |
| apps/typegpu-docs/src/content/docs/ecosystem/typegpu-three/index.mdx | Documents importing Three.js textures via fromTSL and using TypeGPU texture ops. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const builder = builderFor('generate'); | ||
| expect(() => node.build(builder)).not.toThrow(); | ||
| expect(builder.getCodes('fragment')).toContain('textureSample'); | ||
| expect(fragmentUniforms(builder)).toHaveLength(1); |
| function getResourceOutput(dataType: d.AnyWgslData): string | undefined { | ||
| if (dataType.type === 'sampler') { | ||
| return 'sampler'; | ||
| } | ||
| if (dataType.type === 'sampler_comparison') { | ||
| return 'samplerComparison'; | ||
| } | ||
| if (dataType.type === 'texture_external') { | ||
| return 'texture'; | ||
| } | ||
| if (dataType.type.startsWith('texture_storage')) { | ||
| return 'storageTexture'; | ||
| } | ||
| if (dataType.type.startsWith('texture_depth')) { | ||
| return 'depthTexture'; | ||
| } | ||
| if (dataType.type === 'texture_cube' || dataType.type === 'texture_cube_array') { | ||
| return 'cubeTexture'; | ||
| } | ||
| if (dataType.type === 'texture_3d') { | ||
| return 'texture3D'; | ||
| } | ||
| if (dataType.type.startsWith('texture_')) { | ||
| return 'texture'; | ||
| } | ||
| return undefined; | ||
| } |
| import * as TSL from 'three/tsl'; | ||
| import * as t3 from '@typegpu/three'; | ||
|
|
||
| const texture = new THREE.DataTexture(new Uint8Array(4), 1, 1); |
| pointerX = event.clientX / window.innerWidth - 0.5; | ||
| pointerY = event.clientY / window.innerHeight - 0.5; | ||
| }; | ||
| window.addEventListener('pointermove', onPointerMove); |
| void renderer.setAnimationLoop((time) => { | ||
| const seconds = time * 0.001; | ||
| knot.rotation.y = seconds * 0.16 + pointerX * 0.3; | ||
| knot.rotation.x = -0.18 + stdlibLerp(knot.rotation.x + 0.18, -pointerY * 0.2, 0.04); | ||
| halo.rotation.z = -seconds * 0.055; | ||
| renderer.render(scene, camera); | ||
| }); |
| knot.geometry.dispose(); | ||
| material.dispose(); | ||
| halo.geometry.dispose(); | ||
| halo.material.dispose(); |
Resolution Time Benchmark---
config:
themeVariables:
xyChart:
plotColorPalette: "#E63946, #3B82F6, #059669"
---
xychart
title "Random Branching (🔴 PR | 🔵 main | 🟢 release)"
x-axis "max depth" [1, 2, 3, 4, 5, 6, 7, 8]
y-axis "time (ms)"
line [1.05, 2.06, 4.39, 6.95, 7.65, 12.14, 23.91, 25.04]
line [0.98, 2.05, 4.38, 6.94, 8.14, 13.07, 26.46, 24.81]
line [1.03, 2.03, 4.43, 6.50, 7.66, 12.59, 25.99, 24.81]
---
config:
themeVariables:
xyChart:
plotColorPalette: "#E63946, #3B82F6, #059669"
---
xychart
title "Linear Recursion (🔴 PR | 🔵 main | 🟢 release)"
x-axis "max depth" [1, 2, 3, 4, 5, 6, 7, 8]
y-axis "time (ms)"
line [0.31, 0.57, 0.73, 0.84, 1.15, 1.19, 1.47, 1.64]
line [0.29, 0.49, 0.68, 0.81, 1.10, 1.14, 1.37, 1.56]
line [0.29, 0.52, 0.71, 0.88, 1.48, 1.15, 1.43, 1.60]
---
config:
themeVariables:
xyChart:
plotColorPalette: "#E63946, #3B82F6, #059669"
---
xychart
title "Full Tree (🔴 PR | 🔵 main | 🟢 release)"
x-axis "max depth" [1, 2, 3, 4, 5, 6, 7, 8]
y-axis "time (ms)"
line [0.91, 2.44, 4.51, 7.64, 13.48, 26.82, 57.18, 115.69]
line [0.89, 2.24, 4.36, 7.56, 12.73, 26.54, 57.56, 114.35]
line [0.86, 2.16, 3.86, 7.50, 12.65, 26.57, 56.24, 114.01]
|
Bundle size comparison (
|
| 🟢 Decreased | ➖ Unchanged | 🔴 Increased (max 0.18%) | ❔ Unknown |
|---|---|---|---|
| 0 | 311 | 13 | 0 |
import { ... } in PR vs import * as ... in PR (is the library tree-Shakeable?):
| Test | tsdown |
|---|---|
| tgpu_init.ts | 274.51 kB ( |
| tgpu_initFromDevice.ts | 273.97 kB ( |
| tgpu_resolve.ts | 174.80 kB ( |
| tgpu_resolveWithContext.ts | 174.74 kB ( |
| tgpu_bindGroupLayout.ts | 74.47 kB ( |
| tgpu_mutableAccessor.ts | 69.19 kB ( |
| tgpu_accessor.ts | 69.19 kB ( |
| tgpu_privateVar.ts | 67.88 kB ( |
| tgpu_workgroupVar.ts | 67.88 kB ( |
| tgpu_const.ts | 67.30 kB ( |
| tgpu_lazy.ts | 67.10 kB ( |
| tgpu_fragmentFn.ts | 39.65 kB ( |
| tgpu_fn.ts | 39.60 kB ( |
| tgpu_vertexFn.ts | 39.47 kB ( |
| tgpu_computeFn.ts | 39.17 kB ( |
| tgpu_vertexLayout.ts | 28.30 kB ( |
| tgpu_comptime.ts | 15.91 kB ( |
| tgpu_unroll.ts | 1.75 kB ( |
| tgpu_slot.ts | 1.70 kB ( |
If you wish to run a comparison for other, slower bundlers, run the 'Tree-shake test' from the GitHub Actions menu.
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
ℹ️ Minor suggestions only — the resource-handle path is sound, the mapping checks out against three.js internals, and the new tests are meaningful.
Reviewed changes
fromTSL(THREE.Texture, d.WgslTexture)now accepts a rawTHREE.Texture(auto-wrapped viaTSL.texture) and the node path builds resources directly, alongside agetResourceOutput()helper that maps WGSL data types to three.js reference outputs.TSLAccessortreats resource types (sampler,texture*,depthTexture,cubeTexture,storageTexture,texture3D) throughnode.build(builder, resourceOutput)+ ahandle-originrawCodeSnippet, bypassing the scalar bridge variable.- A new docs example +
index.mdxsection demonstratingtextureSample/textureLoadagainst a Three.jsDataTexture.
I verified the resource outputs against three@0.181.2: every value getResourceOutput() emits is a member of WGSLNodeBuilder.isReference, so building a TextureNode with that output returns the materialized handle instead of emitting a sample snippet. A single texture uniform in three's WGSL builder emits both the texture and its _sampler bindings, which explains why the sampler test's toHaveLength(1) is correct rather than accidental. The bridge-var path is unchanged for non-resource types (#resourceOutput is undefined), so existing storage-buffer/scalar accessors are unaffected. Tests and tsc both pass locally.
ℹ️ Nitpicks
- The storage path in
generatestill looks upgetBridgeVar(dep, builder)for every dependency (including resource handles, whose#varis nowundefined), so the loop runs a no-opcontinuefor them. Harmless, just a small structural wart — a comment or aif (!this.#resourceOutput)guard before thebridgeVarlookup would make intent clearer. - The new texture-access example defines its own
stdlibLerp;THREE.MathUtils.lerpalready exists if you want to drop the helper.
DeepSeek Flash (free via Pullfrog for OSS) ｜ 𝕏
|
|
||
| export const fromTSL = tgpu.comptime(((node, type) => { | ||
| const tgpuType = d.isData(type) ? type : (type as (length: number) => d.AnyWgslData)(0); | ||
| ) => TSLAccessor<T, TNode>) & |
There was a problem hiding this comment.
Minor: the convenience overload is typed T extends d.WgslTexture, which excludes WgslStorageTexture and WgslExternalTexture (separate interfaces in typegpu's texture schemas), even though getResourceOutput() explicitly maps texture_storage_* to storageTexture and texture_external to texture. So fromTSL(someTexture, d.textureStorage2d('rgba8unorm')) or fromTSL(texture, d.externalTexture()) won't type-check, and casting past the type wouldn't help either, because the runtime auto-wraps with TSL.texture, a sampling TextureNode, which is not a usable storage/external binding. Consider widening the overload to also accept WgslStorageTexture / WgslExternalTexture and routing those through TSL.storageTexture(...) / an external-texture node, or documenting that only sampled and depth textures are supported via the THREE.Texture overload and everything else must be passed as a TSL node.
There was a problem hiding this comment.
Important
The new WebGL fallback mapping for textureSample / textureLoad only handles the plain scalar-coordinate case and silently emits invalid GLSL for offset, array, cube, and depth forms — valid std calls fail to compile at runtime instead of producing a clear error. The GL fallback also lands as a TEMP: commit, so it reads as work-in-progress; worth deciding on coverage (guard unsupported forms or map them correctly) before this is merged.
Reviewed changes
- Routed
std.textureSample/std.textureLoadcodegen throughctx.gen.emitCall(...), opening the GLSL generator override hook while keeping the WGSL output byte-identical to the previousstitchtemplate (verified against generated WGSL). - Taught
GlslGenerator.emitCallto translatetextureLoad→texelFetchandtextureSample→texture, dropping the separate WGSL texture argument where GLSL fuses texture and sampler into a single uniform. - Extracted
TSLAccessor.buildAccessorNode, and in WebGL mode strips the trailing_samplerfrom sampler-accessor snippets so they reference the fused GLSL sampler uniform. - Added WebGL-backend coverage in
@typegpu/threeand@typegpu/glfor the combined-sampler load/sample cases.
I verified against three@0.181.2 that the WGSL and GLSL builders both emit nodeUniform0_sampler for the sampler snippet (so the _sampler strip is load-bearing), that WebGLBackend sets isWebGLBackend = true and pairs with GLSLNodeBuilder, and that the generated texture(...) / texelFetch(...) pass through three's GLSL node parser as literal GLSL. All tests (typegpu-three: 7, typegpu-gl glslGenerator: 25, typegpu texture: 40) pass.
⚠️ GLSL fallback only supports the plain 2D/3D/cube sampled and loaded forms
Beyond the tested happy path, the new textureSample / textureLoad translation emits uncompilable GLSL with no guard. Confirmed by direct generation: textureSample with the documented offset?: v2i argument becomes texture(nodeUniform0, vec2(0.5), ivec2(1, 0)) (GLSL texture()'s third parameter is a float bias), and an array textureLoad becomes texelFetch(nodeUniform0, uvec2(0), 0, 0) (GLSL texelFetch needs a fused ivec3 coordinate and LOD). Cube and depth sampling are broken the same way (cubemap load needs a fused ivec3; depth textures are sampler2DShadow, which texture(sampler, vec2) can't address). See the inline comment on GlslGenerator.emitCall for the details and suggested mappings.
ℹ️ Nitpicks
- The trailing
_samplerstrip (snippet.replace(/_sampler$/, '')) is a silent no-op if three ever stops suffixing sampler snippets with_sampler(or if the built snippet is a longer expression) — it degrades to emitting a wrong GLSL name rather than failing loudly. A comment asserting the invariant is already there, which helps, but a!endsWith('_sampler')guard that throws would make failure explicit. args.filter((arg) => arg !== undefined)in the newtextureSamplecodegen is dead code —dualImplpasses fully converted snippets, neverundefined. Harmless, just noise.- Commit title
TEMP: GL fallback— if the GL path is meant to ship, it deserves a real title and a coverage decision; if it's exploratory, it shouldn't be on the PR branch.
DeepSeek Flash (free via Pullfrog for OSS) ｜ 𝕏
9464302 to
50d3d3e
Compare
No description provided.