Follow-ups for Gaussian Splat SPZ support

[javagl]

I mentioned that in a comment in the Gaussian Splat SPZ Support PR, but it may be worth tracking this here:

---

There seem to be some assumptions about the structure of the glTF of which I'm not sure whether they will always hold:

cesium/packages/engine/Source/Scene/GaussianSplat3DTileContent.js

Line 381 in 8a362a0

this.splatPrimitive = loader.components.scene.nodes[0].primitives[0];

There is not necessarily a matrix property in that node. The first primitive may not be a Gaussian Splat primitive (or the Gaussian Splat primitive may not be the first one). There may also be multiple Gaussian Splat primitives in one glTF.

---

EDIT/Update: The following was addressed via #12706 (additional test cases may have to be considered):

Modifying the tileset modelMatrix does not seem to have an effect on the gaussian splat primitive. (Or at least not the "right" effect: It seems to affect the position and bounding volumes, but not the orientation).

---

Speaking of orientations: People might say that I'm obsessed with unit cubes. And I wouldn't even concur 😁 : I created a "Unit Cube Splat File". (It's rather a 10x10x10 cube, but unit-y in other ways). And when rendering this in BabylonJS, it looks like this:

Exactly as expected. With the colors and such. 🔴 🟢 🔵

When putting this into a glTF and a tileset and rendering it with CesiumJS, it looks like this:

There may be a million reasons why something is "wrong". For example, the bounding box. This is coming from the tileset JSON, and may therefore be "my fault". The splat is a 10x10x10 cube, and one might expect that the bounding volume has to be
[5.0, 5.0, 5.0, 5.0, 0.0, 0.0, 0.0, 5.0, 0.0, 0.0, 0.0, 5.0]
But that seems to be wrong. The one from the screenshot takes the RUB->LUF coordinates of the SPZ-to-glTF into account, and is
[-5.0, -5.0, 5.0, 5.0, 0.0, 0.0, 0.0, 5.0, 0.0, 0.0, 0.0, 5.0]
but that is also wrong. I've been twiddling with a few axis conversions, and couldn't get it "right" (i.e. matching the splat primitive). But I didn't spend too much time with that, because it's not clear whether the bounding box is wrong, or the rendering/placement of the splat primitive.

What's probably more important:

The orientation of the individual splats seems to be wrong! In the screenshot, the mouse cursor is hovering over the orange splat (orange ... between red and yellow - there's that unit cube again 🙂 ). And this seems to be oriented horizontally. But it should be oriented vertically (because it is that edge of the unit cube). In BabylonJS, this seems to be right.

---

I also noticed that the scaling factor from the tileset model matrix is not applied to the splat when the splat content is in the root tile, but it is applied when the splat content is in a child tile, but that may just be an artifact of the whole ~"matrices" point.

---

Here is an archive with that "Unit Cube Tileset", including the SPZ, GLB, tileset JSON, and Sandcastles for testing (including one with basic tileset modelMatrix editing functionality)

unitCubeTileset-2025-06-21.zip

[keyboardspecialist]

When we tile, we tell SPZ to not do any coordinate conversions, then we do a rotation correction during the glTF load here

cesium/packages/engine/Source/Scene/PrimitiveLoadPlan.js

Line 263 in cc8c6e0

const rots = rotations.typedArray;

Might try commenting that out and see if your splat orientations render as expected.

[javagl]

I'd need much more time to understand what's going on there. (Sure, it will eventually boil down to something simple, but for now, it is just swizzling some quaternion components, and I'd have to check which axis conversion that is and should be). I've seen some code for coordinate conversions in SPZ. (Yes, I see. I'm a nerd. It can imagine the satisfaction of having figured out this oh-so-clever code the generically do the coordinate conversions. But ... *sigh*). It at least looks like axis conversions might be more complicated than that. I'll probably try it out with this conversion commented out (a bit later today)

[javagl]

I had another short look at this. And I created a test data set to dedicatedly test the rotations:

^{(Here, rendered in BabylonJS)}

It contains, from bottom to top (and left to right)

• red splats, scaled along the x-direction (rotated about [0°, 30°, 60°, 90°] around the y-axis)
• green splats, scaled along the y-direction (rotated about [0°, 30°, 60°, 90°] around the z-axis)
• blue splats, scaled along the z-direction (rotated about [0°, 30°, 60°, 90°] around the x-axis)

Here is a GIF, showing the rendering in CesiumJS, once with applying the rotation in the primitive load plan (mentioned above) and once with not applying it:

Both of them look.... at least different from what BabylonJS is showing (regardless of what's "right" or "wrong" here...).

(My concern is that people might now start generating data that is rendered properly by CesiumJS, but that turns out to be "wrong" in another renderer - I think that this should be carefully reviewed...)

EDIT: Nearly forgot the test data (including tileset and sandcastle):

Cesium Splat Rotation 2025-07-01.zip

[javagl]

I'm not sure if spherical harmonics with degree > 0 are supposed to be supported in the current state. But here's some test data:

Cesium Splats SH 2025-07-02.zip

(With PLY, SPZ, GLB, tileset, and sandcastle)

Rendered in BabylonJS:

And in Cesium:

[javagl]

The issue about the spherical harmonics might be caused by https://github.com/drumath2237/spz-loader/blob/4032056c1d2628ae9cf89dc0c2233cc15381e8ee/packages/core/lib/spz-wasm/gaussianCloud.ts#L48 - to be investigated.

[javagl]

(EDITED:)

From my understanding,

• The spz-loader library calls rotate180DegAboutX on the gaussian cloud
• The GltfSpzLoader does not account for that
• The PrimitiveLoadPlan tries to convert this back by negating the y- and z-coordinates and modifying the rotations

I think that there are several possible inconsistencies here that should be investigated. Is the loader library even supposed to apply this rotation?

Related to the first point of nianticlabs/spz#42

[javagl]

When the same glb content (with SPZ) is referred to by two tiles with different transform, then the positions of the splats are all messed up. Apparently, there is some deduplication happening, so it loads the splats only once, but applies wrong/duplicate transforms to them. This even applies across tilesets.

Test data:

Cesium Splats Tile Translation.zip

[javagl]

It is currently not possible to combine splat- and non-splat data in a single tileset. The check for hasGaussianSplatExtension is wrong. Just because the tileset requires this extension does not mean that each glTF is a splat glTF.