Cross-section rendering

[HenryWConklin]

For #5

Adds a renderer that slices a 4D tetrahedron mesh on the GPU in the vertex shader. Works by packing data wherever it fits on Godot's rendering server and reinterpreting it in the shader.

In a decent state but there are a couple issues I'm aware of:

• 3D editor preview is broken, the preview camera's transform isn't passed to the shader so 4D meshes stay fixed in place and don't move with the camera view, probably means adding support for 3D cameras
• UVWs look off, can't tell if they're wrong on the 4D mesh or getting mangled on the way to the shader. Haven't had the brainpower to delve into the fourth dimension and debug that.
• Missing some kind of support for wireframe meshes, should be possible but not quite sure what that looks like. Should be able to re-use the same shader with a mesh using line primitives.
• Missing the "shadow" effect to give some indication of the rest of the mesh, should be doable with another shader on next_pass for the first shader, may need another surface on the mesh as well

[HenryWConklin]

movie_render.mp4

[HenryWConklin]

Man first this formatter says too few newlines at end of file, then it says too many. Needs to make up its mind 😞

[aaronfranke]

UVWs look off, can't tell if they're wrong on the 4D mesh or getting mangled on the way to the shader.

The current UVW maps for the box and orthoplex were tediously hand-unwrapped by me, because I wanted to have some kind of test mesh but didn't have a general algorithm for unwrapping them. It's very possible I made a mistake. But I would be surprised if the whole thing is wrong.

[HenryWConklin]

Ok, I'll take a closer look some time soon. Normals were also messed up in a similar way when I tried passing them in, so could be one of the fields I'm stashing data in is getting messed with and breaking things.

[HenryWConklin]

Did some more digging and found out that the normal is getting normalized somewhere in the pipeline but I couldn't figure out where. Also found some weird processing happening where data is getting clamped into [0,1]: https://github.com/godotengine/godot/blob/45fc515ae3574e9c1f9deacaa6960dec68a7d38b/servers/rendering_server.cpp#L712C1-L717C8

Looks a bit better with a fix for the normalization but there are still some seams in the middle of faces which seems off. They're at least continuous now.

Before:

4d_pre_normalization_fix.mp4

After:

4d_normalization_fix.mp4

Thinking I need to take a different approach. This is definitely hacky and there isn't really enough space to fit all the data I need. What do you think about switching to using ImmediateMesh and doing the cross-section on the CPU? Could maybe do a compute/geometry shader which would only work on the Forward+ renderer as far as I know, but I can't find any examples.

[aaronfranke]

@HenryWConklin I think it's most performant to do it on the GPU, but any approach you can get working is better than wireframe-only, so a CPU-based solution would be welcome.

Also, about the visual of the box, did you test with both 40-cell and 48-cell BoxTetraDecomp settings?

[HenryWConklin]

Figured it out, I was using the wrong UVW coords. Turns out they're not indexed like the positions so I was reading the wrong UVW coords out of the array. The other samples were all 40-cell, 40 and 48 cell cubes look identical now.

4d_fix_40cell.mp4

4d_orthoplex.mp4

Still no smooth normals support, but I added support for Texture3D:

4d_orthoplex_textured.mp4

[aaronfranke]

@HenryWConklin Ah, that makes sense. And yes, they are duplicated to allow for different tets on the same vertices to have different teture coordinates at those vertices. This is vital even for simple shapes.

[aaronfranke]

Thanks for your patience with this! It's a super cool feature for sure.

One important aspect of this module is that I am keeping it able to be compiled as both a module and a GDExtension. As usual when any work happens on this module, this PR has revealed problems with a mismatched API between the engine and GDExtension, so I did a few detours to fix a few of those problems: godotengine/godot#107139, godotengine/godot#107141, godotengine/godot-cpp#1789.

The new render/cross_section/ folder isn't referenced in the GDExtension SConstruct, so it's currently not compiled at all for GDExtension. It needs to be added:

I also added the header builders script from godotengine/godot-cpp#1789 in commit 6e5ae5d (be sure to rebase your PR!) which should allow for env.GLSL_HEADER("../../render/cross_section/cross_section_shader.glsl") in the GDExtension SConstruct. However, I'm getting an error TypeError: 'tuple' object is not callable:, which is strange because I swear the same build system code was working earlier when I was testing on top of your PR before I pushed that commit. I don't know what's different between then and now. Maybe some fresh eyes will help. Either way, ideally, we want this header to be able to generate with the GDExtension build.

[HenryWConklin]

I also added the header builders script from godotengine/godot-cpp#1789 in commit 6e5ae5d (be sure to rebase your PR!) which should allow for env.GLSL_HEADER("../../render/cross_section/cross_section_shader.glsl") in the GDExtension SConstruct. However, I'm getting an error TypeError: 'tuple' object is not callable:, which is strange because I swear the same build system code was working earlier when I was testing on top of your PR before I pushed that commit. I don't know what's different between then and now. Maybe some fresh eyes will help. Either way, ideally, we want this header to be able to generate with the GDExtension build.

Got the GLSL header working, seems like it was just a stray comma in the definition for GLSL_HEADER making a 1-element tuple.

GDExtension complies now with all the source files included. I haven't been able to get it to run to test it out though. I run into some odd errors from the method registration that cause crashes on loading the project. The full editor/module build still works though.

This bit in G4MFItem4D seems like it's shadowing a property from Resource which causes some errors saying the set_name and get_name methods aren't defined, then throwing errors if you do re-bind them:

godot-4d/model/g4mf/structures/g4mf_item_4d.cpp

Line 169 in f46cbf3

ADD_PROPERTY(PropertyInfo(Variant::STRING_NAME, "name"), "set_name", "get_name");

And I get some kind of segfault from a class in physics, seems like it's interpreting something as a pointer that is not a pointer but I didn't dig too far.

[HenryWConklin]

Moved everything to a separate world3d which fixes the weird rendering in the 3D preview, should be more robust overall.

I am getting an error message now on the first frame when running a scene, seems like the 4D rendering server gets called in pre_render_frame and some things are still null in the rendering server. Doesn't seem to cause any problems. Maybe the render_frame method could be moved to _process? Would need to make sure it runs after everything else but might be a better place for it.

[aaronfranke]

@HenryWConklin That might cause things to be a frame late in some cases. I think it's better to check for null things in RenderingServer and silently skip rendering that frame if null.

[HenryWConklin]

Yeah, makes sense. I think there's also a DRAW notification on some types of nodes but seems like that would be a lot of refactoring.

Tried some things and found a work around doing some manual RID stuff.

Actual error happens here: https://github.com/godotengine/godot/blob/e1b4101e3460dd9c6ba0b7f8d88e9751b8383f5b/servers/rendering/renderer_scene_cull.cpp#L488

Coming from here: https://github.com/godotengine/godot/blob/e1b4101e3460dd9c6ba0b7f8d88e9751b8383f5b/servers/rendering/renderer_viewport.cpp#L1236

It's trying to clean up the old world on the viewport but it's somewhere in the middle of being created where the RID is valid but hasn't been initialized or something.

[aaronfranke]

Sorry for being a bit slow to review this. I've made some changes on master which will help with this.

In general I'm happy to merge this sooner rather than later, to get something in which we can build on top of further, even if it's missing features like 4D shadows, lighting, editor gizmos, etc, those could always be added later.

Please run doctool: ./bin/godot.something.exe --doctool . (replace godot.something.exe with your executable) and fill out the descriptions in the generated documentation XML.

[HenryWConklin]

No worries on being slow, I've been meaning to add more to this and haven't been. Have most of the change to add support for wireframes to the cross-section renderer, but I can send that as another PR. Need to do some research for the shadow effect so might be a while but I'm interested in working on it.

Should be in a good state to merge, let me know if you want any other changes!

[aaronfranke]

That's actually not what the Texture4D color source is meant to be used for. Texture4D is when you have a 4D texture which directly maps 4D coordinates to a 4D texture without using the cell UVW map. The TETRA_COLOR_SOURCE_CELL_UVW_ONLY and TETRA_COLOR_SOURCE_CELL_UVW_AND_SINGLE options are for using a cell UVW map to map cells onto a 3D texture. Maybe it could be renamed for clarity.

[HenryWConklin]

Ah ok, fixed. Is texture 4D a thing? Not sure I've seen that as a supported type anywhere. Might need to be a shader or a sprite sheet type thing with some manual sampling.

[aaronfranke]

It's not a thing in Godot yet, but can be done with shaders. See https://www.youtube.com/watch?v=Ir8oft_qAMQ&t=3m35s

[aaronfranke]

Awesome work, thank you!

[aaronfranke]

The TetraColorSource enum is intended for safe user-friendly setting, but when reading, there are bitwise flags available to simplify this. That code can be replaced with this:

	const ColorSourceFlags flags = get_albedo_source_flags();
	const Color albedo = (flags & Material4D::COLOR_SOURCE_FLAG_SINGLE_COLOR) ? _albedo_color : Color(1.0, 1.0, 1.0);
	// Setting to a Nil variant resets to the default texture, which is white.
	const Variant texture = (flags & Material4D::COLOR_SOURCE_FLAG_CELL_UVW) ? Variant(_texture) : Variant();

I've pushed this in commit f2b29bd