Manually generated Normals now missing data (4.2 Beta1 regression)

[Bimbam360]

Godot version

v4.2.beta1.official [b137180]

System information

Godot v4.2.beta1 - Windows 10.0.22000 - Vulkan (Forward+)

Issue description

I wrote a bootleg subdivided plane mesh generation script in 4.2 dev 6 that pushes a generated ArrayMesh into ImporterMesh to return the decimated version. As the LODed version had some seriously stretched flat normals, notably on flat areas (my landscape is tiered), I manually calculated averaged normals and handled a special case for upwards facing normals.

This worked fine in 4.2 Dev 6:

Identical code run on 4.2 Beta 1 causes this:

Edit: Further note, normal maps 'appear' fine until they are enabled in a shader (see below)

Steps to reproduce

Run Main.tscn
Hit "Gen" = On in the inspector (might want to reduce Thread count depending on your machine)
Witness:

Deselect "Normal Map" on Mat in the inspector and Witness again:

Minimal reproduction project

MRP.zip

[clayjohn]

Could you upload an MRP so that others can reproduce and attempt to fix this?

[Bimbam360]

Could you upload an MRP so that others can reproduce and attempt to fix this?

I have attached an MRP and updated the original post with some extra context.

[clayjohn]

I've taken a look at this locally and figured out the issue.

Some changes we made for beta1 exposed a bug in your code. The underlying issue is that normal maps require both normals and tangents (https://docs.godotengine.org/en/latest/classes/class_basematerial3d.html#class-basematerial3d-property-normal-texture):

Note: The mesh must have both normals and tangents defined in its vertex data. Otherwise, the normal map won't render correctly and will only appear to darken the whole surface. If creating geometry with SurfaceTool, you can use SurfaceTool.generate_normals and SurfaceTool.generate_tangents to automatically generate normals and tangents respectively.

By chance, your mesh didn't totally break despite missing tangents. I'm a little surprised actually.

The solution for your project is to ensure you call st.generate_tangents() before committing your mesh. I've confirmed that this works in beta1.

I'm going to continue digging to see if I can restore the old behaviour (despite it hiding a bug in your code) as I'm sure other users will run into the same issue.

[clayjohn]

Okay, debugging a bit further. I have identified the exact cause of the change in behaviour and there is no simple fix. The problem is that we used to provide a default tangent buffer when the user used Tangents, but didn't supply them. The tangent buffer obviously used totally incorrect values, but they worked fine for your project. Now, we can't provide that tangent buffer anymore as we compress the tangents and the normals together. So when you try to access tangents without supplying them in the mesh, you get garbage values.

I think it should be possible to at least warn users when this happens. I'll keep thinking to see if I can find a more helpful fix though

[clayjohn]

Hmmm, I had this stuck on my brain so I made a proof of concept 0e44f0f

It appears to work. I will return to this later

[Bimbam360]

Wow yeah just reread and confirmed the surface tool tutorial literally has a section about Normal maps requiring tangents. No idea how I've managed to miss this before!

Thanks for providing the fix and sorry for wasting your time >.<

[galileolajara]

Hello! I'm using ArrayMesh, not SurfaceTool class and this issue also occurs to me. I'm newbie here in godot, I'm using the master branch, compiled it from source and the ArrayMesh still produces a different-looking mesh compared to the 4.1.2-stable where the normals doesn't seem to properly take effect in the master branch (mesh looks dark).

I don't use Mesh::ARRAY_TANGENT because I do not use normal maps at all. Any would be appreciated!

[clayjohn]

@galileolajara If you are not using normal maps or using TANGENT in your shader, then you are running into a different bug. This issue only triggers from using normal maps.

There have been a few mesh normal regressions fixed in the last couple of weeks (the most recent were only merged yesterday). What version of the engine have you built from?

[galileolajara]

Glad to receive some help! I'm using ArrayMesh, without Mesh::ARRAY_TANGENT, and a simple gdshader that only uses normals in the fragment shader to produce plain vec3 color, and don't use any tangent (in fact, I don't know how to use TANGETNS).

After some trial and error, I'm able to work around the bug in ArrayMesh by specifying an array in Mesh::ARRAY_TANGENT:

.... some codes ....
PackedFloat32Array emptyTangents; // Float32 is a must, other types such as Vec3 causes runtime errors!
emptyTangents.resize(numVertices * 4);
arrays[Mesh::ARRAY_TANGENT] = emptyTangents;
..... some codes .....
arraymesh->add_surface_from_arrays(Mesh::PRIMITIVE_TRIANGLES, arrays);

This workaround results to a mesh that looks exactly like the ones in 4.1.2-stable. Let me know what info you need so I can contribute in figuring this bug out.

[clayjohn]

This workaround results to a mesh that looks exactly like the ones in 4.1.2-stable. Let me know what info you need so I can contribute in figuring this bug out.

Could you let me know what version of the engine you are using? Like I said, this bug might have been fixed already

Also, could you let me know what the normals are in the locations that are turning black?

[galileolajara]

Sorry, I haven't answered that question. I use the "git clone " and compile with scons. I'm at the master branch. The last time I did "git pull" was just about an hour or two ago. The Godot's about window shows "v4.2.beta.custom_build [dd74ffd]". The bug is still there unless I do the emptyTangets workaround.

Also, could you let me know what the normals are in the locations that are turning black?

I'm not a native english speaker so I'm not sure if I understood this correctly. If I dont use the emptyTangents workaround, the resulting mesh seemed that everything in that mesh is black/dark except the last vertex. That one last vertext is not dark and seem correct in its shading.

[clayjohn]

I'm not a native english speaker so I'm not sure if I understood this correctly. If I dont use the emptyTangents workaround, the resulting mesh seemed that everything in that mesh is black/dark except the last vertex. That one last vertext is not dark and seem correct in its shading.

My question is, what values are in the arrays[Mesh::ARRAY_NORMAL] array? Ideally you could share with me the project that exhibits the issue or at least the contents of that array.

[galileolajara]

I just did a git pull today, and noticed that there are some fixes about mesh in the git log. Recompiled Godot and removed the emptyTangents workaround. Now it worked perfectly just like on the 4.1.2-stable version! Thanks @clayjohn

[clayjohn]

I've submitted a tentative fix for this. It doesn't totally restore the old behaviour because the old behaviour was kind of problematic for a few reasons. But it adds a warning for users in this situation, so you get some feedback on the fact that you have done something wrong in your script and are pointed at how to fix it