Support for quads

[pathquester]

Would it be within the scope of the project to also support quads for meshes? That would add tremendous utility.

[lucidrains]

@pathquester hey! yea, maybe i can slot that into my limited time left

what are quads? is it just building meshes out of rectangles?

[lucidrains]

as you can tell, i know nothing about the domain 🤣

[pathquester]

@lucidrains Hi, awesome!! Yeah instead of triangular meshes, quad meshes make modeling a joy since you can add edge loops easily and subdivide further, among other benefits. If it remains limited to triangular meshes, that means we'll very often have to retopologize the mesh generated afterwards.

[lucidrains]

@pathquester if it just means adding an extra vertex to each face (4 vertices instead of 3), that would be easy. do people ever mix quads and triangles in the same mesh? that may complicate things

[lucidrains]

@pathquester wait, i'm geometrically challenged, but isn't it true that every quad can be broken into two triangles?

[lucidrains]

@pathquester let me read up on the subject first to get a surface level understanding. will get back to this maybe near month's end, once i wrap up a project in flight

[pathquester]

@lucidrains They can be mixed yes although quads-only is arguably better than mixed, supporting just that should be ideal. That is fantastic, I would be very happy to help with this. Yes that's true also more quads thus increase the resolution, subdivision surface modifiers are mostly associated with quads.

[lucidrains]

@pathquester maybe the issue is that when you have the attention net spit out the vertices, with 4 it may be difficult for it to maintain that they are within the same plane. yeah i'm not sure if it will work

[lucidrains]

anyways, i know nothing lol

i'll get back to you on this

[pathquester]

@lucidrains Ah yeah there may be challenges. Anyhow, many thanks!!

[MarcusLoppe]

@lucidrains

I brought this up in previous conversations with you, I did some testing and it reduces the face/triangles count by about 50%.
So this would also increase the performance and hardware requirements, and who knows maybe the quads are easier for the model to learn.
Most of the 3D work seems to be quad based and all the 3d meshes can be converted to quads using blender so it would be a extra step to convert the meshes.
I think it would be worth the hassle since it reduces the triangle count to 50% but adds additional vertex per face, but the transformer would save 6 tokens per triangle, e.g from 19 200 tokens to 9 600 tokens.
I don't think you can improve the transformer that much since this would also half the required context length.

Below is example of a 3200 triangle mesh convert into 1620 quads
Triangles:

Quads:

Here is another example,
Face edges reduced from 5768 to 4336
Face count reduced from 3688 to 2256

Triangles

Quads

[lucidrains]

@MarcusLoppe oh yes, now I recall

ok, I think I should be able to do this before mid March

[MarcusLoppe]

@MarcusLoppe oh yes, now I recall

ok, I think I should be able to do this before mid March

@lucidrains
Is this something I can implement? Wouldn't this just require changing the einops expressions, 3's to 4 and the face edge generator?

[lucidrains]

yeah! and also when decoding from attention net, have to terminate on multiple of 4s