Skip to content

feature(rendering)!: add vertex attribute scheme#4634

Merged
jdrueckert merged 53 commits intodevelopfrom
feature/mesh-attribute-scheme
Jun 5, 2021
Merged

feature(rendering)!: add vertex attribute scheme#4634
jdrueckert merged 53 commits intodevelopfrom
feature/mesh-attribute-scheme

Conversation

@pollend
Copy link
Member

@pollend pollend commented Apr 24, 2021

Here is a generalized scheme that maps attributes to a given vertex resource. The attributes would be defined per hardware implementation such as vulkan or opengl. The attribute maps the value correctly into the buffer so hopefully this should generalize well but I could be wrong. I also just write the data straight to the buffer and then pass the contents of the buffer straight to opengl. Should be more efficient since the data is backed by the buffer rather then an array. arbitrary changes to the buffer should let us quickly update the vertex data without rebuilding a new buffer.

Ideally would like to store the vbo pointer with the resource so the buffered data is not duplicated per mesh instance. still not sure if this attribute scheme is the right way to go.

I reverted some of the work for skeletal mesh. this should shrink the scope of this work.
#4627

I would like to build up a scheme of these types of resources and they should be a thin shim for the underlying data. would like to make a variant for FBO, textures etc ...

@pollend
Copy link
Member Author

pollend commented Apr 24, 2021

Comment on lines 24 to 30
GL30.glBindBuffer(GL30.GL_ARRAY_BUFFER, state.vbo);
for (int x = 0; x < state.entries.length; x++) {
if (state.entries[x].version != state.entries[x].resource.getVersion()) {
GL30.glBufferSubData(GL30.GL_ARRAY_BUFFER, state.entries[x].offset, state.entries[x].resource.buffer);
}
}
GL30.glBindBuffer(GL30.GL_ARRAY_BUFFER, 0);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trying to find a mismatch with the marked vbo so the data is correctly updated during the drawing pass.

@pollend pollend marked this pull request as draft April 25, 2021 02:41
@pollend
Copy link
Member Author

pollend commented Apr 25, 2021

I'm just going to use this as reference. there are some problems that I was not able to address

@pollend pollend force-pushed the feature/mesh-attribute-scheme branch from eaab074 to ca10c64 Compare May 3, 2021 04:03
@pollend pollend marked this pull request as ready for review May 5, 2021 04:24
@pollend
Copy link
Member Author

pollend commented May 5, 2021

I though about this some more and I decoupled the resource and the binding itself. so the binding informs how the data is written to the buffer. the buffer should autosize to accommodate the binding which I think is a better scheme.

@lgtm-com
Copy link

lgtm-com bot commented May 5, 2021

This pull request introduces 4 alerts when merging 9e4835a into 0d39a3b - view on LGTM.com

new alerts:

  • 4 for Array index out of bounds

@pollend
Copy link
Member Author

pollend commented May 8, 2021

@pollend
Copy link
Member Author

pollend commented May 9, 2021

looks like DirectByteBuffers has close to no difference between a fixed array. so the binding is read and written directly to the bytebuffer and cuts the transfer that happens where we take an array/allocate a buffer and copy it over. there are some benefits to doing it this way. updating the vertex data is a simple direct memory transfer to opengl so there is no need to allocate and prefill a buffer when transferring data. I think we can get a significant performance boost if we setup something similar for chunk data.

I would like to move this to gestalt.

http://imranrashid.com/posts/profiling-bytebuffers/

Copy link
Member

@skaldarnar skaldarnar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments, but I have absolutely not idea about the code changes itself. I would feel better if anybody from the 3d wizardry department had at least a brief look whether the changes "go in the right direction" - maybe @vampcat has a bit of time?

In other news, I checked out this PR and started it in a local singleplayer game of Iota. Everything looks fine as far as I can tell.

  • OS Linux Mint 20 Cinnamon
  • Linux Kernel 5.4.0
  • CPU AMD Ryzen 7 3700X
  • Memory 32 GB
  • GPU GeForce GTX 1070 (450.119.03)

Comment on lines 56 to 61
// if (!data.normals.isEmpty() && data.normals.size() != data.getVertices().size()) {
// throw new IOException("The number of normals does not match the number of vertices.");
// }
// if (!data.uv0.isEmpty() && data.uv0.size() / 2 != data.getVertices().size() / 3) {
// throw new IOException("The number of tex coords does not match the number of vertices.");
// }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this commented out? If it is no longer needed, remove it, or explain (in the code) why it is commented out.

Comment on lines 9 to 11
/**
*
*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be obvious for someone who knows their way around graphics programming, but for me Javadoc would be really helpful.

If this is a very common pattern, maybe we can give cross links (to something like OpenGL specs or docs or whatever).

import org.lwjgl.BufferUtils;
import java.nio.ByteBuffer;

public class IndexResource {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here - what is an "index resource"?

*
* @param <TARGET> the target object
*/
public class VertexAttribute<TARGET> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, this reads funny. I guess I'm just not used to having full words 😱 as type parameters... I guess you'd typically just use T in Java. What is the reason you chose TARGET?

Comment on lines 56 to 82
// public SkeletalMeshDataBuilder addMesh(Bone bone, MeshBuilder builder) {
// return addMesh(bone, builder.getMeshData());
// }
//
// public SkeletalMeshDataBuilder addMesh(Bone bone, StandardMeshData data) {
// TFloatList meshVertices = data.getVertices();
// TIntList meshIndices = data.getIndices();
// TFloatList texCoord0 = data.uv0;
// int weightsStart = weights.size();
// addBone(bone);
// for (int i = 0; i < meshVertices.size() / 3; i++) {
// float x = meshVertices.get(i * 3);
// float y = meshVertices.get(i * 3 + 1);
// float z = meshVertices.get(i * 3 + 2);
// Vector3f pos = new Vector3f(x, y, z);
// BoneWeight weight = new BoneWeight(new float[]{1}, new int[]{bone.getIndex()});
// // TODO Copy mesh normals
// vertices.add(pos);
// weights.add(weight);
// uvs.add(new Vector2f(texCoord0.get(i * 2), texCoord0.get(i * 2 + 1)));
// }
//
// for (int i = 0; i < meshIndices.size(); i++) {
// indices.add(meshIndices.get(i) + weightsStart);
// }
// return this;
// }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this commented out? Is it no longer needed? If so, just remove it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the concept is you can build a static mesh and attach it to an existing bone :? kind of strange and niche and apparently unused so I dropped it. I want to refactor this to use a vertex shader to instead of constantly updating vertex data on the CPU.

private float uiScale = 1f;

private Matrix4fStack modelMatrixStack = new Matrix4fStack(1000);
private Matrix4fStack modelMatrixStack = new Matrix4fStack(10000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two magic numbers 👀 what is this change good for?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matrix4fStack is preallocated so the matrices are reused. we need to allocate something ahead of time.

Copy link
Contributor

@vampcat vampcat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a review, just a brief gloss-over of the top 20 files or so


@Override
public int getVertexCount() {
return 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this always returning 0?


public MeshBuilder addIndex(int index) {
meshData.getIndices().add(index);
meshData.indices.put(index);//.add(index);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the commented out part?

}

public void put(int value) {
ensureCapacity((posIndex + 1) * Integer.BYTES);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for this function to be called multiple times, say in a loop? If yes, then wouldn't it make more sense to not put it here/have an alternate overload that takes a list and only performs the check once for the entire list?

Copy link
Member Author

@pollend pollend May 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the check should be quick. the buffer is doubled each time you overflow its content. its dynamic in how the data is getting allocated. the point is so you don't have to manage how large the buffer will have to be. I have a reserve method that will allocate a buffer of that size to start with so the contents is not resized constantly.

*/
public class VertexAttribute<T, TImpl extends T> {


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The newlines in this file looks a bit wonky?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep fixed it :D

pollend added a commit to MovingBlocks/TeraNUI that referenced this pull request May 9, 2021
floating point assignment does not work correctly. additional tests
verify that this works correctly.

ref: MovingBlocks/Terasology#4634
@jdrueckert
Copy link
Member

With #4735 being merged into this, I left Terasology/CoreRendering#59 still open

@pollend pollend force-pushed the feature/mesh-attribute-scheme branch from af41af3 to 7628164 Compare June 1, 2021 02:16
@pollend pollend changed the title feature: add vertex attribute scheme feature(rendering)!: add vertex attribute scheme Jun 1, 2021
}

@Override
public int elements() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think elements is a good name for something that returns a number, but maybe this is the right name for this domain 🤷 I'd prefer something like elementCount to emphasize that this is a number, opposed to elements which, to me, suggests that this method returns the elements of this mesh (as in, vertices, edges, or whatever the "elements" of a mesh are).

Comment on lines 186 to 194
meshData.position.put(pos.set(-dimensions.x - offset, -dimensions.y - offset, -dimensions.z - offset)); // 0
meshData.position.put(pos.set(+dimensions.x + offset, -dimensions.y - offset, -dimensions.z - offset)); // 1
meshData.position.put(pos.set(+dimensions.x + offset, -dimensions.y - offset, +dimensions.z + offset)); // 2
meshData.position.put(pos.set(-dimensions.x - offset, -dimensions.y - offset, +dimensions.z + offset)); // 3

meshData.position.put(pos.set(-dimensions.x - offset, +dimensions.y + offset, -dimensions.z - offset)); // 4
meshData.position.put(pos.set(+dimensions.x + offset, +dimensions.y + offset, -dimensions.z - offset)); // 5
meshData.position.put(pos.set(+dimensions.x + offset, +dimensions.y + offset, +dimensions.z + offset)); // 6
meshData.position.put(pos.set(-dimensions.x - offset, +dimensions.y + offset, +dimensions.z + offset)); // 7
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this passes in the same vector object with changed values each time. with a quick search, it was hard for me to see whether we store this element directly and would override previous states, or whether we do a "copy-on-write" somewhere. It would help a lot if we could find a way to document this somewhere (here, on the put method of the VertexAttribute, I think).

Color color = new Color(Color.black);
for (int i = 0; i < 8; i++) {
meshData.color0.addAll(new float[]{0.0f, 0.0f, 0.0f, 1.0f});
meshData.color0.put(color);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to create a new instance of color here beforehand, and can't use

Suggested change
meshData.color0.put(color);
meshData.color0.put(Color.black);

here? Also, do we do a "copy-on-write" here? Otherwise, is it correct that we put the same color instance for all 8 indices?

import org.lwjgl.opengl.GL33;
import org.lwjgl.opengl.GL44;

public enum DrawingMode {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extracted from MeshData.java.

Comment on lines 28 to 29
VertexAttributeBinding<Vector3fc, Vector3f> vertices = this.vertices();
if (elements() == 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use (or omit) this consistently 😉

Comment on lines +56 to +61
int numIndices = 0;
int numVerts = 0;
for (int x = 0; x < rawIndices.size(); x++) {
numIndices += (rawIndices.get(x).length - 2) * 3;
numVerts += rawIndices.get(x).length;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How often is this code called/how performance critical is this? I personally would prefer using a more functional approach using the Streams API:

        final int numVerts = rawIndices.stream().map(x -> x.length).reduce(0, Integer::sum);
        final int numIndices = rawIndices.stream().map(x -> (x.length - 2) * 3).reduce(0, Integer::sum);

And instead of doing the computation for numIndices inside the loop we could do the math and only do it at the end, can't we? This would only do one stream iteration then.

final int numIndices = (numVerts - 2 * rawIndices.size()) * 3;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

umm not that much

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets move this to another PR. its non critical to how this is suppose to work.

skaldarnar
skaldarnar previously approved these changes Jun 3, 2021
Copy link
Member

@skaldarnar skaldarnar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested this together with Terasology/CoreRendering#59 and the games looks quite the same. I could not spot any difference in performance, either. Tested with Iota workspace.

I'll approve this, but @pollend please feel free to do small changes and ping me for a re-approve.

Still to be checked: does this work with the falling block animation?

Comment on lines +27 to +28
public final VertexResource positionBuffer;
public final VertexAttributeBinding<Vector3fc, Vector3f> position;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the relation here between the buffer and the binding?

I'm wondering why both are public. Isn't the binding somehow responsible for filling the buffer/keeping it in sync or something like that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Attribute is what is describing the format of the data
  • AttributeBinding is how that data is getting mapped for that attribute
  • VertexResource is just a basic buffer for how that data is getting stored. The bare minimum of describing this data i.e a stride a buffer and a size. The idea is for vulkan we would have a set of attributes describing vulkan data types and the binding themselves would stay consistent. have some kind of VertexFactory to build the data type for the end user to use but that is something to worry about later.

/**
* factory that is used to define an attribute binding to the backing {@link VertexResource}
*/
public class VertexResourceBuilder {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a bit confused by the name "builder" here, as I usually associate that with a fluent API where each modification method returns the builder again.

Here, we mutate internal state, but return a binding to something which looks like it's not including the added stuff (inStride is only modified afterwards). Is that intended?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a VertexResourceFactory?


import org.lwjgl.opengl.GL30;

public enum TypeMapping {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This an "OpenGL type mapping", right? should we make that more clear in the name or package?

And, as in many places, I'd appreciate some docstring, and if it is only reference to available OpenGL types or something like that.

Copy link
Member Author

@pollend pollend Jun 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to pack more types into this enum for different graphics API's have a unified enum for different mappings but maybe that is the wrong approach. I want to do a standard enum for different texture formats as well so we would use that instead of using the direct types.

/**
* defines the order of vertices to walk for rendering geometry
*
* refrence: https://www.khronos.org/opengl/wiki/Primitive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

return addMesh(bone, builder.getMeshData());
}

public SkeletalMeshDataBuilder addMesh(Bone bone, StandardMeshData data) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where did this go?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't use it in any capacity It was just a way to add meshes to bones. I guess I would want to redo this as a skinned mesh so we're not doing this all on the CPU.

Comment on lines 63 to 66
// ByteBuffer vertexBuffer = resource.buffer();
// vertexBuffer.rewind();
// vertexBuffer.limit(resource.inSize());
// GL30.glBufferSubData(GL30.GL_ARRAY_BUFFER, offset, vertexBuffer);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this commented?

Mesh result = Assets.generateAsset(urn, meshData, Mesh.class);
meshData = new StandardMeshData();
return result;
return Assets.generateAsset(urn, buildMeshData(), Mesh.class);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pollend @DarkWeird is this resolved?

…/shadowMap_frag.glsl

Co-authored-by: Tobias Nett <skaldarnar@googlemail.com>
@pollend
Copy link
Member Author

pollend commented Jun 4, 2021

Tested this together with Terasology/CoreRendering#59 and the games looks quite the same. I could not spot any difference in performance, either. Tested with Iota workspace.

I'll approve this, but @pollend please feel free to do small changes and ping me for a re-approve.

Still to be checked: does this work with the falling block animation?

kind of hard to tell with other larger bottle necks in the pipeline. The pipeline can only run as fast as its slowest stage :/.

@jdrueckert jdrueckert merged commit f51c06f into develop Jun 5, 2021
@jdrueckert jdrueckert deleted the feature/mesh-attribute-scheme branch June 5, 2021 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Breaking Change API breaking change requiring follow-up work in dependant areas Topic: Architecture Requests, Issues and Changes related to software architecture, programming patterns, etc. Type: Improvement Request for or addition/enhancement of a feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants