Improving BufferAttribute (maybe) #17089
Description
While working with GLTF loader a fair bit in the past few months I've hit a few issues that seem to be impossible to fix perfectly short of changing parts of the core library. This issue presents the issues and solicits feedback.
Ideally, GLTF loader should be able to create a WebGL buffer for each bufferView that is present in the file. The glTF spec is structured to make this possible through various limitations on alignment and colocated data in a single view. Given a good glTF processor like gltfpack you can then achieve a minimum number of WebGL buffers used for the entire scene - for example, one buffer for everything (this requires interleaving vertex data), or maybe one buffer per each unique stride (this is commonly how glTF files exported from Sketchfab seem to look). This is independent of the complexity of the scene.
Doing so carries a few benefits - it minimizes the number of buffer objects used to represent the scene, which improves memory consumption, minimizes the cost of switching between geometry buffers during rendering, and in theory unlocks some future accelerated rendering scenarios using multi draw (unfortunately, WEBGL_multi_draw specifically isn't optimal for this, but maybe this can be improved in the future). Additionally, the loading logic can be more straightforward and can become faster for files with lots of meshes.
There are two important features that are required to make this work (see #16802 for what triggered all of this, and GLTFLoader.js for the current somewhat sad workarounds):
-
We need to be able to reference the same WebGL buffer from multiple buffer attribute objects, at different byte offsets. This is currently impossible with BufferAttribute. This seems possible with InterleavedBufferAttribute but it doesn't truly work because all IBAs that refer to the same IB share the same .count property which breaks various assumptions in various parts of three.js.
-
We need to be able to reference the same WebGL buffer from multiple buffer attribute objects with different component types. This is current impossible with BA or IBA - in both cases the backing array on the JS side is a typed array, not an array buffer.
It seems to me that InterleavedBuffer is not pulling its weight. It doesn't solve either of the two problems above - effectively it only works if you want to interleave data of the same component type within a buffer for a single mesh, but doesn't allow mixing component types or packing multiple meshes into a single buffer. WebGL is perfectly capable of all of these, but Three.JS lacks a good interface to this.
Additionally, the fact that there are two different constructs with two different interfaces to represent, conceptually, the exact same thing - an array of typed components used for vertex processing - seems problematic. This results in some number of type checks for which BufferAttribute implementation it is, and not all code can handle both.
Now, if we agree that these problems should be solved, there's still an option of continuing to evolve InterleavedBufferAttribute to support these use cases - it probably involves extending InterleavedBuffer to optionally contain an ArrayBuffer instead of a TypedArray, adding a .count property to InterleavedBufferAttribute that can be set independently, and fixing code that directly works with InterleavedBuffer. If this was done, you'd be able to create InterleavedBuffer objects exclusively in glTF loader.
However, it seems like there's another way to fix this problem - instead of fixing InterleavedBuffer, we can improve BufferAttribute and deprecate InterleavedBuffer instead (it could still potentially exist as a thin wrapper over BufferAttribute to maintain backwards compatibility).
Here's how this could work.
Right now BufferAttribute is created from a typed array. We would add a new Buffer object that can be created from an ArrayBuffer, and a way to create BufferAttribute from a Buffer object with a byte offset and a component type (which would create a typed array for a slice of the buffer object). Existing constructor from a typed array would create a private Buffer object, so existing code would work as is.
BufferAttribute would be extended with a stride; to keep everything working nicely we will require that the stride is divisible by the component size so that it can be expressed in terms of the type of the typed array - this isn't a significant limitation in practice because the largest usable type is 32-bit float, and for various legacy reasons offsets need to be aligned by 4 bytes both in glTF spec and in general use.
Code that directly accesses BufferAttribute.array and isn't aware of the stride will need to be taught about the stride; code that works with BufferAttribute through getX/etc. accessors will continue to work.
After this is done, glTF loader can forget that interleaved buffers exist and create Buffer from each bufferView and BufferAttribute from each geometry accessor. (in glTF specifically there's one more source of inefficiency atm which is the morph target buffer duplication, which I think should be solved at the core level as well but that's a separate discussion) We can then separately deprecate InterleavedBuffer and either just remove it, or reroute IBA & IB methods to work through BA & B - they might just extend these classes and add the relevant functions if necessary.
As a result, three.js can gain a Buffer object that is roughly equal in power to WebGL buffers, glTF loader can stop creating thousands of tiny buffers for scenes with a large number of meshes, and (eventually) the whole "there are two different types of buffer attributes" issue can disappear.
Thoughts?