vec3 initialization from 'some' numpy arrays takes wrong memory

[jimy-byerley]

Hello dear @Zuzu-Typ ! That's me again, with a new buffer protocol case

This seems to be a quite subtle bug, I cannot point exactly what is wrong between 3 modules: pyglm, numpy, plyfile
This is what I'm trying to do:

I'm loading a .ply mesh file, using plyfile. This is giving me an array of triangular faces, returned as a numpy array of objects, whose objects are numpy arrays. (Yeah: weird).
Despite the good looking of those array faces, it seems there is something in it that does not please uvec3 constructor in these loaded arrays.
Issue: When I'm trying to convert that array into a uvec3: The third component of the created vector is always replaced by the third component of the last created vector.

This is the code I use

import numpy as np
from glm import dvec3, uvec3
from plyfile import PlyData, PlyElement

def ply_read(file, **opts):
	mesh_faces = []
	
	data = PlyData.read(file)
	index = {}
	for i,e in enumerate(data.elements):
		index[e.name] = i
	
	# not working with the imported array of arrays
	faces = data.elements[index['face']].data['vertex_indices']
	
	# working with a manually created array of arrays 
	# (it should be the same as the loaded arrays, I cannot spot a difference on shape/strides of them)
	#faces = np.empty(8, object)
	#faces[:] = [
		#np.array([0, 1, 5], dtype=np.uint32), np.array([0, 5, 4], dtype=np.uint32),
		#np.array([1, 2, 6], dtype=np.uint32), np.array([1, 6, 5], dtype=np.uint32),
		#np.array([2, 3, 7], dtype=np.uint32), np.array([2, 7, 6], dtype=np.uint32),
		#np.array([3, 0, 4], dtype=np.uint32), np.array([3, 4, 7], dtype=np.uint32)]
	
	# this shows the bug
	# BUG: the last elements of uvec3 faces created this way will be put to a constant despite its value in the loaded face
	a = uvec3(np.array([3,0,4], dtype=np.uint32))
	b = uvec3(np.array([3,0,10], dtype=np.uint32))
	print('this is working with fresh array:', a, b)
	print('but the magic happens using the last created element from fresh array (value 10)\n')
	
	
	print('will convert from', repr(faces))
	print(repr(faces[0]))
	print('characteristics', faces[0].shape, faces[0].strides, faces[0].data.format, faces[0].data.itemsize)
	
	print('an element apart from the array does not work:', uvec3(faces[0]))
	print('not any element from the array work:')
	for face in faces:
		assert len(face) == 3   # just for this test
		print('    ', repr(face), repr(uvec3(face)), repr(uvec3(*face)))
		mesh_faces.append(face)
	print('however after all this, creating from fresh array works:', uvec3(np.array([3,0,11], dtype=np.uint32)))

	return mesh_faces
	
ply_read('test_io.ply')

Together with test_io.ply

• numpy 1.21.4
• plyfile 0.7.4
• pyglm 2.5.4

I get the following output

this is working with fresh array: uvec3(            3,            0,            4 ) uvec3(            3,            0,           10 )
but the magic happens using the last created element from fresh array (value 10)

will convert from array([array([0, 1, 5], dtype=uint32), array([0, 5, 4], dtype=uint32),
       array([1, 2, 6], dtype=uint32), array([1, 6, 5], dtype=uint32),
       array([2, 3, 7], dtype=uint32), array([2, 7, 6], dtype=uint32),
       array([3, 0, 4], dtype=uint32), array([3, 4, 7], dtype=uint32)],
      dtype=object)
array([0, 1, 5], dtype=uint32)
characteristics (3,) (4,) I 4
an element apart from the array does not work: uvec3(            0,            1,           10 )
not any element from the array work:
     array([0, 1, 5], dtype=uint32) uvec3( 0, 1, 10 ) uvec3( 0, 1, 5 )
     array([0, 5, 4], dtype=uint32) uvec3( 0, 5, 10 ) uvec3( 0, 5, 4 )
     array([1, 2, 6], dtype=uint32) uvec3( 1, 2, 10 ) uvec3( 1, 2, 6 )
     array([1, 6, 5], dtype=uint32) uvec3( 1, 6, 10 ) uvec3( 1, 6, 5 )
     array([2, 3, 7], dtype=uint32) uvec3( 2, 3, 10 ) uvec3( 2, 3, 7 )
     array([2, 7, 6], dtype=uint32) uvec3( 2, 7, 10 ) uvec3( 2, 7, 6 )
     array([3, 0, 4], dtype=uint32) uvec3( 3, 0, 10 ) uvec3( 3, 0, 4 )
     array([3, 4, 7], dtype=uint32) uvec3( 3, 4, 10 ) uvec3( 3, 4, 7 )
however after all this, creating from fresh array works: uvec3(            3,            0,           11 )

I cannot see for my part what is failing exactly in the following:

• plyfile creating an errored array ?
  (its written in pure python so I doubt of it)
• numpy exposing its buffer wrongly in this specific configuration ?
• pyglm not understanding so specificity of the buffer ? or corrupting memory ?
  (I tried to read type_checkers.h, but I failed to understand some of its mechanics)

What's you mind on this ?
Thanks in advance

[Zuzu-Typ]

Hey there (:

Thanks a lot for the detailed information!
I'm sorry you had to go through all that. It's such a silly bug.
It only occurs, when a vector / matrix / quaternion is initialized with a buffer object that is in Fortran memory layout and readonly.
It will always try to store the data of the buffer inside 8 bytes of allocated memory (which happens to be enough to store 2 of the 3 values of the uvec3). You can see why over here.

It should be fixed now.

Cheers
--Zuzu_Typ--

[jimy-byerley]

It works ! thank you very much !
looking at the small mistake it comes from and the specific circumstances, I must say congrats for finding it so fast :)