efficient way to Graphic3_ArrayofPoints() AddVertex population

[andysfd]

Hello, is there currently a more efficent way in pythonocc to populate the pc element rather than calling AddVertex(gp_Pnt(x,y,z)) in a foor loop?

I need to load lidar pointclouds of ~50.000.000 points and are stuck with a slow loop. On my laptop its currently taking 5-7 minutes.
I am not that familiar with this kind of optimization, any help is really welcome!

pc = Graphic3d_ArrayOfPoints(n_points, True)
for idx in range(n_points):
    x, y, z = points[idx]
    r, g, b = colors[idx]
    color = Quantity_Color(r, g, b, Quantity_TOC_RGB)
    pc.AddVertex(gp_Pnt(x, y, z), color)

I already did some research and came across the following topic: #1000
However this solution was more focused on populating meshes. The other solution the user came up with was a cython optimization which i would use as a last resort.

Is there any other smart way to make this class useful also for large (or nowadays rather "normal") sized pound clouds.

Thank you for providing this library and the support!

[andysfd]

I managed to gain a performance increase of ~ 6x using the recommended cython optimization. If any other options or ideas comes up i would be really thankful.

heres the optimized cython code:

cimport cython
cimport numpy

from OCC.Core._gp import new_gp_Pnt
from OCC.Core._Graphic3d import new_Graphic3d_ArrayOfPoints
from OCC.Core._Quantity import new_Quantity_Color, Quantity_TOC_RGB

@cython.boundscheck(False)
@cython.wraparound(False)
def load_pointcloud_cython(int n_points, numpy.ndarray[double, ndim=2] points, numpy.ndarray[double, ndim=2] colors):
    cdef int i
    pc = new_Graphic3d_ArrayOfPoints(n_points, True)
    for i in range(n_points):
        pc.AddVertex(new_gp_Pnt(points[i,0], points[i,1], points[i,2]), new_Quantity_Color(colors[i,0], colors[i,1], colors[i,2], Quantity_TOC_RGB))
    return pc

[tpaviot]

Did you look at https://github.com/tpaviot/pythonocc-demos/blob/master/examples/core_meshDS_numpy.py?

If the cython optimization is not enough, I guess the implementation should be done at the c++ level. An additionnal method could be part of the Graphic3d module available as, for example:

pc = Graphic3d_ArrayOfPoints(n_points, True)
pc.AddVerticesFromNumpy(points, colors)

or something like that

[andysfd]

Thanks for the hint, i looked into it prior but thought it wouldn't be of any help as my large datasets are only colorized point cloud. I'm currently sticking to the cython workflow an will updated if i have the chance to further enhance the procedure. Thank you!

[tpaviot]

ok please leave this ticket open, I'll try to take look at that, that should not be that hard to add a c++ method that takes two numpy arrays

[tpaviot]

ok, I implemented a solution I'm about to push to a test branch. Here are the results for 1 million points, stored in a numpy array, passed to a Graphic3d_ArrayOfPoints.

Total time in python/c++ 0.036586518999683904s

(for information, using pure python 1.575533026998528s)

The fix needs pythonocc to be recompiled

[andysfd]

wow thats fast! thanks for taking that topic up so quickly.

I must admit i'm not a well versed software developer (surveying/geodesy is my main field). I guess i need to build pythonocc after pulling the test branch rather than installing it via conda? Via conda it would be just available after this implementation would be included in a proper release?

[jnwalther]

Hi, this looks really interesting. @tpaviot, am I correct in assuming that the solution you describe is the one implemented here? I guess it didn't make it to the main branch in the end, then. Was there any particular reason behind it?

In a project I'm working on we used Cython to implement a whole bunch of numpy interfaces to a variety of OCC array types (e.g. from TColStd, TColgp, TShort). But this looks much more elegant and performant.