PCLVisualizer point picking weirdness

[victoryc]

I am noticing a weird behavior with point picking in PCLVisualizer. For some clicks, it returns the correct x,y,z coordinates. But, some times, the returned coordinate values are not at all in the area I am clicking; they are way off. Any idea why this could be happening?

I don't expect this to be related to the NaN issue because (a) I am not using the point index in the PointPickingEvent. Instead, I am just using the x,y,z values provided by the getPoint method of the event. (b) The point cloud I am rendering doesn't have NaN values.

I did notice that whenever the returned coordinates are wrong, the corresponding point index in the PointPickingEvent is a single digit incorrect value such as 0, 1, 3 etc. But, as I said, I am not using the point index value.

Once in a while, it returns coordinate values so incorrect that the reported pick point is not even within the bounding box of the point cloud.

PCL version: 1.10.1 on Windows 10
Number of points in the cloud: About 4 million

Thanks.

[taketwo]

Not a definitive answer, but from my experience I have a feeling VTK (our underlying gfx library) does some sort of approximation/heuristics when deciding which actor/primitive was poked. So if you have something in foreground, but click on background near it, then you may get the coordinates of the foreground object.

[victoryc]

I did try removing anything else being drawn by the application other than the point cloud and still got occasionally completely incorrect coordinate values.

[victoryc]

Update: I hacked some code to use vtkWorldPointPicker instead of the vtkPointPicker used by PCL. With vtkWorldPointPicker, the coordinates are indeed coming out right. So, the bug seems to be with either vtkPointPicker or in PCL's use of it. I was motivated to try this out by some old discussions of bugs involving vtkPointPicker that turned up while searching online.

[Apache97]

@victoryc I am facing the same issue. Regarding your fix, did you modified the sourcecode of PCL to use vtkWorldPointPicker instead of vtkPointPick or you did something else? Have you created a stash/shelveset of the changes? Thanks.

[mkaler999]

I am having the same issue where the x, y and z coordinates of the picked point do not even belong to my point cloud.
I also noticed that this behavior becomes much more frequent if I increase PCL_VISUALIZER_POINT_SIZE through setPointCloudRenderingProperties.

[vestri]

@victoryc Hi, I am having the same issue. That's really annoying since we cannot recover correct 3D coordinates even though we are using one point cloud. Any hint or explanation to recover the right x,y,z coordinates?
does #3763 PR will solve it? Thanks

[themightyoarfish]

I also am frequently confused by some points seemingly attracting the viewer, and being unable to pick the one actually under the cursor. Something in the foreground while picking background is a problem, but it even happens at other times.

Currently I'm noticing this behaviour when the data in PCLVIsualizer is many cloud objects. Meanwhile, pcl_viewer with the same data passed as individual files has similar issues, while it does not when the clouds are first merged to one file, then displayed.