Support VTK7 with OpenGL2 backend #1534

taketwo · 2016-02-11T21:49:30Z

This is a work-in-progress to support the new default backend in VTK. Addresses #1275.

Compiles and works on my Arch with VTK7, however I observe very low frame-rates (~15fps) when rotating and panning moderately sized (~300k) point clouds.

VictorLamoine · 2016-02-12T08:18:50Z

I'm building against VTK trunk:

CMakeFiles/pcl_outofcore_viewer.dir/__/src/visualization/outofcore_cloud.cpp.o: In function `OutofcoreCloud::render(vtkRenderer*)':
outofcore_cloud.cpp:(.text+0x278f): undefined reference to `vtkVertexBufferObjectMapper::New()'
outofcore_cloud.cpp:(.text+0x27e9): undefined reference to `vtkVertexBufferObjectMapper::SetInput(vtkPolyData*)'
collect2: error: ld returned 1 exit status
make[2]: *** [bin/pcl_outofcore_viewer] Error 1
make[1]: *** [outofcore/tools/CMakeFiles/pcl_outofcore_viewer.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

VictorLamoine · 2016-02-12T11:05:35Z

What hardware are you testing on?

Here is a VTK test code, use this big mesh (8 M point clouds)

#include <vtkSmartPointer.h>
#include <vtkCallbackCommand.h>
#include <vtkRenderer.h>
#include <vtkRenderWindowInteractor.h>
#include <vtkInteractorStyleTrackballCamera.h>
#include <vtkRenderWindow.h>
#include <vtkTimerLog.h>
#include <vtkPolyDataMapper.h>
#include <vtkActor.h>
#include <vtkPLYReader.h>
#include <vtkProperty.h>

#include <iostream>

static void CallbackFunction(vtkObject* caller, long unsigned int eventId, void* clientData, void* callData);

int main(int, char *argv[])
{
  vtkSmartPointer<vtkPLYReader> reader = vtkSmartPointer<vtkPLYReader>::New();
  reader->SetFileName(argv[1]);
  reader->Update();

  vtkSmartPointer<vtkPolyData> polydata = vtkSmartPointer<vtkPolyData>::New();
  polydata = reader->GetOutput();
  std::cout << "Mesh points: " << polydata->GetPoints()->GetNumberOfPoints() << std::endl;

  vtkSmartPointer<vtkPolyDataMapper> mapper = vtkSmartPointer<vtkPolyDataMapper>::New();
  mapper->SetInputData(polydata);

  vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();
  actor->SetMapper(mapper);
  actor->GetProperty ()->SetColor (0.9, 0.1, 0.2);

  vtkSmartPointer<vtkRenderer> renderer = vtkSmartPointer<vtkRenderer>::New();
  renderer->AddActor(actor);

  vtkSmartPointer<vtkRenderWindow> renderWindow = vtkSmartPointer<vtkRenderWindow>::New();
  renderWindow->AddRenderer(renderer);

  vtkSmartPointer<vtkInteractorStyleTrackballCamera> imageStyle =
      vtkSmartPointer<vtkInteractorStyleTrackballCamera>::New();

  vtkSmartPointer<vtkRenderWindowInteractor> renderWindowInteractor = vtkSmartPointer<vtkRenderWindowInteractor>::New();
  renderWindowInteractor->SetInteractorStyle(imageStyle);
  renderWindowInteractor->SetRenderWindow(renderWindow);

  vtkSmartPointer<vtkCallbackCommand> callback = vtkSmartPointer<vtkCallbackCommand>::New();
  callback->SetCallback(CallbackFunction);
  renderer->AddObserver(vtkCommand::EndEvent, callback);

  // Render and interact
  renderWindow->Render();
  renderWindowInteractor->Start();
  return EXIT_SUCCESS;
}

void CallbackFunction(vtkObject* caller, long unsigned int vtkNotUsed(eventId), void* vtkNotUsed(clientData),
                      void* vtkNotUsed(callData))
{
  vtkRenderer* renderer = static_cast<vtkRenderer*>(caller);

  double timeInSeconds = renderer->GetLastRenderTimeInSeconds();
  double fps = 1.0 / timeInSeconds;
  std::cout << "FPS: " << fps << std::endl;
}

cmake_minimum_required(VERSION 2.8)

PROJECT(vtk_ply_loader)

find_package(VTK REQUIRED)
include(${VTK_USE_FILE})

add_executable(test_vtk_1 MACOSX_BUNDLE test_vtk_1)

if(VTK_LIBRARIES)
  target_link_libraries(test_vtk_1 ${VTK_LIBRARIES})
else()
  target_link_libraries(test_vtk_1 vtkHybrid)
endif()

I see a big improvement between VTK OpenGL and OpenGL2.
Big point cloud: https://mega.nz/#!UZ8wDC6L!jefEj4aZknTYFP3grU2nAPdRGalFORHBsJVy3zGnFI8

taketwo · 2016-02-12T20:42:00Z

Ok, for now I simply disabled the pcl_outofcore_viewer app because it relies on our custom vtkVertexBufferObjectMapper which I have no clue how to adapt for the new backend.

I'll give your example a try.

taketwo · 2016-02-12T20:58:57Z

I did not manage to get the FPS counter below 116, mostly it reports numbers between 1k and 3k. However still for me the motions are not super-smooth. Maybe I'm subjective, so please give a try to pcl_viewer yourself.

VictorLamoine · 2016-02-12T21:32:46Z

outofcore/tools/CMakeLists.txt

@@ -15,7 +15,7 @@ else(NOT VTK_FOUND)
  include("${VTK_USE_FILE}")
  include_directories("${CMAKE_CURRENT_SOURCE_DIR}/include")

-  if("${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}" VERSION_GREATER "5.8")
+  if("${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}" VERSION_GREATER "5.8" AND "${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}" VERSION_LESS "7.0")


I would rather check VTK_RENDERING_BACKEND so that it allows people with OpenGL backend to still use pcl_outofcore_viewer.

taketwo · 2016-02-12T21:47:54Z

Well, it's not meant to be the final solution. I should have clarified, this pull request is more like a hack to make PCL compile with latest VTK. Eventually we will need to figure out how to substitute PCL's custom vtkVertexBufferObjectMapper with the stuff from the new backend (e.g. vtkOpenGLPointGaussianMapper).

The new OpenGL2 backend does not have vktOpenGLHardwareSupport class which was used before.

These can be used at configuration and compilation times to adjust for the currently used backend.

This mapper is a performance optimization that is only needed when OpenGL backend is used. The modern OpenGL2 backend uses vertex buffer objects transparently for us.

taketwo · 2016-02-13T13:55:03Z

Ok, now it's supposed to be the final solution. Vertex buffer objects are used ubiquitously by the new backend, transparently for us. Therefore there is no need in the custom vtkVertexBufferObjectMapper class at all. And we don't need to substitute it with anything explicitly.

This pull request introduces a couple of variables and macros that are checked in scripts and code to conditionally compile the custom mapper only when the old backend is used. They are based on the VTK_RENDERING_BACKEND, not just on VTK version, as suggested by Victor.

I think it's a safe change and should probably be included in 1.8.0, though of course it would be nice to hear back from someone who tried this patch first.

VictorLamoine · 2016-02-13T14:09:26Z

I think it's a safe change and should probably be included in 1.8.0, though of course it would be nice to hear back from someone who tried this patch first.

It would really be great, this is a huge improvement!

I will compile, test and benchmark this code on several machines (at least 3) with different hardware configurations and get back with the results 👍

taketwo · 2016-02-13T14:32:23Z

Great, thanks!

UnaNancyOwen · 2016-02-13T16:58:22Z

I has confirmed that PCL with this fix can build on MSVC14.
Thanks! 👍

	MSVC14 x86	MSVC14 x64
VTK 7.0 with OpenGL back-end	success	success
VTK 7.0 with OpenGL2 back-end	success	success

taketwo · 2016-02-13T17:33:24Z

Did you also try to run pcl_viewer? How is the performance?

UnaNancyOwen · 2016-02-14T12:59:33Z

Where is the test code and test data?

taketwo · 2016-02-14T13:10:08Z

I did not have any specific code or data in mind. Just the standard pcl_viewer.exe tool with some PCD file (e.g. from our "test/" directory).

VictorLamoine · 2016-02-14T13:23:49Z

Test program

#include <pcl/console/parse.h>
#include <pcl/io/vtk_lib_io.h>
#include <pcl/visualization/pcl_visualizer.h>

int main(int argc, char *argv[])
{
  std::vector<int> ply_files = pcl::console::parse_file_extension_argument(argc, argv, ".ply");
  if (ply_files.empty())
  {
    PCL_ERROR("Please provide a PLY mesh to be visualized!\n");
    return (-1);
  }

  pcl::PolygonMesh mesh;
  pcl::io::loadPolygonFilePLY(argv[ply_files[0]], mesh);

  pcl::visualization::PCLVisualizer viewer;
  viewer.addPolygonMesh(mesh);
  viewer.addCoordinateSystem();
  viewer.resetCamera();
  viewer.spin();
  return (0);
}

Test file (8 millions points mesh).

UnaNancyOwen · 2016-02-14T14:21:21Z

I was tested in the following environments.
I think that it have good frame rate.
https://youtu.be/LjZSs7CY5-g

Intel Core i7 4790
NVIDIA GeForce GTX970
Microsoft Windows 10 (x64)
Microsoft Visual Studio 2015
PCL 1.8.0 with this fix + VTK 7.0 OpenGL2 back-end

taketwo · 2016-02-14T15:06:37Z

Brilliant! Thanks for giving it a try. I think we can merge this as soon as Victor reports his results.

VictorLamoine · 2016-02-14T15:22:53Z

Results will be available on Monday. I have 3 configurations at home but two of them crashes with OpenGL2 backend (GLXBadFBConfig), its not PCL related.

At work I know the OpenGL2 back-end works so I will test there.

mschlaffer · 2016-02-14T20:34:59Z

If you're looking for additional benchmarks, my specs:

Current Arch Linux (64bit)
VTK7 with OpenGL2
latest PCL from Git with this fix (everything built correct)
CPU: i5-4210H
Onboard Graphics: Intel HD 4600
Graphics Card: Nvidia GTX 950M (using primusrun and bumblebee)

On Victors last test program and the suzanne_hd.ply mesh I got:

Onboard: ~14 fps during turning
Nvidia: ~2200 fps average, almost never below 1000 fps, the lowest I could find on multiple screenshots was 600

Edit: On the first programm with vtk directly even my onboard went over 1000fps (./main suzanne_hd.ply
Mesh points: 8060918
FPS: 9.57956, FPS: 592.834, FPS: 3862.16, FPS: 4670.72, FPS: 2785.06, FPS: 781.79
) and nvidia reached upwards to 8000 (primusrun ./main suzanne_hd.ply
Mesh points: 8060918
FPS: 6.56069, FPS: 3320.91, FPS: 2545.09, FPS: 2222.74, FPS: 3276.8, FPS: 2976.79, FPS: 6842.26, FPS: 7194.35, FPS: 7084.97), but I'm not so sure about this fps calculation. Onboard felt a little stuttery, which shouldn't be the case on such high fps counts.

VictorLamoine · 2016-02-15T11:02:51Z

Setup

Ubuntu 14.04 bits
gcc 4.9.3
VTK trunk (331d)
PCL Sergey version (https://github.com/taketwo/pcl/tree/vtk-7)
Everything is compiled in Release mode

Hardware

Victor's desktop
- Intel® Core™ i5-4670K CPU @ 3.40GHz × 4
- GeForce GTX 770 (nvidia 352.63 updates)
Dell M6700 laptop
- Intel® Core™ i7-3740QM CPU @ 2.70GHz × 8
- Quadro K3000M/PCIe/SSE2 (nvidia 352.63 updates)

Results

Victor's desktop
- OpenGL: Big lag when switching from surface to point or wireframe. When turning the model the image flickers or is not displayed.
  The model is displayed as a point cloud when turning the view.
- OpenGL2: Everything smooth (at least 40 FPS). No lag
Dell M6700 laptop
- OpenGL: Big lag when switching from surface to point or wireframe. When turning the model the image flickers or is not displayed.
  The model is displayed as a bounding box when turning the view.
- OpenGL2: Small lag when switching from surface to point or wireframe. The model is displayed as a point cloud when turning the view.

I tested two machines where the VTK OpenGL2 backend simly doesn't work (GLXBadFBConfig), I did not manage to fix that, it's most probably a driver issue.

In a general way, the gain is very good when using only VTK. When using PCL the gain is not as big.
But it works 👍

taketwo · 2016-02-15T11:07:48Z

Can you please try to call setUseVbos(true) on visualizer object when in OpenGL backend mode? Does this give any improvement? And if so, can it compete with performance on OpenGL2 backend?

VictorLamoine · 2016-02-15T11:29:28Z

With viewer.setUseVbos(true) (VTK OpenGL backend) the mesh is not displayed... much better performance 😄

taketwo · 2016-02-15T11:33:38Z

Hah, so it was broken anyway.
Thanks for trying, I think we are safe to merge this now.

Support VTK7 with OpenGL2 backend

VictorLamoine reviewed Feb 12, 2016
View reviewed changes

Use vtkTextureUnitManager to query the number of available texture units

9cfb009

The new OpenGL2 backend does not have vktOpenGLHardwareSupport class which was used before.

taketwo force-pushed the vtk-7 branch 2 times, most recently from 8523721 to 8f51a52 Compare February 13, 2016 13:13

taketwo added 3 commits February 13, 2016 14:34

Add CMake variable and C++ macro VTK_RENDERING_BACKEND_OPENGL_VERSION

d0efc67

These can be used at configuration and compilation times to adjust for the currently used backend.

Conditionally compile code that uses vtkVertexBufferObjectMapper

c50c65b

This mapper is a performance optimization that is only needed when OpenGL backend is used. The modern OpenGL2 backend uses vertex buffer objects transparently for us.

Update documentation for setUseVbos() function

641fe1b

taketwo force-pushed the vtk-7 branch from 8f51a52 to 641fe1b Compare February 13, 2016 13:34

taketwo changed the title ~~[WIP] Support VTK7 with OpenGL2 backend~~ Support VTK7 with OpenGL2 backend Feb 13, 2016

VictorLamoine added a commit that referenced this pull request Feb 15, 2016

Merge pull request #1534 from taketwo/vtk-7

9ca3b6b

Support VTK7 with OpenGL2 backend

VictorLamoine merged commit 9ca3b6b into PointCloudLibrary:master Feb 15, 2016

taketwo deleted the vtk-7 branch February 15, 2016 12:11

VictorLamoine mentioned this pull request Feb 15, 2016

Prevent building PCL with VTK and OpenGL2 backend #1533

Closed

This was referenced Feb 15, 2016

Cannot build PCL against VTK with OpenGL2 #1275

Closed

Impossible to compile PCL against VTK 6 trunk #1234

Closed

r9y9 mentioned this pull request Feb 28, 2016

Build error on OSX #1556

Closed

rimmartin mentioned this pull request Feb 12, 2020

PCL 1.9 fails to detect installed VTK (v8.9) #3525

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support VTK7 with OpenGL2 backend #1534

Support VTK7 with OpenGL2 backend #1534

taketwo commented Feb 11, 2016

VictorLamoine commented Feb 12, 2016

VictorLamoine commented Feb 12, 2016

taketwo commented Feb 12, 2016

taketwo commented Feb 12, 2016

VictorLamoine Feb 12, 2016

taketwo commented Feb 12, 2016

taketwo commented Feb 13, 2016

VictorLamoine commented Feb 13, 2016

taketwo commented Feb 13, 2016

UnaNancyOwen commented Feb 13, 2016

taketwo commented Feb 13, 2016

UnaNancyOwen commented Feb 14, 2016

taketwo commented Feb 14, 2016

VictorLamoine commented Feb 14, 2016

UnaNancyOwen commented Feb 14, 2016

taketwo commented Feb 14, 2016

VictorLamoine commented Feb 14, 2016

mschlaffer commented Feb 14, 2016

VictorLamoine commented Feb 15, 2016

taketwo commented Feb 15, 2016

VictorLamoine commented Feb 15, 2016

taketwo commented Feb 15, 2016

Support VTK7 with OpenGL2 backend #1534

Support VTK7 with OpenGL2 backend #1534

Conversation

taketwo commented Feb 11, 2016

VictorLamoine commented Feb 12, 2016

VictorLamoine commented Feb 12, 2016

taketwo commented Feb 12, 2016

taketwo commented Feb 12, 2016

VictorLamoine Feb 12, 2016

Choose a reason for hiding this comment

taketwo commented Feb 12, 2016

taketwo commented Feb 13, 2016

VictorLamoine commented Feb 13, 2016

taketwo commented Feb 13, 2016

UnaNancyOwen commented Feb 13, 2016

taketwo commented Feb 13, 2016

UnaNancyOwen commented Feb 14, 2016

taketwo commented Feb 14, 2016

VictorLamoine commented Feb 14, 2016

Test program

UnaNancyOwen commented Feb 14, 2016

taketwo commented Feb 14, 2016

VictorLamoine commented Feb 14, 2016

mschlaffer commented Feb 14, 2016

VictorLamoine commented Feb 15, 2016

Setup

Hardware

Results

taketwo commented Feb 15, 2016

VictorLamoine commented Feb 15, 2016

taketwo commented Feb 15, 2016