v0.16.0

Changelogs

Features

• Added easy_render module for quick rendering of meshes with PBR materials from gltf, usd, obj imports (#800 , #797 )
• Added new math module with new quaternions math features (#805 )
• Added new physics module with Simplicits method (#806 , #812 )
• Implemented new functions for generating rays from camera (#811 )
• Implemented conversion function from gaussian-rendering cameras to kaolin Camera class (#813 )

Misc

• Support extended to pytorch 2.4.0 and cuda 12.4
• Now generating wheels for cpu only

Contributors (in alphabetical order):

• Clement Fuji Tsang (@Caenorst )
• Vismay Modi (@itsvismay )
• Or Perel (@orperel )
• Masha Shugrina (@shumash )
• Alex Zook (@zookae )

v0.15.0

Changelogs:

Features:

Add methods and property to simplify camera transformation (#751)
Allow camera width and height to be modified without modifying FOV (#751)
Support .obj files with R G B defined (#760)
Make arguments optional in PBRMaterials and add helper methods, and improve import / export of materials for USD (#762)
Add GLTF loader (#763)
Add translation movement in turntable visualizer (#767)
Improve API of usd and obj loaders for triangulation (#774)
Add optional canvas as argument to visualizers (#775)
Add vertex tangents to the mesh class (#763)

Bugfix:

Fix drifting bug in turntable visualizer (#767)

Misc:

Added support for pytorch 2.1.0 and 2.1.1 (#765)
Added support for cuda 12.1 (#765)

Contributors (by last name Alphabetical order):

Sanja Fidler
Clement Fuji Tsang
Or Perel
Tianchang (Frank) Shen
Masha Shugrina
Alexander Zook
@mikerjacobi

v0.14.0

Changelogs:

Features:

Added mesh to spc conversions (#696)
Added interactive visualizers (#704, #723, ##742)
Added high-level SurfaceMesh class (#740)

Bugfix:

Fixed trilinear interpolation gradient when the point is not on voxel (#684)
Simplified and fix material importation from USD and OBJ (#708, #718, #724)
Fixed UV bug in examples using nvdiffrast (#742)

Misc:

Added support for pytorch 2.0.0 and 2.0.1 (#717, #741)
Added support for python 3.10
Dropped support for python 3.7

Contributors (by last name alphabetical order):

Sanja Fidler
Clement Fuji Tsang
Charles Loop
Or Perel
Masha Shugrina
Alexander Zook

v0.13.0

Changelogs:

Features:

Reformat dataset preprocessing with CachedDataset (#626)
Added uniform spherical sampling and conversion between spherical and cartesian (#661)

Add lighting features: (#663)

• Improved spherical harmonics coefficients
• Added Spherical Gaussians features, including diffuse and specular lighting

Added support for internal transforms in USD mesh import (#667)
Add gradients on coordinates for trilinear interpolation (#650)

Improved memory consumption of uniform_laplacian (#643)
Improved ~10% speed on SPC ray tracing (#634)

Bugfix:

Fix bug enabling raytracing with SPC from within the [-1, 1] range (#634)
Fix bug on SPC scan_octrees with sometimes allocation issue (#653)

Tutorials:

Added new recipe for Fast Mesh Sampling preprocessing (#626)
Added new recipe for SPC 3D convolution (#621)
Added Lighting tutorials for diffuse and specular (#663)

Misc

Kaolin is now pip installable.

Contributors (by last name alphabetical order):

Sanja Fidler
Clement Fuji Tsang
Jean-Francois Lafleche
Charles Loop
Or Perel
Masha Shugrina
Towaki Takikawa
Alexander Zook
@Mason-McGough

v0.12.0

Changelogs:

Summary:

With the version 0.12.0 we have added a Camera API, allowing to use all our renderers and multiple coordinate systems.

Checkout our news tutorials:

• plenty of camera recipes in examples/recipes/camera
• a tutorial using the new camera API with nvdiffrast examples/tutorial/camera_and_rasterization.ipynb

Features:

Added Camera API

Bugfix:

Fix bug with kaolin.ops.mesh.check_sign when the point is aligned with a vertice or an edge.
Fix bug on some ops when using 2nd GPU

Tutorials:

Added a bunch of recipes for Camera API
Added a tutorial to show how to use Camera API with nvdiffrast

Contributors (by last name alphabetical order):

Sanja Fidler
Clement Fuji Tsang
Or Perel
Masha Shugrina
Towaki Takikawa
Jiehan Wang
Alexander Zook

v0.11.0

Changelogs:

Summary:

In Kaolin 0.11.0 we are focusing on improving performance for our main renderers, strongly improving SPC raytracing and trilinear interpolation, and integrating nvdiffrast as a backend for DIB-R rasterization. We are also adding a few features such as tetrahedral and triangle mesh subdivision, support for heterogeneous mesh in obj loader, as well as improving Dash3D usability.

Finally, several tutorials and recipes are implemented for new users to quickly get a grasp on Kaolin features.

Features:

Improved Dash3D usability (#538)
Improved SPC raytracing memory usage / speed (~12x less memory used / +33% on a test model at level 11) (#548)
Added tetrahedral mesh subdivision used in DMTet (#551)
Added triangle mesh subdivision used in DMTet (#562)
Allowed SPC unbatched_query to output parents nodes (#559)
Splitted rasterization and dibr soft mask in two functions (#560)
Integrated nvdiffrast for rasterization (measured up to x640 faster on large model from ShapeNet at 1024x1024 resolution) (#560)
Added support for Heterogeneous mesh in obj loader (#563)
Implemented a fused trilinear interpolation kernel with computation of dual of octree (#567)

Tutorials:

Added recipe to convert pointcloud to SPC (#535)
Added recipe for basic explanation of spc's octree (#535)
Added example for fitting a 3D bounding box using differentiable renderer (#543)
Added recipe to compute Occupancy using check_sign (#564)
Showed backend keyword for rasterization in DIB-R tutorial (#569)
Added recipe for the dual octree and trilinear_interpolation

Bug fix:

Fixed DIB-R tutorial due to normalization changing in the Omniverse App (#532)
Fixed issue with uint type issue on Windows (#537)
Fixed f_score reduction bug (#546)
Fixed indexing bug on understanding SPC tutorial (#553)

Contributors (by last name alphabetical order):

Sanja Fidler
Clement Fuji Tsang
Charles Loop
Or Perel
Frank Shen
Masha Shugrina
Gavriel State
Towaki Takikawa
Jiehan Wang
Alexander Zook

v0.10.0

Changelogs:

Summary

In Kaolin 0.10.0 we are focusing on Volumetric rendering, adding new features for tetrahedral meshes, including DefTet volumetric renderer and losses, and Deep Marching Tetrahedrons, and adding new primitive operations for efficient volumetric rendering of Structured Point Clouds, we are also adding support for materials with USD importation.

Finally we are adding two new tutorials to show how to use the latest features from Kaolin:

• How to use DMtet to reconstruct a mesh from point clouds generated by the Omniverse Kaolin App

• An Introduction to Structured Point Clouds, with conversion from mesh and interactive visualization with raytracing.

Features:

Simplify kaolin.ops.spc.unbatched_query API (#442)
Added point to vertice type of distance in kaolin.metrics.trianglemesh.point_to_mesh_distance, with a little speedup (#446, #458)
Added new “thin” mode for kaolin.ops.voxelgrids.extract_surface (#448)
Adding Marching Tetrahedra (#476)
Extend SPC raytracing to output depth (#478)
Refactor SPC raytracing API (#478)
Adding Differentiable Volumetric rendering features for SPC (#485)
Adding “squared” option for Chamfer distance (#486)
Adding Deftet Volumetric Renderer (#488)
Adding Deftet losses (#496)
Adding interpolation of features for point sampling on mesh (#499)
Adding DMtet Tutorial (#492)
Adding SPC Tutorial (#500)
Adding unbatched_pointcloud_to_spc wrapper (#498)
Adding materials support for USD importers (#502)

Bug fix:

Fix small bugs on USD importer / exporter (#441, #445)
Fix trianglemesh_to_voxelgrids when sparse (#455)
Fix bug where Kaolin were not building with submodule CUB (#457)
Fix Preprocessing bug where “name” attributes contains “/” (#469)

Misc:

Define a proper C++ Style Guide and fine-tune codebase accordingly (#470, #471, #472, #477)

Kaolin now featured in:

DIB-R
Deftet
DMtet
GradSim
NeuralLOD
Text2Mesh

Contributors (by last name alphabetical order):

Sanja Fidler
Clement Fuji Tsang
Jun Gao
Jean-François Lafleche
Michael Li
Charles Loop
Or Perel
Frank Shen
Masha Shugrina
Gavriel State
Towaki Takikawa
Jiehan Wang
Alexander Zook
(github) Talmaj
(github) le-Greg

v0.9.1

Changelogs:

Summary

The latest Kaolin release includes a new representation, structured point clouds, an octree-based acceleration data structure, with highly efficient convolution and ray tracing capabilities. This representation is useful for neural implicit representations, popular in 3D DL research today, and beyond, and powers the latest version of NeuralLOD training.

The release is also coming with extended support for 3d dataset like ModelNet / ShapeNet / SHREC, new utility functions to improve usability and speedups on import / export of USD used in checkpoints.
In this version, we added a lightweight visualizer Dash3D for quickly visualizing from a low-end config such as a laptop.

New Additions

• Structured Point Clouds (#397)
  • added new acceleration data structure (see kaolin.ops.spc)
  • supported operations:
    • ray tracing (see kaolin.render.spc)
    • Trilinear interpolation
    • Sparse conv3d / conv_transpose3d (see kaolin.ops.spc)
  • added class container (see kaolin.rep)
• Experimental Dash3D Checkpoint Dashboard (#378)
  • tensorboard-style web dashboard for 3D data written with Kaolin’s Timelapse API (see docs)
• Tutorials (#381)
  • Differentiable Rendering jupyter notebook tutorial (#372)
  • 3D Checkpoint Tutorial

Features

• added pointclouds_to_voxelgrids (#361)
• added support for non-fully connected mesh for uniform_laplacian (#380)
• added mask_IoU on rendered images (#372)
• added support for camera transform matrix (instead of just rotation / translation) (#372)
• support for SHREC (#375)
• support for colors in exporting point clouds in USD (#400)
• support for UsdGeomPoints (#400)
• support for .off (#367)
• support for ModelNet (#382, #410)
• added utility function for loading synthetic data from OV app (#372)
• added material support for ShapeNet (#388)
• added version support for ShapeNet (#399)

Optimizations

• accelerated USD import: ~10-5X (#421)
• accelerated USD export: ~8-4X for exporting and timelapse (#422)
• accelerated backward of index_by_face_vertices (#419)

Bug fix:

• fixing a bug on texture_mapping removing when UVs are out-of-bounds.
  fix some issues with ShapeNet and support for bad models (#391, #411)

Misc:

• Allow users to install Pytorch version out of official support (#390)

Contributors:

• Clement Fuji Tsang
• Masha Shugrina
• Charles Loop
• Towaki Takikawa
• Jiehan Wang
• Michael Li
• @AndresCasado
• @mjd3

doctest

do not use

v0.9.0

Changelogs:

Highlights

The Kaolin 0.9 release include a reformat of the API and various improvment of the performance and the ergonomy of Kaolin.
A reformat was required to be able to have a maintainable, clean and reliable Kaolin in the long term.

Low level API

Mesh class contained too many attributes and methods that were too specific or unused or redundant.
Also given how quickly the field can shift to new methods, having a fixed class representation can be a constraint. We chose to focus on low-level functions with torch tensors as inputs / outputs, to favor reusability.
High-level representation will be added later once the common use cases get more easy to define.

Model Zoo

Maintainable and reliable Kaolin means a more compact library. We decided to move the model zoo out of Kaolin, this model zoo will have a dedicate repository, will rely on release of Kaolin, and so will be maintained separately.

Batching

Kaolin is now fully batched, by default with a fixed topology, but also (with limited support) representation for heterogenous structures using packed and padded approch, see documentation for more details.
We intend to provide more primitive ops for heterogenous structures.

Optimizations

We've been mostly focusing on GPU efficiency.
Among the optimizations, speedups are reported on:

• kaolin.render.mesh.rasterization.dibr_rasterization(height, width, face_vertice_z, face_vertices_image, face_features, face_normals_z) (~1.35x faster).
• GraphConv: added a functionality of pre-normalization of the adjacency matrix kaolin.ops.gcn.GraphConv(node_feat, adj, normalized_adj=False) (~1.85x faster).
• kaolin.ops.mesh.check_sign(vertices, faces, points, hash_resolution): (~2.75x faster).
• kaolin.ops.mesh.sample_points(vertices, faces, num_samples, areas): added a functionality of pre-computation of faces areas (~1.6x faster)
• kaolin.ops.conversions.voxelgrids_to_cubic_meshes(voxelgrids, is_trimesh) (~17x faster on cpu, >10000x faster on gpu)
• kaolin.ops.voxelgrid.downsample(voxelgrids, scale) (~6.2x faster on cpu, ~25x faster on gpu)
• kaolin.ops.voxelgrid.fill(voxelgrids) (~1.3x faster on cpu)
• kaolin.ops.voxelgrid.extract_surface(voxelgrids) (~6.9x faster on cpu, ~37x faster on gpu)
• kaolin.ops.voxelgrid.extract_odms(voxelgrids) (~250x faster on cpu, ~1276x faster on gpu)
• kaolin.ops.voxelgrid.project_odms(odms, voxelgrids, votes) (~125x faster on cpum ~882x faster on gpu)
  We added a cuda implementation of lorensen's marching cube (used in kaolin.ops.conversions.voxelgrids_to_trianglemeshes(voxelgrids, iso_value))
  We added backpropagation to the triangle distance (used in kaolin.metrics.trianglemesh.point_to_mesh_distance(pointcloud, vertices, faces)) and side distance (used in kaolin.metrics.pointcloud.sided_distance(p1, p2), kaolin.metrics.pointcloud.chamfer_distance(p1, p2, w1, w2), kaolin.metrics.pointcloud.f_score(gt_points, pred_points, radius, eps)).

USD Visualization

We now provide importer and exporter to Universal Scene Description files, see the documentation for more information.
You can open those file using the Omniverse companion app, see Kaolin Devpage.

Contributors

In alphabetical order:

Wenzheng Chen
Sanja Fidler
Clement Fuji Tsang
Jason Gorski
Jean-Francois Lafleche
Rev Lebaredian
Jianing Li
Frank Shen
Masha Shugrina
Gavriel State
Jiehan Wang
Tommy Xiang