Pretrained models

[yaorz97]

Hi, thank you for your amazing work :), could you release the pretrained models to reproduce results reported in the paper?

[Wuziyi616]

Hi, thanks for your interest in our work. Since there are so many settings in the experiments (train one model per category, train one model over all categories, different subsets, etc.), it's very hard to share the pre-trained weights. But if you take a look at #1, the user confirmed that by running our code + training commands, he is able to reproduce the results reported in the paper. So I think you can just train the model by yourself.

The training time is not long! Training 1 model on 1 category usually takes less than 6h. Training on all categories usually takes less than 2 days. Hope this information helps.

[yaorz97]

Thank you for your prompt and useful advice. In addition, I found that the point cloud of the fractured part contains not only the surface points but also the internal points. Can you suggest some ways to remove the internal points during the data generation process?

[Wuziyi616]

I'm not sure actually, cause I haven't looked at the generated point clouds. The point clouds are sampled from fracture meshes (faces) right? So if I understand correctly, that means some meshes have faces inside the shape? Is that the reason according to your observation?

[yaorz97]

yes, there are some meshes have faces inside, you can visualize some examples, like bottle in everyday.

[Wuziyi616]

Thanks for the details. I'm pinning the authors who work on the geometry part of the paper to take a look.

[Wuziyi616]

A quick question: you are saying the fracture meshes themselves have inner faces, not the mesh assembled from fractures have inner faces (which is natural)?

At first thought I think that is impossible, because our fracture simulation algorithm (Breaking Good) generates meshes without inner faces. Can you upload a mesh file with this issue as the attachment, and point out the ID (the path to it, basically something like everyday/Bottle/xxx), so that we can localize it faster? We did go through the meshes last year, and I don't think we observe such issues. Do you observe many such cases in our dataset?

[yaorz97]

yes, i found the fracture meshes have inner faces, for example, everyday/Bottle/fdc47f5f8dff184830eaaf40a8a562c1/mode_9/piece_0.obj. The sampled point cloud is visualized in the attached image and there are a lot of points inside the shape.
I randomly select one fracture to visualize and do not know how many issues like this are in the dataset.

[Wuziyi616]

Thanks for the information. We look into the mesh you provided, and indeed there are inner faces. This is because of the fracture simulator we use, which creates mesh cages (See Section 3.5 of Breaking Good if you are interested), will introduce inner faces to the mesh in a pre-processing step. Then, after we break the shape into fractures, we forgot to remove these inner cage faces before exporting them.

Removing inner faces should be simple, and we are working on a script to convert the current data to the inner-face-free version (e.g. we can implement this post-processing step in the decompress.py, so that when you decompress a mesh, you also remove the inner faces before saving it to file). Another solution is to sample point clouds only from surface faces (Idk if this is easy to implement in Python without using those geometry libraries). Overall, thank you for pointing this issue out. We will fix it ASAP and let you know if we have any results.

[Wuziyi616]

OK, so we have a hotfix by updating the decompress.py (see this branch). The new meshes have (nearly) no inner faces. You can try it out (requires a new dependency, please do pip install gpytoolbox).

However, the decompression speed with the new script is 100x slower. We're still looking for a way to accelerate it. Expect to see updates in the following days.

EDIT: we have fixed the speed issue as well. Can you @Pterosaur-Yao kindly run our decompression code and check if the new meshes look all good? From our side, we believe we have removed the inner faces. We are now working on re-running the benchmark results.

[yaorz97]

Ok, I will re-run the updated code and look forward to your new benchmark results.

[yaorz97]

Now, we find that most of the inner faces have been removed. However, in some meshes, there may still be some inner faces

[Wuziyi616]

This is the reply from my teammate:

there can still be some, but getting rid of those would either be slow or we'd need to do it at the data generation step

From my re-running benchmarks (have just finished ~20%, and there are still more to run), removing internal points doesn't seem to have a clear effect on the final performance. This could be due to the use of PointNet encoder, which may eliminate the inner point features after the max-pooling operation. That means if we use e.g. DGCNN, the inner points can possibly cause large differences. But overall, I think keeping these minimal inner points is fine.

Can you share your thoughts here? Also if you want to further improve the results, can you point me to a mesh with inner faces again (but as I said, we cannot guarantee to fix it)?

[yaorz97]

Intuitively, the internal points do not contribute to the pose estimation, but rather reduce the sampling rate of the surface. However, the good news is that there are now very few internal triangles (I have observed at most one), which should have little effect on the results.