Improved Deep Point Cloud Geometry Compression

Authors: Maurice Quach, Giuseppe Valenzise and Frederic Dufaux
Affiliation: Université Paris-Saclay, CNRS, CentraleSupélec, Laboratoire des signaux et systèmes, 91190 Gif-sur-Yvette, France
Funding: ANR ReVeRy national fund (REVERY ANR-17-CE23-0020)
Links: [Paper]

Please feel free to contact me at maurice.quach /at/ l2s.centralesupelec.fr for discussions or questions.

Experimental data

We provide the full experimental data used for the paper. This includes:

In models, all trained models
In results/data.csv, bitrates and objective metric values for all models, point clouds and metrics (D1/D2)
Compressed and decompressed point clouds for all models (c1 to c6, G-PCC trisoup, G-PCC octree)

Prerequisites

Python 3.6.9
Tensorflow 1.15.0 with CUDA 10.0.130 and cuDNN 7.4.2
tensorflow-compression 1.3
MPEG G-PCC codec mpeg-pcc-tmc13: necessary only to compare results with G-PCC, to obtain more recent versions you need to register on the MPEG Gitlab and request the permissions for MPEG/PCC
MPEG metric software v0.12.3 mpeg-pcc-dmetric: available on the MPEG Gitlab, you need to register and request the permissions for MPEG/PCC
MPEG PCC dataset: refer to Common Test Conditions (CTCs) to download the full dataset, you can also get some point clouds from JPEG Pleno.
packages in requirements.txt

Note 1: using a Linux distribution such as Ubuntu is highly recommended
Note 2: CTCs can be found at wg11.sc29.org in All Meetings > Latest Meeting > Output documents "Common test conditions for point cloud compression". For example, "Common test conditions for PCC", in ISO/IEC JTC1/SC29/WG11 MPEG output document N19324 is in the Alpbach meeting 130.

Getting started

Configuration

Adapt the configurations in ev_experiment.yml to your particular setup:

MPEG_TMC13_DIR: G-PCC folder (mpeg-pcc-tmc13)
PCERROR: mpeg-pcc-dmetric folder
MPEG_DATASET_DIR: MPEG PCC dataset folder
TRAIN_DATASET_PATH: Path to training dataset
TRAIN_RESOLUTION: Resolution of the training dataset
EXPERIMENT_DIR: Experiment folder, all results are saved in this folder, it needs to be created manually

Datasets

The dataset is available in the Git repository as ModelNet40_200_pc512_oct3_4k.zip. If you want to reproduce it, steps are provided below.

First, download the ModelNet40 manually aligned dataset: http://modelnet.cs.princeton.edu.
Then, we generate the training dataset specified in our paper (block size 64) with the following commands:

python ds_select_largest.py ~/data/datasets/ModelNet40 ~/data/datasets/pcc_geo_cnn_v2/ModelNet40_200 200
python ds_mesh_to_pc.py ~/data/datasets/pcc_geo_cnn_v2/ModelNet40_200 ~/data/datasets/pcc_geo_cnn_v2/ModelNet40_200_pc512 --vg_size 512
python ds_pc_octree_blocks.py ~/data/datasets/pcc_geo_cnn_v2/ModelNet40_200_pc512 ~/data/datasets/pcc_geo_cnn_v2/ModelNet40_200_pc512_oct3 --vg_size 512 --level 3 
python ds_select_largest.py ~/data/datasets/pcc_geo_cnn_v2/ModelNet40_200_pc512_oct3 ~/data/datasets/pcc_geo_cnn_v2/ModelNet40_200_pc512_oct3_4k 4000

Parameterized experiments

This section contains examples for experiments parameterized using a configuration file. For manual experiments, please go to the next section.

Run G-PCC experiments:

python mp_run.py ev_experiment.yml

Note 1: It is highly recommended to set EXPERIMENT_DIR on an SSD drive. On an HDD, it is recommended to add the --num_parallel 1 option as it can be extremely slow otherwise.

Train all models, run experiments and compare results:

python tr_train_all.py ev_experiment.yml && \
python ev_run_experiment.py ev_experiment.yml --num_parallel 8 && \
python ev_run_compare.py ev_experiment.yml

Note 1: In the provided configuration ev_experiment.yml, a large number of models are trained. This takes about 4 days on an Nvidia GeForce GTX 1080 Ti. Note 2: Adjust the --num_parallel option for ev_run_experiment.py depending on your GPU memory. With 11GB, we've found 8 to leave a fair amount of GPU memory available.

To build the LaTeX tables and gather the figures:

python ut_build_paper.py ev_experiment.yml figs/

To render the point clouds and the visual comparisons:

python ut_run_render.py ev_experiment.yml

To draw the training plots from tfevents data:

python ut_tensorboard_plots.py ev_experiment.yml

Manual experiments

This section contains usage examples for the individual scripts.

Training:

python tr_train.py \
    ~'/data/datasets/pcc_geo_cnn_v2/ModelNet40_200_pc512_oct3_4k/**/*.ply' \
    ~/datassd/experiments/pcc_geo_cnn_v2/models/c4-ws/1.00e-04 \
    --resolution 64 --lmbda 1.00e-04 --alpha 0.75 --gamma 2.0 --batch_size 32 --model_config c3p

Experiment (compress, decompress, remap colors, compute metrics):

python ev_experiment.py \
    --output_dir ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04 \
    --model_dir ~/datassd/experiments/pcc_geo_cnn_v2/models/c4-ws/1.00e-04 \
    --model_config c3p --opt_metrics d1_mse d2_mse --max_deltas inf \
    --pc_name longdress_vox10_1300 \
    --pcerror_path ~/code/MPEG/mpeg-pcc-dmetric/test/pc_error_d \
    --pcerror_cfg_path ~/code/MPEG/mpeg-pcc-tmc13-v10.0/cfg/trisoup-predlift/lossy-geom-lossy-attrs/longdress_vox10_1300/r06/pcerror.cfg \
    --input_pc ~/data/datasets/mpeg_pcc/Static_Objects_and_Scenes/People/longdress_vox10_1300/longdress_vox10_1300.ply \
    --input_norm ~/data/datasets/mpeg_pcc/Static_Objects_and_Scenes/People/longdress_vox10_1300_n/longdress_vox10_1300_n.ply

Compress/decompress (for D1 and D2 optimized point clouds):

python compress_octree.py \
    --input_files ~/data/datasets/mpeg_pcc/Static_Objects_and_Scenes/People/longdress_vox10_1300/longdress_vox10_1300.ply \
    --input_normals ~/data/datasets/mpeg_pcc/Static_Objects_and_Scenes/People/longdress_vox10_1300_n/longdress_vox10_1300_n.ply \
    --output_files ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d1.ply.bin \
        ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d2.ply.bin \
    --checkpoint_dir ~/datassd/experiments/pcc_geo_cnn_v2/models/c4-ws/1.00e-04 \
    --opt_metrics d1_mse d2_mse --resolution 1024 --model_config c3p --octree_level 4 \
    --dec_files ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d1.ply.bin.ply \
        ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d2.ply.bin.ply

Compress/decompress (for D1 optimized point cloud only):

python compress_octree.py \
    --input_files ~/data/datasets/mpeg_pcc/Static_Objects_and_Scenes/People/longdress_vox10_1300/longdress_vox10_1300.ply \
    --output_files ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d1.ply.bin \
    --checkpoint_dir ~/datassd/experiments/pcc_geo_cnn_v2/models/c4-ws/1.00e-04 \
    --opt_metrics d1_mse --resolution 1024 --model_config c3p --octree_level 4 \
    --dec_files ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d1.ply.bin.ply

Transfer colors from original to decompressed point cloud for visualization:

python map_color.py \
    ~/data/datasets/mpeg_pcc/Static_Objects_and_Scenes/People/longdress_vox10_1300/longdress_vox10_1300.ply \
    ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d1.ply.bin.ply \
    ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d1.ply.bin.ply.color.ply

Execute mpeg-pcc-dmetric to compute distortion metrics:

~/code/MPEG/mpeg-pcc-dmetric/test/pc_error_d \
    -a ~/data/datasets/mpeg_pcc/Static_Objects_and_Scenes/People/longdress_vox10_1300/longdress_vox10_1300.ply \
    -b ~/datassd/experiments/pcc_geo_cnn_v2/longdress_vox10_1300/c4-ws/1.00e-04/longdress_vox10_1300_d1.ply.bin.ply \
    -n ~/data/datasets/mpeg_pcc/Static_Objects_and_Scenes/People/longdress_vox10_1300_n/longdress_vox10_1300_n.ply \
    --resolution 1023 --dropdups 2 --neighborsProc 1

Overview

├── requirements.txt                            package requirements
└── src
    ├── compress_octree.py                      [Coding] Compress a point cloud
    ├── decompress_octree.py                    [Coding] Decompress a point cloud
    ├── ds_mesh_to_pc.py                        [Dataset] Convert mesh to point cloud
    ├── ds_pc_octree_blocks.py                  [Dataset] Divide a point cloud into octree blocks
    ├── ds_select_largest.py                    [Dataset] Select the N largest files from a folder
    ├── ev_compare.py                           [Eval] Compare results (curves, BD-Rates and BD-PSNRs...)
    ├── ev_experiment.py                        [Eval] Run pipeline for a point cloud (compress, decompress, eval)
    ├── ev_experiment.yml                       [Config] Experimental configuration
    ├── ev_run_compare.py                       [Eval] Compare results for the experimental configuration
    ├── ev_run_experiment.py                    [Eval] Run pipelines for the experimental configuration
    ├── map_color.py                            [Utils] Transfer colors from a point cloud onto another
    ├── model_configs.py                        [Model] Model configurations
    ├── model_opt.py                            [Model] Threshold optimization
    ├── model_syntax.py                         [Model] Bitsream specification
    ├── model_transforms.py                     [Model] Transforms configurations
    ├── model_types.py                          [Model] Model types
    ├── mp_report.py                            [MPEG] Generate JSON report for a G-PCC folder
    ├── mp_run.py                               [MPEG] Run G-PCC for the experimental configuration
    ├── tr_train.py                             [Train] Train a compression model
    ├── tr_train_all.py                         [Train] Train compression models for the experimental configuration
    ├── ut_build_paper.py                       [Utils] Create LaTeX tables and gather figures
    ├── ut_run_render.py                        [Utils] Render point clouds and visual comparisons
    ├── ut_tensorboard_plots.py                 [Utils] Generate plots from tfevents data
    └── utils
        ├── bd.py                               BD-Rate and BD-PSNR computation
        ├── colorbar.py                         Colorbar generation
        ├── experiment.py                       Experimental utilities
        ├── focal_loss.py                       Focal loss
        ├── matplotlib_utils.py                 Matplotlib utilities
        ├── mpeg_parsing.py                     MPEG log files parsing
        ├── o3d.py                              Open3D utilities (rendering)
        ├── octree_coding.py                    Octree coding
        ├── parallel_process.py                 Parallel processing
        ├── patch_gaussian_conditional.py       Patch tensorflow-compression GaussianConditional to get debug tensors
        ├── pc_io.py                            Point Cloud Input/Output
        ├── pc_metric.py                        Point Cloud geometry distortion metrics (D1 and D2)
        ├── pc_to_camera_params.py              Generate camera parameters for a Point Cloud
        └── pc_to_img.py                        Convert a Point Cloud to an image using predefined camera parameters

Citation

@misc{quach2020improved,
    title={Improved Deep Point Cloud Geometry Compression},
    author={Maurice Quach and Giuseppe Valenzise and Frederic Dufaux},
    year={2020},
    eprint={2006.09043},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

Improved Deep Point Cloud Geometry Compression

Experimental data

Prerequisites

Getting started

Configuration

Datasets

Parameterized experiments

Manual experiments

Overview

Citation

Files

README.md

Latest commit

History

README.md

File metadata and controls

Improved Deep Point Cloud Geometry Compression

Experimental data

Prerequisites

Getting started

Configuration

Datasets

Parameterized experiments

Manual experiments

Overview

Citation