Datasets for Dex-NeRF

The repo contains the datasets for the paper:

"Dex-NeRF: Using a Neural Radiance field to Grasp Transparent Objects," Jeffrey Ichnowski*, Yahav Avigal*, Justin Kerr, and Ken Goldberg, Conference on Robot Learning (CoRL), 2020

The dataset zip files can be found in release section. The Blender source files for the simulated datasets can be found in a separate release section.

Physical Datasets (For Grasping)

• Bottle
• Flask
• Safety Glasses
• Tape Dispenser
• Wineglass
• Lion Figurine

Physical Datasets (Cluttered Scenes)

• Dishwasher
• Table

Simulated Datasets (Singulated)

The rendered images below were created using a single Blender file with rendering of objects enabled and disabled as appropriate.

• Mount1
• Pawn
• Pipe Connector
• Turbine Housing
• Wineglass (Upright)
• Wineglass (On Side)

Simulated Datasets (Cluttered Scenes)

• Cluttered with Light Array (Rendered images) (Blender file)
• Cluttered with Single Light (Rendered images) (Blender file)

Blender Notes

To match the render settings used for the datasets, use the Cycles renderer, and tune the Samples in the render settings. Also, tune the settings under Light Path / Max Bounces, particularly Transmission and Transparent. Higher settings generally produce more realistic renderings at the cost of compute time. To generate the datasets, we performed an exponential search comparing results until there was no difference in rendered results. With GPU-assisted rendering with the tuned settings, rendering all images takes many hours.

The singulated objects blend file contains all the objects used in the singulated scenes. To render different singulated objects, turn rendering of objects on/off as desired.

The Blender files have a keyframe for the camera to align with the transforms in the rendered datasets. Render Animation will thus render all 400 images needed for a dataset. Rendered frames 1 to 100 correspond to train/r_0.png to train/r_99.png. Frames 101 to 300 correspond to test/r_0.png to test/r_199.png. Frames 301 to 400 correspond to val/r_0.png to val/r_99.png. As an example, you can run the following shell script in the directory containing the rendered images to reorganize into the dataset directory structure:

mkdir train test val
for ((i=1 ; i<=400 ; ++i)) ; do
    file=$(printf "frame_%04d.png" "$i")
    if [ "$i" -le 100 ] ; then
        mv -v "$file" "train/r_$((i-1)).png"
    elif [ "$i" -le 300 ] ; then
        mv -v "$file" "test/r_$((i-101)).png"
    else
        mv -v "$file" "val/r_$((i-301)).png"
    fi
done

Citing

If you use this in your work, please cite:

@inproceedings{IchnowskiAvigal2021DexNeRF,
  title={{Dex-NeRF}: Using a Neural Radiance field to Grasp Transparent Objects},
  author={Ichnowski*, Jeffrey and Avigal*, Yahav and Kerr, Justin and Goldberg, Ken},
  booktitle={Conference on Robot Learning (CoRL)},
  year={2020}
}