SfMLearner Chainer version

This codebase implements the system described in the paper:

Unsupervised Learning of Depth and Ego-Motion from Video link
See the project webpage for more details.

TF code: https://github.com/tinghuiz/SfMLearner

Preparing training data

In order to train the model using the provided code, the data needs to be formatted in a certain manner.

Depth

For KiTTI, first download the dataset using this script provided on the official website, and then run the following command

python data/prepare_train_data.py /path/to/KITTI_raw --dataset-format kitti_raw --static-frames ./data/static_frames.txt  --dump-root /path/to/KITTI_formatted --height 128 --width 416 --num-threads 8

Odometry

This script generates only training data.
Remove '2011_09_26_drive_0067' sequence because there is no data at kitti server.

python data/prepare_train_data.py /path/to/KITTI_raw --dataset-format kitti_odom --static-frames ./data/static_frames.txt  --dump-root /path/to/KITTI_formatted --height 128 --width 416 --num-threads 8

Training using KiTTI Raw Dataset

Once the data are formatted following the above instructions, you should be able to train the model by running the following command

Depth

python3 train.py experiments/sfm_learner_v1.yml

Odometry

python3 train.py experiments/sfm_learner_v1_odom.yml

Evaluation using KiTTI Raw Dataset

If you finish training models using above scripts, you should be able to evaluate your trained model.

Depth

You can obtain the single-view depth predictions on the KITTI eigen test split formatted properly for evaluation by running.
You could download pretrained model from here

python evaluate.py experiments/sfm_learner_v1_eval.yml

Odometry

You can obtain the 5-snipped odometry predictions on the KITTI odometry dataset. This scripts use kitti raw dataset directly.

python evaluate.py experiments/sfm_learner_v1_odom_eval.yml --mode odom

abs_rel	sq_rel	rms	log_rms	a1	a2	a3
0.1779	1.3594	6.2696	0.2570	0.7390	0.9075	0.9647

Inference using KiTTI Raw Dataset

Depth

You could download pretrained model from here

# For kitti formatted dataset
python inference.py experiments/sfm_learner_v1_test.yml
# For a image
python inference.py experiments/sfm_learner_v1_test.yml --img_path /path/to/img --save 1 --width 416 --height 128

odometry

# Create predicted trajectory
python inference.py experiments/sfm_learner_v1_odom_test.yml --mode odom
# Visualize trajectories
python inference.py experiments/sfm_learner_v1_odom_test.yml --mode odom --gt_file ./kitti_eval/pose_data/ground_truth/10_full.txt --pred_file ./test.txt