This repository contains the official implementation of LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraints
Requires:
- Python ≥ 3.6
- PyTorch ≥ 1.6
pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113
pip3 install -r requirements.txt
The latest version of Argoverse requires Python ≥ 3.7
If using Python 3.6, you can install Argoverse v1.0
https://github.com/argoai/argoverse-api
pip3 install nuscenes-devkit
-
Download the nuScenes dataset. For this project we just need the following.
- Metadata for the Trainval split (v1.0)
- Map expansion pack (v1.3)
-
Organize the nuScenes root directory as follows
└── nuScenes/
├── maps/
| ├── basemaps/
| ├── expansion/
| ├── prediction/
| ├── 36092f0b03a857c6a3403e25b4b7aab3.png
| ├── 37819e65e09e5547b8a3ceaefba56bb2.png
| ├── 53992ee3023e5494b90c316c183be829.png
| └── 93406b464a165eaba6d9de76ca09f5da.png
└── v1.0-trainval
├── attribute.json
├── calibrated_sensor.json
...
└── visibility.json
Suppose the training data of Argoverse motion forecasting is at ./train/data/
.
OUTPUT_DIR=checkpoints/models.laformer.1; \
GPU_NUM=8; \
python src/train.py --future_frame_num 30 --do_train --data_dir ./train/data/ \
--output_dir ${OUTPUT_DIR} --topk 2 --hidden_size 128 --train_all --vector_size 32 \
--train_batch_size 128 --use_map --num_train_epochs 10 --lane_loss_weight 10 --core_num 32 \
--subdivide_length 5 --use_centerline --distributed_training ${GPU_NUM} \
--other_params semantic_lane direction step_lane_score enhance_global_graph point_level-4-3
Add --reuse_temp_file to skip re-listing the map file for the second time running or use --temp_file_dir ${path} to specify the pre-processed data path.
Suppose the training data of Argoverse motion forecasting is at ./train/data/
.
OUTPUT_DIR=checkpoints/models.laformer.s2.1; \
GPU_NUM=8; \
MODEL_PATH=checkpoints/models.laformer.1/model_save/model.10.bin; \
python src/train.py --future_frame_num 30 --do_train --data_dir ./train/data/ \
--output_dir ${OUTPUT_DIR} --hidden_size 128 --topk 2 \
--vector_size 32 --train_batch_size 128 --use_map --num_train_epochs 9 \
--lane_loss_weight 10 --core_num 32 --subdivide_length 5 --learning_rate 0.0003\
--train_all --use_centerline --distributed_training ${GPU_NUM} \
--other_params semantic_lane direction step_lane_score enhance_global_graph \
point_level-4-3 stage-two-train_recover=${MODEL_PATH} stage-two-epoch=9 stage_two
Suppose the validation data of Argoverse motion forecasting is at ./val/data/
.
Example:
OUTPUT_DIR=checkpoints/models.laformer.s2.1; \
GPU_NUM=1; \
python src/eval.py --future_frame_num 30 --eval_batch_size 128\
--output_dir ${OUTPUT_DIR} --hidden_size 128 --train_batch_size 128 \
--lane_loss_weight 10 --topk 2 --use_map --vector_size 32 --model_recover_path 9 \
--core_num 16 --use_centerline --distributed_training ${GPU_NUM}\
--subdivide_length 5 --other_params semantic_lane direction step_lane_score enhance_global_graph \
point_level-4-3 stage-two-train_recover=${MODEL_PATH} stage-two-epoch=9 stage_two \
--do_eval --data_dir_for_val ./val/data/
- Run the following script to extract pre-processed data. This speeds up training significantly.
python src/datascripts/dataloader_nuscenes.py --DATAROOT path/to/nuScenes/root/directory --STOREDIR path/to/directory/with/preprocessed/data
Suppose the processed training data of NuScenes is at checkpoints/models.laformer.nuscenes.1/temp_file
.
OUTPUT_DIR=checkpoints/models.laformer.nuscenes.1; \
GPU_NUM=8; \
python src/train.py --do_train --future_frame_num 12 --output_dir ${OUTPUT_DIR} \
--topk 2 --hidden_size 64 --train_batch_size 32 --num_train_epochs 50 \
--lane_loss_weight 0.9 --distributed_training ${GPU_NUM} --reuse_temp_file \
--other_params semantic_lane direction step_lane_score enhance_global_graph \
point_level-4-3 nuscenes nuscenes_mode_num=5
Add --reuse_temp_file or use --temp_file_dir ${path} to specify the pre-processed data path.
Suppose the processed training data of NuScenes is at checkpoints/models.laformer.nuscenes.1/temp_file
.
OUTPUT_DIR=checkpoints/models.laformer.nuscenes.1; \
GPU_NUM=8; \
MODEL_PATH=checkpoints/models.laformer.1/model_save/model.50.bin; \
python src/train.py --future_frame_num 12 --do_train --output_dir ${OUTPUT_DIR}\
--hidden_size 64 --topk 2 --train_batch_size 32 --num_train_epochs 50 \
--lane_loss_weight 0.9 --reuse_temp_file --distributed_training ${GPU_NUM} \
--other_params semantic_lane direction step_lane_score enhance_global_graph \
point_level-4-3 stage-two-train_recover=${MODEL_PATH} stage-two-epoch=50 stage_two \
nuscenes nuscenes_mode_num=5
To train a model with 10 modes, you can set nuscenes_mode_num=10, --topk 4, and --num_train_epochs 100, the other settings is the same as aforementioned command.
Suppose the processed test data of NuScenes is at checkpoints/models.laformer.nuscenes.1/temp_file
.
Example:
OUTPUT_DIR=checkpoints/models.laformer.nuscenes.1; \
GPU_NUM=1; \
MODEL_PATH=checkpoints/models.laformer.1/model_save/model.50.bin; \
python src/eval.py --future_frame_num 12 --eval_batch_size 128 \
--output_dir ${OUTPUT_DIR} --hidden_size 64 --train_batch_size 32 \
--lane_loss_weight 0.9 --topk 2 --reuse_temp_file --model_recover_path 50 \
--distributed_training ${GPU_NUM} --other_params step_lane_score stage_two \
semantic_lane direction enhance_global_graph subdivide new laneGCN \
point_level-4-3 stage-two-train_recover=${MODEL_PATH} stage-two-epoch=50 \
nuscenes nuscenes_mode_num=5 --do_eval
We provide the pretrained LAformer in checkpoints/. You can evaluate the pretrained models using the aforementioned evaluation command.
For this repository, the expected performance on Argoverse 1.1 validation set is:
Models | minADE | minFDE | MR |
---|---|---|---|
LAformer | 0.64 | 0.92 | 0.08 |
The expected performance on nuScenes Test set is:
Models | minADE_5 | minFDE_5 | minADE_10 | minFDE_10 |
---|---|---|---|---|
LAformer | 1.19 | 2.31 | 0.93 | 1.50 |
If you found this repository useful, please consider citing our work:
@inproceedings{liu2024laformer,
title={Laformer: Trajectory prediction for autonomous driving with lane-aware scene constraints},
author={Liu, Mengmeng and Cheng, Hao and Chen, Lin and Broszio, Hellward and Li, Jiangtao and Zhao, Runjiang and Sester, Monika and Yang, Michael Ying},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={2039--2049},
year={2024}
}
This repository is licensed under Apache 2.0.