Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -149,4 +149,10 @@ phoenix*
debug/
*projs/

core.*
core.*

# mmcv package
mmcv

# vscode config file
.vscode/
49 changes: 0 additions & 49 deletions .vscode/launch.json

This file was deleted.

20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ UniAD is trained in two stages. Pretrained checkpoints of both stages will be re

| Method | Encoder | Tracking<br>AMOTA | Mapping<br>IoU-lane | config | Download |
| :---: | :---: | :---: | :---: | :---:|:---:|
| UniAD-B | R101 | 0.390 | 0.297 | [base-stage1](projects/configs/stage1_track_map/base_track_map.py) | [base-stage1](https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/uniad_base_track_map.pth) |
| UniAD-B | R101 | 0.394 | 0.294 | [base-stage1](projects/configs/stage1_track_map/base_track_map.py) | [base-stage1](https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/uniad_base_track_map.pth) |



Expand All @@ -107,7 +107,7 @@ Pre-trained models and results under main metrics are provided below. We refer y

| Method | Encoder | Tracking<br>AMOTA | Mapping<br>IoU-lane | Motion<br>minADE |Occupancy<br>IoU-n. | Planning<br>avg.Col. | config | Download |
| :---: | :---: | :---: | :---: | :---:|:---:| :---: | :---: | :---: |
| UniAD-B | R101 | 0.363 | 0.313 | 0.705 | 63.7 | 0.29 | [base-stage2](projects/configs/stage2_e2e/base_e2e.py) | [base-stage2](https://github.com/OpenDriveLab/UniAD/releases/download/v1.0.1/uniad_base_e2e.pth) |
| UniAD-B | R101 | 0.380 | 0.314 | 0.794 | 64.0 | 0.29 | [base-stage2](projects/configs/stage2_e2e/base_e2e.py) | [base-stage2](https://github.com/OpenDriveLab/UniAD/releases/download/v1.0.1/uniad_base_e2e.pth) |

> Planning results on the nuScense benchmark

Expand All @@ -131,14 +131,14 @@ Pre-trained models and results under main metrics are provided below. We refer y
<tr>
<td>UniAD-B</td>
<td>R101</td>
<td>0.48</td>
<td>0.96</td>
<td>1.65</td>
<td>1.03</td>
<td>0.05</td>
<td>0.17</td>
<td>0.71</td>
<td>0.31</td>
<td>0.29</td>
<td>0.89</td>
<td>1.53</td>
<td>0.90</td>
<td>0.15</td>
<td>0.20</td>
<td>0.53</td>
<td>0.29</td>
</tr>
</table>

Expand Down
13 changes: 9 additions & 4 deletions docs/DATA_PREP.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,13 @@ mkdir data

**Prepare UniAD data info**

*Option1: We have already prepared the off-the-shelf data infos for you:*
*Option1: We have already prepared the off-the-shelf data infos for you in [HuggingFace::OpenDriveLab/UniAD2.0_R101_nuScenes](https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/tree/main/data):*
```shell
cd UniAD/data
mkdir infos && cd infos
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/nuscenes_infos_temporal_train.pkl # train_infos
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/nuscenes_infos_temporal_val.pkl # val_infos
wget https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/resolve/main/data/nuscenes_infos_temporal_train.pkl # train_infos

wget https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/resolve/main/data/nuscenes_infos_temporal_val.pkl # val_infos
```


Expand All @@ -33,10 +34,12 @@ mkdir infos
```

**Prepare Motion Anchors**

We already upload motion anchors in [HuggingFace::OpenDriveLab/UniAD2.0_R101_nuScenes](https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/tree/main/data)
```shell
cd UniAD/data
mkdir others && cd others
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/motion_anchor_infos_mode6.pkl
wget https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/resolve/main/data/motion_anchor_infos_mode6.pkl
```

**The Overall Structure**
Expand All @@ -54,10 +57,12 @@ UniAD
│ ├── nuscenes/
│ │ ├── can_bus/
│ │ ├── maps/
│ │ ├── lidarseg/
│ │ ├── samples/
│ │ ├── sweeps/
│ │ ├── v1.0-test/
│ │ ├── v1.0-trainval/
│ │ ├── v1.0-mini/
│ ├── infos/
│ │ ├── nuscenes_infos_temporal_train.pkl
│ │ ├── nuscenes_infos_temporal_val.pkl
Expand Down
47 changes: 16 additions & 31 deletions docs/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,33 +12,14 @@ conda activate uniad2.0
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
```

**c. GCC: Make sure gcc>=5 in conda env.**
**c. Install mmcv-series packages.**
```shell
# If gcc is not installed:
# conda install -c omgarcia gcc-6 # gcc-6.2

export PATH=YOUR_GCC_PATH/bin:$PATH
# Eg: export PATH=/mnt/gcc-5.4/bin:$PATH
```

**d. CUDA: Before installing MMCV family, you need to set up the CUDA_HOME (for compiling some operators on the gpu).**
```shell
export CUDA_HOME=YOUR_CUDA_PATH/
# Eg: export CUDA_HOME=/mnt/cuda-11.8/
```


**e. Install mmcv-series packages.**
```shell
git clone https://github.com/open-mmlab/mmcv.git & cd mmcv
git checkout v1.6.0
export MMCV_WITH_OPS=1 MMCV_CUDA_ARGS=-std=c++17
pip install -v -e .
pip install -v mmcv-full==1.6.1 -f https://download.openmmlab.com/mmcv/dist/cu118/torch2.0/index.html
pip install mmdet==2.26.0 mmsegmentation==0.29.1 mmdet3d==1.0.0rc6
```


**h. Install UniAD.**
**d. Install UniAD.**
```shell
cd ~
git clone https://github.com/OpenDriveLab/UniAD.git
Expand All @@ -47,19 +28,23 @@ pip install -r requirements.txt
```


**i. Prepare pretrained weights.**
**e. Prepare pretrained weights.**

We release our pretrained weights in [HuggingFace::OpenDriveLab/UniAD2.0_R101_nuScenes](https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/tree/main/ckpts)

```shell
mkdir ckpts && cd ckpts
mkdir ckpts & cd ckpts
# r101_dcn_fcos3d_pretrain.pth (from bevformer)
wget https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/resolve/main/ckpts/r101_dcn_fcos3d_pretrain.pth

# Pretrained weights of bevformer
# Also the initial state of training stage1 model
wget https://github.com/zhiqi-li/storage/releases/download/v1.0/bevformer_r101_dcn_24ep.pth
# bevformer_r101_dcn_24ep.pth
wget https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/resolve/main/ckpts/bevformer_r101_dcn_24ep.pth

# Pretrained weights of stage1 model (perception part of UniAD)
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/uniad_base_track_map.pth
# uniad_base_track_map.pth
wget https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/resolve/main/ckpts/uniad_base_track_map.pth

# Pretrained weights of stage2 model (fully functional UniAD)
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0.1/uniad_base_e2e.pth
# uniad_base_e2e.pth
wget https://huggingface.co/OpenDriveLab/UniAD2.0_R101_nuScenes/resolve/main/ckpts/uniad_base_e2e.pth
```

---
Expand Down
17 changes: 13 additions & 4 deletions docs/TRAIN_EVAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Please make sure you have prepared the environment and the nuScenes dataset. You can check it by simply evaluating the pre-trained first-stage(track_map) model as follows:
```shell
cd UniAD
./tools/uniad_dist_eval.sh ./projects/configs/stage1_track_map/base_track_map.py ./ckpts/uniad_base_track_map.pth 8
./tools/uniad_dist_eval.sh ./projects/configs/stage1_track_map/base_track_map.py ./ckpts/uniad_base_track_map.pth 8 # (8 is the number of GPUs)

# For slurm users:
# ./tools/uniad_slurm_eval.sh YOUR_PARTITION ./projects/configs/stage1_track_map/base_track_map.py ./ckpts/uniad_base_track_map.pth 8
Expand All @@ -13,9 +13,9 @@ If everything is prepared properly, the output results should be:

```
Aggregated results:
AMOTA 0.390
AMOTP 1.300
RECALL 0.489
AMOTA 0.394
AMOTP 1.316
RECALL 0.484
```

**Note**: If you evaluate with different number of GPUs rather than 8, the results might be slightly different.
Expand Down Expand Up @@ -45,6 +45,15 @@ The first-stage training takes ~ 50 GB GPU memory, ~ 2 days for 6 epochs on 8 A1

The second-stage training takes ~ 17 GB GPU memory, ~ 4 days for 20 epochs on 8 A100 GPUs.
* **NOTE**: Compared to the first-stage, much less GPU memory is required because we freeze the BEV encoder in this stage to focus on learning task-specific queries. Due to this, you can run the second-stage training on V100 or 3090 devices. -->
### Bevformer Backbone Training
we integrated BEVFormer training (you also can use other BEV backbone) into the UniAD code repository in this version.
```shell
# Training
./tools/uniad_dist_train.sh ./projects/configs/bevformer/bevformer_base.py N_GPUS

# Evaluation
./tools/uniad_dist_train.sh ./projects/configs/bevformer/bevformer_base.py UniAD/ckpts/bevformer_r101_dcn_24ep.pth N_GPUS
```


### Training Command
Expand Down
Loading