##Check out our new ULSD-ISPRS

Unified Line Segment Detection

This repository contains the official PyTorch implementation of the paper: ULSD: Unified Line Segment Detection across Pinhole, Fisheye, and Spherical Cameras.

Introduction

ULSD is a unified line segment detection method for both distorted and undistorted images from pinhole, fisheye or spherical cameras. With a novel line segment representation based on the Bezier curve, our method can detect arbitrarily distorted line segments. Experimental results on the pinhole, fisheye, and spherical image datasets validate the superiority of the proposed ULSD to the SOTA methods both in accuracy and efficiency.

Network Architecture

Results

Pinhole Image Datasets

Quantitative Comparisons

Method	Wireframe Dataset							YorkUrban Dataset							FPS
	sAP5	sAP10	sAP15	msAP	mAPJ	APH	FH	sAP5	sAP10	sAP15	msAP	mAPJ	APH	FH
LSD	8.3	10.8	12.7	10.6	17.2	54.3	61.5	8.5	10.6	12.2	10.4	15.4	49.7	60.0	50.9
DWP	5.8	7.6	8.8	7.4	38.6	65.9	72.2	2.3	3.2	4.1	3.2	23.4	51.6	62.3	2.3
AFM	21.2	26.8	30.2	26.1	24.3	70.1	77.0	8.0	10.3	12.1	10.1	12.5	48.5	63.2	14.3
L-CNN	60.7	64.1	65.6	63.5	59.3	80.3	76.9	25.3	27.2	28.5	27.0	30.3	57.8	61.6	13.7
HAWP	64.5	67.7	69.2	67.1	60.2	83.2	80.2	27.3	29.5	30.8	29.2	31.7	58.8	64.8	30.9
ULSD1(ours)	65.3	69.0	70.6	68.3	61.6	82.3	80.6	26.6	29.2	30.9	28.9	31.3	56.6	63.4	38.3
ULSD2(ours)	65.3	69.2	70.9	68.5	61.5	82.5	80.4	27.3	30.2	32.0	29.8	32.3	56.6	63.6	37.2
ULSD3(ours)	65.0	68.9	70.5	68.1	61.6	82.2	80.1	26.1	28.6	30.4	28.4	31.0	56.1	63.3	37.6
ULSD4(ours)	65.3	69.2	70.9	68.5	61.4	82.2	80.4	27.7	30.4	32.0	30.0	31.5	56.9	63.8	37.2

Qualitative Comparisons

More Results of ULSD

Fisheye Image Datasets

Quantitative Comparisons

Method	F-Wireframe Dataset					F-YorkUrban Dataset					FPS
	sAP5	sAP10	sAP15	msAP	mAPJ	sAP5	sAP10	sAP15	msAP	mAPJ
SHT	0.4	0.9	1.3	0.8	2.2	0.4	0.8	1.1	0.8	2.3	0.3
L-CNN	40.0	43.4	45.2	42.9	44.2	18.2	19.9	20.8	19.6	26.5	14.3
HAWP	42.5	46.3	48.0	45.6	43.8	19.5	21.5	22.5	21.2	26.4	31.5
ULSD2(ours)	59.4	64.3	66.3	63.3	56.4	27.7	30.7	32.4	30.3	33.6	37.2
ULSD3(ours)	59.7	64.7	66.7	63.7	56.0	27.1	30.2	32.0	29.8	32.9	37.1
ULSD4(ours)	59.4	64.3	66.3	63.3	56.1	28.8	32.0	33.8	31.5	33.9	36.9

Qualitative Comparisons

More Results of ULSD

Spherical Image Dataset

Quantitative Comparisons

Method	SUN360 Dataset					FPS
	sAP5	sAP10	sAP15	msAP	mAPJ
SHT	0.8	1.6	2.3	1.5	4.0	0.2
L-CNN	39.8	42.4	43.6	41.9	41.2	13.4
HAWP	41.6	44.7	45.8	44.0	39.2	25.4
ULSD2(ours)	63.1	69.4	71.5	68.0	57.8	23.9
ULSD3(ours)	61.9	69.1	71.1	67.3	57.2	23.7
ULSD4(ours)	63.8	70.1	71.8	68.6	57.9	23.8

Qualitative Comparisons

More Results of ULSD

Video

Requirements

• python3
• pytorch==1.6.0
• CUDA==10.1
• opencv, numpy, scipy, matplotlib, argparse, yacs, tqdm, json, multiprocessing, sklearn, tensorboardX

Step-by-step installation

conda create --name ulsd python=3.7
conda activate ulsd

cd <ulsd-path>
git clone https://github.com/lh9171338/Unified-Line-Segment-Detection.git
cd Unified-Line-Segment-Detection

pip install -r requirements.txt
conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.1 -c pytorch

Quickstart with the pretrained model

• There are 3 pretrained models (pinhole.pkl, fisheye.pkl, and spherical.pkl) in Google drive . Please download them and put in the model/ folder.
• There are some testing images in dataset/ folder.

python test.py --config_file pinhole.yaml --dataset_name pinhole --save_image

• The result is saved in output/ folder.

Training & Testing

Data Preparation

• Download the json-format dataset.
• Convert the dataset from json-format to npz-format.

cd dataset/
python json2npz.py --config_file fisheye.yaml --dataset_name fwireframe --order 2

• Generate the ground truth for evaluation.

cd dataset/
python json2npz_gt.py --config_file fisheye.yaml --dataset_name fwireframe

Train

python train.py --config_file fisheye.yaml --dataset_name fwireframe --order 2 [--gpu 0]

Test

python test.py --config_file fisheye.yaml --dataset_name fwireframe --order 2 --model_name best.pkl [--gpu 0] [--save_image] [--evaluate]

Evaluation

• Evaluate mAP^J, sAP, and FPS

python test.py --config_file pinhole.yaml --dataset_name wireframe --evaluate

• Evaluate AP^H

cd metric/
python eval_APH.py --config_file pinhole.yaml --dataset_name wireframe

Citation

@misc{li2020ulsd,
      title={ULSD: Unified Line Segment Detection across Pinhole, Fisheye, and Spherical Cameras}, 
      author={Hao Li and Huai Yu and Wen Yang and Lei Yu and Sebastian Scherer},
      year={2020},
      eprint={2011.03174},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}