Skip to content

Commit

Permalink
re-organizing repo
Browse files Browse the repository at this point in the history
  • Loading branch information
Cartucho committed Mar 16, 2019
1 parent 6a14cf3 commit eb8a5ac
Show file tree
Hide file tree
Showing 271 changed files with 102 additions and 48 deletions.
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# mAP (mean Average Precision)

[![New](https://img.shields.io/badge/2018-NEW-brightgreen.svg)](https://github.com/Cartucho/mAP/commits/master)
[![GitHub stars](https://img.shields.io/github/stars/Cartucho/mAP.svg?style=social&label=Stars)](https://github.com/Cartucho/mAP)

This code will evaluate the performance of your neural net for object recognition.
Expand Down Expand Up @@ -74,43 +73,44 @@ git clone https://github.com/Cartucho/mAP
Step by step:

1. [Create the ground-truth files](#create-the-ground-truth-files)
2. Move the ground-truth files into the folder **ground-truth/**
3. [Create the predicted objects files](#create-the-predicted-objects-files)
4. Move the predictions files into the folder **predicted/**
2. Copy the ground-truth files into the folder **input/ground-truth/**
3. [Create the detection-result files](#create-the-detection-result-files)
4. Copy the detection-result files into the folder **input/predicted/**
5. Run the code:
```
python main.py
```

Optional (if you want to see the **animation**):

6. Insert the images into the folder **images/**
6. Insert the images into the folder **input/images/**


#### PASCAL VOC, Darkflow and YOLO users

In the [extra](https://github.com/Cartucho/mAP/tree/master/extra) folder you can find additional scripts to convert **PASCAL VOC**, **darkflow** and **YOLO** files into the required format.
In the [scripts/extra](https://github.com/Cartucho/mAP/tree/master/scripts/extra) folder you can find additional scripts to convert **PASCAL VOC**, **darkflow** and **YOLO** files into the required format.

#### Create the ground-truth files

- Create a separate ground-truth text file for each image.
- Use **matching names** (e.g. image: "image_1.jpg", ground-truth: "image_1.txt"; "image_2.jpg", "image_2.txt"...).
- Use **matching names** for the files (e.g. image: "image_1.jpg", ground-truth: "image_1.txt").
- In these files, each line should be in the following format:
```
<class_name> <left> <top> <right> <bottom> [<difficult>]
```
- The `difficult` parameter is optional, use it if you want to ignore a specific prediction.
- The `difficult` parameter is optional, use it if you want the calculation to ignore a specific detection.
- E.g. "image_1.txt":
```
tvmonitor 2 10 173 238
book 439 157 556 241
book 437 246 518 351 difficult
pottedplant 272 190 316 259
```
#### Create the predicted objects files
- Create a separate predicted objects text file for each image.
- Use **matching names** (e.g. image: "image_1.jpg", predicted: "image_1.txt"; "image_2.jpg", "image_2.txt"...).
#### Create the detection-result files
- Create a separate detection-result text file for each image.
- Use **matching names** for the files (e.g. image: "image_1.jpg", detection-result: "image_1.txt").
- In these files, each line should be in the following format:
```
<class_name> <confidence> <left> <top> <right> <bottom>
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
45 changes: 32 additions & 13 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,18 @@
parser.add_argument('--set-class-iou', nargs='+', type=str, help="set IoU for a specific class.")
args = parser.parse_args()

'''
0,0 ------> x (width)
|
| (Left,Top)
| *_________
| | |
| |
y |_________|
(height) *
(Right,Bottom)
'''

# if there are no classes to ignore then replace None by empty list
if args.ignore is None:
args.ignore = []
Expand All @@ -29,10 +41,15 @@
if args.set_class_iou is not None:
specific_iou_flagged = True

# make sure that the cwd() is the location of the python script (so that every path makes sense)
os.chdir(os.path.dirname(os.path.abspath(__file__)))

GT_PATH = os.path.join(os.getcwd(), 'input', 'ground-truth')
DR_PATH = os.path.join(os.getcwd(), 'input', 'predicted')
# if there are no images then no animation can be shown
img_path = 'images'
if os.path.exists(img_path):
for dirpath, dirnames, files in os.walk(img_path):
IMG_PATH = os.path.join(os.getcwd(), 'input', 'images')
if os.path.exists(IMG_PATH):
for dirpath, dirnames, files in os.walk(IMG_PATH):
if not files:
# no image files found
args.no_animation = True
Expand Down Expand Up @@ -337,7 +354,7 @@ def draw_plot_func(dictionary, n_classes, window_title, plot_title, x_label, out
Create a list of all the class names present in the ground-truth (gt_classes).
"""
# get a list with the ground-truth files
ground_truth_files_list = glob.glob('ground-truth/*.txt')
ground_truth_files_list = glob.glob(GT_PATH + '/*.txt')
if len(ground_truth_files_list) == 0:
error("Error: No ground-truth files found!")
ground_truth_files_list.sort()
Expand All @@ -347,11 +364,12 @@ def draw_plot_func(dictionary, n_classes, window_title, plot_title, x_label, out

for txt_file in ground_truth_files_list:
#print(txt_file)
file_id = txt_file.split(".txt",1)[0]
file_id = txt_file.split(".txt", 1)[0]
file_id = os.path.basename(os.path.normpath(file_id))
# check if there is a correspondent predicted objects file
if not os.path.exists('predicted/' + file_id + ".txt"):
error_msg = "Error. File not found: predicted/" + file_id + ".txt\n"
temp_path = os.path.join(DR_PATH, (file_id + ".txt"))
if not os.path.exists(temp_path):
error_msg = "Error. File not found: {}\n".format(temp_path)
error_msg += "(You can avoid this error message by running extra/intersect-gt-and-pred.py)"
error(error_msg)
lines_list = file_lines_to_list(txt_file)
Expand Down Expand Up @@ -438,7 +456,7 @@ def draw_plot_func(dictionary, n_classes, window_title, plot_title, x_label, out
Load each of the predicted files into a temporary ".json" file.
"""
# get a list with the predicted files
predicted_files_list = glob.glob('predicted/*.txt')
predicted_files_list = glob.glob(DR_PATH + '/*.txt')
predicted_files_list.sort()

for class_index, class_name in enumerate(gt_classes):
Expand All @@ -448,9 +466,10 @@ def draw_plot_func(dictionary, n_classes, window_title, plot_title, x_label, out
# the first time it checks if all the corresponding ground-truth files exist
file_id = txt_file.split(".txt",1)[0]
file_id = os.path.basename(os.path.normpath(file_id))
temp_path = os.path.join(GT_PATH, (file_id + ".txt"))
if class_index == 0:
if not os.path.exists('ground-truth/' + file_id + ".txt"):
error_msg = "Error. File not found: ground-truth/" + file_id + ".txt\n"
if not os.path.exists(temp_path):
error_msg = "Error. File not found: {}\n".format(temp_path)
error_msg += "(You can avoid this error message by running extra/intersect-gt-and-pred.py)"
error(error_msg)
lines = file_lines_to_list(txt_file)
Expand Down Expand Up @@ -500,16 +519,16 @@ def draw_plot_func(dictionary, n_classes, window_title, plot_title, x_label, out
file_id = prediction["file_id"]
if show_animation:
# find ground truth image
ground_truth_img = glob.glob1(img_path, file_id + ".*")
ground_truth_img = glob.glob1(IMG_PATH, file_id + ".*")
#tifCounter = len(glob.glob1(myPath,"*.tif"))
if len(ground_truth_img) == 0:
error("Error. Image not found with id: " + file_id)
elif len(ground_truth_img) > 1:
error("Error. Multiple image with id: " + file_id)
else: # found image
#print(img_path + "/" + ground_truth_img[0])
#print(IMG_PATH + "/" + ground_truth_img[0])
# Load image
img = cv2.imread(img_path + "/" + ground_truth_img[0])
img = cv2.imread(IMG_PATH + "/" + ground_truth_img[0])
# load image with draws of multiple detections
img_cumulative_path = results_files_path + "/images/" + ground_truth_img[0]
if os.path.isfile(img_cumulative_path):
Expand Down
File renamed without changes.
File renamed without changes.
12 changes: 8 additions & 4 deletions extra/convert_gt_xml.py → scripts/extra/convert_gt_xml.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,15 @@
import glob
import xml.etree.ElementTree as ET

# make sure that the cwd() in the beginning is the location of the python script (so that every path makes sense)
os.chdir(os.path.dirname(os.path.abspath(__file__)))

# change directory to the one with the files to be changed
path_to_folder = '../ground-truth'
#print(path_to_folder)
os.chdir(path_to_folder)
parent_path = os.path.abspath(os.path.join(os.getcwd(), os.pardir))
parent_path = os.path.abspath(os.path.join(parent_path, os.pardir))
GT_PATH = os.path.join(parent_path, 'input','ground-truth')
#print(GT_PATH)
os.chdir(GT_PATH)

# old files (xml format) will be moved to a "backup" folder
## create the backup dir if it doesn't exist already
Expand All @@ -33,5 +37,5 @@
bottom = bndbox.find('ymax').text
new_f.write("%s %s %s %s %s\n" % (obj_name, left, top, right, bottom))
# 2. move old file (xml format) to backup
os.rename(tmp_file, "backup/" + tmp_file)
os.rename(tmp_file, os.path.join(backup, tmp_file))
print("Conversion completed!")
12 changes: 8 additions & 4 deletions extra/convert_gt_yolo.py → scripts/extra/convert_gt_yolo.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
import glob
import cv2


def convert_yolo_coordinates_to_voc(x_c_n, y_c_n, width_n, height_n, img_width, img_height):
## remove normalization given the size of the image
x_c = float(x_c_n) * img_width
Expand All @@ -21,6 +20,9 @@ def convert_yolo_coordinates_to_voc(x_c_n, y_c_n, width_n, height_n, img_width,
bottom = int(y_c + half_height) + 1
return left, top, right, bottom

# make sure that the cwd() in the beginning is the location of the python script (so that every path makes sense)
os.chdir(os.path.dirname(os.path.abspath(__file__)))

# read the class_list.txt to a list
with open("class_list.txt") as f:
obj_list = f.readlines()
Expand All @@ -30,9 +32,11 @@ def convert_yolo_coordinates_to_voc(x_c_n, y_c_n, width_n, height_n, img_width,
#print(obj_list[0])

# change directory to the one with the files to be changed
path_to_folder = '../ground-truth'
#print(path_to_folder)
os.chdir(path_to_folder)
parent_path = os.path.abspath(os.path.join(os.getcwd(), os.pardir))
parent_path = os.path.abspath(os.path.join(parent_path, os.pardir))
GT_PATH = os.path.join(parent_path, 'input','ground-truth')
#print(GT_PATH)
os.chdir(GT_PATH)

# old files (YOLO format) will be moved to a new folder (backup/)
## create the backup dir if it doesn't exist already
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,15 @@
import glob
import json

# make sure that the cwd() in the beginning is the location of the python script (so that every path makes sense)
os.chdir(os.path.dirname(os.path.abspath(__file__)))

# change directory to the one with the files to be changed
path_to_folder = '../predicted'
#print(path_to_folder)
os.chdir(path_to_folder)
parent_path = os.path.abspath(os.path.join(os.getcwd(), os.pardir))
parent_path = os.path.abspath(os.path.join(parent_path, os.pardir))
DR_PATH = os.path.join(parent_path, 'input','predicted')
#print(DR_PATH)
os.chdir(DR_PATH)

# old files (darkflow json format) will be moved to a "backup" folder
## create the backup dir if it doesn't exist already
Expand Down
13 changes: 11 additions & 2 deletions extra/convert_pred_yolo.py → scripts/extra/convert_pred_yolo.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
import os
import re

# make sure that the cwd() in the beginning is the location of the python script (so that every path makes sense)
os.chdir(os.path.dirname(os.path.abspath(__file__)))

IN_FILE = 'result.txt'
OUTPUT_DIR = os.path.join('..', 'predicted')

# change directory to the one with the files to be changed
parent_path = os.path.abspath(os.path.join(os.getcwd(), os.pardir))
parent_path = os.path.abspath(os.path.join(parent_path, os.pardir))
DR_PATH = os.path.join(parent_path, 'input','predicted')
#print(DR_PATH)
os.chdir(DR_PATH)

SEPARATOR_KEY = 'Enter Image Path:'
IMG_FORMAT = '.jpg'
Expand All @@ -22,7 +31,7 @@
if outfile is not None:
outfile.close()
# open a new file
outfile = open(os.path.join(OUTPUT_DIR, image_name + '.txt'), 'w')
outfile = open(os.path.join(DR_PATH, image_name + '.txt'), 'w')
elif outfile is not None:
# split line on first occurrence of the character ':' and '%'
class_name, info = line.split(':', 1)
Expand Down
12 changes: 10 additions & 2 deletions extra/find_class.py → scripts/extra/find_class.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
import os
import glob

# make sure that the cwd() in the beginning is the location of the python script (so that every path makes sense)
os.chdir(os.path.dirname(os.path.abspath(__file__)))

if len(sys.argv) != 2:
print("Error: wrong format.\nUsage: python find_class.py [class_name]")
sys.exit(0)
Expand Down Expand Up @@ -29,9 +32,14 @@ def find_class(class_name):
if not file_found:
print(" No file found with that class")

parent_path = os.path.abspath(os.path.join(os.getcwd(), os.pardir))
parent_path = os.path.abspath(os.path.join(parent_path, os.pardir))
GT_PATH = os.path.join(parent_path, 'input','ground-truth')
DR_PATH = os.path.join(parent_path, 'input','predicted')

print("Ground-Truth folder:")
os.chdir("../ground-truth")
os.chdir(GT_PATH)
find_class(searching_class_name)
print("\nPredicted folder:")
os.chdir("../predicted")
os.chdir(DR_PATH)
find_class(searching_class_name)
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,26 @@
## not in the intersection into a backup folder (backup_no_matches_found).
## This will retain only files that have the same name in both folders.

# change directory to the one with the files to be changed
path_to_gt = '../ground-truth'
path_to_pred = '../predicted'

# make sure that the cwd() in the beginning is the location of the python script (so that every path makes sense)
os.chdir(os.path.dirname(os.path.abspath(__file__)))

parent_path = os.path.abspath(os.path.join(os.getcwd(), os.pardir))
parent_path = os.path.abspath(os.path.join(parent_path, os.pardir))
GT_PATH = os.path.join(parent_path, 'input','ground-truth')
DR_PATH = os.path.join(parent_path, 'input','predicted')

backup_folder = 'backup_no_matches_found' # must end without slash

os.chdir(path_to_gt)
os.chdir(GT_PATH)
gt_files = glob.glob('*.txt')
if len(gt_files) == 0:
print("Error: no .txt files found in", path_to_gt)
print("Error: no .txt files found in", GT_PATH)
sys.exit()
os.chdir(path_to_pred)
os.chdir(DR_PATH)
pred_files = glob.glob('*.txt')
if len(pred_files) == 0:
print("Error: no .txt files found in", path_to_pred)
print("Error: no .txt files found in", DR_PATH)
sys.exit()

gt_files = set(gt_files)
Expand All @@ -46,8 +52,8 @@ def backup(src_folder, backup_files, backup_folder):
for file in backup_files:
os.rename(file, backup_folder + '/' + file)

backup(path_to_gt, gt_backup, backup_folder)
backup(path_to_pred, pred_backup, backup_folder)
backup(GT_PATH, gt_backup, backup_folder)
backup(DR_PATH, pred_backup, backup_folder)
if gt_backup:
print('total ground-truth backup files:', len(gt_backup))
if pred_backup:
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 comments on commit eb8a5ac

Please sign in to comment.