Added disclaimer (nutonomy#58)

* Added disclaimer
* Added actual warning in the code

python-sdk/nuscenes/eval/README.md

@@ -1,6 +1,11 @@
 # nuScenes detection task
 In this document we present the rules, results format, classes, evaluation metrics and challenge tracks of the nuScenes detection task.
 
+
+## Disclaimer
+Please note that the challenge design and evaluation protocol described in this document are an initial proposal to discuss our design choices with the community.
+The final design will be released once the community feedback has been taken into account.
+
 ## Overview
 - [Introduction](#introduction)
 - [General rules](#general-rules)

python-sdk/nuscenes/eval/nuscenes_eval.py

@@ -8,6 +8,7 @@
 from typing import List, Tuple, Dict, Callable
 import random
 import argparse
+import warnings
 
 import numpy as np
 import tqdm
@@ -100,6 +101,9 @@ def __init__(self, nusc: NuScenes, result_path: str, eval_set: str, class_names:
         # Load and store GT and predictions.
         self.sample_tokens, self.all_annotations, self.all_results = self.load_boxes()
 
+        # Print a warning that the evaluation protocol is not finalized.
+        warnings.warn("Warning: Please note the evaluation protocol is not finalized and may be subject to change in the future.")
+
     def load_boxes(self) -> Tuple[List[str], Dict[str, List[Dict]], Dict[str, List[Dict]]]:
         """
         Loads the GT and EST boxes used in this class.