adds fresh download links and metadata readmes

README.md

@@ -36,11 +36,19 @@ We recommend using the mattersim [Dockerfile](Dockerfile) to install the simulat
 
 ### CVDN Dataset Download
 
+Download the `train`, `val_seen`, `val_unseen`, and `test` splits of the whole CVDN dataset by executing the following script:
+```
+tasks/CVDN/data/download.sh
+```
+
+### NDH Task Data Download
+
 Download the `train`, `val_seen`, `val_unseen`, and `test` splits for NDH by executing the following script:
 ```
 tasks/NDH/data/download.sh
 ```
 
+
 ### Matterport3D Dataset Download
 
 To use the simulator you must first download the [Matterport3D Dataset](https://niessner.github.io/Matterport/) which is available after requesting access [here](https://niessner.github.io/Matterport/). The download script that will be provided allows for downloading of selected data types. 

tasks/CVDN/data/README.md

@@ -0,0 +1,32 @@
+# Cooperative Vision-and-Dialog Navigation (CVDN) Dataset
+
+A brief overview of the metadata available in each CVDN instance:
+
+#### Always Available:
+| Metadata | Explanation |
+|---|---|
+| `idx` | The unique index of this dialog. |
+| `scan` | The unique scan ID of the house in which this dialog took place. |
+| `target` | The target object. |
+| `start_pano` | The navigation node from which the dialog began. |
+
+#### Only Available at Training Time:
+| Training Metadata | Explanation |
+|---|---|
+| `problem` | The number (0-2) of workers who reported technical problems in this game. |
+| `end_panos` | The navigation nodes that compose the end region. |
+| `navigator` | The unique (across dialogs) ID assigned to the navigator when this game began. |
+| `navigator_mturk` | The unique worker ID of the navigator. |
+| `navigator_quality` | The 1-5 rating received by the navigator from the oracle in this game. |
+| `navigator_avg_quality` | The average 1-5 rating received by the navigator across all games in which they were involved. |
+| `oracle` | The unique (across dialogs) ID assigned to the oracle when this game began. |
+| `oracle_mturk` | The unique worker ID of the oracle. |
+| `oracle_quality` | The 1-5 rating received by the oracle from the navigator in this game. |
+| `oracle_avg_quality` | The average 1-5 rating received by the oracle across all games in which they were involved. |
+| `nav_camera` | Indexed by the `nav_idx` into the `nav_steps` and `dia_idx` into the `dialog_history` when the corresponding question was asked. The `message` is a list of camera heading adjustments that occurred since the navigator moved to the most recent navigation node (i.e., looking around before asking a question). |
+| `nav_steps` | The navigation nodes traversed by the navigator to the end region. |
+| `dialog_history` | A list of turns. Each turn has a `nav_idx` (the `player_path` list index where the utterance was transmitted), a `role` (either 'oracle' or 'navigator'), and a `message` (the utterance). |
+| `stop_history` | The `nav_steps` indices where the navigator guessed they had reached the goal region. |
+| `planner_nav_steps` | The navigation nodes along the shortest path from the starting point to the end region. | 
+| `R2R_spl` | The R2R SPL metric of the nav steps against the end region. |
+| `R2R_oracle_spl` | The R2R SPL metric calculated as though the player path stopped within three meters of a node in the end region, if they passed near it. |

tasks/CVDN/data/download.sh

@@ -0,0 +1,5 @@
+#!/bin/sh
+
+wget https://cvdn.dev/dataset/CVDN/train_val/train.json -P tasks/NDH/data/
+wget https://cvdn.dev/dataset/CVDN/train_val/val_seen.json -P tasks/NDH/data/
+wget https://cvdn.dev/dataset/CVDN/train_val/val_unseen.json -P tasks/NDH/data/

tasks/NDH/data/README.md

@@ -0,0 +1,30 @@
+# Navigation from Dialog History (NDH) Task Data
+
+A brief overview of the metadata available in each NDH instance:
+
+#### Always Available:
+| Metadata | Explanation |
+|---|---|
+| `inst_idx` | The unique index of this task instance. |
+| `scan` | The unique scan ID of the house in which this instance took place. |
+| `target` | The target object for the dialog this instance was drawn from. |
+| `start_pano` | The `heading`, `elevation`, and panorama id `pano` of the position from which the navigator asked the last question. |
+| `nav_camera` | A list of camera heading adjustments that occurred since the navigator moved to the most recent navigation node (i.e., looking around before asking a question). |
+| `dialog_history` | A list of turns. Each turn has a `nav_idx` (the `nav_history` list index where the utterance was transmitted), a `role` (either 'oracle' or 'navigator'), and a `message` (the utterance). |
+| `nav_history` | The navigation nodes traversed by the navigator before the latest question. |
+
+#### Only Available at Training Time:
+| Training Metadata | Explanation |
+|---|---|
+| `game_idx` | The unique index of the dialog from which this instance was drawn. |
+| `end_panos` | The navigation nodes that compose the end region. |
+| `player_path` | The navigation nodes traversed by the navigator in response to the latest answer. |
+| `planner_path` | The navigation nodes shown to the oracle in response to the most recent question (first 5 shortest path steps towards the `end_panos`, if there is no dialog history). | 
+| `navigator_game_quality` | The 1-5 rating received by the navigator from the oracle in this game. |
+| `navigator_avg_quality` | The average 1-5 rating received by the navigator across all games in which they were involved. |
+| `oracle_game_quality` | The 1-5 rating received by the oracle from the navigator in this game. |
+| `oracle_avg_quality` | The average 1-5 rating received by the oracle across all games in which they were involved. |
+| `R2R_success` |  The R2R success metric of the player path calculated against the last node in the planner path. |
+| `R2R_spl` | The R2R SPL metric of the player path against the end node of the planner path. |
+| `R2R_oracle_success` | The R2R success metric calculated as though the player path stopped within three meters of the last node in the planner path, or 0 if it never got close. |
+| `R2R_oracle_spl` | The R2R SPL metric calculated as though the player path stopped within three meters of the last node in the planner path, or 0 if it never got close. |

tasks/NDH/data/download.sh

@@ -1,5 +1,5 @@
 #!/bin/sh
 
-wget https://mp-dialog.s3-us-west-2.amazonaws.com/data/v1/ndh/train.json -P tasks/NDH/data/
-wget https://mp-dialog.s3-us-west-2.amazonaws.com/data/v1/ndh/val_seen.json -P tasks/NDH/data/
-wget https://mp-dialog.s3-us-west-2.amazonaws.com/data/v1/ndh/val_unseen.json -P tasks/NDH/data/
+wget https://cvdn.dev/dataset/NDH/train_val/train.json -P tasks/NDH/data/
+wget https://cvdn.dev/dataset/NDH/train_val/val_seen.json -P tasks/NDH/data/
+wget https://cvdn.dev/dataset/NDH/train_val/val_unseen.json -P tasks/NDH/data/