data-ingest-and-monitoring updates (#88)

* data-ingest-and-monitoring updates * updates requested by Dave
wmo-im · Oct 3, 2023 · 105cadf · 105cadf
1 parent 4129c30
commit 105cadf
Show file tree

Hide file tree

Showing 17 changed files with 196 additions and 108 deletions.
diff --git a/documentation/docs/assets/img/grafana-homepage.png b/documentation/docs/assets/img/grafana-homepage.png
diff --git a/documentation/docs/assets/img/grafana_success.png b/documentation/docs/assets/img/grafana_success.png
diff --git a/documentation/docs/assets/img/minio-admin-create-new-path.png b/documentation/docs/assets/img/minio-admin-create-new-path.png
diff --git a/documentation/docs/assets/img/minio-admin-uploaded-bufr-file.png b/documentation/docs/assets/img/minio-admin-uploaded-bufr-file.png
diff --git a/documentation/docs/assets/img/minio-admin-uploaded-file.png b/documentation/docs/assets/img/minio-admin-uploaded-file.png
diff --git a/documentation/docs/assets/img/minio-second-tab.png b/documentation/docs/assets/img/minio-second-tab.png
diff --git a/documentation/docs/practical-sessions/accessing-your-student-vm.md b/documentation/docs/practical-sessions/accessing-your-student-vm.md
@@ -4,9 +4,13 @@ title: Accessing your student VM
 
 # Accessing your student VM
 
-!!! abstract
+!!! abstract "Learning outcomes"
 
-    In this session you will practice accessing your student VM using SSH and WinSCP.
+    By the end of this session, you will be able to:
+
+    - access your student VM over SSH and WinSCP
+    - verify the required software for the practical exercises is installed
+    - verify you have access to exercise materials for this training on your local student VM
 
 ## Introduction
 
@@ -22,7 +26,7 @@ Your student VM has the following software pre-installed:
 
 !!! note
 
-    If you want to run this training outside of a local training session, you can provide your own instance using any Cloud Provider:
+    If you want to run this training outside of a local training session, you can provide your own instance using any Cloud Provider, for example:
 
     - GCP (Google Cloud Platform) VM instance `e2-medium`
     - AWS (Amazon Web Services)  ec2-instance `t3a.medium` 
@@ -39,13 +43,21 @@ Your student VM has the following software pre-installed:
 
     You can always find the latest 'wis2box-setup' archive at [https://github.com/wmo-im/wis2box/releases](https://github.com/wmo-im/wis2box/releases).
 
-    The exercise material as used in this training can be downloaded as follows:
+    The exercise material used in this training can be downloaded as follows:
 
     ```bash
     wget https://training.wis2box.wis.wmo.int/exercise-materials.zip
     unzip exercise-materials.zip
     ```
 
+    The following additional python modules are required to run the exercise materials:
+
+    ```bash
+    pip3 install minio
+    pip3 install pywiscat
+    pip3 install pywis-pubsub
+    ```
+
     If you are using the student VM provided during local WIS2 training sessions, the required software will already be installed.
 
 ## Connect to your student VM on the local training network

diff --git a/documentation/docs/practical-sessions/configuring-station-metadata.md b/documentation/docs/practical-sessions/configuring-station-metadata.md
@@ -4,9 +4,14 @@ title: Configuring station metadata
 
 # Configuring station metadata
 
-!!! abstract
+!!! abstract "Learning outcomes"
 
-    In this session you will practice configuring station metadata for your dataset.
+    By the end of this session, you will be able to:
+
+    - create an authorization token for the `collections/stations` endpoint
+    - add station metadata to wis2box
+    - review stations associated to datasets in the **wis2box-ui**
+    - update/delete station metadata using the **wis2box-webapp**
 
 ## Introduction
 

diff --git a/documentation/docs/practical-sessions/connecting-to-mqtt.md b/documentation/docs/practical-sessions/connecting-to-mqtt.md
@@ -4,9 +4,13 @@ title: Connecting to WIS2 over MQTT
 
 # Connecting to WIS2 over MQTT
 
-!!! abstract
+!!! Learning outcomes
 
-    In this session you will practice connecting to the WIS2 Global Broker using MQTT Explorer.
+    By the end of this session, you will be able to:
+
+    - connect to the WIS2 Global Broker using MQTT Explorer
+    - review the WIS2 topic structure
+    - review the WIS2 notification message structure
 
 ## Introduction
 

diff --git a/documentation/docs/practical-sessions/data-ingest-and-monitoring.md b/documentation/docs/practical-sessions/data-ingest-and-monitoring.md
diff --git a/documentation/docs/practical-sessions/initializing-wis2box.md b/documentation/docs/practical-sessions/initializing-wis2box.md
@@ -188,7 +188,7 @@ This is the wis2box API (running via the **wis2box-api** container).
 
      What collections are currently available?
 
-??? note "Click to reveal answer"
+??? success "Click to reveal answer"
 
     To view collections currently available through the API, click `View the collections in this service`:
 
@@ -204,7 +204,7 @@ This is the wis2box API (running via the **wis2box-api** container).
 
     How many data notifications have been published?
 
-??? note "Click to reveal answer"
+??? success "Click to reveal answer"
 
     Click on "Data notifications", then click on `Browse through the items of "Data Notifications"`. 
 

diff --git a/documentation/docs/sample-data/WIGOS_0-454-2-AWSBILIRA_new.csv b/documentation/docs/sample-data/WIGOS_0-454-2-AWSBILIRA_new.csv
diff --git a/documentation/docs/sample-data/WIGOS_0-454-2-AWSCHIKANGAWA_new.csv b/documentation/docs/sample-data/WIGOS_0-454-2-AWSCHIKANGAWA_new.csv
diff --git a/documentation/docs/sample-data/aws-example.csv b/documentation/docs/sample-data/aws-example.csv
@@ -0,0 +1,4 @@
+wsi_series,wsi_issuer,wsi_issue_number,wsi_local,wmo_block_number,wmo_station_number,station_type,year,month,day,hour,minute,latitude,longitude,station_height_above_msl,barometer_height_above_msl,station_pressure,msl_pressure,geopotential_height,thermometer_height,air_temperature,dewpoint_temperature,relative_humidity,method_of_ground_state_measurement,ground_state,method_of_snow_depth_measurement,snow_depth,precipitation_intensity,anemometer_height,time_period_of_wind,wind_direction,wind_speed,maximum_wind_gust_direction_10_minutes,maximum_wind_gust_speed_10_minutes,maximum_wind_gust_direction_1_hour,maximum_wind_gust_speed_1_hour,maximum_wind_gust_direction_3_hours,maximum_wind_gust_speed_3_hours,rain_sensor_height,total_precipitation_1_hour,total_precipitation_3_hours,total_precipitation_6_hours,total_precipitation_12_hours,total_precipitation_24_hours
+0,20000,0,60351,15,15,1,2022,3,31,1,0,47.77706163,23.94046026,503,,100940,101040,1448,5,298.15,294.55,80,3,1,1,0,0.004,10,-10,30,3,30,5,40,9,20,11,2,4.7,5.3,7.9,9.5,11.4
+0,20000,0,60355,15,15,1,2022,3,31,1,0,47.77706163,23.94046026,503,504.43,100940,101040,1448,5,298.15,294.55,80,3,1,1,0,0.004,10,-10,30,3,30,5,40,9,20,11,2,4.7,5.3,7.9,9.5,11.4
+0,20000,0,60360,15,15,1,2022,3,31,1,0,47.77706163,23.94046026,503,504.43,100940,101040,1448,5,298.15,294.55,80,3,1,1,0,0.004,10,-10,30,3,30,5,40,9,20,11,2,4.7,5.3,7.9,9.5,11.4
diff --git a/documentation/docs/sample-data/bufr-example.bin b/documentation/docs/sample-data/bufr-example.bin
diff --git a/documentation/docs/sample-data/synop-202307.txt b/documentation/docs/sample-data/synop-202307.txt
@@ -0,0 +1,4 @@
+SICG20 FCBB 030900
+AAXX 03094
+60355 42460 71004 10285 20245 30113 40133 8493/
+    333 59005 83813 81930 87363 94966 95836=
diff --git a/documentation/docs/sample-data/synop-202308.txt b/documentation/docs/sample-data/synop-202308.txt
@@ -0,0 +1,4 @@
+SICG20 FCBB 030900
+AAXX 03094
+60355 42460 71004 10285 20245 30113 40133 8493/
+    333 59005 83813 81930 87363 94966 95836=