Improved auto BLE reconnect and added state transition async updates

README.md

@@ -64,6 +64,52 @@ local Greengrass components.
 **Base Control Topic:** aws-greengrass/things/THING_NAME/ble/control/  
 **Error Response Topic:** aws-greengrass/things/THING_NAME/ble/error/  
 
+#### BLE Device State Transition Topic and Connection Workflow
+
+The AWS Greengrass Bluetooth (BLE) IoT Gateway manages the connection state of BLE devices that have received a connect request as 
+determined by thier MAC address. To connect or disconnect from a BLE device, publish the required message to the 
+connect and disconnect control topics respectiovly as described in the following sections. 
+
+Once a connect attempt has been requested, the AWS Greengrass Bluetooth (BLE) IoT Gateway will either successfully connect
+or continue re-trying the connection every 5 seconds until a disconnect message has been received and the BLE Device is 
+removed from the active devices list. The same is true if a connected device shuts down or becomes unreachable and is disconnected.
+
+The AWS Greengrass Bluetooth (BLE) IoT Gateway provides an asychronious connection state control topic which receives a 
+message every time a active BLE Device transistions from one connected state to another. 
+
+The State Transition Topic is: **aws-greengrass/things/THING_NAME/ble/state**
+Subscribe to this topic on either IPC or MQTT message busses to receive asychronious BLE device connectivity state transitions. 
+
+An example of a BLE state transition message is as per below: 
+```
+{
+  "control-command": "ble-conection-state-changed",
+  "ble-mac": "3C:61:05:12:EE:0A",
+  "updated": "20211209153659670182",
+  "data": {
+    "previous-state": {
+      "connection-state": "disc",
+      "addr-type": "N/A"
+    },
+    "current_state": {
+      "connection-state": "conn",
+      "addr-type": "public"
+    }
+  }
+}
+```
+
+Valid connection-states are:
+* **waiting-init** - Waiting for connection to be initilised,
+* **conn** - connected,
+* **disc** - disconnected, 
+* **scan** BLE scan in progress
+* **tryconn** - connecting.
+
+When connecting or disconecting to a BLE device, the AWS Greengrass Bluetooth (BLE) IoT Gateway will provide a synchronious (immediate) 
+acknowledgemnet response that the requst is being provessed on the connect/response and disconnect/resonse topics. This is just so
+applicactin logic can move to the next task while it monitors the state transition topic for the requested change to be inacted. 
+
 #### BLE Device Connect:  
 
 Requests the AWS Greengrass BLE Gateway attempt to pair with BLE device with the given MAC address. 
@@ -73,8 +119,12 @@ fails it will re-attempt to pair again until a disconnect request is received.
 The connected devices list does not survive a Greengrass device reboot. Future iterations will 
 write this to a local shadow instead but in this initial release, you must monitor BLE device connection state.
 
-**Note:** If the device doesn’t exist, it will continue to try for up to 30 seconds before returning a failed connection. 
-If the delay causes a problem for application logic then perform a scan first to ensure the device exists and is reachable.  
+**Note:** If the device doesn’t exist, it will continue to try for up to 40 seconds before returning a failed connection. 
+
+If the delay causes a problem for application logic then perform a scan first to ensure the device exists and is reachable.
+To minimise the impact of long lived connect processes, the AWS Greengrass BLE Gateway attempt provides an immediate 
+acknowledgement of this request in the connect/response topic, the application shold monitor the state transition topic 
+for an asynchronious update indicating the requesting connection has been made. 
 
 **Connect Request Topic:** aws-greengrass/things/**THING_NAME**/ble/control/connect  
 **Connect Response Topic:** aws-greengrass/things/**THING_NAME**/ble/control/connect/response  
@@ -89,14 +139,15 @@ If the delay causes a problem for application logic then perform a scan first to
 **Sample Responses:**  
 ```
 {
-    "status": 200,
-    "data": {
-        "ble-mac": "XX:XX:XX:XX:XX:XX",
-        "connect-status": "success"
-    }
+  "status": 200,
+  "data": {
+    "ble-mac": "3C:61:05:12:EE:0A",
+    "connect_status": "request-accepted"
+  }
 }
 ```
 
+
 **BLE Device Disconnect:**  
 
 The BLE Device disconnect command instructs the AWS Greengrass BLE Gateway to terminate 
@@ -115,14 +166,17 @@ the BLE connection if active and remove the BLE Device MAC from the locally main
 **Sample Response:**  
 ```
 {
-    "status": 200,
-    "data": {
-        "ble-mac": "XX:XX:XX:XX:XX:XX",
-        "disconnect-status": "success"
-    }
+  "status": 200,
+  "data": {
+    "ble-mac": "3C:61:05:12:EE:0A",
+    "disconnect_status": "request-accepted"
+  }
 }
 ```
 
+As per the state transition and connect sections above. This response is only an acknowledgemnet of the request to 
+disconnect a device. The application should monitor the state transition topic for when it has been completed. 
+
 **BLE Device List:**   
 The BLE Device list command returns the connection state of all devices in the connected device list. 
 That is, all BLE devices that have received a connect command and not a subsequent disconnect command. 
@@ -153,7 +207,10 @@ have had a connection request, scan will return information on all devices withi
 **BLE Device Scan:**  
 
 The BLE Device scan returns details of all BLE devices within range of the AWS Greengrass BLE Gateway. 
-By default, the scan is set for 5 seconds and so the response will take at least that long. 
+By default, the scan is set for 5 seconds and so the response will take at least that long.
+
+**Note:** Running a BLE Scan will disconnect existing BLE Peripherals. The application will automatically 
+attempt to re-connect once the scan is complete.
 
 BLE devices can advertise a number of supported data types including a description (BLE Ad-Type: 255) and 
 the BLE Complete Local Name (BLE Ad-Type: 9). In the ESP32 micro-python code sample provided, both of these are 
@@ -419,7 +476,29 @@ Record the Mac address of this device and send a connect command as described ab
 }
 ```
 
-2. Toggle LED:  
+2. Request entire message is reflected / returned in RX Data Topic (good to test latency and MTU).
+Request:
+```
+{
+  "command" : "reflect-message",
+  "data" : {
+    "Message 01" : "Some Data Message",
+    "Message 02" : "Some Other Data Message"
+  }
+}
+```
+Response:
+```
+{
+  "command" : "reflect-message",
+  "data" : {
+    "Message 01" : "Some Data Message",
+    "Message 02" : "Some Other Data Message"
+  }
+}
+```
+
+3. Toggle LED:  
 ```
 {
   "command" : "toggle_led"
@@ -436,7 +515,7 @@ Response:
 }
 ```
 
-3. Request BLE Device Processor Board Temperature
+4. Request BLE Device Processor Board Temperature
 ```
 {
   "command" : "get-processor-board-temp"
@@ -454,7 +533,7 @@ Response:
 }
 ```
 
-4. Request a HW reset of the BLE devices.
+5. Request a HW reset of the BLE devices.
 ```
 {
   "command" : "hw-reset-micro"

ble-test-messages.txt

@@ -12,7 +12,7 @@
 ### Request CONNECT BLE device:
 Publish Topic: aws-greengrass/things/THING_NAME/ble/control/connect
 Response Topic: aws-greengrass/things/THING_NAME/ble/control/connect/response
-Error Response Topic: aws-greengrass/things//THING_NAME/ble/error
+Error Response Topic: aws-greengrass/things/THING_NAME/ble/error
 
 Message: 
 {
@@ -118,17 +118,26 @@ to provide a response or action the request on the uController.
   "command" : "describe-ble-device"
 }
 
-2. Toggle LED:
+2. Request entire message is reflected / returned in RX Data Topic (good to test latency and MTU).
+{
+  "command" : "reflect-message",
+  "data" : {
+    "Message 01" : "Some Data Message",
+    "Message 02" : "Some Other Data Message"
+  }
+}
+
+3. Toggle LED:
 {
   "command" : "toggle_led"
 }
 
-2. Request BLE Device Processor Board Temperature
+4. Request BLE Device Processor Board Temperature
 {
   "command" : "get-processor-board-temp"
 }
 
-4. Request a HW reset of the BLE devices. 
+5. Request a HW reset of the BLE devices. 
 {
   "command" : "hw-reset-micro"
 }

esp32-ble-device/ble/ble_uart_peripheral.py

@@ -71,7 +71,7 @@ def __init__(self, ble_name="BLE_Device", ble_specific_id='BLE_Device', rxbuf=15
             self._rx_buffer = bytearray()
             self._handler = None
 
-            # Optionally add services=[_UART_UUID], but this is likely to make the payload too large.
+            # Optionally add services=[_UART_UUID], but this is possible to make the payload too large.
             self._payload = advertising_payload(name=ble_name, manufacturer_specific=ble_specific_id, appearance=_ADV_APPEARANCE_GENERIC_COMPUTER)
             self._advertise()
 
@@ -130,5 +130,5 @@ def close(self):
         self._connections.clear()
 
     def _advertise(self, interval_us=100000):
-        print("STARTED BLE ADVERTISMENT")
+        print('Starting BLE Advertisment from BLE Mac: {}'.format(self.ble_mac))
         self._ble.gap_advertise(interval_us, adv_data=self._payload)

esp32-ble-device/serial-link-control-notes.txt

@@ -0,0 +1,54 @@
+### Raspberry Pi USB connection to ESP32 Serial and Flash control
+###
+
+## Serial Terminal using SCREEN
+
+## To connect to serial interface (assuming ESP connect to Pi USB)
+# Note: May need to press boot button to get the terminal
+screen /dev/ttyUSB0 115200
+
+# Note: On Mac or linux the serial tty interface will be different.
+# i.e: On mac: /dev/cu.SLAB_USBtoUART
+
+# to Exit Screen session
+ctl a, d
+
+# To view active / detacted screen sessions 
+screen -ls
+
+# To reconect a Screen session
+screen -r [SCREEN SESSION ID]
+
+## Erase FLASH and load Micropython using ESPTOOL
+
+## To erase flash
+## Note: press both power and boot buttons together once requestng connection begins
+esptool.py --port /dev/ttyUSB0 erase_flash 
+
+# To load new Micropythin version:
+# Note: Don't click the boot button. should make the connection after a few seconds.
+cd [to dir with uPy imaage]
+esptool.py --chip esp32 --port /dev/ttyUSB0 write_flash -z 0x1000 esp32-20210902-v1.17.bin
+
+## To access Flash as mounted directory using RSHELL
+rshell -p /dev/ttyUSB0
+
+# ESP32 Flash will be mounted at /pyboard
+
+# To see ESP32 loaded files
+ls -l /pyboard
+
+# To delete, move, etc just as normal file system
+rm /pyboard/some-file
+mv /pyboard/some-file
+
+# or can cd into the directory
+cd /pyboard
+
+# To upload ESP32 code just copy as mormal
+cd esp32-ble-peripheral
+cp main.py /pyboard
+cp -r ble /pyboard
+
+# To exit RSHELL
+ctl a, \