Description
Hello @luebbe and @bertmelis
I am happy because thanks to espMqttClient library I have finally managed to add TLS to MQTT to my ESP32 project and it is working great.
I have started from the example for TLS MQTT fo ESP32 and while I tested it isolated it was working great but when I included TLS in my code I had 3 big issues. I am reporting the issues and how I was able to go around with some strategies.
My ESP32 includes:
- Webserver implemented with this method called ESP32-data-to-local-webpage
- OTA implemented with this repository, called ESP32-update-both-OTA-Firmware-and-OTA-SPIFFS-with-Arduino
- HTTPS POST requests to some php scripts on my server implemented with this repository called ESP32-Arduino-HTTPS-POST-request-to-PHP-MySQL-database
To introduce the TLS code I had to make some changes to my MQTT code that I had from async-mqtt-client library
The changes made are below:
#include "espMqttClient.h" //instead than AsyncMqttClient.h
/*add your real certificate here*/
const char rootCA[] PROGMEM = "-----BEGIN CERTIFICATE-----\n" \
"MIIDwzCCAqugAwIBAgIUbTivEoSEFfAFAFWfzwFSFSFESFESFSEFSEQYJKoZIhvcNAQEL\n"\
"BQAwcTELMAkGA1UEBhMCSVQxDzANBgNVBADSF4EFAEFAEFMA0GA1UEBwDVmVy\n"\
"bouESFSFESFSEChNw==\n"\
"-----END CERTIFICATE-----\n";
espMqttClientSecure mqttClient; //insted of AsyncMqttClient mqttClient
// before it was void onMqttDisconnect(AsyncMqttClientDisconnectReason reason) now it changed as below
void onMqttDisconnect(espMqttClientTypes::DisconnectReason reason) ;
//before the function was void onMqttSubscribe(uint16_t packetId, uint8_t qos) now it is
void onMqttSubscribe(uint16_t packetId, const espMqttClientTypes::SubscribeReturncode* codes, size_t len) ;
// before the function was void onMqttMessage(char* topic, char* payload, AsyncMqttClientMessageProperties properties, size_t len, size_t index, size_t total) now it is as below
void onMqttMessage(const espMqttClientTypes::MessageProperties& properties, const char* topic, const uint8_t* payload, size_t len, size_t index, size_t total);
void launchMQTT() {
mqttReconnectTimer = xTimerCreate("mqttTimer", pdMS_TO_TICKS(2000), pdFALSE, (void*)0, reinterpret_cast<TimerCallbackFunction_t>(connectToMqtt)); //I make sure MQTT will reconnect if it loses connection
mqttClient.setCACert(rootCA); //added line
mqttClient.setCredentials(MQTT_USERNAME, MQTT_PASSWORD);
mqttClient.onConnect(onMqttConnect);
mqttClient.onDisconnect(onMqttDisconnect);
mqttClient.onSubscribe(onMqttSubscribe);
mqttClient.onUnsubscribe(onMqttUnsubscribe);
mqttClient.onPublish(onMqttPublish);
mqttClient.onMessage(onMqttMessage);
mqttClient.setServer(MQTT_HOST, MQTT_PORT);
mqttClient.setCleanSession(true); //added line
connectToMqtt();
}
Once I uploaded the code with these variations I had many other things that stopped working:
ISSUE 1: web server gave me back a blank page, I had already experience with it. For details look at this my old issue.
The string for the webpage from now started to go into overflow. I did't have enough heap because TLS MQTT was taking I think 52KB of RAM while connecting
I have sorted like this. sending the webpage in pieces
server.sendHeader("Cache-Control", "no-cache, no-store, must-revalidate");
server.sendHeader("Pragma", "no-cache");
server.sendHeader("Expires", "-1");
server.setContentLength(CONTENT_LENGTH_UNKNOWN);
server.send(200, "text/html", ""); // here begin chunked transfer
server.sendContent("<html> <head> <title> ..................."); //put a piece of your webpage here
server.sendContent("<body> <row> ...................</html>"); //put another piece of your webpage here
server.client().stop();
Serial.printf("Free heap: %u\n", ESP.getFreeHeap());
ISSUE 2: OTA HTTPS call stopped to work and I could not update my ESP32 wirelessly, error below
I have solved disconnecting MQTT just before OTA and putting a delay before launching the update function
disconnectMQTT()
delay(3000);
handleUpdate();
setFlagUpdate(false);
I have the timer to reconnect MQTT so I just disconnected than it will automatically reconnect
ISSUE 3:
The more problematic issues. Post request returned error -1 as in the picture
I have solved disconnecting MQTT before each post request and putting a small delay
setPostInProgress(true);
disconnectMQTT();
delay(100);
This is the full code for the post request.
#include <HTTPClient.h>
#include <WiFiClientSecure.h>
bool sendCounterDataPhp()
{
setPostInProgress(true);
disconnectMQTT();
delay(100);
bool successFlag = false;
WiFiClientSecure *client3 = new WiFiClientSecure;
if (client3) {
client3->setCACert(rootCACert);
{ // Add a scoping block for HTTPClient https to make sure it is destroyed before WiFiClientSecure *client is
HTTPClient https;
Serial.print("[HTTPS] begin...\n");
if (https.begin(*client3, "PUT_YOUR_WEBLINK_TO_YOUR_phpScript.php")) {
/*start POST request*/
https.addHeader("Content-Type", "application/x-www-form-urlencoded");
String sendCounters = "my data";
Serial.println(sendCounters);
int httpResponseCode = https.POST(sendCounters);
if (httpResponseCode > 0) {
String response = https.getString(); //Get the response to the request
Serial.println(httpResponseCode); //Print return code
Serial.println(response); //Print request answer
response.trim(); //remove spaces and carrige return from string
setMyResponse(response);
successFlag = true;
}
else {
Serial.print("Error on sending POST: ");
Serial.println(httpResponseCode);
}
/*end POST request*/
https.end();
}
else {
Serial.printf("[HTTPS] Unable to connect\n");
}
}//for loop
} // End extra scoping block
delete client3;
}
else {
Serial.println("Unable to create client");
}
if (successFlag) {
setPostInProgress(false);
return true;
}
else {
setPostInProgress(false);
return false;
}
}
So my conclusion is that setting the MQTT TLS certificate was going into conflict with my other certificates that I use for HTTPS OTA to retrive my .bin file and for HTTPS POSTs. All the libraries from what I understand use WiFiClientSecure.h
The certificates were not the same and not on the same server. My MQTT mosquitto server has a certificate, my other server - where I have the .bin files - have another certificate.
I believe having two certificates was making the situation messy so I just disconnected MQTT every time I made another call.
I hope this helps as I have worked a lot to sort my code out. Now all works great, MQTT is encrypted and everything I had is working as before.