[FR] Secure MQTT for ESP32 Devices

[chr2002qw]

Will ESP Easy get MQTT over SSL like Tasmota has.

On ESP32 Devices Secure MQTT works fine at Tasmota. With ESP8266 its possible, too but very very unstable.

How about adding SSL to ESPeasy ?

[TD-er]

I have been working on it: #3788
The reason it has not yet been saved is because of the extreme strange hoops I had to jump through for MbedTLS to make sure it doesn't leak memory.

You can try the last test build made for this PR: https://github.com/letscontrolit/ESPEasy/actions/runs/5968797472

[chr2002qw]

How can i download this build for the common ESP32 ? Theres no Link to a File.

[TD-er]

I literally put a link for you to download.
See this screenshot:

I highlighted the ESP32 download link

[chr2002qw]

Oh yes, found it. My Browser had Issues. Used another and it worked. Just flashed this. But i cant see any SSL or TLS Stuff in controller Settings.

This is the current firmware :

Build:⋄	20230824 - Mega32
System Libraries:⋄	ESP32 SDK 4.4.5.230614
Git Build:⋄	HEAD_567931a
Plugin Count:⋄	48 [Normal]
Build Origin:	GitHub Actions
Build Time:⋄	Aug 24 2023 23:03:24
Binary Filename:⋄	ESP_Easy_mega_20230824_normal_ESP32_4M316k
Build Platform:⋄	Linux-5.15.0-1041-azure-x86_64-with-glibc2.35
Git HEAD:⋄	HEAD_567931a

[TD-er]

You need to test either the "max" build (for 16M flash units) or the "custom" build (which doesn't have all plugins included).
But since it is a test for the MQTT controller, you can simply test with "dummy" or "sysinfo" plugins to test the controller.

[chr2002qw]

Hmm, on Max. Wifi is broken

RESET: Warm boot, reset count: 1
RESET: formatting...
RESET: formatting done...
RESET: FORMAT FS FAILED!
3943 : Info : ESPEasy console using ESPEasySerial
3944 : Info : INIT : Free RAM:224852
3978 : Info : ESPEasy console using ESPEasySerial
3980 : Info : INFO : Plugins: 144 [Normal][MAX ESP32] (ESP32 SDK 4.4.5.230614)
3982 : Error : WIFI : No valid wifi settings
3983 : Info : WIFI : Arduino wifi status: WL_NO_SHIELD 255 ESPeasy internal wifi status: DISCONNECTED
3988 : Info : Webserver: start
3989 : Info : WiFi : WiFiConnected(), start AP
4093 : Info : WIFI : Set WiFi to AP
4095 : Info : WIFI : Cannot set mode!!!!!
4196 : Info : WIFI : Cannot set mode!!!!!
4298 : Info : WIFI : mode not yet set
4398 : Info : WIFI : mode not yet set
4601 : Error : WIFI : [AP] softAPConfig failed!
4603 : Error : WIFI : Error while starting AP Mode with SSID: ESPEasy- IP: 192.168.4.1
4706 : Info : WIFI : Set WiFi to STA
4707 : Info : WIFI : Cannot set mode!!!!!
4808 : Info : WIFI : Cannot set mode!!!!!
4909 : Info : WIFI : mode not yet set
5009 : Info : WIFI : mode not yet set
5211 : Info : WiFi : WiFiConnected(), start AP
5314 : Info : WIFI : Set WiFi to AP
5315 : Info : WIFI : Cannot set mode!!!!!
5416 : Info : WIFI : Cannot set mode!!!!!
5517 : Info : WIFI : mode not yet set
5617 : Info : WIFI : mode not yet set
5820 : Error : WIFI : [AP] softAPConfig failed!
5821 : Error : WIFI : Error while starting AP Mode with SSID: ESPEasy- IP: 192.168.4.1
5823 : Info : WIFI : Arduino wifi status: WL_NO_SHIELD 255 ESPeasy internal wifi status: DISCONNECTED
5913 : Info : WD : Uptime 0 ConnectFailures 0 FreeMem 217680 WiFiStatus WL_NO_SHIELD 255 ESPeasy internal wifi status: DISCONNECTED
35334 : Info : WD : Uptime 1 ConnectFailures 0 FreeMem 217528 WiFiStatus WL_NO_SHIELD 255 ESPeasy internal wifi status: DISCONNECTED
65342 : Info : WD : Uptime 1 ConnectFailures 0 FreeMem 217528 WiFiStatus WL_NO_SHIELD 255 ESPeasy internal wifi status: DISCONNECTED

[TD-er]

Hmm are you sure about the power supply of that unit?
Can you just try to set the WiFi credentials via the serial port and save them, then power cycle the ESP?

wifissid,YourWiFiSSID
wifikey,YourSecretWiFiPassword
save
wifidisconnect

WiFiDisconnect will then force a WiFi restart
If that's not working, you can power cycle the ESP.

[chr2002qw]

Doesnt work.

I´ll drop on this. I need a stable solution for MQTT TLS and will wait until its merged,
Theres a Problem with the FS on the Max Builds. So i need to wait.

[chr2002qw]

Seems to work with "custom" but whyyyyyyyyyyyyyyyyyyyy MQTT-Import is removed :-( :-(
I Only need Import and Framed Display.

[TD-er]

OK, but that's something we can change....
Which build do you need?

[chr2002qw]

I flashed this "ESP_Easy_mega_20230824_custom_ESP32_4M316k.factory.bin" for testing. It seems, MQTT-TLS gets connected 👍

Is it easy to add MQTT-Import to that build ?

Thank you 👍

[TD-er]

I will have a look later this evening.
It sure isn't hard to do (maybe Ton will do it as he always has some window open with the ESPEasy code :) ), but I have to get behind a PC that won't take forever to make a build.

[TD-er]

Just thought about a quick work-around :)

I just edited the file describing the Custom builds on ESP32 and triggered a GH Actions build:
https://github.com/letscontrolit/ESPEasy/actions/runs/6123368939

So it will be ready in about 40 - 45 minutes.

[chr2002qw]

Had to do a break from this stuff due to massive work :-(
I tried it, and this is working very good with SSL and my external MQTT broker.

Today i tried to compile a custom myself and checked out the Code with this PR. But i get following error in PIO if i try to compile a custom build :

"error initializations for multiple members of 'ProtocolStruct::"

Can i compile a build myself or is that not as esay with this PRed Version ?

[TD-er]

Depending on what time you pulled the code, you may have pulled the code that still had a merge conflict.
I solved that later this morning.

[chr2002qw]

I pulled again and the error gone 👍 But iam to dumb to compile it. I switched to custom_env and renamed the custom-sample.h and selected the Plugins i need.
Then i selected the controller "Homeasisstant MQTT" The bin compiles w/o errors and was uploaded to the ESP32.
But no Wifi Hotspot is showing up. No Idea what iam doing wrong :-(

EDIT: I managed to get the Wifi up Via Serial.

But i think, it makes no sense to try to compile a Firmware with this PR and a custom.h.
Its still incomplete and some stuff is missing. I cant find SSL in my build.
I wait and hope until this is merged to master, then i´ll try again.

The PR works fine with my Broker, MQTT Import and framed oled.

[TD-er]

Maybe it is easier to remove your Custom.h file and just edit the file tools/pio/pre_custom_esp32.py
As you can see in that file, there is a line like this:

"-DFEATURE_MQTT_TLS=1",

This define will enable the feature MQTT TLS.

Or just add to your Custom.h:

#define FEATURE_MQTT_TLS   1

[chr2002qw]

Thanks, this worked 👍 Everything is running very good with that PR on my ESP32. Not tested with ESP8266.