Current status of MQTT SSL

Running latest master branch as of today, I'm not sure anymore if the SSL support still conflicts with the web UI regardless of the platform used (I'm building with 180).

It would be nice to have a current document on SSL on its own page, as well as tweaks to enable HTTP access for the UI but MQTT over SSL, to minimize the performance requirements.

[mcspr]

Some notes after testing:

• Core 2.4.2 / async-mqtt-client
  is ok'ish with both mqtt and web. can save settings, can view debug log. once crashed on page reload
• Core 2.4.2 / lwip2 (low/high) / pubsubclient / modified mqtt module to support bearssl wificlient
  works ok with basic payloads, but crashes with mere 260 bytes of json. upping pubsubclient mqtt packet buffer size does not help, neither is tuning in/out fragment length of the wificlient. web ok
• Core 2.5.0 beta (platform staging) / lwip2 low / same pubsubclient - works ok for both basic and json. could not get web to crash.

All tests done with local server, because I could not get cloudmqtt.com to work. It just crashes the board (or fails to connect at the best of times). Tried async/axtls builds, new sync/bearssl, even tried different firmware from mongoose os - nothing works with them :/

Pubsubclient + bearssl combo can also use CA cert instead of fingerprinting, either as hardcoded string or spiffs file. Also, fingerprinting api is different and there is no need to manually do it (client.connect, check fp, mqtt.setClient, mqtt.connect repeat)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue will be auto-closed because there hasn't been any activity for two months. Feel free to open a new one if you still experience this problem.

[Niek]

My experience with core 2.4.2:

• aync-mqtt-client works, but disconnects randomly when sending or receiving messages. I thought it was related to setWill() cause mqtt could not connect with SSL enable marvinroger/async-mqtt-client#107, but disabling setWill() doesn't help
• pubsubclient doesn't work at all for me, it keeps disconnecting at connection stage - maybe interfering with web? (edit: pretty sure this is due to conflicts with ESPAsyncTCP)
• pubsubclient + bearssl is no option because the binary size grows with 100KB+ - maybe using BearSSL::WiFiClientSecure results in both SSL variants to be compiled in?

I tried testing with core 2.5.0 as well, but unfortunately the binary grows too large to do OTA on a 1MB board - no matter which MQTT library I use.

[mcspr]

pretty sure this is due to conflicts with ESPAsyncTCP

Like, what conflicts? I've seen the comment, but I still do not quite understand what conflict is there.

In general with BearSSL:

• size grows, true. so is feature-set. OTA is possible only with two-stage, uploading small .bin first and then the full one. I glanced over it as my main use-case are Wemos boards with 4MB flash.
• X509 date constraint checks require proper timestamp. see

  espurna/code/espurna/blynk.ino

  Line 54 in 0949414

  setX509Time(ntpLocal2UTC(now()));

  for an example i did with blynk.
• another important thing not mentioned - you are pretty much required to change CPU speed to 160MHz and free up as much ram as possible.
  platformio.ini doc: https://docs.platformio.org/en/latest/platforms/espressif8266.html#cpu-frequency
  ArduinoIDE has a menu entry doing the same thing
• when running non-cloud solution, mosquitto with recent openssl, MFLN is an option: https://arduino-esp8266.readthedocs.io/en/latest/esp8266wifi/bearssl-client-secure-class.html#mfln-or-maximum-fragment-length-negotiation-saving-ram
  IDK if google cloud allows that. blynk's java server based on netty/boringssl, from example above, does not.

[Niek]

MFLN is interesting - if you compile your broker against OpenSSL 1.1 it should work out of the box. It should be quite easy to do a couple of quick probeMaxFragmentLength() calls in the MQTT module before connecting.

As for conflicts: whenever I try to use PubSubClient (MQTT_USE_ASYNC=0) I cannot connect to any MQTT broker - the _mqtt_client_secure.connect() call always fails. Since it's suggested in the header files, I assumed the WifiClientSecure conflicts with WifiClient used in ESPAsyncTCP.

[Niek]

Just an update after lots of experimentation:
PubSubClient SSL (MQTT_USE_ASYNC=0) does not work in any of my cases. When I compile with -DDEBUG_ESP_SSL -DDEBUG_ESP_TLS_MEM I see the following errors:

[MQTT] Connecting as user X
please start sntp first !
State:	sending Client Hello (1)
Error: connection lost

When using AsyncMqttClient, SSL works but only a few packets. It disconnects and then reconnects in a loop every x packets.
I tested with Arduino core 2.4.2 and 2.5.0 (axTLS only, BearSSL is too heavy) and LWIP 1.4 and 2.0 - no difference.