- dtu Gateway for Hoymiles HMS-800W-2T (2T series)
The new series of Hoymiles inverter with internal wireless access point and wireless client have no direct API to include this endpoint in smarthome installations/ IFTT environments.
Usually there should be no need for an extra device to "translate" the connection to common APIs or bindings. Unfortunately the interface on the dtu is unlikely unstable/ or not really stable.
E.g. there is a treshhold of ~ 31 seconds before you can send new data to the dtu (e.g. new power limit), otherwise the connection hangs and an internal restart/ reboot (???) leads to an offline time of ~ 30 minutes. Data from dtu can be read in a very short time, but it has to be tested how often a request leads to the problem before.
On a manual way you can be back on track, if you are logging in to the local access point of the dtu and resend your local wifi login data to (it seems) initiate a reboot. With this way you can be back online in ~ 1:30 minutes.
So I decided to put this abstraction in an ESP8266 to have a stable abstraction to an existing smart home environment.
hint: the whole project could be also implemented on a small server and translated to e.g. python see here for an example and also the sources below
- Abstract the interface to the dtu (inverter connection endpoint) with different possibilities to connect to other systems. (push/ pull)
- Very stable interface with no dependencies to an environment/ a system with a stand alone application based on an arduino board (ESP8266).
- TODO: Ability to change running wifi to connect to dtu over local network or direct access point.
- Use this need to create a full enivronment for an ESP8266 based project. (see features below)
- base connection to retrieve data from inverter e.g.
- power (Watts), voltage (V), current (A) for the PV input data (PV0,PV1) and the grid
- energy counter (kWh) for all 3 sources (day and total)
- temperature and wifi rssi of the dtu
- setting the target inverter power limit dynamically
- for testing purposes the time between each request is adjustable (default 31 seconds)
- syncing time of gateway with the local time of the dtu to prevent wrong restart counters
- configurable 'cloud pause' - see experiences - to prevent missing updates by the dtu to the hoymiles cloud
- automatic reboot of DTU, if there is an error detected (e.g. inplausible not changed values)
- serving the readed data per /api/data
- configuration of bindings with seperate activation and login data setting
- binding: updating openHab instance with readed data and pulling set data from the instance
- binding: updating to a MQTT broker with readed data incl. set PowerLimit over MQTT
- 2 ways to configure - simple mqtt publishing with base topic or together with HA MQTT AutoDiscovery based
- for all publishing retain flag is set (keeping last seen data in broker)
-
selectable (and storable) over advanced web config1 or per serial com and at directly at start up coming from factory mode ( see first-setup-with-access-point )
selectDisplay 0= OLED (default)selectDisplay 1= round TFT -
display SSH1106 OLED 1,3" 128x64 (other sizes with same driver (SSH1106) and resolution should also directly work)
- segmented in 3 parts
- header:
- left: wifi quality dtuGateway to local wifi
- mid: current time of dtuGateway
- right: wifi quality of dtu connection to local wifi
- main:
- small left: current power limit of inverter
- big mid/ right: current power of inverter
- footer:
- left: current daily yield
- right: current total yield
- header:
- additonal features
- small screensaver to prevent burn-in effect with steady components on the screen (shifting the whole screen every minute with 1 pixel in a 4 step rotation)
- smooth brightness control for changed main value - increase to max after change and then dimmming smooth back to the default level
- segmented in 3 parts
-
display GAGC9A01 round TFT 1,28" 240x240
- setup screen for first start (factory mode)
- status screen with the (current) most important data
- serving own access point in factory mode for first setup
- web application will be directly served by the system
- settings of needed user data over the web app (stored in a json-file in local flash file system - extensions of user setup will not lead to breakable changes)
- select found local wifi and enter/ save the needed wifi password
- change dtu connection data (e.g. host IP in local network, wireless user/ pass for dtu access point)
- configurable data for openhab item settings
- configurable data for MQTT settings incl. HomeAssistant AutoDiscovery
- advanced web config1 for all config parameter (http://IP_domain/config) - expert mode
- display selection (0 - OLED, 1 - round TFT)
- timeZone Offset -xxxx sec <-> xxxx sec e.g. 3600 for CET(+1h) /7200 for CEST(+2)/-21600 for CST
- OTA with direct connection to the github build pipeline - available updates will be checked by web app and device. Notification in web app, if update available and user can decide for direct online update
expand to see json example
{
"localtime": 1704110892,
"ntpStamp": 1707640484,
"lastResponse": 1704063600,
"dtuConnState": 1,
"dtuErrorState": 0,
"starttime": 1707593197,
"inverter": {
"pLim": 0,
"pLimSet": 101,
"temp": 0.00,
"uptodate": 0
},
"grid": {
"v": 0.00,
"c": 0.00,
"p": 0.00,
"dE": 0.000,
"tE": 0.000
},
"pv0": {
"v": 0.00,
"c": 0.00,
"p": 0.00,
"dE": 0.000,
"tE": 0.000
},
"pv1": {
"v": 0.00,
"c": 0.00,
"p": 0.00,
"dE": 0.000,
"tE": 0.000
}
}expand to see json example
{
"chipid": 12345678,
"host": "hoymilesGW_12345678",
"initMode": 0,
"firmware": {
"version": "1.0.0022",
"versiondate": "10.02.2024 - 19:23:57",
"versionServer": "1.0.0051",
"versiondateServer": "10.02.2024 - 19:23:57",
"versionServerRelease": "checking",
"versiondateServerRelease": "...",
"selectedUpdateChannel": "0",
"updateAvailable": 0
},
"dtuConnection": {
"dtuHostIpDomain": "192.168.0.249",
"dtuSsid": "DTUBI-12345678",
"dtuPassword": "dtubiPassword",
"dtuRssi": 0,
"dtuDataCycle": 32,
"dtuResetRequested": 0,
"dtuCloudPause": 1,
"dtuCloudPauseTime": 40
},
"openHabConnection": {
"ohHostIp": "192.168.1.100",
"ohItemPrefix": "inverter"
},
"wifiConnection": {
"networkCount": 2,
"foundNetworks": [
{
"name": "Name1 Wlan",
"wifi": 62,
"rssi": -69,
"chan": 1
},
{
"name": "name2-wifi",
"wifi": 48,
"rssi": -76,
"chan": 3
}
],
"wifiSsid": "myWifiSSID",
"wifiPassword": "myPass",
"rssiGW": 87
}
}-
set the IP to your openhab instance - data will be read with http://<your_openhab_ip>:8080/rest/items//state
-
set the prefix () of your openhab items
-
list of items that should be available in your openhab config
- read your given power set value from openhab with "_PowerLimit_Set"
- set openhab items with data from dtu:
expand to see to details
- grid data:
- "Grid_U"
- "Grid_I"
- "Grid_P"
- "PV_E_day"
- "PV_E_total"
- panel 1 data:
- "PV1_U"
- "PV1_I"
- "PV1_P"
- "PV1_E_day"
- "PV1_E_total"
- panel 2 data:
- "PV2_U"
- "PV2_I"
- "PV2_P"
- "PV2_E_day"
- "PV2_E_total"
- inverter status:
- "_Temp"
- "_PowerLimit" //current read power limit from dtu
- "_WifiRSSI"
-
set the IP to your MQTT broker
-
set the MQTT user and MQTT password
-
set the main topic e.g. 'dtu_12345678' for the pubished data (default: is
dtu_<ESP chip id>and has to be unique in your environment) -
to set the Power Limit from your environment
- you have to publish to
dtu_<ESP chip id>/inverter/PowerLimit_Seta value between 2...100 (possible range at DTU) - the incoming value will be checked for this interval and locally corrected to 2 or 100 if exceeds
- (optional: with retain flag, to get the last set value after restart / reconnect of the dtuGateway)
- you have to publish to
-
data will be published as following ('dtu_12345678' is configurable in the settings):
dtu_12345678/timestamp dtu_12345678/grid/U dtu_12345678/grid/I dtu_12345678/grid/P dtu_12345678/grid/dailyEnergy dtu_12345678/grid/totalEnergy dtu_12345678/pv0/U dtu_12345678/pv0/I dtu_12345678/pv0/P dtu_12345678/pv0/dailyEnergy dtu_12345678/pv0/totalEnergy dtu_12345678/pv1/U dtu_12345678/pv1/I dtu_12345678/pv1/P dtu_12345678/pv1/dailyEnergy dtu_12345678/pv1/totalEnergy dtu_12345678/inverter/Temp dtu_12345678/inverter/PowerLimit dtu_12345678/inverter/PowerLimit_Set // <-- this topic will be subscribed to get the power limit to set from your broker dtu_12345678/inverter/WifiRSSI -
Home Assistant Auto Discovery
- you can set HomeAssistant Auto Discovery, if you want to auto configure the dtuGateway for your HA installation
- switch to ON means - with every restart/ reconnection of the dtuGateway the so called config messages will be published for HA and HA will configure all the given entities of dtuGateway incl. the set value for PowerLimit
- switch to OFF means - all the config messages will be deleted and therefore the dtuGateway will be removed from HA (base publishing of data will be remain the same, if MQTT is activated)
- sometimes out-of-memory resets with instant reboots (rare after some hours or more often after some days)
- ESP8266 based board
- optional display SSH1106 OLED 1,3" 128x64 (e.g. link):
-
connect SSH1106 driven OLED display (128x64) with your ESP8266 board (VCC, GND, SCK, SCL)
-
pinning for different boards (display connector to ESPxx board pins)
dev board ESP family VCC GND SCK SDA tested AZDelivery D1 Board NodeMCU ESP8266MOD-12F ESP8266 3.3V GND D15/GPIO5/SCL/D3 D14/GPIO4/SDA/D4 OK AZDelivery NodeMCU V2 WiFi Amica ESP8266 ESP-12F ESP8266 3.3V GND D1/GPIO5/SCL D2/GPIO4/SDA OK AZDelivery D1 Mini NodeMcu mit ESP8266-12F ESP8266 3V3 G D1/GPIO5/SCL D2/GPIO4/SDA OK
-
- optional display GAGC9A01 round TFT 1,28" 240x240 (e.g. link):
-
connect SSH1106 driven round TFT display (240x240) with your ESP8266 board (VCC, GND, SCL, SDA, DC, CS, RST)
-
pinning for different boards (display connector to ESPxx board pins)
dev board ESP family VCC GND SCL SDA DC CS RST tested AZDelivery D1 Board NodeMCU ESP8266MOD-12F ESP8266 3.3V GND t.b.d. t.b.d. t.b.d. t.b.d. t.b.d. t.b.c. AZDelivery NodeMCU V2 WiFi Amica ESP8266 ESP-12F ESP8266 3.3V GND D5/GPI14/SCLK D7/GPIO13/MOSI D3/GPIO0/Flash D8/GPIO15/CS 3V32 OK AZDelivery D1 Mini NodeMcu mit ESP8266-12F ESP8266 3V3 G D5/GPI14/SCLK D7/GPIO13/MOSI D3/GPIO0/Flash D8/GPIO15/CS 3V32 OK
-
- download the preferred release as binary (see below)
- HAS TO BE VERIFIED [only once] flash the esp8266 board with the (esp download tool)[https://www.espressif.com/en/support/download/other-tools]
- choose bin file at address 0x0
- crystal frequency to 26 Mhz
- SPI speed 40 MHz
- SPI Mode QIO
- Flash Size 32 MBit-C1
- select your COM port and baudrate = 921600
- press start ;-)
- all further updates are done by OTA (see chapters above)
prequesite: If you have directly attached a display, then in factory mode the used display is unknown. Default is OLED Display. To get the TFT running in factory mode, a change with each reboot is implemented. Means if you are powering on the first time the OLED will be choosen internally. The next power up the TFT will be chosen. And so on. So the 'first start' screen will be shown until the wifi settings will be changed over the webinterface.
- connect with the AP hoymilesGW_ (on smartphone sometimes you have to accept the connection explicitly with the knowledge there is no internet connectivity)
- open the website http://192.168.4.1 (or http://hoymilesGW.local) for the first configuration
- choose your wifi
- type in the wifi password - save
- in webfrontend setting your DTU IP adress within your local network (currently the user and password for dtu are not needed, for later integration relevant for a direct connection to the dtu over their access point)
- then you can configure your needed binding
- openhab -> set the IP of your openhab instance and the prefix for the dtu items according to your configured item file in openhab
- mqtt -> set the IP and port (e.g. 192.178.0.42:1883) of your mqtt broker and the user and passwort that your hacve for this instance
- after this one time configuration, the connection to the dtu should be established and the data displayed in the webfrontend and (if connected on the display) according to your setup transmitted to the target instance
- connect your ESP with serial (115200 baud) in a COM terminal
- check if you receive some debug data from the device
- type in
resetToFactory 1 - response of the device will be
reinitialize UserConfig data and reboot ... - after reboot the device starting again in AP mode for first setup
latest release - changes will documented by commit messages https://github.com/ohAnd/dtuGateway/releases/latest
(to be fair, the amount of downloads is the count of requests from the client to check for new firmware for the OTA update)
snapshot with latest build https://github.com/ohAnd/dtuGateway/releases/tag/snapshot
- if the config file is corrupted due to whatever reason with unexpected behavior - connect with serial terminal and type in the command
resetToFactory 1- the config file be rewritten with the default values - if in the first startup mode a wrong ssid/ password was entered, then also
resetToFactory 1
(not fully investigated yet)
If there to much requests of setting the power limit minutes later the connection is broken and cannot be directly established again - with current experience the dtu resets itself after ~ 30 min and is accessable again.
With the manual login to dtu access point and forcing the storing of local wifi connect data again, then dtu is back online and accessable in your local network. (This is a possible feature that can be implemented in future - needed protocol sequence has to be investigated)
[2024-03-24]
- lot of single updates for power setting within few seconds (< 2-3) without any reading of values (e.g. realdata) -> it seems this creating no problems
- therefore current setup -> no time limit for power setting, but reading data only every 31 seconds is running fine
- sometimes hanging or full shutdown/ break of DTU will be prevented by sending an active reboot request to dtu (hanging detection at this time over grid voltage, should be changing at least within 10 consecutive incoming data)
- with this setup: now the device is running for days without any stops (overall system point of view: target settings will be performed everytime, readed data will be available, no manual steps needed to recover the dtu connection)
- everey 15 min (0,15,30,45) -> timestamp update
- after 7 min 40 s update of graph data (if wifi not reachable, also reset of wifi AP)
- https://github.com/henkwiedig/Hoymiles-DTU-Proto
- https://github.com/suaveolent/hoymiles-wifi/tree/main
- tbnobody/OpenDTU#1430
fully covered with github actions
building on push to develop and serving as a snapshot release with direct connection to the device - available updates will be locally checked and offered to the user for installation
hint: referring to Error Build in platform.io - buildnumber file not found #6 for local building:
For automatic versioning there is a file called ../include/buildnumber.txt expected. With the content "localDev" or versionnumber e.g. "1.0.0" in first line. (File is blocked by .gitignore for GitHub actions to run.)
- creating dev release (https://blog.derlin.ch/how-to-create-nightly-releases-with-github-actions)