Release 1.0.5

Examples/docker-compose.yml

@@ -65,11 +65,20 @@ services:
       - pvmqttauth=False
 
       - pvgridrelay=False
-      - pvgridrelaykenterean=000000000000000000
-      - pvgridrelaykentermeterid=0000000000
       - pvgridrelaykenteruser=my@username.com
       - pvgridrelaykenterpasswd=secretpassword
 
+      - pvgridrelaysysname=transformer01
+      - pvgridrelaykenterean=000000000000000000
+      - pvgridrelaykentermeterid=0000000000
+
+      - pvgridrelaysys02enabled=False
+      - pvgridrelaysysname02=transformer02
+      - pvgridrelaykenterean02=000000000000000000
+      - pvgridrelaykentermeterid02=0000000000
+
+
+
     volumes:
       - /etc/localtime:/etc/localtime:ro
 

README.md

@@ -1,5 +1,5 @@
 # Huawei FusionSolar Kiosk to InfluxDB, MQTT, PVOutput and Home Assistant relay
-This is a python project intended to fetch data from the **Huawei FusionSolar** public **kiosk** and relay it to **InfluxDB** and/or **PVOutput.org** and/or **MQTT** and/or **Home Assistant (hass)**. 
+This is a python project intended to fetch data from the **Huawei FusionSolar** public **kiosk** and relay it to **InfluxDB**/**VictoriaMetrics** and/or **PVOutput.org** and/or **MQTT** and/or **Home Assistant (hass)**. 
 
 Additionally this project can also fetch and relay grid usage data from the Dutch meetdata.nl API provider by **Kenter**.
 
@@ -84,15 +84,19 @@ Fusion solar data fetching is planned by cron in order to exactly specify at wha
 | mqttpasswd | pvmqttpasswd | MQTT Password | fusionsolar |
 | mqtttopic | pvmqtttopic | MQTT Topic for publishing | energy/pyfusionsolar |
 | gridrelay | pvgridrelay | Can be `True` or `False`, determines if data is fetched from Kenter's meetdata.nl API | False |
-| gridrelaysysname | pvgridrelaysysname | Grid transformer name for InfluxDB transformer data | transformer01 |
 | gridrelayinterval | pvgridrelayinterval | Interval in seconds to fetch data from meetdata.nl and post to PVOutput and InfluxDB | 43200 |
 | gridrelaykenterurl | pvgridrelaykenterurl | Kenter API url for fetching transformer grid measurements | [Click url](https://webapi.meetdata.nl) |
-| gridrelaykenterean | pvgridrelaykenterean | EAN code for transformer on Kenter's www.meetdata.nl | XXX |
-| gridrelaykentermeterid | pvgridrelaykentermeterid | MeterID as shown on Kenter's www.meetdata.nl | XXX |
 | gridrelaykenteruser | pvgridrelaykenteruser | Username for Kenter's API | user |
 | gridrelaykenterpasswd | pvgridrelaykenterpasswd | Password for Kenter's API | passwd |
 | gridrelaydaysback | pvgridrelaydaysback | Kenter's meetdata.nl does not provide live data. Data is only available up until an X amount of days back. May vary per transformer. | 3 |
 | gridrelaypvoutputspan | pvgridrelaypvoutputspan | In my case meetdata.nl has datapoints for each 15mins. Setting this to a value of 2, will calculate averages over 2 datapoints spanning half an hour before posting to PVOutput. This way the datapoint interval between the grid usage data and fusionsolar PV production data matches, resulting in nice diagrams on PVOutput.org | 2 |
+| gridrelaysysname | pvgridrelaysysname | Grid transformer name for InfluxDB transformer data | transformer01 |
+| gridrelaykenterean | pvgridrelaykenterean | EAN code for transformer on Kenter's www.meetdata.nl | XXX |
+| gridrelaykentermeterid | pvgridrelaykentermeterid | MeterID as shown on Kenter's www.meetdata.nl | XXX |
+| gridrelaysys02enabled | pvgridrelaysys02enabled | Can be `True` or `False`, determines if a secondary transformer is configured for InfluxDB output | False |
+| gridrelaysysname02 | pvgridrelaysysname02 | Grid transformer name for InfluxDB transformer data | transformer02 |
+| gridrelaykenterean02 | pvgridrelaykenterean02 | EAN code for transformer on Kenter's www.meetdata.nl | XXX |
+| gridrelaykentermeterid02 | pvgridrelaykentermeterid02 | MeterID as shown on Kenter's www.meetdata.nl | XXX |
 
 # Grafana dashboard example
 A grafana dashboard export is included in the Examples subfolder in the Git repository.
@@ -118,6 +122,8 @@ Result:
 # Changelog
 | Version | Description |
 | --- | --- |
+| 1.0.5 | Added InfluxDB support for an optional secondary grid telemetry EAN configuration (pvoutput output is only supported on the primary EAN) |
+| 1.0.5 | Bugfix for InfluxDB v1 implementation and removed auto-database creation for VictoriaMetrics compatibility |
 | 1.0.3 | Grid transformer usage measurement polling from Kenter's meetdata.nl API has been implemented |
 | 1.0.3 | Changed docker-compose.yml template not to use host networking mode |
 | 1.0.3 | pv.py now uses separate threads for PvRelay and GridRelay classes |
@@ -126,5 +132,7 @@ Result:
 
 
 Released under [MIT](/LICENSE) by [@JasperE84](https://github.com/JasperE84).
+
 This project has been partly developed in time donated by [Contour - Sheet metal supplier](https://www.contour.eu/en/)
+
 Dit project is deels ontwikkeld ontwikkeld in de tijd van [Contour - Plaatwerkleverancier](https://www.contour.eu/)

gridkenter.py

@@ -11,19 +11,20 @@ def __init__(self, conf: PvConf, logger):
         self.logger = logger
         self.logger.debug("GridKenter class instantiated")
 
-    def fetch_gridkenter_data(self, days_back):
+    def fetch_gridkenter_data(self, sysname, ean, meterid, days_back):
         self.logger.info(
             "Requesting data for {} from GridKenter API...".format(
-                self.conf.gridrelaysysname
+                sysname
             )
         )
+
         req_time = datetime.now() - timedelta(days=days_back)
         req_year = req_time.strftime("%Y")
         req_month = req_time.strftime("%m")
         req_day = req_time.strftime("%d")
 
         try:
-            url = f"{self.conf.gridrelaykenterurl}/api/1/measurements/{self.conf.gridrelaykenterean}/{self.conf.gridrelaykentermeterid}/{req_year}/{req_month}/{req_day}"
+            url = f"{self.conf.gridrelaykenterurl}/api/1/measurements/{ean}/{meterid}/{req_year}/{req_month}/{req_day}"
             self.logger.debug(f"Fetching URL: {url}")
 
             response = requests.get(
@@ -59,9 +60,9 @@ def fetch_gridkenter_data(self, days_back):
             )
 
         grid_data_obj = {
-            "sysname": self.conf.gridrelaysysname,
-            "ean": self.conf.gridrelaykenterean,
-            "meter_id": self.conf.gridrelaykentermeterid,
+            "sysname": sysname,
+            "ean": ean,
+            "meter_id": meterid,
             "grid_net_consumption": [],
         }
 

gridrelay.py

@@ -27,9 +27,16 @@ def start(self):
 
         while 1:
             try:
-                grid_measurement_data = self.gridkenter.fetch_gridkenter_data(self.conf.gridrelaydaysback)
+                grid_measurement_data = self.gridkenter.fetch_gridkenter_data(self.conf.gridrelaysysname, self.conf.gridrelaykenterean, self.conf.gridrelaykentermeterid, self.conf.gridrelaydaysback)
                 self.write_gridkenter_to_influxdb(grid_measurement_data)
                 self.write_gridkenter_to_pvoutput(grid_measurement_data)
+
+                if self.conf.gridrelaysys02enabled:
+                    grid_measurement_data = self.gridkenter.fetch_gridkenter_data(self.conf.gridrelaysysname02, self.conf.gridrelaykenterean02, self.conf.gridrelaykentermeterid02, self.conf.gridrelaydaysback)
+                    self.write_gridkenter_to_influxdb(grid_measurement_data)
+                    #No support for pvoutput on 2 EAN codes yet (needs summing of kenter data or pvoutput support for 2 distinct systems)
+                    #self.write_gridkenter_to_pvoutput(grid_measurement_data)
+
             except:
                 self.logger.exception(
                     "Uncaught exception in GridRelay data processing loop."

pv.py

@@ -13,7 +13,7 @@
 formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
 streamHandler.setFormatter(formatter)
 logger.addHandler(streamHandler)
-logger.info("PyFusionSolarDataRelay 1.0.4 started")
+logger.info("PyFusionSolarDataRelay 1.0.5 started")
 
 # Config
 conf = PvConf(logger)

pvconf.py

@@ -33,26 +33,38 @@ def apply_default_settings(self):
         # Gridrelay default
         # Please note that local server or docker container needs to be in same timezone als meetdata.nl in order for kenter data to work correctly
         self.gridrelay = False
-        self.gridrelaysysname = "transformer01"
         self.gridrelayinterval = 43200
         self.gridrelaykenterurl = "https://webapi.meetdata.nl"
-        self.gridrelaykenterean = "XXX"
-        self.gridrelaykentermeterid = "XXX"
         self.gridrelaykenteruser = "user"
         self.gridrelaykenterpasswd = "passwd"
         # Grid infrastructure measurements in The Netherlands, show up in the API with a 3-5 days delay.
         self.gridrelaydaysback = 3
         # If fusionsolar updates every 30mins and meetdata.nl has values per 15min, set this to 2 so that intervals between two datasources match to avoid weird pvoutput graphs.
         self.gridrelaypvoutputspan = 2
 
+        self.gridrelaysysname = "transformer01"
+        self.gridrelaykenterean = "XXX"
+        self.gridrelaykentermeterid = "XXX"
+
+        self.gridrelaysys02enabled = False
+        self.gridrelaysysname02 = "transformer02"
+        self.gridrelaykenterean02 = "XXX"
+        self.gridrelaykentermeterid02 = "XXX"
+
         # Influxdb default
         self.influx = False
-        self.influx2 = True
         self.ifhost = "localhost"
         self.ifport = 8086
+
+        # Set to True to enable InfluxDB v2, or to False for InfluxDB v1 or VictoriaMetrics
+        self.influx2 = True
+
+        # Influxdb v1 settings
         self.if1dbname = "fusionsolar"
         self.if1user = "fusionsolar"
         self.if1passwd = "fusionsolar"
+
+        # Influxdb v2 settings
         self.if2protocol = "https"
         self.if2org = "acme"
         self.if2bucket = "fusionsolar"
@@ -108,15 +120,21 @@ def print(self):
         self.logger.info(f"Topic: {self.mqtttopic}")
         self.logger.info(f"_GridRelay")
         self.logger.info(f"Enabled: {self.gridrelay}")
-        self.logger.info(f"System name: {self.gridrelaysysname}")
         self.logger.info(f"Interval: {self.gridrelayinterval}")
         self.logger.info(f"PVOutput span: {self.gridrelaypvoutputspan}")
         self.logger.info(f"Kenter URL: {self.gridrelaykenterurl}")
-        self.logger.info(f"Kenter EAN: {self.gridrelaykenterean}")
-        self.logger.info(f"Kenter MeterId: {self.gridrelaykentermeterid}")
+        self.logger.info(f"Days back: {self.gridrelaydaysback}")
         self.logger.info(f"Kenter User: {self.gridrelaykenteruser}")
         self.logger.info(f"Kenter Passwd: {self.gridrelaykenterpasswd}")
-        self.logger.info(f"Days back: {self.gridrelaydaysback}")
+
+        self.logger.info(f"System name 01: {self.gridrelaysysname}")
+        self.logger.info(f"Kenter EAN 01: {self.gridrelaykenterean}")
+        self.logger.info(f"Kenter MeterId 01: {self.gridrelaykentermeterid}")
+
+        self.logger.info(f"System name 02: {self.gridrelaysysname02}")
+        self.logger.info(f"Kenter EAN 02: {self.gridrelaykenterean02}")
+        self.logger.info(f"Kenter MeterId: {self.gridrelaykentermeterid02}")
+
 
     def getenv(self, envvar):
         envval = os.getenv(envvar)
@@ -190,16 +208,10 @@ def apply_environment_settings(self):
 
         if os.getenv("pvgridrelay") != None:
             self.gridrelay = self.getenv("pvgridrelay") == "True"
-        if os.getenv("pvgridrelaysysname") != None:
-            self.gridrelaysysname = self.getenv("pvgridrelaysysname")
         if os.getenv("pvgridrelayinterval") != None:
             self.gridrelayinterval = int(self.getenv("pvgridrelayinterval"))
         if os.getenv("pvgridrelaykenterurl") != None:
             self.gridrelaykenterurl = self.getenv("pvgridrelaykenterurl")
-        if os.getenv("pvgridrelaykenterean") != None:
-            self.gridrelaykenterean = self.getenv("pvgridrelaykenterean")
-        if os.getenv("pvgridrelaykentermeterid") != None:
-            self.gridrelaykentermeterid = self.getenv("pvgridrelaykentermeterid")
         if os.getenv("pvgridrelaykenteruser") != None:
             self.gridrelaykenteruser = self.getenv("pvgridrelaykenteruser")
         if os.getenv("pvgridrelaykenterpasswd") != None:
@@ -209,3 +221,19 @@ def apply_environment_settings(self):
         if os.getenv("pvgridrelaypvoutputspan") != None:
             self.gridrelaypvoutputspan = int(self.getenv("pvgridrelaypvoutputspan"))
 
+        if os.getenv("pvgridrelaysysname") != None:
+            self.gridrelaysysname = self.getenv("pvgridrelaysysname")
+        if os.getenv("pvgridrelaykenterean") != None:
+            self.gridrelaykenterean = self.getenv("pvgridrelaykenterean")
+        if os.getenv("pvgridrelaykentermeterid") != None:
+            self.gridrelaykentermeterid = self.getenv("pvgridrelaykentermeterid")
+
+        if os.getenv("pvgridrelaysys02enabled") != None:
+            self.gridrelaysys02enabled = self.getenv("pvgridrelaysys02enabled") == "True"
+        if os.getenv("pvgridrelaysysname02") != None:
+            self.gridrelaysysname02 = self.getenv("pvgridrelaysysname02")
+        if os.getenv("pvgridrelaykenterean02") != None:
+            self.gridrelaykenterean02 = self.getenv("pvgridrelaykenterean02")
+        if os.getenv("pvgridrelaykentermeterid02") != None:
+            self.gridrelaykentermeterid02 = self.getenv("pvgridrelaykentermeterid02")
+

pvinflux.py

@@ -107,12 +107,14 @@ def initialize_v1(self):
                 timeout=3,
                 username=self.conf.if1user,
                 password=self.conf.if1passwd,
+                database=self.conf.if1dbname
             )
         except Exception as e:
             raise Exception(
                 "Error instantiating InfluxDB v1 client library: '{}'".format(str(e))
             )
 
+        '''
         try:
             self.logger.debug("Fetching influxdb database list")
             databases = [db["name"] for db in self.influxclient.get_list_database()]
@@ -146,6 +148,7 @@ def initialize_v1(self):
                 self.conf.if1dbname
             )
         )
+        '''
 
     def pvinflux_write_pvdata(self, response_json_data):
         ifjson = self.make_influx_pvdata_jsonrecord(response_json_data)
@@ -161,7 +164,7 @@ def pvinflux_write_pvdata(self, response_json_data):
                 )
             else:
                 self.logger.debug("Writing PvData to InfluxDB v1...")
-                self.conf.influxclient.write_points(ifjson, time_precision="s")
+                self.influxclient.write_points(ifjson, time_precision="s")
         except Exception as e:
             self.logger.exception("InfluxDB PvData write error: '{}'".format(str(e)))
 
@@ -198,7 +201,7 @@ def pvinflux_write_griddata(self, grid_data_obj):
                 )
             else:
                 self.logger.debug("Writing GridData to InfluxDB v1...")
-                self.conf.influxclient.write_points(ifjson, time_precision="s")
+                self.influxclient.write_points(ifjson, time_precision="s")
         except Exception as e:
             self.logger.exception("InfluxDB GridData write error: '{}'".format(str(e)))