You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The container image is based on Alpine Linux with python3 interpreter.
33
34
The tool is written in python and connects to a MQTT server and subscripes to one ore more topics.
34
-
All messages in that topic will be pushed to an elasticsearch server.
35
+
All messages in that topic will be pushed to an elasticsearch or opensearch server.
35
36
36
37
# Prerequisites to run the docker container
37
38
1. You need a MQTT server to read the data from the topics.
38
-
2. You need an elasticsearch v8 database to store the data inside.
39
+
2. You need an elasticsearch v8 / or opensearch v2 database to store the data inside.
39
40
40
41
# Configuration
41
42
## Container configuration
@@ -53,9 +54,9 @@ The container grab some configuration via environment variables.
53
54
54
55
The path and filename to the general configuration file can be set via environment variable `CONFIG_FILE`. By default, the script will use `/app/etc/mqtt2elasticsearch.json`.
55
56
56
-
Inside this file we need to configure the Elasticsearch and MQTT server connection parameters.
57
+
Inside this file we need to configure the Elasticsearch (or Opensearch) and MQTT server connection parameters.
57
58
58
-
### Example
59
+
### Example with Elasticsearch
59
60
60
61
```json
61
62
{
@@ -77,6 +78,38 @@ Inside this file we need to configure the Elasticsearch and MQTT server connecti
77
78
}
78
79
```
79
80
81
+
### Example with Opensearch
82
+
83
+
```json
84
+
{
85
+
"DEBUG": true,
86
+
"removeIndex": false,
87
+
"opensearch": {
88
+
"hosts": [
89
+
{
90
+
"host": "opensearch",
91
+
"port": 9200
92
+
}
93
+
],
94
+
"tls": true,
95
+
"verify_certs": true,
96
+
"username": "admin",
97
+
"password": "admin"
98
+
},
99
+
"mqtt": {
100
+
"client_id": "mqtt2elasticsearch",
101
+
"user": "mqtt2elasticsearch",
102
+
"password": "myPassword",
103
+
"server": "test.mosquitto.org",
104
+
"port": 1883,
105
+
"tls": false,
106
+
"hostname_validation": true,
107
+
"protocol_version": 3
108
+
}
109
+
}
110
+
```
111
+
112
+
80
113
### Field description
81
114
82
115
| Field | Type | Description |
@@ -85,6 +118,16 @@ Inside this file we need to configure the Elasticsearch and MQTT server connecti
85
118
|`removeIndex`| Boolean | If this flag is set to `true`, the script will remove the Elasticsearch index and exits. |
86
119
|`elasticsearch`| Object | Contains Elasticsearch specific configuration parameters. |
87
120
|`elasticsearch.cluster`| Array | Contains a list of Eleasticsearch cluster node URLs. |
121
+
|`elasticsearch.api_ley`| String | Optional api\_key if Elasticsearch requires authentication. |
122
+
|`opensearch`| Object | Contains Opensearch specific configuration parameters. |
123
+
|`opensearch.hosts`| Array | Contains a list of Opensearch hosts objects. |
124
+
|`opensearch.hosts[].host`| String | Opensearch hostname (fqdn) or IP address. |
125
+
|`opensearch.hosts[].port`| String | Optional Opensearch TCP port (Default: 9200). |
126
+
|`opensearch.username`| String | Optional username for authentication (Default: null). |
127
+
|`opensearch.password`| String | Optional password for authentication (Default: null. |
128
+
|`opensearch.tls`| Boolean | Optional if a TLS encrpted communication should be established or not (Default: false). |
129
+
|`opensearch.verify_certs`| Boolean | Optional to validate the server certificate ort not (Default: false). |
130
+
|`opensearch.ca_certs_path`| String | Optional path to CA certs (Default: "/etc/ssl/certs/ca-certificates.crt"). |
88
131
|`mqtt`| Object | Contains MQTT specific configuration parameters. |
89
132
|`mqtt.client_id`| String | The MQTT client identifier. |
90
133
|`mqtt.user`| String | The username to authenticate to the MQTT server. |
@@ -96,16 +139,16 @@ Inside this file we need to configure the Elasticsearch and MQTT server connecti
96
139
|`mqtt.protocol_version`| Integer | The MQTT protocol version. Can be 3 (for MQTTv311) or 5 (for MQTTv5). |
97
140
98
141
99
-
## Elasticsearch index configuration file
142
+
## Elasticsearch/Opensearch index configuration file
100
143
101
-
The path and filename to the Elasticsearch index configuration file can be set via environment variable `ELASTICSEARCH_MAPPING_FILE`.
144
+
The path and filename to the Elasticsearch/Opensearch index configuration file can be set via environment variable `ELASTICSEARCH_MAPPING_FILE`.
102
145
By default, the script will use `/app/etc/mqtt2elasticsearch-mappings.json`.
103
146
104
-
Inside this file we need to configure the MQTT topic and the associated Elasticsearch index with it's configuration.
147
+
Inside this file we need to configure the MQTT topic and the associated Elasticsearch/Opensearch index with it's configuration.
105
148
106
149
### Example
107
150
108
-
This is minimal example. The JSON file contains the MQTT topic as **key**. Every topic contains the associated Elasticsearch index and an optional index configuration.
151
+
This is minimal example. The JSON file contains the MQTT topic as **key**. Every topic contains the associated Elasticsearch/Opensearch index and an optional index configuration.
109
152
You can use placeholder inside the index name. Following will be translated on the fly:
110
153
-`{Y}`: the 4-digit year
111
154
-`{m}`: the 2-digit month
@@ -121,7 +164,9 @@ You can use placeholder inside the index name. Following will be translated on t
121
164
}
122
165
```
123
166
124
-
A full blown example can be found here: [speedtest2mqtt-elasticsearch-mapping.json](./examples/speedtest2mqtt-elasticsearch-mapping.json).
0 commit comments