This repository has been archived by the owner on May 17, 2019. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 119
/
out_forward.txt
494 lines (331 loc) · 13.7 KB
/
out_forward.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
# forward Output Plugin
The `out_forward` Buffered Output plugin forwards events to other fluentd nodes. This plugin supports load-balancing and automatic fail-over (a.k.a. active-active backup). For replication, please use the [out_copy](out_copy) plugin.
The `out_forward` plugin detects server faults using a “φ accrual failure detector” algorithm. You can customize the parameters of the algorithm. When a server fault recovers, the plugin makes the server available automatically after a few seconds.
The `out_forward` plugin supports at-most-once and at-least-once semantics. The default is at-most-once.
## Example Configuration
`out_forward` is included in Fluentd's core. No additional installation process is required.
:::text
<match pattern>
@type forward
send_timeout 60s
recover_wait 10s
hard_timeout 60s
<server>
name myserver1
host 192.168.1.3
port 24224
weight 60
</server>
<server>
name myserver2
host 192.168.1.4
port 24224
weight 60
</server>
...
<secondary>
@type file
path /var/log/fluent/forward-failed
</secondary>
</match>
NOTE: Please see the LINK:[Config File](config-file) article for the basic structure and syntax of the configuration file.
## Supported modes
* Synchronous
* Asynchronous
See [Output Plugin Overview](output-plugin-overview) for more details.
## Plugin helpers
* [socket](api-plugin-helper-socket)
* [server](api-plugin-helper-server)
* [timer](api-plugin-helper-timer)
* [thread](api-plugin-helper-thread)
* [compat_parameters](api-plugin-helper-compat_parameters)
## Parameters
[Common Parameters](plugin-common-parameters)
### @type
The value must be `forward`.
### <server> (at least one is required)
| required | multi | version |
|:--------:|:-----:|:-------:|
| true | true | 0.14.5 |
The destination servers. Each server has following parameters.
- host
- name
- port
- shared_key
- username
- password
- standby
- weight
#### host
| type | default | version |
|:------:|:------------------:|:-------:|
| string | required parameter | 0.14.5 |
The IP address or host name of the server.
#### name
| type | default | version |
|:------:|:-------:|:-------:|
| string | nil | 0.14.5 |
The name of the server. Used for logging and certificate verification in TLS transport (when host is address).
#### port
| type | default | version |
|:-------:|:-------:|:-------:|
| integer | 24224 | 0.14.5 |
The port number of the host. Note that both TCP packets (event stream) and UDP packets (heartbeat message) are sent to this port.
#### shared_key
| type | default | version |
|:------:|:-------:|:-------:|
| string | nil | 0.14.5 |
The shared key per server.
#### username
| type | default | version |
|:------:|:-----------------:|:-------:|
| string | "" (empty string) | 0.14.5 |
The username for authentication.
#### password
| type | default | version |
|:------:|:-----------------:|:-------:|
| string | "" (empty string) | 0.14.5 |
The password for authentication.
#### standby
| type | default | version |
|:----:|:-------:|:-------:|
| bool | false | 0.14.5 |
Marks a node as the standby node for an Active-Standby model between Fluentd nodes. When an active node goes down, the standby node is promoted to an active node. The standby node is not used by the `out_forward` plugin until then.
:::text
<match pattern>
@type forward
...
<server>
name myserver1
host 192.168.1.3
weight 60
</server>
<server> # forward doesn't use myserver2 until myserver1 goes down
name myserver2
host 192.168.1.4
weight 60
standby
</server>
...
</match>
#### weight
| type | default | version |
|:-------:|:-------:|:-------:|
| integer | 60 | 0.14.5 |
The load balancing weight. If the weight of one server is 20 and the weight of the other server is 30, events are sent in a 2:3 ratio. The default weight is 60.
### require_ack_response
| type | default | version |
|:----:|:-------:|:-------:|
| bool | false | 0.14.0 |
Change the protocol to at-least-once. The plugin waits the ack from destination's in_forward plugin.
### ack_response_timeout
| type | default | version |
|:----:|:-------:|:-------:|
| time | 190 | 0.14.0 |
This option is used when `require_ack_response` is `true`. This default value is based on popular `tcp_syn_retries`.
If set `0`, this plugin doesn't wait the ack response.
### send_timeout
| type | default | version |
|:----:|:-------:|:-------:|
| time | 60 | 0.14.0 |
The timeout time when sending event logs.
### recover_wait
| type | default | version |
|:----:|:-------:|:-------:|
| time | 10 | 0.14.0 |
The wait time before accepting a server fault recovery.
### heartbeat_type
| type | default | available | version |
|:----:|:---------:|:-------------------------:|:-------:|
| enum | transport | transport, tcp, udp, none | 0.14.12 |
The transport protocol to use for heartbeats. Set "none" to disable heartbeat.
### heartbeat_interval
| type | default | version |
|:----:|:-------:|:-------:|
| time | 1 | 0.14.0 |
The interval of the heartbeat packer.
### phi_failure_detector
| type | default | version |
|:----:|:-------:|:-------:|
| bool | true | 0.14.0 |
Use the "Phi accrual failure detector" to detect server failure.
### phi_threshold
| type | default | version |
|:-------:|:-------:|:-------:|
| integer | 16 | 0.14.0 |
The threshold parameter used to detect server faults.
NOTE: `phi_threshold` is deeply related to `heartbeat_interval`. If you are using longer `heartbeat_interval`, please use the larger `phi_threshold`. Otherwise you will see frequent detachments of destination servers. The default value 16 is tuned for `heartbeat_interval` 1s.
### hard_timeout
| type | default | version |
|:----:|:-------:|:-------:|
| time | 60 | 0.14.0 |
The hard timeout used to detect server failure. The default value is equal to the `send_timeout` parameter.
### expire_dns_cache
| type | default | version |
|:----:|:----------------------:|:-------:|
| time | nil (persistent cache) | 0.14.0 |
Set TTL to expire DNS cache in seconds. Set 0 not to use DNS Cache.
### dns_round_robin
| type | default | version |
|:----:|:-------:|:-------:|
| bool | false | 0.14.0 |
Enable client-side DNS round robin. Uniform randomly pick an IP address to send data when a hostname has several IP addresses.
NOTE: `heartbeat_type udp` is not available with `dns_round_robin true`. Use `heartbeat_type tcp` or `heartbeat_type none`.
### ignore_network_errors_at_startup
| type | default | version |
|:----:|:-------:|:-------:|
| bool | false | 0.14.12 |
Ignore DNS resolution and errors at startup time.
### tls_version
| type | default | available | version |
|:----:|:-------:|:---------------:|:-------:|
| enum | TLSv1_2 | TLSv1_1, TLSv1_2| 0.14.12 |
The default version of TLS transport.
### tls_ciphers
| type | default | version |
|:------:|:--------------------------------------------------:|:-------:|
| string | ALL:!aNULL:!eNULL:!SSLv2 (OpenSSL > 1.0.0 default) | 0.14.12 |
The cipher configuration of TLS transport.
### tls_insecure_mode
| type | default | version |
|:----:|:-------:|:-------:|
| bool | false | 0.14.12 |
Skip all verification of certificates or not.
### tls_allow_self_signed_cert
| type | default | version |
|:----:|:-------:|:-------:|
| bool | false | 0.14.12 |
Allow self signed certificates or not.
### tls_verify_hostname
| type | default | version |
|:----:|:-------:|:-------:|
| bool | true | 0.14.12 |
Verify hostname of servers and certificates or not in TLS transport.
### tls_cert_path
| type | default | version |
|:---------------:|:-------:|:-------:|
| array of string | nil | 0.14.12 |
The additional CA certificate path for TLS.
### tls_client_cert_pat
| type | default | version |
|:------:|:-------:|:-------:|
| string | nil | 1.3.2 |
The client certificate path for TLS
### tls_client_private_key_path
| type | default | version |
|:------:|:-------:|:-------:|
| string | nil | 1.3.2 |
The client private key path for TLS.
### tls_client_private_key_passphrase
| type | default | version |
|:------:|:-------:|:-------:|
| string | nil | 1.3.2 |
The client private key passphrase for TLS.
### keepalive
| type | default | version |
|:----:|:-------:|:-------:|
| bool | fales | 1.4.3 |
Enable keepalive connection.
### keepalive_timeout
| type | default | version |
|:----:|:-------:|:-------:|
| time | nil | 1.4.3 |
Expired time of keepalive. Default value is nil, which means to keep connection as long as possible.
### <security> section
| required | multi | version |
|:--------:|:-----:|:-------:|
| false | false | 0.14.5 |
This section contains parameters related to authentication.
- self_hostname
- shared_key
#### self_hostname
| type | default | version |
|:------:|:------------------:|:-------:|
| string | required parameter | 0.14.5 |
The hostname.
#### shared_key
| type | default | version |
|:------:|:------------------:|:-------:|
| string | required parameter | 0.14.5 |
Shared key for authentication. If you want to specify `shared_key` for specific server, use `<server>` section.
### <secondary>
| required | multi | version |
|:--------:|:-----:|:-------:|
| false | false | 0.14.0 |
The backup destination that is used when all servers are unavailable.
For more details, see [Secondary Output](output-plugin-overview#secondary-output).
### verify_connection_at_startup
| type | default | version |
|:----:|:-------:|:-------:|
| bool | false | 1.3.1 |
Verify that a connection can be made with one of out_forward nodes at the time of startup.
## Tips & Tricks
### How to connect to a TLS/SSL enabled server
If you've [set up TLS/SSL encryption in the receiving server](in_forward#how-to-enable-tls/ssl-encryption), you need to tell the output forwarder to use encryption by setting the `transport` parameter:
```
<match debug.**>
@type forward
transport tls
<server>
host 192.168.1.2
port 24224
</server>
</match>
```
If you're using a self-singed certificate, copy the certificate file to the forwarding server, then add the following settings:
```
<match debug.**>
@type forward
transport tls
tls_cert_path /path/to/fluentd.crt # Set the path to the certificate file.
tls_verify_hostname true # Set false to ignore cert hostname.
<server>
host 192.168.1.2
port 24224
</server>
</match>
```
After updating the settings, please confirm that the forwarded data is being received by the destination node properly.
### How to Enable Password Authentication
If you want to connect to [a server that requires password authentication](in_forward#how-to-enable-password-authentication), you need to set your credentials in the configuration file.
<match debug.**>
@type forward
<server>
host 192.168.1.2
port 24224
</server>
<security>
self_hostname HOSTNAME
shared_key secret
</security>
</match>
Note that, as to the option `self_hostname`, you need to set the name of the server on which your `out_forward` instance is running. In the current implementation, it is considered invalid if your `in_forward` and `out_forward` shares the same hostname.
### How to enable gzip compression
Since v0.14.7, Fluentd supports transparent data compression. You can use this feature to reduce the transferred payload size.
To enable this feature, set the `compress` option as follows:
```
<match debug.**>
@type forward
compress gzip
<server>
host 192.168.1.2
port 24224
</server>
</match>
```
You don't need any configuration in the receiving server. Data compression is auto-detected and handled transparently by the destination node.
### What is a Phi accrual failure detector?
Fluentd implements an adaptive failure detection mechanism called "Phi accrual failure detector". Here is how it works:
1. Each `in_forward` node sends heartbeat packets to its `out_foward` server at a regular interval.
2. The `out_forward` server records the arrival time of heartbeat packets sent by each node.
3. If the server does not receive a heartbeat from one of its nodes for "a long time", it assumes the node is down.
But how long should the server wait before detaching a node? The phi accrual failure detector answers this question by computing the probability of a node being down based on the assumption that heartbeat intervals follow normal distribution. Internally it represent the confidence of a node being down by a continuous function *φ(t)* which grows as the time from the last packet increases.
For example, suppose that the historical average interval is 1 seconds and the standard deviation is 1, it's not likely that the node is still being active when its heartbeat does not arrive for the last 10 seconds.
For details, please read the original paper: [Hayashibara, Naohiro, et al. "The φ accrual failure detector." IEEE, 2004.](https://scholar.google.com/scholar?cluster=12946656837229314866)
## Troubleshooting
### "no nodes are available"
Please make sure that you can communicate with port 24224 using **not only TCP, but also UDP**. These commands will be useful for checking the network configuration.
:::term
$ telnet host 24224
$ nmap -p 24224 -sU host
Please note that there is one [known issue](http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2019944) where VMware will occasionally lose small UDP packets used for heartbeat.