You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: input/tail.md
+106Lines changed: 106 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -17,6 +17,16 @@ It is included in Fluentd's core.
17
17
<parse>
18
18
@type apache2
19
19
</parse>
20
+
<group>
21
+
rate_period 30s
22
+
<rule>
23
+
match {
24
+
namespace: /space1|space2|space2/,
25
+
podname: /app.*/,
26
+
}
27
+
limit 200
28
+
</rule>
29
+
</group>
20
30
</source>
21
31
```
22
32
@@ -402,6 +412,80 @@ The `@log_level` option allows the user to set different levels of logging for e
402
412
403
413
Refer to the [Logging](../deployment/logging.md) for more details.
404
414
415
+
416
+
### `<group>` Section \(highly recommended\)
417
+
418
+
The `in_tail` plugin can assign each log file to a group, based on user defined rules. The `limit` parameter controls the total number of lines collected for a group within a `rate_period` time interval.
Specifies the regular expression for extracting metadata (namespace, podname) from log file path. Default value of the pattern regexp extracts information about `namespace`, `podname`, `docker_id`, `container` of the log (K8s specific).
454
+
455
+
You can also add custom named captures in `pattern` for custom grouping of log files. For example,
456
+
```text
457
+
pattern /^\/home\/logs\/(?<file>.+)\.log$/
458
+
```
459
+
In this example, filename will be extracted and used to form groups.
460
+
461
+
#### `rate_period`
462
+
463
+
| type | default | version |
464
+
| :--- | :--- | :--- |
465
+
| time | 60 \(seconds\)| 1.15 |
466
+
467
+
Time period in which the group line limit is applied. `in_tail` resets the counter after every `rate_period` interval.
If this article is incorrect or outdated, or omits critical information, please [let us know](https://github.com/fluent/fluentd-docs-gitbook/issues?state=open). [Fluentd](http://www.fluentd.org/) is an open-source project under [Cloud Native Computing Foundation \(CNCF\)](https://cncf.io/). All components are available under the Apache 2 License.
471
555
556
+
### What happens when a file can be assigned to more than one group?
557
+
558
+
Example,
559
+
560
+
```text
561
+
<rule> ## Rule1
562
+
match {
563
+
namespace: /monitoring/
564
+
}
565
+
limit 100
566
+
</rule>
567
+
568
+
<rule> ## Rule2
569
+
match {
570
+
namespace: /monitoring/,
571
+
podname: /logger/,
572
+
}
573
+
limit 2000
574
+
</rule>
575
+
```
576
+
577
+
In this case, rules with more constraints, i.e., greater number of `match` hash keys will be given a higher priority. So a file will be assigned to `Rule2` if it can be assigned to both `Rule1` and `Rule2`.
0 commit comments