Skip to content

Commit 9fb0bfd

Browse files
github-actions[bot]alltilla
authored andcommitted
NEWS: generate for 4.1.1
Signed-off-by: github-actions <41898282+github-actions@users.noreply.github.com>
1 parent abdb77a commit 9fb0bfd

File tree

1 file changed

+6
-310
lines changed

1 file changed

+6
-310
lines changed

NEWS.md

Lines changed: 6 additions & 310 deletions
Original file line numberDiff line numberDiff line change
@@ -1,315 +1,10 @@
1-
4.1.0
1+
4.1.1
22
=====
33

4-
## Highlights
5-
6-
### PROXY protocol v2 support ([#4211](https://github.com/syslog-ng/syslog-ng/pull/4211))
7-
8-
We've added support for PROXY protocol v2 (`transport(proxied-tcp)`), a protocol
9-
used by network load balancers, such as Amazon Elastic Load Balancer and
10-
HAProxy, to carry original source/destination address information, as described
11-
in https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
12-
13-
### Metrics revised
14-
15-
#### Prometheus metric format ([#4325](https://github.com/syslog-ng/syslog-ng/pull/4325))
16-
17-
A new metric system has been introduced to syslog-ng, where metrics are
18-
identified by names and partitioned by labels, which is similar to the
19-
[Prometheus data model](https://prometheus.io/docs/concepts/data_model/).
20-
21-
The `syslog-ng-ctl stats prometheus` command can be used to query syslog-ng
22-
metrics in a format that conforms to the Prometheus text-based exposition
23-
format.
24-
25-
`syslog-ng-ctl stats prometheus --with-legacy-metrics` displays legacy metrics
26-
as well. Legacy metrics do not follow Prometheus' metric and label conventions.
27-
28-
#### Classification (metadata-based metrics) ([#4318](https://github.com/syslog-ng/syslog-ng/pull/4318))
29-
30-
`metrics-probe()`, a new parser has also been added, which counts messages
31-
passing through based on the metadata of each message. The parser creates
32-
labeled metrics based on the fields of the message.
33-
34-
Both the key and labels can be set in the config, the values of the labels can
35-
be templated. E.g.:
36-
```
37-
parser p_metrics_probe {
38-
metrics-probe(
39-
key("custom_key") # adds "syslogng_" prefix => "syslogng_custom_key"
40-
labels(
41-
"custom_label_name_1" => "foobar"
42-
"custom_label_name_2" => "${.custom.field}"
43-
)
44-
);
45-
};
46-
```
47-
With this config, it creates counters like these:
48-
```
49-
syslogng_custom_key{custom_label_name_1="foobar", custom_label_name_2="bar"} 1
50-
syslogng_custom_key{custom_label_name_1="foobar", custom_label_name_2="foo"} 1
51-
syslogng_custom_key{custom_label_name_1="foobar", custom_label_name_2="baz"} 3
52-
```
53-
54-
The minimal config creates counters with the key
55-
`syslogng_classified_events_total` and labels `app`, `host`, `program` and
56-
`source`. E.g.:
57-
```
58-
parser p_metrics_probe {
59-
metrics-probe();
60-
};
61-
```
62-
With this config, it creates counters like these:
63-
```
64-
syslogng_classified_events_total{app="example-app", host="localhost", program="baz", source="s_local_1"} 3
65-
syslogng_classified_events_total{app="example-app", host="localhost", program="bar", source="s_local_1"} 1
66-
syslogng_classified_events_total{app="example-app", host="localhost", program="foo", source="s_local_1"} 1
67-
```
68-
69-
#### Named log paths (path ingress/egress metrics) ([#4344](https://github.com/syslog-ng/syslog-ng/pull/4344))
70-
71-
It is also possible to create named log paths, for example:
72-
73-
```
74-
log top-level {
75-
source(s_local);
76-
77-
log inner-1 {
78-
filter(f_inner_1);
79-
destination(d_local_1);
80-
};
81-
82-
log inner-2 {
83-
filter(f_inner_2);
84-
destination(d_local_2);
85-
};
86-
};
87-
```
88-
89-
Each named log path counts its ingress and egress messages:
90-
```
91-
syslogng_log_path_ingress{id="top-level"} 114
92-
syslogng_log_path_ingress{id="inner-1"} 114
93-
syslogng_log_path_ingress{id="inner-2"} 114
94-
syslogng_log_path_egress{id="top-level"} 103
95-
syslogng_log_path_egress{id="inner-1"} 62
96-
syslogng_log_path_egress{id="inner-2"} 41
97-
```
98-
99-
Note that the egress statistics only count the messages which have been have not been filtered out from the related
100-
log path, it does care about whether there are any destinations in it or that any destination delivers or drops the
101-
message.
102-
103-
The above three features are experimental; the output of `stats prometheus`
104-
(names, labels, etc.) and the metrics created by `metrics-probe()` and named log
105-
paths may change in the next 2-3 releases.
106-
107-
## Features
108-
109-
* `$(format-date)`: add a new template function to format time and date values
110-
111-
`$(format-date [options] format-string [timestamp])`
112-
113-
`$(format-date)` takes a timestamp in the DATETIME representation and
114-
formats it according to an strftime() format string. The DATETIME
115-
representation in syslog-ng is a UNIX timestamp formatted as a decimal
116-
number, with an optional fractional part, where the seconds and the
117-
fraction of seconds are separated by a dot.
118-
119-
If the timestamp argument is missing, the timestamp of the message is
120-
used.
121-
122-
Options:
123-
`--time-zone <TZstring>` -- override timezone of the original timestamp
124-
([#4202](https://github.com/syslog-ng/syslog-ng/pull/4202))
125-
126-
* `syslog-parser()` and all syslog related sources: accept unquoted RFC5424
127-
SD-PARAM-VALUEs instead of rejecting them with a parse error.
128-
129-
`sdata-parser()`: this new parser allows you to parse an RFC5424 style
130-
structured data string. It can be used to parse this relatively complex
131-
format separately.
132-
([#4281](https://github.com/syslog-ng/syslog-ng/pull/4281))
133-
134-
* `system()` source: the `system()` source was changed on systemd platforms to
135-
fetch journal messages that relate to the current boot only (e.g. similar
136-
to `journalctl -fb`) and to ignore messages generated in previous boots,
137-
even if those messages were succesfully stored in the journal and were not
138-
picked up by syslog-ng. This change was implemented as the journald access
139-
APIs work incorrectly if time goes backwards across reboots, which is an
140-
increasingly frequent event in virtualized environments and on systems that
141-
lack an RTC. If you want to retain the old behaviour, please bypass the
142-
`system()` source and use `systemd-journal()` directly, where this option
143-
can be customized. The change is not tied to `@version` as we deemed the new
144-
behaviour fixing an actual bug. For more information consult #2836.
145-
146-
`systemd-journald()` source: add `match-boot()` and `matches()` options to
147-
allow you to constrain the collection of journal records to a subset of what
148-
is in the journal. `match-boot()` is a yes/no value that allows you to fetch
149-
messages that only relate to the current boot. `matches()` allows you to
150-
specify one or more filters on journal fields.
151-
152-
Examples:
153-
154-
```
155-
source s_journal_current_boot_only {
156-
systemd-source(match-boot(yes));
157-
};
158-
159-
source s_journal_systemd_only {
160-
systemd-source(matches(
161-
"_COMM" => "systemd"
162-
)
163-
);
164-
};
165-
```
166-
([#4245](https://github.com/syslog-ng/syslog-ng/pull/4245))
167-
168-
* `date-parser()`: add `value()` parameter to instruct `date-parser()` to store
169-
the resulting timestamp in a name-value pair, instead of changing the
170-
timestamp value of the LogMessage.
171-
172-
`datetime` type representation: typed values in syslog-ng are represented as
173-
strings when stored as a part of a log message. syslog-ng simply remembers
174-
the type it was stored as. Whenever the value is used as a specific type in
175-
a type-aware context where we need the value of the specific type, an
176-
automatic string parsing takes place. This parsing happens for instance
177-
whenever syslog-ng stores a datetime value in MongoDB or when
178-
`$(format-date)` template function takes a name-value pair as parameter.
179-
The datetime() type has stored its value as the number of milliseconds since
180-
the epoch (1970-01-01 00:00:00 GMT). This has now been enhanced by making
181-
it possible to store timestamps up to nanosecond resolutions along with an
182-
optional timezone offset.
183-
184-
`$(format-date)`: when applied to name-value pairs with the `datetime` type,
185-
use the timezone offset if one is available.
186-
([#4319](https://github.com/syslog-ng/syslog-ng/pull/4319))
187-
188-
* `stats`: Added `syslog-stats()` global `stats()` group option.
189-
190-
E.g.:
191-
```
192-
options {
193-
stats(
194-
syslog-stats(no);
195-
);
196-
};
197-
```
198-
199-
It changes the behavior of counting messages based on different syslog-proto fields,
200-
like `SEVERITY`, `FACILITY`, `HOST`, etc...
201-
202-
Possible values are:
203-
* `yes` => force enable
204-
* `no` => force disable
205-
* `auto` => let `stats(level())` decide (old behavior)
206-
([#4337](https://github.com/syslog-ng/syslog-ng/pull/4337))
207-
208-
* `kubernetes` source: Added `key-delimiter()` option.
209-
210-
Some metadata fields can contain `.`-s in their name. This does not work with syslog-ng-s macros, which
211-
by default use `.` as a delimiter. The added `key-delimiter()` option changes this behavior by storing
212-
the parsed metadata fields with a custom delimiter. In order to reach the fields, the accessor side has
213-
to use the new delimiter format, e.g. `--key-delimiter` option in `$(format-json)`.
214-
([#4213](https://github.com/syslog-ng/syslog-ng/pull/4213))
215-
2164
## Bugfixes
2175

218-
* Fix conditional evaluation with a dangling filter
219-
220-
We've fixed a bug that caused conditional evaluation (if/else/elif) and certain logpath flags (`final`, `fallback`)
221-
to occasionally malfunction. The issue only happened in certain logpath constructs; examples can be found in the
222-
PR description.
223-
([#4058](https://github.com/syslog-ng/syslog-ng/pull/4058))
224-
* `python`: Fixed a bug, where `PYTHONPATH` was ignored with `python3.11`.
225-
([#4298](https://github.com/syslog-ng/syslog-ng/pull/4298))
226-
* `disk-buffer`: Fixed disk-queue file becoming corrupt when changing `disk-buf-size()`.
227-
228-
`syslog-ng` now continues with the originally set `disk-buf-size()`.
229-
Note that changing the `disk-buf-size()` of an existing disk-queue was never supported,
230-
but could cause errors, which are fixed now.
231-
([#4308](https://github.com/syslog-ng/syslog-ng/pull/4308))
232-
* `dqtool`: fix `dqtool assign`
233-
([#4355](https://github.com/syslog-ng/syslog-ng/pull/4355))
234-
* `example-diskq-source`: Fixed failing to read the disk-queue content in some cases.
235-
([#4308](https://github.com/syslog-ng/syslog-ng/pull/4308))
236-
* `default-network-drivers()`: Added support for the `log-iw-size()` option with a default value of 1000.
237-
Making it possible to adjust the `log-iw-size()` for the TCP/TLS based connections, when changing the `max-connections()` option.
238-
([#4328](https://github.com/syslog-ng/syslog-ng/pull/4328))
239-
* `apache-accesslog-parser()`: fix rawrequest escaping binary characters
240-
([#4303](https://github.com/syslog-ng/syslog-ng/pull/4303))
241-
* `dqtool`: Fixed `dqtool cat` failing to read the content in some cases.
242-
([#4308](https://github.com/syslog-ng/syslog-ng/pull/4308))
243-
* Fixed a rare main loop related crash on FreeBSD.
244-
([#4262](https://github.com/syslog-ng/syslog-ng/pull/4262))
245-
* Fix a warning message that was displayed incorrectly:
246-
"The actual number of worker threads exceeds the number of threads estimated at startup."
247-
([#4282](https://github.com/syslog-ng/syslog-ng/pull/4282))
248-
* Fix minor memory leak related to tznames
249-
([#4334](https://github.com/syslog-ng/syslog-ng/pull/4334))
250-
251-
## Packaging
252-
253-
* `dbparser`: libdbparser.so has been renamed to libcorrelation.so.
254-
([#4294](https://github.com/syslog-ng/syslog-ng/pull/4294))
255-
* `systemd-journal`: Fixed a linker error, which occurred, when building with `--with-systemd-journal=optional`.
256-
([#4304](https://github.com/syslog-ng/syslog-ng/pull/4304))
257-
([#4302](https://github.com/syslog-ng/syslog-ng/pull/4302))
258-
259-
## Notes to developers
260-
261-
* `LogThreadedSourceDriver` and `Fetcher`: implement source-side batching
262-
support on the input path by assigning a thread_id to dynamically spawned
263-
input threads (e.g. those spawned by LogThreadedSourceDriver) too. To
264-
actually improve performance the source driver should disable automatic
265-
closing of batches by setting `auto_close_batches` to FALSE and calling
266-
log_threaded_source_close_batch() explicitly.
267-
([#3969](https://github.com/syslog-ng/syslog-ng/pull/3969))
268-
269-
## Other changes
270-
271-
* stats related options: The stats related options have been groupped to a new `stats()` block.
272-
273-
This affects the following global options:
274-
* `stats-freq()`
275-
* `stats-level()`
276-
* `stats-lifetime()`
277-
* `stats-max-dynamics()`
278-
279-
These options have been kept for backward compatibility, but they have been deprecated.
280-
281-
Migrating from the old stats options to the new ones looks like this.
282-
```
283-
@version: 4.0
284-
285-
options {
286-
stats-freq(1);
287-
stats-level(1);
288-
stats-lifetime(1000);
289-
stats-max-dynamics(10000);
290-
};
291-
```
292-
```
293-
@version: 4.1
294-
295-
options {
296-
stats(
297-
freq(1)
298-
level(1)
299-
lifetime(1000)
300-
max-dynamics(10000)
301-
);
302-
};
303-
```
304-
305-
**Breaking change**
306-
For more than a decade `stats()` was a deprecated alias to `stats-freq()`, now it is used as the name
307-
of the new block. If you have been using `stats(xy)`, use `stats(freq(xy))` instead.
308-
([#4337](https://github.com/syslog-ng/syslog-ng/pull/4337))
309-
* `kubernetes` source: Improved error logging, when the pod was unreachable through the python API.
310-
([#4305](https://github.com/syslog-ng/syslog-ng/pull/4305))
311-
* APT repository: Added .gz, .xz and .bz2 compression to the Packages file.
312-
([#4313](https://github.com/syslog-ng/syslog-ng/pull/4313))
6+
* `db-parser()`, `grouping-by()`: Fixed a crash introduced in 4.1.0.
7+
([#4366](https://github.com/syslog-ng/syslog-ng/pull/4366))
3138

3149
## Credits
31510

@@ -322,5 +17,6 @@ of syslog-ng, contribute.
32217

32318
We would like to thank the following people for their contribution:
32419

325-
Attila Szakacs, Balazs Scheidler, Bálint Horváth, Gergo Ferenc Kovacs,
326-
Hofi, László Várady, Ronny Meeus, Szilard Parrag
20+
Attila Szakacs, Balazs Scheidler, Kyle Brown
21+
22+

0 commit comments

Comments
 (0)