Disk buffer updates for 4.0

content/headless/chunk/option-description-destination-diskbuffer-prealloc.md

@@ -0,0 +1,5 @@
+By default, {{% param "product.abbrev" %}} doesn’t reserve the disk space for the disk-buffer file, since in a properly configured and sized environment the disk-buffer is practically empty, so a large preallocated disk-buffer file is just a waste of disk space. But a preallocated buffer can prevent other data from using the intended buffer space (and elicit a warning from the OS if disk space is low), preventing message loss if the buffer is actually needed. To avoid this problem, when using {{% param "product.abbrev" %}} 4.0 or later, you can preallocate the space for your disk-buffer files by setting `prealloc(yes)`.
+
+In addition to making sure that the required disk space is available when needed, preallocated disk-buffer files provide radically better (3-4x) performance as well: in case of an outage the amount of messages stored in the disk-buffer is continuously growing, and using large continuous files is faster, than constantly waiting on a file to change its size.
+
+If you are running {{% param "product.abbrev" %}} on a dedicated host (always recommended for any high-volume settings), use `prealloc(yes)`.

content/headless/chunk/option-destination-diskbuffer.md

@@ -76,6 +76,15 @@ If the `dir()` path provided by the user does not exist, {{% param "product.ose"
 
 *Description:* Use this option if the option `reliable()` is set to `yes`. This option contains the size of the messages in bytes that is used in the memory part of the disk buffer. It replaces the old `log-fifo-size()` option. It does not inherit the value of the global `log-fifo-size()` option, even if it is provided. Note that this option will be ignored if the option `reliable()` is set to `no`.
 
+### prealloc() {#diskbuf-prealloc}
+
+| Type:        | yes/no    |
+|--------------|-----------|
+| Default:     | no        |
+
+*Description:* {{< include-headless "chunk/option-description-destination-diskbuffer-prealloc.md" >}}
+
+Available in {{% param "product.abbrev" %}} 4.0 and later.
 
 ### qout-size()
 
@@ -87,6 +96,27 @@ If the `dir()` path provided by the user does not exist, {{% param "product.ose"
 
 Options `reliable()` and `disk-buf-size()` are required options.
 
+### truncate-size-ratio() {#diskbuf-trunkate-size-ratio}
+
+| Type:        | number((between 0 and 1))    |
+|--------------|-----------|
+| Default:     | 1 (do not truncate)   |
+
+*Description:* Limits the truncation of the disk-buffer file. Truncating the disk-buffer file can slow down the disk IO operations, but it saves disk space. By default, {{% param "product.abbrev" %}} version 4.0 and later doesn't truncate disk-buffer files by default (`truncate-size-ratio(1)`). Earlier versions freed the disk-space when at least 10% of the disk-buffer file could be freed (`truncate-size-ratio(0.1)`).
+
+{{% param "product.abbrev" %}} only truncates the file if the possible disk gain is more than `truncate-size-ratio()` times `disk-buf-size()`.
+
+- Smaller values free disk space quicker.
+- Larger ratios result in better performance.
+
+If you want to avoid performance fluctuations:
+
+- use `truncate-size-ratio(1)` (never truncate), or
+- use `prealloc(yes)` to [reserve the entire size of the disk-buffer on disk](#diskbuf-prealloc).
+
+{{% alert title="Warning" color="warning" %}}
+{{% param "product.companyabbrev" %}} does not recommend you to change `truncate-size-ratio()`. Only change its value if you understand the performance implications of doing so.
+{{% /alert %}}
 
 ### Example: Examples for using disk-buffer()
 
@@ -123,17 +153,3 @@ In the following case normal disk-buffer() is used.
         );
     };
 ```
-
-### truncate-size-ratio() {#diskbuf-trunkate-size-ratio}
-
-| Type:        | number((between 0 and 1))    |
-|--------------|-----------|
-| Default:     | 0.1 (10%)   |
-
-*Description:* Limits the truncation of the disk-buffer file. Truncating the disk-buffer file can slow down the disk IO operations, but it saves disk space, so AxoSyslog only truncates the file, if the possible disk gain is more than `truncate-size-ratio()` times `disk-buf-size()`.
-
-{{% alert title="Warning" color="warning" %}}
-
-{{% param "product.companyabbrev" %}} does not recommend you to change `truncate-size-ratio()`. Only change its value if you know the performance implications of doing so.
-
-{{% /alert %}}

content/headless/chunk/topic-about-disk-queue-files.md

@@ -10,23 +10,20 @@ Both reliable and normal disk-buffers employ an in-memory output queue (set in `
 
 ## Size of the queue files
 
-Disk queue files tend to grow. Each may take up to `disk-buf-size()` bytes on the disk. Due to the nature of reliable queue files, all the messages traversing the queue are written to disk, constantly increasing the size of the queue file.
+Disk queue files grow. Each may take up to `disk-buf-size()` bytes on the disk. Due to the nature of reliable queue files, all the messages traversing the queue are written to disk, constantly increasing the size of the queue file.
 
-The disk-buffer file's size should be considered as the configured `disk-buf-size()` at any point of time, even if it does not have messages in it. Truncating the disk-buffer file can slow down disk I/O operations, so {{% param "product.abbrev" %}} does not always truncate the file when it would be possible (see the `truncate-size-ratio()` option). If a large disk-buffer file is not desirable, you should set the `disk-buf-size()` option to a smaller value.
-
-{{% alert title="Warning" color="warning" %}}
-
-{{% param "product.companyabbrev" %}} recommends that you do not build upon the current truncating logic of the disk-buffer files, because {{% param "product.abbrev" %}} might pre-allocate the disk-buffer files and never truncate them in the future.
-
-{{% /alert %}}
+The disk-buffer file's size should be considered as the configured `disk-buf-size()` at any point of time, even if it does not have messages in it. Truncating the disk-buffer file can slow down disk I/O operations, so {{% param "product.abbrev" %}} does not always truncate the file when it would be possible (see the `truncate-size-ratio()` option). If a large disk-buffer file is not desirable, you should set the `disk-buf-size()` option to a smaller value. Note that {{% param "product.abbrev" %}} version 4.0 and later doesn't truncate disk-buffer files by default.
 
+Starting with {{% param "product.abbrev" %}} version 4.0, you can [preallocate disk-buffer files](#preallocate).
 
 {{% alert title="Note" color="info" %}}
-
 The disk-buffer file's size does not strictly correlate to the number of stored messages. If you want to get information about the disk-buffer, use `dqtool` (for more information, see {{% xref "/chapter-routing-filters/concepts-diskbuffer/get-information-about-disk-buffer-files/get-information-about-disk-buffer-files-getting-status-info/_index.md" %}}).
+{{% /alert %}}
 
-{{% /alert %}} {{% alert title="Note" color="info" %}}
-
+{{% alert title="Note" color="info" %}}
 If a queue file becomes corrupt, {{% param "product.abbrev" %}} starts a new one. This might lead to the queue files consuming more space in total than their maximal configured size and the number of configured queue files multiplied together.
-
 {{% /alert %}}
+
+## Preallocating disk-buffer files {#preallocate}
+
+{{< include-headless "chunk/option-description-destination-diskbuffer-prealloc.md" >}}