Skip to content

Commit 579cf59

Browse files
committed
Added new storage configuration section with table for storage service options and linked to DLQ documentation in service section docs. Applies to #2207.
Signed-off-by: Eric D. Schabell <[email protected]>
1 parent c8ed628 commit 579cf59

File tree

1 file changed

+19
-0
lines changed

1 file changed

+19
-0
lines changed

administration/configuring-fluent-bit/yaml/service-section.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,27 @@ The `service` section defines global properties of the service. The available co
2626
| `sp.convert_from_str_to_num` | If enabled, the stream processor converts strings that represent numbers to a numeric type. | `true` |
2727
| `windows.maxstdio` | If specified, adjusts the limit of `stdio`. Only provided for Windows. Values from `512` to `2048` are allowed. | `512` |
2828

29+
### Storage configuration
30+
31+
The following storage-related keys can be set in the `service` section:
32+
33+
| Key | Description | Default Value |
34+
| --- | ----------- | ------------- |
35+
| `storage.path` | Set a location in the file system to store streams and chunks of data. Required for filesystem buffering. | _none_ |
36+
| `storage.sync` | Configure the synchronization mode used to store data in the file system. Accepted values: `normal` or `full`. | `normal` |
37+
| `storage.checksum` | Enable data integrity check when writing and reading data from the filesystem. Accepted values: `off` or `on`. | `off` |
38+
| `storage.max_chunks_up` | Set the maximum number of chunks that can be `up` in memory when using filesystem storage. | `128` |
39+
| `storage.backlog.mem_limit` | Set the memory limit for backlog data chunks. | `5M` |
40+
| `storage.backlog.flush_on_shutdown` | Attempt to flush all backlog chunks during shutdown. Accepted values: `off` or `on`. | `off` |
41+
| `storage.metrics` | Enable storage layer metrics on the HTTP endpoint. Accepted values: `off` or `on`. | `off` |
42+
| `storage.delete_irrecoverable_chunks` | Delete irrecoverable chunks during runtime and at startup. Accepted values: `off` or `on`. | `off` |
43+
| `storage.keep.rejected` | Enable the dead letter queue (DLQ) to preserve chunks that fail to be delivered. Accepted values: `off` or `on`. | `off` |
44+
| `storage.rejected.path` | Subdirectory name under `storage.path` for storing rejected chunks. | `rejected` |
45+
2946
For scheduler and retry details, see [scheduling and retries](../../scheduling-and-retries.md#Scheduling-and-Retries).
3047

48+
For storage and buffering details, see [buffering and storage](../../buffering-and-storage.md).
49+
3150
## Configuration example
3251

3352
The following configuration example that defines a `service` section with [hot reloading](../../hot-reload.md) enabled and a pipeline with a `random` input and `stdout` output:

0 commit comments

Comments
 (0)