You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Added new storage configuration section with table for storage service options and linked to DLQ documentation in service section docs. Applies to #2207.
Copy file name to clipboardExpand all lines: administration/configuring-fluent-bit/yaml/service-section.md
+19Lines changed: 19 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,8 +26,27 @@ The `service` section defines global properties of the service. The available co
26
26
|`sp.convert_from_str_to_num`| If enabled, the stream processor converts strings that represent numbers to a numeric type. |`true`|
27
27
|`windows.maxstdio`| If specified, adjusts the limit of `stdio`. Only provided for Windows. Values from `512` to `2048` are allowed. |`512`|
28
28
29
+
### Storage configuration
30
+
31
+
The following storage-related keys can be set in the `service` section:
32
+
33
+
| Key | Description | Default Value |
34
+
| --- | ----------- | ------------- |
35
+
|`storage.path`| Set a location in the file system to store streams and chunks of data. Required for filesystem buffering. |_none_|
36
+
|`storage.sync`| Configure the synchronization mode used to store data in the file system. Accepted values: `normal` or `full`. |`normal`|
37
+
|`storage.checksum`| Enable data integrity check when writing and reading data from the filesystem. Accepted values: `off` or `on`. |`off`|
38
+
|`storage.max_chunks_up`| Set the maximum number of chunks that can be `up` in memory when using filesystem storage. |`128`|
39
+
|`storage.backlog.mem_limit`| Set the memory limit for backlog data chunks. |`5M`|
40
+
|`storage.backlog.flush_on_shutdown`| Attempt to flush all backlog chunks during shutdown. Accepted values: `off` or `on`. |`off`|
41
+
|`storage.metrics`| Enable storage layer metrics on the HTTP endpoint. Accepted values: `off` or `on`. |`off`|
42
+
|`storage.delete_irrecoverable_chunks`| Delete irrecoverable chunks during runtime and at startup. Accepted values: `off` or `on`. |`off`|
43
+
|`storage.keep.rejected`| Enable the dead letter queue (DLQ) to preserve chunks that fail to be delivered. Accepted values: `off` or `on`. |`off`|
44
+
|`storage.rejected.path`| Subdirectory name under `storage.path` for storing rejected chunks. |`rejected`|
45
+
29
46
For scheduler and retry details, see [scheduling and retries](../../scheduling-and-retries.md#Scheduling-and-Retries).
30
47
48
+
For storage and buffering details, see [buffering and storage](../../buffering-and-storage.md).
49
+
31
50
## Configuration example
32
51
33
52
The following configuration example that defines a `service` section with [hot reloading](../../hot-reload.md) enabled and a pipeline with a `random` input and `stdout` output:
0 commit comments