Skip to content

Commit 037f2c6

Browse files
committed
chore: add doc for flatten_json_object
1 parent a799764 commit 037f2c6

File tree

2 files changed

+110
-0
lines changed

2 files changed

+110
-0
lines changed

docs/user-guide/logs/write-logs.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,62 @@ Here are some example of using `custom_time_index` assuming the time variable is
223223
- "2025-07-17T10:00:00+0800": `custom_time_index=input_ts;datestr;%Y-%m-%dT%H:%M:%S%z`
224224
- "2025-06-27T15:02:23.082253908Z": `custom_time_index=input_ts;datestr;%Y-%m-%dT%H:%M:%S%.9f%#z`
225225

226+
227+
#### Flatten JSON Objects
228+
229+
If flattening a JSON object into a single-level structure is needed, add the `x-greptime-pipeline-params` header to the request and set `flatten_json_object` to `true`.
230+
231+
Here is a sample request:
232+
233+
```shell
234+
curl -X "POST" "http://localhost:4000/v1/events/logs?db=<db-name>&table=<table-name>&pipeline_name=greptime_identity&version=<pipeline-version>" \
235+
-H "Content-Type: application/x-ndjson" \
236+
-H "Authorization: Basic {{authentication}}" \
237+
-H "x-greptime-pipeline-params: flatten_json_object=true" \
238+
-d "$<log-items>"
239+
```
240+
241+
With this configuration, GreptimeDB will automatically flatten each field of the JSON object into separate columns. For example:
242+
243+
```JSON
244+
{
245+
"a": {
246+
"b": {
247+
"c": [1, 2, 3]
248+
}
249+
},
250+
"d": [
251+
"foo",
252+
"bar"
253+
],
254+
"e": {
255+
"f": [7, 8, 9],
256+
"g": {
257+
"h": 123,
258+
"i": "hello",
259+
"j": {
260+
"k": true
261+
}
262+
}
263+
}
264+
}
265+
```
266+
267+
Will be flattened to:
268+
269+
```json
270+
{
271+
"a.b.c": [1,2,3],
272+
"d": ["foo","bar"],
273+
"e.f": [7,8,9],
274+
"e.g.h": 123,
275+
"e.g.i": "hello",
276+
"e.g.j.k": true
277+
}
278+
```
279+
280+
281+
226282
## Variable hints in the pipeline context
227283

228284
Starting from `v0.15`, the pipeline engine now recognizes certain variables, and can set corresponding table options based on the value of the variables.

i18n/zh/docusaurus-plugin-content-docs/current/user-guide/logs/write-logs.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,60 @@ DESC pipeline_logs;
223223
- "2025-07-17T10:00:00+0800": `custom_time_index=input_ts;datestr;%Y-%m-%dT%H:%M:%S%z`
224224
- "2025-06-27T15:02:23.082253908Z": `custom_time_index=input_ts;datestr;%Y-%m-%dT%H:%M:%S%.9f%#z`
225225

226+
227+
#### 展开 json 对象
228+
229+
如果用户希望将 JSON 对象展开为单层结构,可以在请求的 header 中添加 `x-greptime-pipeline-params` 参数,设置 `flatten_json_object``true`
230+
231+
以下是一个示例请求:
232+
233+
```shell
234+
curl -X "POST" "http://localhost:4000/v1/events/logs?db=<db-name>&table=<table-name>&pipeline_name=greptime_identity&version=<pipeline-version>" \
235+
-H "Content-Type: application/x-ndjson" \
236+
-H "Authorization: Basic {{authentication}}" \
237+
-H "x-greptime-pipeline-params: flatten_json_object=true" \
238+
-d "$<log-items>"
239+
```
240+
241+
这样,GreptimeDB 将自动将 JSON 对象的每个字段展开为单独的列。比如
242+
243+
```JSON
244+
{
245+
"a": {
246+
"b": {
247+
"c": [1, 2, 3]
248+
}
249+
},
250+
"d": [
251+
"foo",
252+
"bar"
253+
],
254+
"e": {
255+
"f": [7, 8, 9],
256+
"g": {
257+
"h": 123,
258+
"i": "hello",
259+
"j": {
260+
"k": true
261+
}
262+
}
263+
}
264+
}
265+
```
266+
267+
将被展开为:
268+
269+
```json
270+
{
271+
"a.b.c": [1,2,3],
272+
"d": ["foo","bar"],
273+
"e.f": [7,8,9],
274+
"e.g.h": 123,
275+
"e.g.i": "hello",
276+
"e.g.j.k": true
277+
}
278+
```
279+
226280
## Pipeline 上下文中的 hint 变量
227281

228282
`v0.15` 开始,pipeline 引擎可以识别特定的变量名称,并且通过这些变量对应的值设置相应的建表选项。

0 commit comments

Comments
 (0)