Skip to content

Commit b8b6545

Browse files
KyleKingKyleKing
committed
fix: patch dotted key access for single line
TODO: should be upstreamed into corallium for re-use
1 parent 3de9e44 commit b8b6545

File tree

7 files changed

+73
-13
lines changed

7 files changed

+73
-13
lines changed

‎docs/README.md‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,37 +15,42 @@ If you are looking for more functionality, there are many good alternatives: [hu
1515

1616
## Installation
1717

18-
[Install with `pipx`](https://pypi.org/project/pipx/)
18+
Install with [`pipx`](https://pypi.org/project/pipx), [`uv tool`](https://docs.astral.sh/uv/guides/tools), [`mise`](https://mise.jdx.dev/getting-started.html), or your other tool of choice for Python packages
1919

2020
```sh
21+
# Choose one:
2122
pipx install tail-jsonl
23+
uv tool install tail-jsonl # or: uvx tail-jsonl
24+
mise use -g pipx:tail-jsonl
2225
```
2326

2427
## Usage
2528

2629
Pipe JSONL output from any file, kubernetes (such as [stern](https://github.com/stern/stern)), Docker, etc.
2730

31+
Tip: use `|&` to ensure that stderr and stdout are formatted (if using latest versions of zsh/bash), but all of these examples only require `|`
32+
2833
```sh
2934
# Example piping input in shell
30-
echo '{"message": "message", "timestamp": "2023-01-01T01:01:01.0123456Z", "level": "debug", "data": true, "more-data": [null, true, -123.123]}' | tail-jsonl
31-
cat tests/data/logs.jsonl | tail-jsonl
35+
echo '{"message": "message", "timestamp": "2023-01-01T01:01:01.0123456Z", "level": "debug", "data": true, "more-data": [null, true, -123.123]}' |& tail-jsonl
36+
cat tests/data/logs.jsonl |& tail-jsonl
3237

3338
# Optionally, pre-filter or format with jq, grep, awk, or other tools
34-
cat tests/data/logs.jsonl | jq '.record' --compact-output | tail-jsonl
39+
cat tests/data/logs.jsonl | jq '.record' --compact-output |& tail-jsonl
3540

3641
# An example stern command (also consider -o=extjson)
37-
stern envvars --context staging --container gateway --since="60m" --output raw | tail-jsonl
42+
stern envvars --context staging --container gateway --since="60m" --output raw |& tail-jsonl
3843

3944
# Or with Docker Compose (note that awk, cut, and grep all buffer. For awk, add '; system("")')
40-
docker compose logs --follow | awk 'match($0, / \| \{.+/) { print substr($0, RSTART+3, RLENGTH); system("") }' | tail-jsonl
45+
docker compose logs --follow | awk 'match($0, / \| \{.+/) { print substr($0, RSTART+3, RLENGTH); system("") }' |& tail-jsonl
4146
```
4247

4348
## Configuration
4449

4550
Optionally, specify a path to a custom configuration file. For an example configuration file see: [./tests/config_default.toml](https://github.com/KyleKing/tail-jsonl/blob/main/tests/config_default.toml)
4651

4752
```sh
48-
echo '...' | tail-jsonl --config-path=~/.tail-jsonl.toml
53+
echo '...' |& tail-jsonl --config-path=~/.tail-jsonl.toml
4954
```
5055

5156
## Project Status

‎docs/docs/CHANGELOG.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,10 @@
1-
## Unreleased
1+
## 1.4.1 (2025-09-06)
2+
3+
### Fix
4+
5+
- handle lists in matched fields for LogTape
6+
7+
## 1.4.0 (2025-09-06)
28

39
### Feat
410

‎docs/docs/DEVELOPER_GUIDE.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,11 @@ poetry config pypi-token.pypi ...
4949
|-------------------------------------------|-----------:|--------:|---------:|---------:|
5050
| `tail_jsonl/__init__.py` | 4 | 0 | 0 | 100.0% |
5151
| `tail_jsonl/_private/__init__.py` | 0 | 0 | 0 | 100.0% |
52-
| `tail_jsonl/_private/core.py` | 45 | 1 | 0 | 95.9% |
52+
| `tail_jsonl/_private/core.py` | 54 | 1 | 0 | 97.0% |
5353
| `tail_jsonl/_runtime_type_check_setup.py` | 13 | 0 | 37 | 100.0% |
5454
| `tail_jsonl/config.py` | 23 | 0 | 0 | 100.0% |
5555
| `tail_jsonl/scripts.py` | 16 | 0 | 18 | 94.4% |
56-
| **Totals** | 101 | 1 | 55 | 97.2% |
56+
| **Totals** | 110 | 1 | 55 | 97.6% |
5757

58-
Generated on: 2025-09-06
58+
Generated on: 2025-09-10
5959
<!-- {cte} -->

‎tail_jsonl/_private/core.py‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,14 @@ def print_record(line: str, console: Console, config: Config) -> None:
7272
if (_this_level := get_level(name=record.level)) == logging.NOTSET and record.level:
7373
record.data['_level_name'] = record.level
7474

75+
# PLANNED: Consider moving to Corallium
76+
for dotted_key in config.keys.on_own_line:
77+
if '.' not in dotted_key:
78+
continue
79+
if value := dotted.get(record.data, dotted_key):
80+
record.data[dotted_key] = value if isinstance(value, str) else str(value)
81+
dotted.remove(record.data, dotted_key)
82+
7583
printer_kwargs = {
7684
'message': record.message,
7785
'is_header': False,
@@ -85,6 +93,6 @@ def print_record(line: str, console: Console, config: Config) -> None:
8593
keys = set(printer_kwargs)
8694
rich_printer(
8795
**printer_kwargs, # type: ignore[arg-type]
88-
# Ensure that there is no repeat keyword arguments
89-
**{f'_{key}' if key in keys else key: value for key, value in record.data.items()},
96+
# Try to print all values and avoid name collision
97+
**{f' {key}' if key in keys else key: value for key, value in record.data.items()},
9098
)

‎tests/_private/__snapshots__/test_core.ambr‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,5 +138,27 @@
138138
ZeroDivisionError: integer division or modulo by zero
139139

140140

141+
'''
142+
# ---
143+
# name: test_core[5]
144+
'''
145+
2025-09-10T15:31:37.651Z [ERROR ] ['Failed to load comments'] category=['app'] host=localhost:8080 method=GET
146+
path=/partials/comments referer=http://localhost:8080/comments requestId=4c58cd34-f521-4af1-8c6d-e61914216710
147+
url=http://localhost:8080/partials/comments error={'name': 'SourceError', 'message': 'Invalid for loop'}
148+
request={'method': 'GET', 'path': '/partials/comments'}
149+
∟ error.stack: SourceError: Invalid for loop
150+
at forTag (https://deno.land/x/[email protected]/plugins/for.ts:74:11)
151+
at Environment.compileTokens (https://deno.land/x/[email protected]/core/environment.ts:291:31)
152+
at Environment.compile (https://deno.land/x/[email protected]/core/environment.ts:136:19)
153+
at https://deno.land/x/[email protected]/core/environment.ts:250:21
154+
at eventLoopTick (ext:core/01_core.js:179:7)
155+
at async Environment.load (https://deno.land/x/[email protected]/core/environment.ts:255:12)
156+
at async renderTemplate (file:///Users/kyleking/Developer/kyleking/app-template/src/templates/engine.ts:8:20)
157+
at async file:///Users/kyleking/Developer/kyleking/app-template/src/partials/commentsRouter.ts:12:18
158+
at async dispatch
159+
(file:///Users/kyleking/Developer/kyleking/app-template/node_modules/.deno/[email protected]/node_modules/hono/dist/compose.js:
160+
22:17)
161+
at async file:///Users/kyleking/Developer/kyleking/app-template/src/app.ts:55:7
162+
141163
'''
142164
# ---

‎tests/_private/test_core.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,3 +85,21 @@ def test_core_wrap(console: Console):
8585
if platform.system() != 'Windows':
8686
expected = '<no timestamp> [NOTSET ] <no message> 0=--- 1=--- 2=---'
8787
assert result.strip() == expected
88+
89+
90+
def test_core_error_stack_on_own_line(console: Console):
91+
line = (
92+
'{"timestamp":"2025-09-10T15:31:37.651Z","level":"error","category":["app"],'
93+
'"message":["Failed to load comments"],"host":"localhost:8080","method":"GET","path":"/partials/comments",'
94+
'"referer":"http://localhost:8080/comments","requestId":"4c58cd34-f521-4af1-8c6d-e61914216710",'
95+
'"url":"http://localhost:8080/partials/comments","error":{"name":"SourceError","message":"Invalid for loop",'
96+
'"stack":"SourceError: Invalid for loop\\n at forTag (https://deno.land/x/[email protected]/plugins/for.ts:74:11)"},'
97+
'"request":{"method":"GET","path":"/partials/comments"}}'
98+
)
99+
print_record(line, console, Config())
100+
result = console.end_capture()
101+
# Should have promoted error.stack out of the error mapping
102+
assert 'error.stack:' in result or 'error.stack=' in result
103+
assert 'SourceError: Invalid for loop' in result
104+
# Original nested stack should not appear inside error={...}
105+
assert 'stack":' not in result.split('error={')[1] if 'error={' in result else True

‎tests/data/logs.jsonl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@
33
{"text": "2023-01-31 06:16:00.911 | WARNING | dodo:<module>:33 - warning-level log\n", "record": {"elapsed": {"repr": "0:00:00.341294", "seconds": 0.341294}, "exception": null, "extra": {}, "file": {"name": "dodo.py", "path": "/Users/kyleking/Developer/packages/tail-jsonl/dodo.py"}, "function": "<module>", "level": {"icon": "⚠️", "name": "WARNING", "no": 30}, "line": 33, "message": "warning-level log", "module": "dodo", "name": "dodo", "process": {"id": 16268, "name": "MainProcess"}, "thread": {"id": 8821244160, "name": "MainThread"}, "time": {"repr": "2023-01-31 06:16:00.911034-05:00", "timestamp": 1675160160.911034}}}
44
{"text": "2023-01-31 06:16:00.911 | ERROR | dodo:<module>:34 - error-level log\n", "record": {"elapsed": {"repr": "0:00:00.341591", "seconds": 0.341591}, "exception": null, "extra": {}, "file": {"name": "dodo.py", "path": "/Users/kyleking/Developer/packages/tail-jsonl/dodo.py"}, "function": "<module>", "level": {"icon": "❌", "name": "ERROR", "no": 40}, "line": 34, "message": "error-level log", "module": "dodo", "name": "dodo", "process": {"id": 16268, "name": "MainProcess"}, "thread": {"id": 8821244160, "name": "MainThread"}, "time": {"repr": "2023-01-31 06:16:00.911331-05:00", "timestamp": 1675160160.911331}}}
55
{"text": "2023-01-31 06:16:00.911 | ERROR | dodo:<module>:38 - exception-level log\nTraceback (most recent call last):\n\n File \"/Users/kyleking/Developer/packages/tail-jsonl/.venv/bin/doit\", line 8, in <module>\n sys.exit(main())\n │ │ └ <function main at 0x103c3a2a0>\n │ └ <built-in function exit>\n └ <module 'sys' (built-in)>\n File \"/Users/kyleking/Developer/packages/tail-jsonl/.venv/lib/python3.11/site-packages/doit/__main__.py\", line 8, in main\n sys.exit(DoitMain().run(sys.argv[1:]))\n │ │ │ │ └ ['/Users/kyleking/Developer/packages/tail-jsonl/.venv/bin/doit', 'run', 'test']\n │ │ │ └ <module 'sys' (built-in)>\n │ │ └ <class 'doit.doit_cmd.DoitMain'>\n │ └ <built-in function exit>\n └ <module 'sys' (built-in)>\n File \"/Users/kyleking/Developer/packages/tail-jsonl/.venv/lib/python3.11/site-packages/doit/doit_cmd.py\", line 294, in run\n return command.parse_execute(args)\n │ │ └ ['test']\n │ └ <function Command.parse_execute at 0x1037e0fe0>\n └ <doit.cmd_run.Run object at 0x103876f50>\n File \"/Users/kyleking/Developer/packages/tail-jsonl/.venv/lib/python3.11/site-packages/doit/cmd_base.py\", line 150, in parse_execute\n return self.execute(params, args)\n │ │ │ └ ['test']\n │ │ └ {'dep_file': '.doit.db', 'backend': 'dbm', 'codec_cls': 'json', 'check_file_uptodate': 'md5', 'dodoFile': 'dodo.py', 'cwdPath...\n │ └ <function DoitCmdBase.execute at 0x1037e1da0>\n └ <doit.cmd_run.Run object at 0x103876f50>\n File \"/Users/kyleking/Developer/packages/tail-jsonl/.venv/lib/python3.11/site-packages/doit/cmd_base.py\", line 524, in execute\n self.loader.setup(params)\n │ │ │ └ {'dep_file': '.doit.db', 'backend': 'dbm', 'codec_cls': 'json', 'check_file_uptodate': 'md5', 'dodoFile': 'dodo.py', 'cwdPath...\n │ │ └ <function DodoTaskLoader.setup at 0x1037e1800>\n │ └ <doit.cmd_base.DodoTaskLoader object at 0x102f748d0>\n └ <doit.cmd_run.Run object at 0x103876f50>\n File \"/Users/kyleking/Developer/packages/tail-jsonl/.venv/lib/python3.11/site-packages/doit/cmd_base.py\", line 394, in setup\n self.namespace = dict(inspect.getmembers(loader.get_module(\n │ │ │ │ │ └ <function get_module at 0x102f6fce0>\n │ │ │ │ └ <module 'doit.loader' from '/Users/kyleking/Developer/packages/tail-jsonl/.venv/lib/python3.11/site-packages/doit/loader.py'>\n │ │ │ └ <function getmembers at 0x102f1dbc0>\n │ │ └ <module 'inspect' from '/opt/homebrew/Cellar/[email protected]/3.11.1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/insp...\n │ └ None\n └ <doit.cmd_base.DodoTaskLoader object at 0x102f748d0>\n File \"/Users/kyleking/Developer/packages/tail-jsonl/.venv/lib/python3.11/site-packages/doit/loader.py\", line 96, in get_module\n return importlib.import_module(os.path.splitext(file_name)[0])\n │ │ │ │ │ └ 'dodo.py'\n │ │ │ │ └ <function splitext at 0x102d0cea0>\n │ │ │ └ <module 'posixpath' (frozen)>\n │ │ └ <module 'os' (frozen)>\n │ └ <function import_module at 0x102f1cc20>\n └ <module 'importlib' from '/opt/homebrew/Cellar/[email protected]/3.11.1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/im...\n\n File \"/opt/homebrew/Cellar/[email protected]/3.11.1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py\", line 126, in import_module\n return _bootstrap._gcd_import(name[level:], package, level)\n │ │ │ │ │ └ 0\n │ │ │ │ └ None\n │ │ │ └ 0\n │ │ └ 'dodo'\n │ └ <function _gcd_import at 0x102c53d80>\n └ <module '_frozen_importlib' (frozen)>\n\n File \"<frozen importlib._bootstrap>\", line 1206, in _gcd_import\n File \"<frozen importlib._bootstrap>\", line 1178, in _find_and_load\n File \"<frozen importlib._bootstrap>\", line 1149, in _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line 690, in _load_unlocked\n File \"<frozen importlib._bootstrap_external>\", line 940, in exec_module\n File \"<frozen importlib._bootstrap>\", line 241, in _call_with_frames_removed\n\n> File \"/Users/kyleking/Developer/packages/tail-jsonl/dodo.py\", line 36, in <module>\n 1 // 0\n\nZeroDivisionError: integer division or modulo by zero\n", "record": {"elapsed": {"repr": "0:00:00.341759", "seconds": 0.341759}, "exception": {"type": "ZeroDivisionError", "value": "integer division or modulo by zero", "traceback": true}, "extra": {}, "file": {"name": "dodo.py", "path": "/Users/kyleking/Developer/packages/tail-jsonl/dodo.py"}, "function": "<module>", "level": {"icon": "❌", "name": "ERROR", "no": 40}, "line": 38, "message": "exception-level log", "module": "dodo", "name": "dodo", "process": {"id": 16268, "name": "MainProcess"}, "thread": {"id": 8821244160, "name": "MainThread"}, "time": {"repr": "2023-01-31 06:16:00.911499-05:00", "timestamp": 1675160160.911499}}}
6+
{"timestamp":"2025-09-10T15:31:37.651Z","level":"error","category":["app"],"message":["Failed to load comments"],"host":"localhost:8080","method":"GET","path":"/partials/comments","referer":"http://localhost:8080/comments","requestId":"4c58cd34-f521-4af1-8c6d-e61914216710","url":"http://localhost:8080/partials/comments","error":{"name":"SourceError","message":"Invalid for loop","stack":"SourceError: Invalid for loop\n at forTag (https://deno.land/x/[email protected]/plugins/for.ts:74:11)\n at Environment.compileTokens (https://deno.land/x/[email protected]/core/environment.ts:291:31)\n at Environment.compile (https://deno.land/x/[email protected]/core/environment.ts:136:19)\n at https://deno.land/x/[email protected]/core/environment.ts:250:21\n at eventLoopTick (ext:core/01_core.js:179:7)\n at async Environment.load (https://deno.land/x/[email protected]/core/environment.ts:255:12)\n at async renderTemplate (file:///Users/kyleking/Developer/kyleking/app-template/src/templates/engine.ts:8:20)\n at async file:///Users/kyleking/Developer/kyleking/app-template/src/partials/commentsRouter.ts:12:18\n at async dispatch (file:///Users/kyleking/Developer/kyleking/app-template/node_modules/.deno/[email protected]/node_modules/hono/dist/compose.js:22:17)\n at async file:///Users/kyleking/Developer/kyleking/app-template/src/app.ts:55:7"},"request":{"method":"GET","path":"/partials/comments"}}

0 commit comments

Comments
 (0)