You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: expand README with new utility scripts and usage examples
Updated the README.md to include detailed descriptions and usage instructions for new utility scripts:
- `kill_long_running_tasks.py`: A Dagster utility for managing long-running jobs.
- SQLite utilities: `create_index.py`, `list_tables.py`, `list_indexes.py`, and `qry.py` for database performance optimization and inspection.
Added quick examples for each script to enhance user guidance and improve overall documentation clarity.
Copy file name to clipboardExpand all lines: scripts/README.md
+90-4Lines changed: 90 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,15 +8,67 @@ The scripts in this directory provide helpful utilities for common administrativ
8
8
9
9
## Structure
10
10
11
-
### `sqlite/`
12
-
Contains SQLite-specific utility scripts for users running Anomstack with SQLite as their database backend.
11
+
### `kill_long_running_tasks.py`
12
+
Dagster utility for terminating or marking as failed any runs that exceed the configured timeout.
13
13
14
-
### `utils/`
15
-
Contains general utility scripts for system management and maintenance.
14
+
**Features:**
15
+
- Uses the same timeout configuration as the timeout sensor
16
+
- Gracefully handles unreachable user code servers
17
+
- Provides detailed logging of termination actions
18
+
- Automatically marks stuck runs as failed when termination isn't possible
19
+
20
+
**Usage:**
21
+
```bash
22
+
cd scripts/
23
+
python kill_long_running_tasks.py
24
+
```
16
25
17
26
### `posthog_example.py`
18
27
Runs the PostHog metrics ingest function to ensure your PostHog credentials work.
19
28
29
+
### `sqlite/`
30
+
Contains SQLite-specific utility scripts for users running Anomstack with SQLite as their database backend:
31
+
32
+
#### `create_index.py`
33
+
Automatically creates performance indexes on common metric columns (`metric_timestamp`, `metric_batch`, `metric_type`) for all tables in the SQLite database.
34
+
35
+
**Usage:**
36
+
```bash
37
+
cd scripts/sqlite/
38
+
python create_index.py
39
+
```
40
+
41
+
#### `list_tables.py`
42
+
Lists all tables in the SQLite database with their names and details.
43
+
44
+
**Usage:**
45
+
```bash
46
+
cd scripts/sqlite/
47
+
python list_tables.py
48
+
```
49
+
50
+
#### `list_indexes.py`
51
+
Lists all indexes in the SQLite database with their names and associated tables.
52
+
53
+
**Usage:**
54
+
```bash
55
+
cd scripts/sqlite/
56
+
python list_indexes.py
57
+
```
58
+
59
+
#### `qry.py`
60
+
Executes the SQL query from `qry.sql` file and returns results as a DataFrame. Useful for running ad-hoc SQL queries against your SQLite database.
61
+
62
+
**Usage:**
63
+
```bash
64
+
cd scripts/sqlite/
65
+
# Edit qry.sql with your query first
66
+
python qry.py
67
+
```
68
+
69
+
### `utils/`
70
+
Contains general utility scripts for system management and maintenance.
71
+
20
72
### `utils/reset_docker.sh`
21
73
Comprehensive Docker reset utility with multiple cleanup levels:
22
74
-**Gentle**: Rebuild containers with fresh images (safest)
@@ -31,10 +83,16 @@ Can be run interactively or with specific reset levels via Makefile targets.
31
83
These scripts are typically used for:
32
84
33
85
-**Database Setup**: Initialize databases and create required tables
86
+
-**Database Performance**: Create indexes to improve query performance (`create_index.py`)
87
+
-**Database Inspection**: View table and index structures (`list_tables.py`, `list_indexes.py`)
34
88
-**Data Migration**: Move data between different storage backends
89
+
-**Data Analysis**: Run ad-hoc SQL queries for troubleshooting (`qry.py`)
35
90
-**Maintenance**: Clean up old data, optimize performance
91
+
-**Task Management**: Terminate stuck or long-running Dagster jobs (`kill_long_running_tasks.py`)
36
92
-**Development**: Setup development environments and test data
37
93
-**Deployment**: Automate deployment tasks and configuration
94
+
-**System Reset**: Clean up Docker environments with various levels of data preservation (`reset_docker.sh`)
95
+
-**Credential Validation**: Test external service integrations (`posthog_example.py`)
0 commit comments