Skip to content

Commit 72d4af1

Browse files
committed
docs: expand README with new utility scripts and usage examples
Updated the README.md to include detailed descriptions and usage instructions for new utility scripts: - `kill_long_running_tasks.py`: A Dagster utility for managing long-running jobs. - SQLite utilities: `create_index.py`, `list_tables.py`, `list_indexes.py`, and `qry.py` for database performance optimization and inspection. Added quick examples for each script to enhance user guidance and improve overall documentation clarity.
1 parent 40800f7 commit 72d4af1

File tree

1 file changed

+90
-4
lines changed

1 file changed

+90
-4
lines changed

scripts/README.md

Lines changed: 90 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,67 @@ The scripts in this directory provide helpful utilities for common administrativ
88

99
## Structure
1010

11-
### `sqlite/`
12-
Contains SQLite-specific utility scripts for users running Anomstack with SQLite as their database backend.
11+
### `kill_long_running_tasks.py`
12+
Dagster utility for terminating or marking as failed any runs that exceed the configured timeout.
1313

14-
### `utils/`
15-
Contains general utility scripts for system management and maintenance.
14+
**Features:**
15+
- Uses the same timeout configuration as the timeout sensor
16+
- Gracefully handles unreachable user code servers
17+
- Provides detailed logging of termination actions
18+
- Automatically marks stuck runs as failed when termination isn't possible
19+
20+
**Usage:**
21+
```bash
22+
cd scripts/
23+
python kill_long_running_tasks.py
24+
```
1625

1726
### `posthog_example.py`
1827
Runs the PostHog metrics ingest function to ensure your PostHog credentials work.
1928

29+
### `sqlite/`
30+
Contains SQLite-specific utility scripts for users running Anomstack with SQLite as their database backend:
31+
32+
#### `create_index.py`
33+
Automatically creates performance indexes on common metric columns (`metric_timestamp`, `metric_batch`, `metric_type`) for all tables in the SQLite database.
34+
35+
**Usage:**
36+
```bash
37+
cd scripts/sqlite/
38+
python create_index.py
39+
```
40+
41+
#### `list_tables.py`
42+
Lists all tables in the SQLite database with their names and details.
43+
44+
**Usage:**
45+
```bash
46+
cd scripts/sqlite/
47+
python list_tables.py
48+
```
49+
50+
#### `list_indexes.py`
51+
Lists all indexes in the SQLite database with their names and associated tables.
52+
53+
**Usage:**
54+
```bash
55+
cd scripts/sqlite/
56+
python list_indexes.py
57+
```
58+
59+
#### `qry.py`
60+
Executes the SQL query from `qry.sql` file and returns results as a DataFrame. Useful for running ad-hoc SQL queries against your SQLite database.
61+
62+
**Usage:**
63+
```bash
64+
cd scripts/sqlite/
65+
# Edit qry.sql with your query first
66+
python qry.py
67+
```
68+
69+
### `utils/`
70+
Contains general utility scripts for system management and maintenance.
71+
2072
### `utils/reset_docker.sh`
2173
Comprehensive Docker reset utility with multiple cleanup levels:
2274
- **Gentle**: Rebuild containers with fresh images (safest)
@@ -31,10 +83,16 @@ Can be run interactively or with specific reset levels via Makefile targets.
3183
These scripts are typically used for:
3284

3385
- **Database Setup**: Initialize databases and create required tables
86+
- **Database Performance**: Create indexes to improve query performance (`create_index.py`)
87+
- **Database Inspection**: View table and index structures (`list_tables.py`, `list_indexes.py`)
3488
- **Data Migration**: Move data between different storage backends
89+
- **Data Analysis**: Run ad-hoc SQL queries for troubleshooting (`qry.py`)
3590
- **Maintenance**: Clean up old data, optimize performance
91+
- **Task Management**: Terminate stuck or long-running Dagster jobs (`kill_long_running_tasks.py`)
3692
- **Development**: Setup development environments and test data
3793
- **Deployment**: Automate deployment tasks and configuration
94+
- **System Reset**: Clean up Docker environments with various levels of data preservation (`reset_docker.sh`)
95+
- **Credential Validation**: Test external service integrations (`posthog_example.py`)
3896

3997
## Usage
4098

@@ -48,6 +106,34 @@ cd scripts/
48106
python script_name.py
49107
```
50108

109+
### Quick Examples
110+
111+
**Database performance optimization:**
112+
```bash
113+
cd scripts/sqlite/
114+
python create_index.py # Create performance indexes
115+
```
116+
117+
**Database inspection:**
118+
```bash
119+
cd scripts/sqlite/
120+
python list_tables.py # See all tables
121+
python list_indexes.py # See all indexes
122+
```
123+
124+
**Clean up stuck Dagster jobs:**
125+
```bash
126+
cd scripts/
127+
python kill_long_running_tasks.py
128+
```
129+
130+
**Complete Docker environment reset:**
131+
```bash
132+
make reset-interactive # Interactive mode
133+
# or
134+
make reset-nuclear # Nuclear reset
135+
```
136+
51137
## Database-Specific Scripts
52138

53139
Different database backends may have specific requirements and utilities:

0 commit comments

Comments
 (0)