Enhance CLAUDE.md with comprehensive development guidance

Add detailed documentation for future Claude Code instances:

- Add Documentation Commands section with Docusaurus workflow
- Expand Testing & Quality section with all test commands
- Add comprehensive Fly.io deployment commands section
- Enhance Reset & Cleanup Operations with all cleanup levels
- Add Container Architecture section explaining 3-container design
- Expand Jobs Workflow from 5 to all 12 job types
- Add Dagster Sensors section documenting monitoring sensors
- Enhance Database Seeding & Examples with command references
- Improve Entry Points with config.py documentation
- Expand Metric Examples with 26+ available examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

Author: andrewm4894

CLAUDE.md

@@ -27,14 +27,25 @@ Anomstack is an open-source anomaly detection system built on Dagster and FastHT
 - `make docker-stop` - Stop all containers
 
 ### Testing & Quality
-- `pytest` or `make tests` - Run test suite
-- `make pre-commit` - Run pre-commit hooks (ruff linting)
+- `pytest` or `make tests` - Run test suite (includes documentation link checking)
+- `make test-examples` - Run only example ingest function tests
 - `make coverage` - Run tests with coverage report
+- `make pre-commit` - Run pre-commit hooks (ruff linting)
+
+### Documentation Commands
+- `make docs` or `make docs-start` - Start Docusaurus dev server with live reload
+- `make docs-build` - Build static documentation site (includes broken link checking)
+- `make docs-test` - Test documentation for broken links
+- `make docs-serve` - Serve built documentation locally
+- `make docs-clear` - Clear documentation build cache
+- `make docs-install` - Install documentation dependencies
 
-### Database Seeding
+### Database Seeding & Examples
 - `make seed-local-db` - Seed local DB with python_ingest_simple data
 - `make seed-local-db-all` - Seed with all example metric batches
 - `make seed-local-db-custom BATCHES='batch1,batch2' DB_PATH='path/to/db'`
+- `make run-example EXAMPLE=<name>` - Test individual example ingest functions
+- `make list-examples` - List all 26+ available examples
 
 ### Configuration & Hot Reload
 - `make reload-config` - Reload configuration without restarting containers
@@ -44,11 +55,40 @@ Anomstack is an open-source anomaly detection system built on Dagster and FastHT
 ### Reset & Cleanup Operations
 - `make reset-interactive` - Interactive reset with guided options
 - `make reset-gentle` - Rebuild containers (safest reset)
-- `make reset-nuclear` - Remove everything including data
-- `make dagster-cleanup-standard` - Clean up old Dagster runs
+- `make reset-medium` - Remove containers, keep data volumes
+- `make reset-nuclear` - Remove everything including local data
+- `make reset-full-nuclear` - Nuclear + full docker system cleanup (maximum cleanup)
+- `make dagster-cleanup-status` - Show current Dagster storage usage
+- `make dagster-cleanup-minimal` - Remove old logs only (safe)
+- `make dagster-cleanup-standard` - Clean up old Dagster runs (older than 30 days)
+- `make dagster-cleanup-aggressive` - Remove runs older than 7 days
+- `make kill-long-runs` - Manually kill any Dagster runs exceeding configured timeout
+
+### Fly.io Deployment Commands
+- `make fly-preview` - Preview environment variables that will be set as Fly secrets
+- `make fly-deploy` - Deploy to Fly.io (reads .env file automatically)
+- `make fly-deploy-demo` - Deploy with demo profile (enables demo metric batches)
+- `make fly-deploy-production` - Deploy with production profile
+- `make fly-deploy-development` - Deploy with development profile (all examples enabled)
+- `make fly-deploy-demo-fresh` - Deploy with fresh build (clears Docker cache first)
+- `make fly-build-test` - Test Fly.io build locally before deploying
+- `make fly-docker-clean` - Clean Docker cache for Fly builds
+- `make fly-cleanup` - Run disk cleanup on Fly instance (requires SSH access)
+- `make fly-cleanup-preview` - Preview cleanup on Fly instance (dry run)
+- `make fly-status` - Check Fly.io app status (requires FLY_APP env var)
+- `make fly-logs` - View Fly.io app logs (requires FLY_APP env var)
+- `make fly-ssh` - SSH into Fly.io app (requires FLY_APP env var)
 
 ## Architecture
 
+### Container Architecture
+Anomstack uses a simplified 3-container Docker architecture:
+- **anomstack_webserver**: Consolidated Dagster webserver with embedded user code (no separate gRPC server)
+- **anomstack_daemon**: Dagster daemon for job scheduling and execution
+- **anomstack_dashboard**: FastHTML dashboard for metrics visualization
+
+This consolidated approach eliminates the previous gRPC code server, reducing network overhead and improving reliability through direct Python module loading.
+
 ### Core Components
 - **anomstack/**: Main application code
   - `main.py`: Dagster definitions and job orchestration
@@ -71,9 +111,22 @@ Metrics are organized into "batches" - collections of related metrics with share
 ### Jobs Workflow
 1. **Ingest**: Run SQL/Python to collect metrics
 2. **Train**: Train PyOD anomaly detection models
-3. **Score**: Score new data points for anomalies  
+3. **Score**: Score new data points for anomalies
 4. **Alert**: Send email/Slack alerts for detected anomalies
-5. **Plot**: Generate visualizations in Dagster UI
+5. **LLM Alert**: LLM-based anomaly detection and alerting using anomaly-agent
+6. **Plot**: Generate visualizations in Dagster UI
+7. **Change**: Change detection for metrics
+8. **Summary**: Daily summary emails
+9. **Delete**: Delete old metrics
+10. **Reload**: Configuration hot-reload job
+11. **Cleanup**: Disk space management
+12. **Retention**: Custom retention for SQLite
+
+### Dagster Sensors
+Three key sensors monitor the system:
+1. **email_on_run_failure**: Sends email notifications when Dagster runs fail
+2. **kill_long_running_runs**: Automatically terminates runs exceeding configured timeout (default 15 minutes, configurable via `ANOMSTACK_KILL_RUN_AFTER_MINUTES`)
+3. **config_file_watcher**: Smart file watcher that detects configuration changes and triggers reloads (enabled via `ANOMSTACK_CONFIG_WATCHER=true`)
 
 ### Database Storage
 All data stored in long-format "metrics" table with columns:
@@ -106,9 +159,10 @@ ANOMSTACK__PYTHON_INGEST_SIMPLE__ALERT_METHODS=email
 - `metrics/defaults/defaults.yaml`: Default parameters for all metric batches
 - `pyproject.toml`: Ruff linting configuration
 
-### Entry Points  
-- `anomstack/main.py`: Main Dagster definitions
+### Entry Points
+- `anomstack/main.py`: Main Dagster definitions (defines all jobs, schedules, and sensors)
 - `dashboard/app.py`: FastHTML dashboard application
+- `anomstack/config.py`: Configuration loading and environment variable override logic
 
 ### Database Connectors
 - `anomstack/external/`: Connectors for BigQuery, Snowflake, ClickHouse, DuckDB, SQLite
@@ -126,9 +180,8 @@ ANOMSTACK__PYTHON_INGEST_SIMPLE__ALERT_METHODS=email
 
 ### Testing
 - Tests in `tests/` directory
-- Use `pytest` or `make tests` for running tests
-- `make test-examples` - Run only example ingest function tests
 - Test coverage tracking with badges in README
+- See "Testing & Quality" section for test commands
 
 ### Deployment Options
 - Local Python environment
@@ -141,8 +194,15 @@ ANOMSTACK__PYTHON_INGEST_SIMPLE__ALERT_METHODS=email
 The `metrics/examples/` directory contains ready-to-use examples:
 - HackerNews story metrics via API
 - Weather data from Open Meteo
-- Stock prices from Yahoo Finance
+- Stock prices from Yahoo Finance (yfinance)
 - System metrics from Netdata
-- Simple Python-generated test metrics
+- Simple Python-generated test metrics (python_ingest_simple)
+- Earthquake data from USGS
+- ISS location tracking
+- PostHog analytics (requires credentials)
+- Currency exchange rates
+- And 26+ total examples
+
+When adding new metrics, follow existing patterns in examples and ensure proper `.yaml` configuration with required fields like `metric_batch`, `db`, and cron schedules.
 
-When adding new metrics, follow existing patterns in examples and ensure proper `.yaml` configuration with required fields like `metric_batch`, `db`, and cron schedules.
+Use `make run-example EXAMPLE=<name>` to test individual examples or `make list-examples` to see all available examples.