Skip to content

Conversation

@r33drichards
Copy link
Owner

Summary

This PR adds an in-memory heap storage implementation for V8 heap snapshots, providing a fast, zero-configuration storage option ideal for development, testing, and ephemeral workloads.

Changes

New Implementation

  • InMemoryHeapStorage: Thread-safe in-memory storage using HashMap with Arc<RwLock>
  • Implements the HeapStorage trait for seamless integration
  • Added to AnyHeapStorage enum as InMemory variant

CLI Updates

  • Added --in-memory flag to enable in-memory storage
  • Proper conflict handling with other storage options (--s3-bucket, --directory-path, --stateless)
  • Enhanced logging to show which storage backend is being used

API Features

Core methods:

  • put(name, data) - Store heap snapshot
  • get(name) - Retrieve heap snapshot

Additional utilities:

  • len() - Get number of stored items
  • is_empty() - Check if storage is empty
  • clear() - Remove all stored data
  • contains_key(name) - Check if key exists
  • remove(name) - Remove specific item
  • keys() - Get all stored keys

Testing

  • Comprehensive unit tests covering:
    • Basic put/get operations
    • Error handling for missing keys
    • Utility method functionality
    • Clone behavior (shared storage)
    • Integration with AnyHeapStorage

Documentation

  • Added IN_MEMORY_STORAGE.md with:
    • Usage examples
    • Comparison with other storage backends
    • Limitations and best practices
    • Architecture details

Benefits

  • Performance: Fastest storage option (no I/O overhead)
  • Simplicity: No external dependencies or configuration
  • Development: Perfect for local testing and iteration
  • Thread-safe: Concurrent access with proper locking

Limitations

  • Data is not persisted (lost on process termination)
  • Memory bounded by available RAM
  • Not suitable for production with persistent data requirements
  • Single-process only (cannot be shared across instances)

Usage

# Run with in-memory storage
cargo run --bin server -- --in-memory

# With HTTP transport
cargo run --bin server -- --in-memory --http-port 8080

# With SSE transport
cargo run --bin server -- --in-memory --sse-port 8080

Testing

cargo test -p server heap_storage

- Implemented InMemoryHeapStorage using HashMap with Arc<RwLock>
- Added --in-memory CLI flag to use in-memory storage
- Provides thread-safe, zero-config storage for development and testing
- Added comprehensive unit tests for in-memory storage
- Added IN_MEMORY_STORAGE.md documentation
- Updated main.rs to support in-memory storage option
- InMemory variant added to AnyHeapStorage enum

Benefits:
- Fastest performance (no I/O overhead)
- No external dependencies
- Useful for testing, development, and ephemeral workloads
- Rich API with utility methods (len, keys, clear, contains_key, remove)

Limitations:
- Data lost on process termination
- Not suitable for production with persistent data needs
- Memory bounded by available RAM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants