-
Notifications
You must be signed in to change notification settings - Fork 973
Add TimesFM 2.5: First Foundation Model Integration #2934
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
Hi @nehalecky and thanks for giving this a go 🚀 Before reviewing, I would actually like to take a step back and think about how we want to do this properly in Darts. We have started a discussion in #2933, feel free to join and share your feedback :) |
Implements TimesFMModel wrapper for Google's TimesFM 2.5-200M foundation model: - Zero-shot forecasting without training (pre-trained on 100B+ time points) - Auto device detection (CPU/CUDA/MPS for Apple Silicon) - Univariate time series support with arbitrary forecast horizons - Batch prediction on multiple series - Proper integration with Darts TimeSeries API and historical_forecasts Model configuration: - Configurable context length (default 1024, must be divisible by 32) - Lazy model loading from HuggingFace (google/timesfm-2.5-200m-pytorch) - Inherits from GlobalForecastingModel for full Darts compatibility Addresses issue unit8co#2359 (Pre-trained foundational models) Co-Authored-By: Claude <[email protected]> test: add comprehensive test suite for TimesFM model Comprehensive test coverage across three test classes: - TestTimesFMModelConstruction: Model initialization, validation, device detection - TestTimesFMModelFit: Fit behavior, univariate validation, zero-shot mode - TestTimesFMModelPredict: Forecasting, batch prediction, time index handling Key test scenarios: - Zero-shot forecasting without calling fit() first - Batch forecasting on multiple series - Multivariate component selection for notebook examples - Proper time index continuation - Historical forecasts compatibility (extreme_lags 7-tuple format) All 25 tests passing. Co-Authored-By: Claude <[email protected]> build: add TimesFM as optional dependency setup.py changes: - Added 'timesfm' extra with git+https source for timesfm[torch] - Users install with: pip install "darts[timesfm]" requirements/timesfm.txt: - Pinned dependency file for TimesFM PyTorch version - Installed from Google Research GitHub repository TimesFM is optional to avoid forcing large dependency on all users. Model gracefully falls back if timesfm not installed. Co-Authored-By: Claude <[email protected]>
Create comprehensive foundation models infrastructure: - FoundationForecastingModel base class with optional PEFT support - PEFT utilities following Hugging Face patterns (LoRA, adapters) - Device utilities for automatic detection (CUDA, MPS, CPU) - Move TimesFM to foundation/timesfm.py - Update imports across codebase Architecture: - darts/models/forecasting/foundation/__init__.py - darts/models/forecasting/foundation/base.py (base class) - darts/models/forecasting/foundation/peft_utils.py (LoRA support) - darts/models/forecasting/foundation/device_utils.py (utilities) - darts/models/forecasting/foundation/timesfm.py (moved from root) This establishes the pattern for future foundation models (Chronos 2, etc.) and enables parameter-efficient fine-tuning via Hugging Face PEFT library. Related: unit8co#2359, unit8co#2933 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> refactor: update imports for foundation package reorganization Update all TimesFM imports to use new foundation package location: - darts/models/__init__.py: Import from foundation package - test_timesfm_model.py: Update all 25 test imports Preserves backward compatibility - TimesFMModel still exported from darts.models as before. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Installation documentation: - INSTALL.md: Step-by-step installation via git clone + pip install - README.md: Foundation Models section with TimesFM quick-start example Architecture documentation: - docs/architecture/foundation-models.md: Design decisions and API patterns - docs/userguide/foundation_models.md: User guide with examples and use cases - Auto-generated API reference for TimesFMModel CHANGELOG updates: - Added TimesFMModel under Unreleased > New Models section Documentation highlights zero-shot capabilities, when to use foundation models vs traditional models, and integration with existing Darts workflows. Co-Authored-By: Claude <[email protected]> docs: add TimesFM foundation model example notebook Comprehensive Jupyter notebook demonstrating TimesFM capabilities: Section 1: Zero-Shot Forecasting - Immediate forecasting without training - 6.59% MAPE vs 8.10% for ExponentialSmoothing (18.6% better) Section 2: Comparison with Traditional Models - Analysis of when foundation models outperform traditional approaches - Trade-offs between zero-shot convenience and specialized tuning Section 3: Dataset Characteristics - Sunspots example showing TimesFM winning on irregular patterns (82.94% vs 108.27%) - Air Passengers showing traditional models excel on structured seasonality Section 4: Batch Forecasting - Multiple series forecasting with single model - Component selection from multivariate datasets Section 5: Backtesting - Rolling 12-month forecasts with overlapping windows (4.84% avg MAPE) - Production-ready validation with last_points_only=False - Zoomed visualization with distinct colors and dashed lines All sections tested and working end-to-end with actual performance metrics. Co-Authored-By: Claude <[email protected]> docs: fix notebook number in CHANGELOG (16 → 25) The TimesFM notebook is examples/25-TimesFM-foundation-model.ipynb, not examples/16. Correcting the CHANGELOG entry. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> docs: comprehensive foundation models documentation overhaul Major documentation improvements addressing 47 identified gaps: User Guide (docs/userguide/foundation_models.md): - FIX BLOCKER: Clarify fit() is optional (not removed) for API compatibility - Add comprehensive PEFT/LoRA fine-tuning section with code examples - Add 10+ external hyperlinks (TimesFM paper, Google blog, HuggingFace, PEFT docs) - Update roadmap section to focus on Chronos 2 (Q1 2026) - Add "Learn More" section with internal cross-references Chronos 2 Roadmap (docs/roadmap/foundation_models.md): - NEW: Comprehensive Chronos 2 integration plan - Architecture comparison table (TimesFM vs Chronos 2) - 3-phase implementation plan with success criteria - Technical challenges and solutions - Community engagement section CHANGELOG.md: - Update TimesFM entry to mention PEFT support - Add foundation/ package entry - Add comprehensive documentation entry - Reference Chronos 2 roadmap Key improvements: - All prose moved from print statements to markdown - Consistent terminology (fine-tuning vs finetune vs fine_tune) - Proper academic citations with hyperlinks - Clear distinction between fit() (API compat) and PEFT (fine-tuning) - Focus on Chronos 2 as next model (per unit8co#2933 demand) Closes unit8co#2359 (partial - TimesFM complete, Chronos 2 planned) References unit8co#2933 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> docs: correct foundation models documentation to match implementation CRITICAL: Remove documentation for non-existent capabilities. User Guide Corrections: - REMOVED: PEFT/LoRA fine-tuning documentation (not implemented) - REMOVED: Claims about FoundationForecastingModel (TimesFM uses GlobalForecastingModel) - CLARIFIED: fit() is for API compatibility only (no training occurs) - CLARIFIED: Univariate only (no cross-series dependencies) - CLARIFIED: Multivariate = cross-dependencies (Chronos 2 feature, not TimesFM) CHANGELOG Corrections: - REMOVED: PEFT fine-tuning claims - CLARIFIED: foundation/ package is infrastructure (not user-facing features) - ACCURATE: Only zero-shot forecasting currently works Roadmap Corrections: - SEPARATED: "What Works Today" vs "Infrastructure for Future" - CLARIFIED: PEFT utils raise NotImplementedError (architectural prep only) - NOTE: TimesFM extends GlobalForecastingModel (not FoundationForecastingModel) What Actually Works: ✅ TimesFM 2.5 zero-shot forecasting ✅ Univariate series only ✅ Device detection (CUDA, MPS, CPU) ✅ Darts integration (historical_forecasts, backtest) What Doesn't Work (Yet): ❌ Fine-tuning / PEFT ❌ Multivariate forecasting ❌ Covariates ❌ FoundationForecastingModel integration The infrastructure exists for future development (Chronos 2, etc.) but is not user-facing. Documentation now accurately reflects reality. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> docs: clarify PEFT is future infrastructure, not current feature - Remove Phase 3: PEFT Support from Chronos 2 roadmap - Remove PEFT from success criteria and technical metrics - Clarify PEFT utilities are for future research - Update contribution opportunities to accurately reflect current state - Change 'Executive Summary' to 'Summary' in architecture doc This ensures we don't overpromise capabilities and clearly distinguish between current features (zero-shot forecasting) and future research directions (fine-tuning with PEFT). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> docs: enhance foundation models guide with TSFM acronym and semantic intelligence **Key Enhancements:** - Introduce Time Series Foundation Models (TSFMs) acronym upfront - Add broader bodies of knowledge references (NLP, CV foundation models) - Emphasize "zero-shot revolution" and solution-oriented tone - Drastically improve covariates vs examples distinction: * TIME-DISTINGUISHED (covariates): temporal causality * SHAPE TEMPLATES (examples): pattern recognition through cycles/seasonality * Highlight TSFMs' semantic intelligence in distinguishing influence vs resemblance - Add info panels for design decisions: * Why GlobalForecastingModel (compatibility-first) * fit() method purpose (validation, not training) * Lazy loading pattern for true zero-shot - Convert ASCII table to proper markdown - Remove GenAI writing patterns ("it's not just X—it's Y") - Remove references to moved architecture/roadmap docs - Add experimental test confirming TimesFM is univariate-only **References Added:** - Foundation Models Survey (arxiv.org/abs/2108.07258) - BERT paper (arxiv.org/abs/1810.04805) - Vision foundation models (arxiv.org/abs/2010.11929) - Chronos paper (arxiv.org/abs/2403.07815) This documentation now accurately reflects TSFMs' robustness and their sophisticated understanding of semantic differences in time series context. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Experimental test confirmed TimesFM is univariate-only (raises "TimesFM only supports univariate series"). This is now properly documented in user guide. Removed test to keep test suite clean - the univariate validation is already tested through the implementation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> fix(foundation): make psutil optional dependency for device utilities Make psutil an optional import with graceful fallback for device_utils module. This prevents ImportError when psutil is not installed while still providing full functionality when available. Changes: - Add try/except block for psutil import with HAS_PSUTIL flag - Update get_device_memory_info() to check HAS_PSUTIL before use - Add debug logging when psutil is unavailable - Update docstring to mention optional psutil installation Impact: - CUDA memory reporting still works (uses torch APIs, no psutil needed) - MPS/CPU memory reporting degrades gracefully to zero values if psutil absent - No breaking changes to function signatures or return types Note: TimesFM currently doesn't use device_utils (has own _detect_device), but this infrastructure is planned for Chronos 2 integration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> chore: remove NotebookEdit test cell from TimesFM notebook
c2c065f to
5863792
Compare
|
Hi @dennisbader! I'm closing out this PR in favor of the new implementation at #2942 that addresses the design discussion from #2933. The new approach evolved from this TimesFM-only implementation into a unified foundation models architecture with a The full design document is at https://gist.github.com/nehalecky/4807ffe147321914387858dd4633c5c7 and I've posted details in the issue thread at #2933 (comment). Looking forward to your feedback on the unified architecture! |
|
Superseded by #2942 |
Add TimesFM 2.5: First Foundation Model Integration
Fixes #2359
Summary
This PR integrates TimesFM 2.5, Google Research's foundation model for time series forecasting, as the first foundation model in Darts. TimesFM 2.5 is a 200M parameter decoder-only transformer trained on 100 billion time points, enabling zero-shot forecasting without training.
Implementation
Core Changes:
darts/models/forecasting/foundation/package containing:TimesFMModel: Main model class extendingGlobalForecastingModelFoundationForecastingModel: Base class included as foundation for the full suite of foundation models (Chronos 2, etc.)historical_forecasts,backtest, metrics)Testing:
Documentation:
docs/userguide/foundation_models.md:examples/25-TimesFM-foundation-model.ipynbKey Features
Technical Decisions
Why GlobalForecastingModel?
TimesFM 2.5 extends
GlobalForecastingModelrather than a custom base class to ensure compatibility with existing Darts utilities and workflows. TheFoundationForecastingModelbase class is included as foundation for the full suite of foundation models (Chronos 2 and others) that may require different patterns.Why does fit() exist?
For API compatibility with Darts utilities that require fit() before predict(). Currently, the method validates inputs and loads the model but doesn't perform training (pre-trained weights remain frozen). In future implementations, fit() can be overloaded to support fine-tuning using parameter-efficient techniques like LoRA through the PEFT ecosystem, maintaining the familiar Darts API while enabling state-of-the-art adaptation workflows.
Lazy loading:
Calling predict() without fit() automatically loads the model, enabling pure zero-shot workflows.
Performance
Zero-shot on Air Passengers dataset: MAPE 6.59% (no training required)
Comparison: TimesFM (6.59%) outperforms fitted ExponentialSmoothing (8.10%) despite never seeing the series before.
Installation
See INSTALL.md for detailed installation instructions.
Future Work
This establishes patterns for integrating additional foundation models mentioned in #2359:
The
FoundationForecastingModelbase class and PEFT utilities are included as foundations for integrating the full suite of foundation models and enabling future fine-tuning research, but are not currently exposed to users.Breaking Changes
None - purely additive feature.
Known Limitations
Acknowledgments and Future Collaboration
Thank you to the Darts maintainers for building such an excellent library. This integration represents a first contribution in bringing foundation model capabilities to the time series community, and I'm excited to collaborate on future enhancements.
Related community requests:
Happy to contribute to additional foundation model integrations and continue improving this implementation based on maintainer and community feedback. The patterns established here are designed to make future integrations straightforward while maintaining Darts' excellent API design principles.