Devops/slim flavour requirements (#2906)

* small fix on assert condition

* Moved XGB, SF to notorch flavor

* Update CHANGELOG.md

* minor updates

* PR fixes

* minor updates

* removed core tests from develop workflow

---------

Co-authored-by: dennisbader <[email protected]>

Author: jakubchlapek

CHANGELOG.md

@@ -17,7 +17,7 @@ but cannot always guarantee backwards compatibility. Changes that may **break co
 - Added `add_regressor_configs` parameter to the `Prophet` model, enabling component-specific control over `prior_scale`, `mode`, and `standardize` for the future covariates. [#2882](https://github.com/unit8co/darts/issues/2882) by [Ramsay Davis](https://github.com/RamsayDavisWL).
 - 🔴 Increased the decimal places for quantile component names from 2 to 3 for more precise quantiles. (e.g. `component_name_q0.500` for quantile 0.5). This affects quantile forecasts as well as quantiles computed with `TimeSeries.quantile()`. [#2887](https://github.com/unit8co/darts/pull/2786) by [He Weilin](https://github.com/cnhwl).
 - Added parameter `load_best` to `TorchForecastingModel.fit()` and `fit_from_dataset()` which, when `True`, will automatically load (and use) the best model on the validation set at the end of the training process. [#2903](https://github.com/unit8co/darts/pull/2903) by [He Weilin](https://github.com/cnhwl).
-- Added model creation parameters `random_errors` and `error` to `ExponentialSmoothing` that give control over how probabilistic forecasts are generated. [#2290491](https://github.com/unit8co/darts/pull/2904) by [Jakub Chłapek](https://github.com/jakubchlapek)
+- Added model creation parameters `random_errors` and `error` to `ExponentialSmoothing` that give control over how probabilistic forecasts are generated. [#2904](https://github.com/unit8co/darts/pull/2904) by [Jakub Chłapek](https://github.com/jakubchlapek)
 - Added parameter `val_length` to `ForecastingModel.historical_forecasts()`, `backtest()` and `residuals()` which will extract a validation set of length `val_length` after the end of each training set when `retrain=True`. The validation set is then used to fit the underlying forecasting model if it supports it. This is especially useful for early stopping mechanisms to reduce overfitting and / or training times. [#2894](https://github.com/unit8co/darts/pull/2894) by [Dennis Bader](https://github.com/dennisbader).
 - 🔴 Renamed the `RegressionEnsembleModel` ensemble model attribute from `regression_model` to `ensemble_model` to make it more clear that this model is used to combine the predictions of the base models. [#2894](https://github.com/unit8co/darts/pull/2894) by [Dennis Bader](https://github.com/dennisbader).
 
@@ -29,6 +29,7 @@ but cannot always guarantee backwards compatibility. Changes that may **break co
 **Dependencies**
 
 - We raised the minimum pytorch-lightning version to `pytorch-lightning>=2.0.0`. [#2888](https://github.com/unit8co/darts/pull/2888) by [Dennis Bader](https://github.com/dennisbader).
+- We made the Darts core and `torch` packages lighter by removing XGBoost and StatsForecast from the dependencies. All our forecasting models wrapping around these libraries will still be supported. To use them simply install the packages manually or via `u8darts[notorch]` and `u8darts[all]`. [#2906](https://github.com/unit8co/darts/pull/2906) by [Jakub Chłapek](https://github.com/jakubchlapek)
 
 ### For developers of the library:
 

INSTALL.md

@@ -3,7 +3,7 @@
 Below, we detail how to install Darts using either `conda` or `pip`.
 
 ## From PyPI
-Install Darts with all models except the ones from optional dependencies (Prophet, LightGBM, CatBoost, see more on that [here](#enabling-optional-dependencies)): `pip install darts`.
+Install Darts with all models except the ones from optional dependencies (Prophet, LightGBM, CatBoost, XGBoost, StatsForecast see more on that [here](#enabling-optional-dependencies)): `pip install darts`.
 
 If this fails on your platform, please follow the official installation
 guide for [PyTorch](https://pytorch.org/get-started/locally/), then try installing Darts again.
@@ -12,8 +12,8 @@ As some dependencies are relatively big or involve non-Python dependencies,
 we also maintain the `u8darts` package, which provides the following alternate lighter install options:
 
 * Install Darts with all available models: `pip install "u8darts[all]"`
-* Install core only (without neural networks, Prophet, LightGBM and Catboost): `pip install u8darts`
-* Install core + Prophet + LightGBM + CatBoost: `pip install "u8darts[notorch]"`
+* Install core only (without neural networks, Prophet, LightGBM, Catboost, XGBoost and StatsForecast): `pip install u8darts`
+* Install core + Prophet + LightGBM + CatBoost + XGBoost + StatsForecast: `pip install "u8darts[notorch]"`
 * Install core + neural networks (PyTorch): `pip install "u8darts[torch]"` (equivalent to `pip install darts`)
 
 ## From conda-forge
@@ -29,16 +29,15 @@ Activate the environment
 As some models have relatively heavy dependencies, we provide four conda-forge packages:
 
 * Install Darts with all available models: `conda install -c conda-forge -c pytorch u8darts-all`
-* Install core only (without neural networks, Prophet, LightGBM and Catboost): `conda install -c conda-forge u8darts`
-* Install core + Prophet + LightGBM + CatBoost: `conda install -c conda-forge u8darts-notorch`
+* Install core only (without neural networks, Prophet, LightGBM, Catboost, XGBoost and StatsForecast): `conda install -c conda-forge u8darts`
+* Install core + Prophet + LightGBM + CatBoost + XGBoost + StatsForecast: `conda install -c conda-forge u8darts-notorch`
 * Install core + neural networks (PyTorch): `conda install -c conda-forge -c pytorch u8darts-torch`
 
 
 ## Other Information
 
 ### Enabling Optional Dependencies
-As of version 0.25.0, the default `darts` package does not install Prophet, CatBoost, and LightGBM dependencies anymore, because their
-build processes were too often causing issues. We continue supporting the model wrappers `Prophet`, `CatBoostModel`, and `LightGBMModel` in Darts though. If you want to use any of them, you will need to manually install the corresponding packages (or install a Darts flavor as described above).
+As of version 0.38.0, we made the default `darts` package more lightweight. Packages Prophet, CatBoost, LightGBM, XGBoost and StatsForecast will not be installed anymore. Don't worry though, we keep supporting our model wrappers `Prophet`, `CatBoostModel`, `LightGBMModel`, `XGBoost` and `StatsForecast` in Darts. If you want to use any of them, you will need to manually install the corresponding packages (or install a Darts flavor as described above).
 
 #### Prophet
 Install the `prophet` package (version 1.1.1 or more recent) using the [Prophet install guide](https://facebook.github.io/prophet/docs/installation.html#python)
@@ -49,6 +48,12 @@ Install the `catboost` package (version 1.0.6 or more recent) using the [CatBoos
 #### LightGBMModel
 Install the `lightgbm` package (version 3.2.0 or more recent) using the [LightGBM install guide](https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html)
 
+#### XGBoost
+Install the `xgboost` package (version 2.1.4 or more recent) using the [XGBoost install guide](https://xgboost.readthedocs.io/en/stable/install.html)
+
+#### StatsForecast
+Install the `statsforecast` package (version 1.4 or more recent) using the [StatsForecast install guide](https://nixtlaverse.nixtla.io/statsforecast/index.html#installation)
+
 ### Enabling GPU support
 Darts relies on PyTorch for the neural network models.
 For GPU support, please follow the instructions to install CUDA in the [PyTorch installation guide](https://pytorch.org/get-started/locally/).

darts/tests/conftest.py

@@ -17,6 +17,58 @@
     logger.warning("Torch not installed - Some tests will be skipped.")
     TORCH_AVAILABLE = False
 
+try:
+    import catboost
+    import lightgbm
+    import xgboost  # noqa: F401
+
+    GBM_AVAILABLE = True
+except ImportError:
+    logger.warning(
+        "Gradient Boosting Models not installed - Some tests will be skipped."
+    )
+    GBM_AVAILABLE = False
+
+try:
+    import xgboost  # noqa: F401
+
+    XGB_AVAILABLE = True
+except ImportError:
+    logger.warning("XGBoost not installed - Some tests will be skipped.")
+    XGB_AVAILABLE = False
+
+try:
+    import lightgbm  # noqa: F401
+
+    LGBM_AVAILABLE = True
+except ImportError:
+    logger.warning("LightGBM not installed - Some tests will be skipped.")
+    LGBM_AVAILABLE = False
+
+try:
+    import catboost  # noqa: F401
+
+    CB_AVAILABLE = True
+except ImportError:
+    logger.warning("CatBoost not installed - Some tests will be skipped.")
+    CB_AVAILABLE = False
+
+try:
+    import prophet  # noqa: F401
+
+    PROPHET_AVAILABLE = True
+except ImportError:
+    logger.warning("Prophet not installed - Some tests will be skipped.")
+    PROPHET_AVAILABLE = False
+
+try:
+    import statsforecast  # noqa: F401
+
+    SF_AVAILABLE = True
+except ImportError:
+    logger.warning("StatsForecast not installed - Some tests will be skipped.")
+    SF_AVAILABLE = False
+
 try:
     import onnx  # noqa: F401
     import onnxruntime  # noqa: F401

darts/tests/explainability/test_shap_explainer.py

@@ -23,11 +23,12 @@
     SKLearnModel,
     XGBModel,
 )
+from darts.tests.conftest import (
+    GBM_AVAILABLE,
+    LGBM_AVAILABLE,
+    XGB_AVAILABLE,
+)
 from darts.utils.timeseries_generation import linear_timeseries
-from darts.utils.utils import NotImportedModule
-
-lgbm_available = not isinstance(LightGBMModel, NotImportedModule)
-cb_available = not isinstance(CatBoostModel, NotImportedModule)
 
 
 def extract_year(index):
@@ -138,43 +139,49 @@ class TestShapExplainer:
         np.concatenate([fut_cov_1.reshape(-1, 1), fut_cov_2.reshape(-1, 1)], axis=1),
     )
 
-    def test_creation(self):
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+    @pytest.mark.skipif(not GBM_AVAILABLE, reason="requires gradient boosting models")
+    @pytest.mark.parametrize(
+        "model",
+        [
+            {
+                "model_cls": LightGBMModel,
+                "config": {
+                    "lags": 4,
+                    "lags_past_covariates": [-1, -2, -3],
+                    "lags_future_covariates": [0],
+                    "output_chunk_length": 4,
+                    "add_encoders": add_encoders,
+                },
+            },
+            {
+                "model_cls": CatBoostModel,
+                "config": {
+                    "lags": 4,
+                    "lags_past_covariates": [-1, -2, -6],
+                    "lags_future_covariates": [0],
+                    "output_chunk_length": 4,
+                },
+            },
+            {
+                "model_cls": XGBModel,
+                "config": {
+                    "lags": 4,
+                    "lags_past_covariates": [-1, -2, -3],
+                    "lags_future_covariates": [0],
+                    "output_chunk_length": 4,
+                    "add_encoders": add_encoders,
+                },
+            },
+        ],
+    )
+    def test_gbm_creation(self, model):
+        model_cls = model["model_cls"]
+        config = model["config"]
         # Model should be fitted first
-        m = model_cls(
-            lags=4,
-            lags_past_covariates=[-1, -2, -3],
-            lags_future_covariates=[0],
-            output_chunk_length=4,
-            add_encoders=self.add_encoders,
-        )
-        with pytest.raises(ValueError):
-            ShapExplainer(m, self.target_ts, self.past_cov_ts, self.fut_cov_ts)
-
-        # Model should be a SKLearnModel
-        m = ExponentialSmoothing()
-        m.fit(self.target_ts["price"])
-        with pytest.raises(ValueError):
-            ShapExplainer(m)
+        m = model_cls(**config)
 
-        # For now, multi_models=False not allowed
-        m = LinearRegressionModel(lags=1, output_chunk_length=2, multi_models=False)
-        m.fit(
-            series=self.target_ts,
-        )
         with pytest.raises(ValueError):
-            ShapExplainer(
-                m,
-                self.target_ts,
-            )
-
-        m = model_cls(
-            lags=4,
-            lags_past_covariates=[-1, -2, -3],
-            lags_future_covariates=[0],
-            output_chunk_length=4,
-            add_encoders=self.add_encoders,
-        )
+            ShapExplainer(m, self.target_ts, self.past_cov_ts, self.fut_cov_ts)
 
         m.fit(
             series=self.target_ts,
@@ -214,6 +221,28 @@ def test_creation(self):
                 shap_explain.explainers.explainers[0][0], shap.explainers.Tree
             )
 
+        # Bad choice of shap explainer
+        with pytest.raises(ValueError):
+            ShapExplainer(m, shap_method="bad_choice")
+
+    def test_creation(self):
+        # Model should be a SKLearnModel
+        m = ExponentialSmoothing()
+        m.fit(self.target_ts["price"])
+        with pytest.raises(ValueError):
+            ShapExplainer(m)
+
+        # For now, multi_models=False not allowed
+        m = LinearRegressionModel(lags=1, output_chunk_length=2, multi_models=False)
+        m.fit(
+            series=self.target_ts,
+        )
+        with pytest.raises(ValueError):
+            ShapExplainer(
+                m,
+                self.target_ts,
+            )
+
         # Linear model - also not a MultiOutputRegressor
         m = LinearRegressionModel(
             lags=1,
@@ -257,33 +286,8 @@ def test_creation(self):
         shap_explain = ShapExplainer(m)
         assert isinstance(shap_explain.explainers.explainers, shap.explainers.Linear)
 
-        # CatBoost
-        model_cls = CatBoostModel if cb_available else XGBModel
-        m = model_cls(
-            lags=4,
-            lags_past_covariates=[-1, -2, -6],
-            lags_future_covariates=[0],
-            output_chunk_length=4,
-        )
-        m.fit(
-            series=self.target_ts,
-            past_covariates=self.past_cov_ts,
-            future_covariates=self.fut_cov_ts,
-        )
-        shap_explain = ShapExplainer(m)
-        if m._supports_native_multioutput:
-            assert isinstance(shap_explain.explainers.explainers, shap.explainers.Tree)
-        else:
-            assert isinstance(
-                shap_explain.explainers.explainers[0][0], shap.explainers.Tree
-            )
-
-        # Bad choice of shap explainer
-        with pytest.raises(ValueError):
-            ShapExplainer(m, shap_method="bad_choice")
-
     def test_explain(self):
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         m = model_cls(
             lags=4,
             lags_past_covariates=[-1, -2, -3],
@@ -440,7 +444,7 @@ def test_explain(self):
         assert isinstance(shap_explain.explain(), ShapExplainabilityResult)
 
     def test_explain_with_lags_future_covariates_series_of_same_length_as_target(self):
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         model = model_cls(
             lags=4,
             lags_past_covariates=[-1, -2, -3],
@@ -476,7 +480,7 @@ def test_explain_with_lags_future_covariates_series_extending_into_future(self):
         fut_cov = np.random.normal(0, 1, len(days)).astype("float32")
         fut_cov_ts = TimeSeries.from_times_and_values(days, fut_cov.reshape(-1, 1))
 
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         model = model_cls(
             lags=4,
             lags_past_covariates=[-1, -2, -3],
@@ -512,7 +516,7 @@ def test_explain_with_lags_covariates_series_older_timestamps_than_target(self):
         past_cov = np.random.normal(0, 1, len(days)).astype("float32")
         past_cov_ts = TimeSeries.from_times_and_values(days, past_cov.reshape(-1, 1))
 
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         model = model_cls(
             lags=None,
             lags_past_covariates=[-1, -2],
@@ -539,7 +543,7 @@ def test_explain_with_lags_covariates_series_older_timestamps_than_target(self):
             assert explanation.start_time() == self.target_ts.start_time()
 
     def test_plot(self):
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         m_0 = model_cls(
             lags=4,
             lags_past_covariates=[-1, -2, -3],
@@ -641,7 +645,7 @@ def test_plot(self):
         plt.close()
 
     def test_feature_values_align_with_input(self):
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         model = model_cls(
             lags=4,
             output_chunk_length=1,
@@ -668,7 +672,7 @@ def test_feature_values_align_with_input(self):
         )
 
     def test_feature_values_align_with_raw_output_shap(self):
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         model = model_cls(
             lags=4,
             output_chunk_length=1,
@@ -695,7 +699,7 @@ def test_feature_values_align_with_raw_output_shap(self):
         ), "The shape of the feature values should be the same as the shap values"
 
     def test_shap_explanation_object_validity(self):
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         model = model_cls(
             lags=4,
             lags_past_covariates=2,
@@ -719,13 +723,13 @@ def test_shap_explanation_object_validity(self):
 
     @pytest.mark.parametrize(
         "config",
-        [
-            (XGBModel, {}),
-            (
-                LightGBMModel if lgbm_available else XGBModel,
-                {"likelihood": "quantile", "quantiles": [0.5]},
-            ),
-        ],
+        [(LinearRegressionModel, {})]
+        + ([(XGBModel, {})] if XGB_AVAILABLE else [])
+        + (
+            [(LightGBMModel, {"likelihood": "quantile", "quantiles": [0.5]})]
+            if LGBM_AVAILABLE
+            else []
+        ),
     )
     def test_shap_selected_components(self, config):
         """Test selected components with and without Darts' MultiOutputRegressor"""
@@ -768,7 +772,7 @@ def test_shap_selected_components(self, config):
 
     def test_shapley_with_static_cov(self):
         ts = self.target_ts_with_static_covs
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         model = model_cls(
             lags=4,
             output_chunk_length=1,
@@ -813,7 +817,7 @@ def test_shapley_with_static_cov(self):
             ]
 
     def test_shapley_multiple_series_with_different_static_covs(self):
-        model_cls = LightGBMModel if lgbm_available else XGBModel
+        model_cls = LightGBMModel if LGBM_AVAILABLE else LinearRegressionModel
         model = model_cls(
             lags=4,
             output_chunk_length=1,