[ENH] Add a "SupportScaler" or "SupportTransformer"

[joshdunnlime]

Is your feature request related to a problem? Please describe.
Yes! When trying to use an skpro GridSearch across distributions you can run into lots of support issues if your target variable is not scaled to the smallest support range. For example, if you use xgblss with target range (-inf, inf) and search across ["Normal", "Gamma"], the GridSearch will fail or return NaNs for the Gamma search. Scaling to the range of the of the smallest leads to less interpretable skpro/scipy distribution parameter for the normal distribution in this case.

Describe the solution you'd like
I have implemented a something locally that works rather nicely:

# imports ...

full_reals = {
    "Laplace",
    "Logistic",
    "Normal",
    "SkewNormal",
    "TDistribution",
    "TruncatedNormal",
}


class SupportTransformer(TransformerMixin, BaseEstimator, OneToOneFeatureMixin):

    def __init__(self, dist, rtol):
        ...

    def _get_skpro_distr(self, distr):
        """Copied from xgblss code"""
        ...

    def _get_support(self):
        # logic to get scipy rvs which includes support
        # calls _get_skpro_distr
        return rvs.support(**sc_params)

    def fit(self, X, y=None):
        if self.dist in full_reals:
            # no fit needed
            return self

        self.support = self._get_support()

        # check if X is within support
        if any(
            [np.any(X.max() >= self.support[1]), np.any(X.min() <= self.support[0])]
        ):
            # some more implementation logic
            self.mms = MinMaxScaler((support_lower, support_upper))
            self.mms.fit(X)

            self.scale_ = mms.scale_

        return self

    def transform(self, X):
        if hasattr(self, "mms"):
            return self.mms.transform(X)
        else:
            return X

    def inverse_transform(self, X):
        if hasattr(self, "mms"):
            return self.mms.inverse_transform(X)
        else:
            return X

Usage is as follows:

ttr = TransformedTargetRegressorProba(
    xgboostlss.XGBoostLSS(),
    SupportTransformer(),
)

param_distributions = [
    {"regressor__dist": "Normal", "transformer__dist": "Normal"},
    {"regressor__dist": "Gamma", "transformer__dist": "Gamma"},
]

rscv = GridSearchCV(
    estimator=ttr,
    param_grid=param_distributions,
    cv=cv,
    scoring=CRPS(),
    error_score='raise',
)

# for some -inf < y < inf
rscv.fit(X, y)

Describe alternatives you've considered
Horrible, horrible loops 😆

[fkiraly]

can we add this by PR? Sounds really useful!

The example should also be added to one of the notebooks, I think it is very nice!

[fkiraly]

Though, this would require inspectable support for the distributions, right?

We have recognized the need for that previously, it would be great if we could add this!

For issue on inspectable support, see:
#244

I think this should simply be a method support that we attach to all distributions

[joshdunnlime]

In my basic implementation I did a bit of hack where I passed some dummy parameters to scipy rvs like so rvs.support(**sc_params_dum). It worked for the distributions I wanted to search over, but having explicit support property for the skpro distribution class I feel is the right way to go about it.

My current solution only handles continuous distributions. I've not worked with discrete distributions so am struggling for a use case here. Maybe if you have the set {-2, -1, 0, 1, 2} you would want this to map to {1, 2, 3, 4, 5}? I assume this would be the same for all discrete distributions one would wish to search over though.

Is there a use case for mixing discrete and continuous? (I can't think of one).

I'll open a PR for this.

[fkiraly]

Is there a use case for mixing discrete and continuous? (I can't think of one).

Yes, I know it can happen in hurdle or zero-inflated or clipped distributions, although the discrete distributions there typically have a finite support. Mostly subcases where you apply max(x, const), or min(x, const) to values x from a distribution, which leads to masses at lower or upper bounds.

Not sure about other use cases though - typically, you would have the one or the other.

Maybe @felipeangelimvieira or @tingiskhan have some examples in mind.

[joshdunnlime]

Initially I could add functionality for the support property and continuous supports and then add further functionality for discrete/integer support. Discrete and continuous might need to be handled slightly differently under-the-hood. I am using the MinMaxScaler for the continuous, but this might need further consideration for the discrete. Any differences should become pretty clear after adding the support property to the distribution.

I assume empirical distributions can handle varied supports natively.