Conformal Predictions

[dennisbader]

Conformal predictions could be a valuable addition to darts.
It would require some brain storming/planning of how (or if) we can integrate this into our API / extend the API.

Some links:

• https://valeman.medium.com/probabilistic-forecasting-with-conformal-prediction-and-neuralprophet-af9c87901d94
• https://towardsdatascience.com/time-series-forecasting-with-conformal-prediction-intervals-scikit-learn-is-all-you-need-4b68143a027a

[jlopezpena]

Perhaps Darts could leverage MAPIE for implementing conformal inference?

[lsorber]

For those interested, Conformal Tights is a Python package that adds conformal prediction to Darts: https://github.com/radix-ai/conformal-tights

[dennisbader]

Working on it. For the beginning we were thinking about the Naive Conformal Model and Conformalized Quantile Regression.

[AugustComte]

Could you add a facility for weighting seasonality i.e. yearly, and decay weighting for recency, within the Naive approach? I find these to be useful.

[dennisbader]

Hmm.. For the first version probably not (to keep things simple for a start, it's already getting quite big ;) ). But the goal is to make it easy to define a custom CP model/class with the ability to customize how to calibrate the intervals based on the residuals.

After the first version, we're more than happy to further improve / expand on it.

[AugustComte]

That makes sense; I also found it was a large amount of work, especially with PyTorch models. But then I was trying to hack PUNCC and MAPIE...

I'm unsure if this is useful, but I have found this YouTube channel a useful reference: https://github.com/mtorabirad/MLBoost

Anyhow, thank you for the update, Dennis. I'm excited about this. We use darts in production, and your work is greatly appreciated!

[dennisbader]

It took a while, but conformal prediction has been added now with #2552 :) 🚀