Understanding default values for tune, draws, and chains in nutpie.sample

[trendelkampschroer]

I am comparing nutpie vs pymc

+---------+--------+-------+
| kwarg   | nutpie | pymc  |
+=========+========+=======+
| chains  |       6|      4|
+---------+--------+-------+
| tune    |     300|   1000|
+---------+--------+-------+
| draws   |    1000|   1000|   
+---------+--------+-------+

Technically for pymc chains is max(2, min(num. CPU, 4)) which evaluates to 4 on most modern machines with multiple cores...

I'd be glad to get some intuition on the difference in chains and tune compared to pymc.

Thanks a lot.

[aseyboldt]

The changes to the mass matrix adaptation in nutpie compared to pymc have the effect that usually it can get by with fewer tuning steps, and still end up with a good mass matrix and step size. I experimented with posteriordb (a collection of posterior distributions) to find a good default number of tuning steps.
Since then, I've come across a couple of models outside of posteriordb where 300 really wasn't enough, and I've been thinking about increasing it to something like 600 to make it more robust.

The change to 6 chains was also based on results from posteriordb. I usually ran 10 chains, and was surprised by how often 1 or 2 of the chains still showed issues (with stan and with nutpie). If we only have 4 chains, I think we can easily miss problems in the model.

I don't think there really is a "correct" way to choose these defaults though.

[trendelkampschroer]

Thanks a lot for the detailed explanation @aseyboldt . Really helps a lot to build up context knowledge.

I understand that there is no guarantee or correct choice here. The fact that it works across problems/models in posteriordb is already a very good indication of a somewhat general applicability.