Training progress in tmux pauses unexpectedly until keypress / new process is triggered

[GFODK]

Hi, I’m running symbolic regression on a remote Linux server via SSH (macOS → server) inside a tmux session.

The process does not crash or exit — CPU usage and load average in top stay similar to when training is running normally. However, the terminal output (hall of fame updates) suddenly stops, and outputs//hall_of_fame.csv stops being rewritten at the same time. It looks like the training loop is stuck or waiting internally, even though the process is still alive.

I first tried to press Enter multiple times inside the tmux session. In some cases this helped after waiting 10–20 minutes, but often nothing happened — which made me suspect the server or process had temporarily stopped.

To verify whether the server was still responsive, I opened a new tmux session and started a dummy run of the same Python code (copy of the script). Surprisingly, after doing this, the previously “frozen” session started running again on its own after a short delay (around 10 minutes) — the terminal output resumed and hall_of_fame.csv began updating again. So it seems the original process was not dead, but somehow unblocked or awakened by starting a second, similar process in another tmux session.

Has anyone seen similar behavior when running PySR inside tmux or SSH sessions on long runs? How can I make sure that a long training run (e.g., niterations = 10,000) never gets stuck or paused mid-run unless explicitly interrupted (Ctrl+C)?

[MilesCranmer]

Weird. I'm not sure if this is specific to PySR or not. Might tmux have some feature that "freezes" a process given some combination of inputs?