Upgrading from Ubuntu 22.04 to 24.04 breaks TLJH

[brucat]

Bug description

TLJH fails on upgrade form Ubuntu 22.04.5 LTS to 24.04.2 LTS

How to reproduce

A TLJH on Ubuntu 22.04 was upgraded to 24.04. The jupyter.service fails to start after upgrade.
I tried rerunning the installer but that failed. I removed the old TLJH install and tried a new install. That seemed to work, but neither the traefik nor jupyter services start

Expected behaviour

TLJH should continue to function on a Ubuntu version upgrade to the latest LTS

Actual behaviour

TLJH ails on upgrade

Your personal set up

Ubuntu 24.04.2 on a Thinkserver R640

• OS:

Ubuntu 24.04.2 on a Thinkserver R640

• Version(s):

Full environment

# paste output of `pip freeze` or `conda list` here

Configuration

# jupyterhub_config.py

Logs

[manics]

What version of TLJH did you have before you upgraded your OS?

[brucat]

I had JupyterHub 4.1.5 20250306174759 I "fixed" the issue by completely burning down the tljh installation and reinstalling from scratch. That worked, but was painful. Thanks anyway! ​ PJ Brucat Department of Chemistry / University of Florida

…

________________________________ From: Simon Li ***@***.***> Sent: Saturday, May 17, 2025 09:22 To: jupyterhub/the-littlest-jupyterhub ***@***.***> Cc: PJ Brucat ***@***.***>; Author ***@***.***> Subject: Re: [jupyterhub/the-littlest-jupyterhub] Upgrading from Ubuntu 22.04 to 24.04 breaks TLJH (Issue #1036) [External Email] [https://avatars.githubusercontent.com/u/1644105?s=20&v=4]manics left a comment (jupyterhub/the-littlest-jupyterhub#1036)<#1036 (comment)> What version of TLJH did you have before you upgraded your OS? — Reply to this email directly, view it on GitHub<#1036 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKLXVZS3ZWNK6BGYBIQZEF3264ZZZAVCNFSM6AAAAAB5IYROVKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQOBYGM4TANJRGM>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[janikmu]

You could try this, which worked back when I was upgrading OS to 22.04:
#613 (comment)

Basically, the system environment and the hub environment are out of sync after Ubuntu updates Python deps.

[minrk]

Since the hub env uses the system Python, it is unfortunately expected that this would happen when the system Python version changes. With Python's increased release cadence, this is now likely to happen on pretty much every LTS->LTS upgrade. At the very least, we need to document this and how to recover from it (recreating the Hub env shouldn't be a complex process, but it should be clearly documented; the user env should not be affected). It's possible that re-running bootstrap should be the minimally invasive "ensure it's all correct" process, but we should ensure that's the case.

If we want to be more robust, we could change to install Python ourselves, e.g. from Python Standalone Builds or conda or another source, so the Python version of the hub env is not sensitive to OS upgrades. But regardless of where we get the Python, upgrading Python in the hub env is likely to break things, and we need to document the process for doing that (even if it's just "run bootstrap again after upgrade").