How to fix immich_[container] exited with code 135 (and/or code 0)

[erlenddahl]

After struggling with an Immich update, I probably corrupted my containers somehow, and kept getting logs like this when I ran docker compose up:

immich_postgres exited with code 0
immich_machine_learning exited with code 0
immich_redis exited with code 0
[10 lines with other stuff]
immich_postgres exited with code 0
immich_redis exited with code 0
immich_machine_learning exited with code 0
immich_proxy exited with code 0
[6 lines with other stuff]
immich_postgres exited with code 135
immich_redis exited with code 135
immich_proxy exited with code 135
immich_machine_learning exited with code 135

Running docker compose down -v to purge everything and reset Immich completely did not work, I still got the same error. Running older versions of Immich (that had worked before) did not work either.

In the end, what I did to fix this was to completely remove all Immich related docker images from my server, to make docker re-download everything completely. (I am a complete docker newbie, so I didn't know this, but docker apparently downloads images and stores them on your hard drive so that it can re-use them later.)

In case others get the same problem, this is how I fixed it.

Please note that this completely resets your Immich -- you'll lose all the images and videos and users and settings that are on your server, so don't do this if you don't have them elsewhere.

First, stop Immich, and remove the containers and volumes. This will remove all users and settings (basically everything stored in your database).
docker compose down -v

Next, make sure no Immich related containers are running:
docker ps -a

On my server, this outputs this list:

CONTAINER ID   IMAGE                                                COMMAND                  CREATED        STATUS                  PORTS                                                                                                      NAMES
abd7822fd26a   ghcr.io/immich-app/immich-proxy:release              "/docker-entrypoint.…"   10 hours ago   Up 10 hours             0.0.0.0:2283->8080/tcp, :::2283->8080/tcp                                                                  immich_proxy
6ca60e82d866   ghcr.io/immich-app/immich-server:release             "/bin/sh start-serve…"   10 hours ago   Up 10 hours             3001/tcp                                                                                                   immich_server
f85a8f8bafd2   ghcr.io/immich-app/immich-server:release             "/bin/sh start-micro…"   10 hours ago   Up 10 hours             3001/tcp                                                                                                   immich_microservices
c219be67e5cf   redis:6.2                                            "docker-entrypoint.s…"   10 hours ago   Up 10 hours             6379/tcp                                                                                                   immich_redis
23c1b46bab87   typesense/typesense:0.24.0                           "/opt/typesense-serv…"   10 hours ago   Up 10 hours             8108/tcp                                                                                                   immich_typesense
145bff8da9d3   postgres:14                                          "docker-entrypoint.s…"   10 hours ago   Up 10 hours             5432/tcp                                                                                                   immich_postgres
142d55adcc84   ghcr.io/immich-app/immich-web:release                "/bin/sh entrypoint.…"   10 hours ago   Up 10 hours             3000/tcp                                                                                                   immich_web
7248d04b20a5   ghcr.io/immich-app/immich-machine-learning:release   "python src/main.py"     10 hours ago   Up 10 hours                                                                                                                        immich_machine_learning
[other non-Immich docker containers]

You can see in the rightmost column that all of the above are indeed Immich-related. So if you see something like this, then you didn't properly stop the containers. You can try docker compose down -v again, make sure to run it from where your docker-compose.yml is located. If they're still running, you can start killing them using docker rm [container ID], for example docker rm abd7822fd26a.

When you're happy that nothing Immich-related is running, it's time to start removing the Immich-related docker images. First, you can list all the images docker has stored on your server like this:
docker image ls

On my server, this outputs this list:

REPOSITORY                                   TAG                IMAGE ID       CREATED        SIZE
redis                                        6.2                de0285e3559c   17 hours ago   113MB
postgres                                     14                 cb988f73c278   18 hours ago   377MB
mariadb                                      10                 75b68210cde0   4 days ago     411MB
ghcr.io/immich-app/immich-server             release            4bb3b205a3fa   9 days ago     472MB
ghcr.io/immich-app/immich-web                release            bfb960507d73   9 days ago     231MB
ghcr.io/immich-app/immich-machine-learning   release            16a97caf2b9e   11 days ago    1.65GB
ghcr.io/immich-app/immich-proxy              release            6ac7b18138b2   2 weeks ago    142MB
[other non-Immich docker images]

All of the above images are Immich-related. How can you tell? Some of them contain "immich", but not all of them. In the end, you need to know which images are listed in the docker-compose.yml file to be sure. My strategy was simply to remove everything that wasn't in use by my other (still running) docker containers. If you try to remove an image that is in use, you will get an error message.

Remove all images (one by one) by running:
docker image rm [repository]:[tag]

For example:

docker image rm redis:6.2
docker image rm ghcr.io/immich-app/immich-server:release

If you struggle with removing an image (for example images with no tag), you can also use the IMAGE ID column like this:
docker image rm de0285e3559c

If you try to remove an image that is being used, it will look like this:
Error response from daemon: conflict: unable to remove repository reference "redis:6.2" (must force) - container c219be67e5cf is using its referenced image de0285e3559c

The error message tells us which container is using it, so we can use docker ps -a to find the relevant container, and/or docker rm [container id] to remove it (as long as you're sure it isn't related to something other than Immich -- be careful!).

When you have managed to remove all Immich related docker images and containers, you can go back to your docker-compose.yml and .env files, and run the normal Immich commands:
docker-compose pull && docker-compose up -d # Or docker compose up -d

If you had the same issue as me, your problem should now be solved. Personally, I had to do this twice before it worked, probably because I missed a docker image or two.

Hope that can help somebody, and thanks again to Alex and jrasm91 for helping me through this!