-
Notifications
You must be signed in to change notification settings - Fork 42
Description
Hello,
I am reporting an issue where I can't install Quickwit chart when the kubernetes cluster is in IPv6.
The reason is due to the healthchecks port, not listening for IPv6.
My value file.
❯ cat quickwit.yaml
environment:
# QW_LISTEN_ADDRESS: "::"
QW_METASTORE_URI: s3://quickwit/quickwit-indexes
config:
# listen_address: "::"
default_index_root_uri: s3://quickwit/quickwit-indexes
storage:
s3:
flavor: minio
endpoint: http://[fd00:cafe::6]:9001
region: eu-east-1
access_key_id: admin
secret_access_key: xxxx
tolerations:
- key: "kubernetes.io/hostname"
operator: "Equal"
value: "toybox"
effect: "NoSchedule"The helm command used to install quickwit
helm install quickwit quickwit/quickwit -f quickwit.yaml --wait --atomicAll pods fails on the startup probe
Startup probe failed: Get "http://[fd01::430]:7280/health/livez": dial tcp [fd01::430]:7280: connect: connection refused
When I get into the machine to check the openned port (I use nsenter to get access to the network namespace)
erebe@toybox ~ [SIGINT]> sudo nsenter -n -t (ps aux | grep 'metastore' | grep -v grep | awk '{print $2}') -- ss -lntp
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 0.0.0.0:7280 0.0.0.0:* users:(("quickwit",pid=200495,fd=13))
LISTEN 0 128 0.0.0.0:7281 0.0.0.0:* users:(("quickwit",pid=200495,fd=12))
We see that Quickwit is listening on 0.0.0.0 but not on [::] for IPv6.
We can confirm with curl
# IPv4 works
erebe@toybox ~ [1]> sudo nsenter -n -t (ps aux | grep 'metastore' | grep -v grep | awk '{print $2}') -- curl http://10.42.4.34:7280/health/livez
true⏎
#IPv6 hangs
sudo nsenter -n -t (ps aux | grep 'metastore' | grep -v grep | awk '{print $2}') -- curl http://[fd01::430]:7280/health/livez
xxxx
Changing the listening address to :: in the config makes everything works and should be working also for both IPv4 and IPv6 networks
erebe@toybox ~ [SIGINT]> sudo nsenter -n -t (ps aux | grep 'metastore' | grep -v grep | awk '{print $2}') -- ss -lntp
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 128 *:7280 *:* users:(("quickwit",pid=204513,fd=13))
LISTEN 0 128 *:7281 *:* users:(("quickwit",pid=204513,fd=12))
Expected result
The chart should use by default for listening address :: instead of 0.0.0.0
One strange thing to note, I don't know if it is linked to the base image of the container, but usually 0.0.0.0 bind also on IPv6 in order to ease transition from IPv4 to IPv6.
In the case of Quickwit, it does not happen and only bind for IPv4
