Calico (via Tigera) and KubePrism

[docbobo]

I was trying to configure my Calico setup to properly use KubePrism, but I am struggling a little bit with the setup. I've configure the kubernetes-services-endpoint CM so that it points to https://localhost:7445. While the ´calico-nodepods come up properly, thecalico-kube-controllers` fail to do so:

2024-12-01 09:28:01.768 [ERROR][1] kube-controllers/client.go 320: Error getting cluster information config ClusterInformation="default" error=Get "https://localhost:7445/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp [::1]:7445: connect: connection refused
2024-12-01 09:28:01.768 [INFO][1] kube-controllers/client.go 248: Unable to initialize ClusterInformation error=Get "https://localhost:7445/apis/crd.projectcalico.org/v1/clusterinformations/default": dial tcp [::1]:7445: connect: connection refused
...

Anybody having experience with this setup who can help?

[smira]

This localhost endpoint would only work for host networking pods. If the pods run with CNI, they should use default kubernetes service. Follow documentation for your CNI for more details

[docbobo]

Well, that is exactly the problem here. Although Talos documentation suggests that the CNI should be configured to use KubePrism, it's hard to find anybody who has actually done that with Calico, especially using the Tigera operator. That's why I was looking for "anybody having experience with this setup" 😉

FWIW, the Talos documentation says:

When using CNI components that require access to the Kubernetes API server, the KubePrism endpoint should be passed to the CNI configuration (e.g. Cilium, Calico CNIs).

I did that, exactly as written in the Calico documentation, but as I just found out by your comment, apparently that's not the whole story because that endpoint only works for host networking pods.

[smira]

It's pretty obvious that the localhost endpoint in the pod with pod networking would point to the pod itself.

But usually CNIs have different components, and some should run with host networking, and those would benefit from KubePrism endpoint.