Prometheus scrape source with headless service as endpoint

[adelmoradian]

Hi,

I would like to use vector prometheus scrape source to scrape metrics from various endpoints in my kubernetes clusters. It works pretty well for kubelet and api server metrics and single pods that are behind a service. But what if I have multiple replicas of a pod? If I scrape the service of type ClusterIP I think vector will miss metrics as the service will probably do something like round-robin... What if my endpoint is a headless service? Would vector in that case scrape all the pods that are behind the headless service? My end goal is to use vector instead of prometheus to scrape metrics from multiple kubernetes clusters and ship all metrics to a centralized prometheus.

[pront]

I think vector will miss metrics as the service will probably do something like round-robin

Can you expand? Vector supports DNS IP resolution with dns::Resolver.lookup_ip (currently not used in prometheus_scrape) which returns a set of IP addresses.

My end goal is to use vector instead of prometheus to scrape metrics from multiple kubernetes clusters and ship all metrics to a centralized prometheus.

You can use an prometheus_exporter for this.

[sairamreddy66]

Hi, May I know what is the exact source block code that you are using to scrape Pod level metrics from kubelet