Vector Config Server

[nazarovkv]

Hi,

We run vector in on-prem cloud solution, in order to control vector configuration we've implemented approach when vector configuration is pulled from git repo, config validation performed and in case of success - configuration applied to vector instances.

Thing that makes me consider using other approach, made me come to some extended approach, looked up in Spring Boot(framework for Java based applications)

Spring Boot has special service called "Spring Boot Cloud Config", purpose of this service is to allow services to connect to special endpoint in order to retrieve configuration, in other words services have central place for configuration.

In my case, I need to make many changes in vector configs and apply new configuration to all instances(thankfully it happens automatically), but wrapper is not stable as it should be, due to many external factors.

For that reason, I would like to start discussion reg implementing new feature "Vector Config Server". vector instances would have configuration that provides URL to config server instances(maybe some extra options, like vector_group), and config server should expose via REST API configuration for vector which requests for configuration.

[pront]

Hi @nazarovkv, we have an HTTP config provider feature but it's not in the docs because we didn't plan to invest more time in it. You can see the configuration here:

https://github.com/vectordotdev/vector/blob/master/src/providers/http.rs#L36-L56

Example:

provider:
  type: "http"
  url: "http://localhost/vector.yaml"

I believe you can experiment with this feature, but as I mentioned above it's an experimental feature.

[nazarovkv]

Hi, @pront

Thanks for response.

Actually it works, one thing to highlight - at the moment only TOML format supported of downloaded config supported. It's not a big problem at the moment, since I can convert config from YAML to TOML and expose TOML version of config.

[pront]

at the moment only TOML format supported of downloaded config supported

If you are motivated to add YAML support, I am happy to review a PR.

[nazarovkv]

at the moment only TOML format supported of downloaded config supported

If you are motivated to add YAML support, I am happy to review a PR.

Let's try #22864