Support full spec of env specification

[xrl]

Right now the .Values.metastore.extraEnv is expected as only literal K/V pairs and does not allow for the pulling in an environment variable from a secret. For me it is important to have the power to remap keys from a secret, I can't just use envFrom because the keys don't match what is expected by quickwit.

I don't think the helm chart should take such a strong opinion on the environment variables, allow the user to write YAML which gets substituted in. Something like:

possible values spec:

        metastore:
          extraEnv:
           -  name: QW_METASTORE_URI
              valueFrom:
                secretRef:
                  name: quickwittracing-postgres-direct-secret
                  key: POSTGRES_URL
          - name: SOMETHING_ELSE
            value: "asdf"

and then in the metastore-deployment.yaml:

# SNIP
spec:
   # snip
  template:
    # snip
    spec:
      # snip
      containers:
        - name: {{ .Chart.Name }}
          # snip
          env:
            {{- include "quickwit.environment" . | nindent 12 }}
            {{- .Values.metastore.extraEnv | nindent 12}}

and now the helm chart can support mapping/remapping ENV vars from literal, configmaps, and secrets. This would be a breaking change -- perhaps we want to add a new config variable while continuing to support the K/V style?

[guilload]

Hi @xrl,

What about using extraEnvFrom for secrets?

[xrl]

I need to remap the key of a secret. My kubernetes cluster has a service for provisioning/owning credentials used to access a postgres instance. These are my current set of keys:

% k -n quickwit get secret k3quickwitlogs-postgres-direct-secret -o json | jq -r '.data | keys'
[
  "POSTGRES_DB",
  "POSTGRES_HOST",
  "POSTGRES_PASSWORD",
  "POSTGRES_URL",
  "POSTGRES_USER"
]

but quickwit expects QW_METASTORE_URI not POSTGRES_URL, hence I need to remap the key from the secret I do not control the ENV var expected by the metastore. I need something like this:

        metastore:
          extraEnv:
           -  name: QW_METASTORE_URI
              valueFrom:
                secretKeyRef:
                  name: k3quickwitlogs-postgres-direct-secret
                  key: POSTGRES_URL

[arlyon]

Similar deal with environment:

        image:
          tag: edge
        searcher:
          replicaCount: 2
        environment:
          QW_ENABLE_OPENTELEMETRY_OTLP_EXPORT: "true"
          OTEL_EXPORTER_OTLP_ENDPOINT: http://localhost:7281
          QW_ENABLE_JAEGER_ENDPOINT: "true"
          QW_METASTORE_URI:
            valueFrom:
              secretKeyRef:
                name: default-pguser-default
                key: uri

I want to get the metastore uri from a secret but don't own the secret itself. It seems like we'd be better off just supporting the native k8s syntax here instead of mapping it. Because the value is turned into a string, the metastore uri becomes file:///quickwit/map[valueFrom:map[secretKeyRef:map[key:uri name:default-pguser-default]]] instead