Index already exists error when upgrading Quickwit on Kubernetes with S3 storage.

[rampanwar97]

Hi Quickwit team,

I'm running Quickwit on a Kubernetes environment with indexes stored in an S3 bucket. During the initial deployment, the index creation works as expected.

However, when I upgrade my environment (via Helm or otherwise), the Quickwit pod fails with an error indicating that the index already exists.

Here’s the command I’m using inside the pod:

`bash
Copy
Edit
/bin/bash -c "quickwit source describe --index {{ .index }} --source {{ .source.source_id }} --endpoint ${QW_CLUSTER_ENDPOINT} || quickwit source create --index {{ .index }} --source-config {{ .source.source_id }}.yaml --endpoint ${QW_CLUSTER_ENDPOINT}"

This logic assumes that if the source doesn’t exist, it should create it—but in case the index already exists, Quickwit still fails. It seems like the source create or something internally is checking for the index even before the source creation.
`

Expected behavior:

When upgrading the environment, if the index already exists, Quickwit should not fail but continue gracefully.

Alternatively, the CLI should provide a way to check and skip creation if the index already exists.

Suggestion:

Add idempotency or a built-in flag like --skip-if-exists for quickwit index create or quickwit source create.

Or provide a command to safely check and conditionally create both the index and source.

Let me know if I can provide any logs or more details to help reproduce the issue.

Thanks for the great work on Quickwit!

[guilload]

You mention having an issue for creating an index but the snippet you pasted is for creating a source, so I'm slightly confused.

Nevertheless, we recently added a --create option to the quickwit index upgrade command to work around this issue. It is not yet released but when it is, we should use that instead.

[rampanwar97]

Thanks for quick response.

Sorry, my mistake. I was referring to the index creation step. When I run the helm upgrade command, it throws an error saying the index already exists. Is there an option to skip index creation if it already exists, so that on the next helm upgrade it checks and skips it if the index is present?

[ramsinghtmdc]

Hi team,

I noticed in recent discussions that a --create option has been added to the quickwit index upgrade command to work around the current upgrade issue. Could you please let me know in which upcoming release this option will be available?

Thanks for your help and for all the great work on Quickwit!