Init container doesn't support restart_policy option (to transform it in a sidecar container)

[arthurgur]

Terraform Version, Provider Version and Kubernetes Version

Terraform version: 1.6.2
Kubernetes provider version: 2.27.0
Kubernetes version: 1.19

Affected Resource(s)

Terraform Configuration Files

resource "kubernetes_deployment_v1" "example" {
  metadata {
    name = "terraform-example"
    labels = {
      test = "MyExampleApp"
    }
  }

  spec {
    replicas = 3

    selector {
      match_labels = {
        test = "MyExampleApp"
      }
    }

    template {
      metadata {
        labels = {
          test = "MyExampleApp"
        }
      }

      spec {
        container {
          name  = "example"
          image = "nginx:1.21.6"

        # Init container
        init_container {
          name  = "init-setup"
          image = "busybox:latest"
          restart_policy = "Always"
        }
      }
    }
  }
}

Debug Output

Terraform v1.6.2
on linux_amd64
Initializing plugins and modules...
╷
│ Error: Unsupported argument
│ 
│   on app.tf line XX, in resource "kubernetes_deployment_v1" "this":
│   XX:             restart_policy = "Always"
│ 
│ An argument named "restart_policy" is not expected here.
╵
Operation failed: failed running terraform plan (exit 1)

Panic Output

Steps to Reproduce

1. Create a ressource kubernetes_deployment_v1 with init container and set the option restart_policy of the init container to "Always" to transform it in a sidecar.
2. Apply your code

Expected Behavior

Pod with sidecar init container created

Actual Behavior

Option non supported

Important Factoids

References

• https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[BBBmau]

Enabled by default with Kubernetes 1.29, a [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/) named SidecarContainers allows you to specify a restartPolicy for containers listed in a Pod's initContainers field. 
These restartable sidecar containers are independent with other [init containers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) and main application container within the same pod.
These can be started, stopped, or restarted without effecting the main application container and other init containers.

Hello @arthurgur, thanks for opening this issue! With this being marked as beta in v1.29, we can proceed with adding this into the provider. You are open to creating a PR that adds this since this is just adding a new field within

terraform-provider-kubernetes/kubernetes/schema_container.go

Line 256 in bff3ae2

func containerFields(isUpdatable bool) map[string]*schema.Schema {

we can guide you in adding the field and also adding tests that check for this.

[arthurgur]

Hello @BBBmau, thanks for your reply. Unfortunately I don't have rights to create a branch (or I dont know how, it's my first contribution).

[BBBmau]

@arthurgur no worries, be sure to look at our contributing guidelines., I would look into also the section on Pull Request Lifecycle since this is your first contribution.

[uyilmaz]

Hello all! Is there a way to use this now without waiting for v3 major release?

[baraksalomon]

If it helps anyone, a workaround can be creating a kubernetes_deployment with eveything but restart_policy for the init_container and adding a dependent kubernetes_manifest with a patch that adds restart_policy to the init_container.

it still has an issue since you cannot add an ignore_changes for restart_policy specifically in the kubernetes_deployment.
but you can add an ignore_changes to the init_container which is sub-optional.

[jedney0901]

Massive help thank you @baraksalomon.

Hopefully this gets resolved soon, I went down so many rabbit holes trying to understand this issue.

[MathieuDR]

If it helps anyone, a workaround can be creating a kubernetes_deployment with eveything but restart_policy for the init_container and adding a dependent kubernetes_manifest with a patch that adds restart_policy to the init_container.

it still has an issue since you cannot add an ignore_changes for restart_policy specifically in the kubernetes_deployment. but you can add an ignore_changes to the init_container which is sub-optional.

The problem I have with this, is that the manifest only gets applied when I apply it twice, it doesn't get applied the first time when the deployment changes

resource "kubernetes_deployment_v1" "backend_deployment" {
  #NOTE : We don't wait for the rollout because of the side-car limitation
  wait_for_rollout = false
  ...
        init_container {
          name  = "cloud-sql-proxy"
          image = "gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.14.1"
          #https://github.com/hashicorp/terraform-provider-kubernetes/issues/2446
          #restart_policy = "Always"

          args = [
            "...:...:pg-staging",
            "--port=5431"
          ]

          security_context {
            run_as_non_root = true
          }
        }
...
}

# manifest
# Patch the deployment to add restart_policy to the cloud-sql-proxy init container
# This works around the Terraform Kubernetes provider limitation
resource "kubernetes_manifest" "backend_deployment_sidecar_patch" {
  manifest = {
    apiVersion = "apps/v1"
    kind       = "Deployment"
    metadata = {
      name      = local.instance_name
      namespace = var.namespace
      annotations = {
        # tried to 'force' this patch using the app version, but it doesn't see the drift on restartPolicy
        "terraform.io/force-patch" = var.app_version
      }
    }
    spec = {
      template = {
        spec = {
          initContainers = [
            {
              name          = "cloud-sql-proxy"
              restartPolicy = "Always"
            },
            # needs to reference this, otherwise second init container gets removed.
            { name = "${local.instance_name}-migration" }
          ]
        }
      }
    }
  }

  computed_fields = [
    "metadata.annotations",
    # tries to ignore / compute second init container. Not sure if needed
    "spec.template.spec.initContainers[1]"
  ]

  field_manager {
    name            = "terraform-patch"
    force_conflicts = true
  }

  depends_on = [
    kubernetes_deployment_v1.backend_deployment
  ]
}

I even tried to manage it completely through the manifest but I always need to apply twice.