Pool member state is not being refreshed

[asmajlovic]

Environment

• TMOS/Bigip Version: 17.5.1 Build 0.0.7 Final
• Terraform Version: 1.13.1
• Terraform bigip provider Version: 1.24.0

Summary

Manual (Configuration tool / web UI) modification of pool member state is not picked up by a Terraform state refresh meaning that pool members will silently end up in an inconsistent / undesired state.

Steps To Reproduce

Steps to reproduce the behavior:

variables.tf

variable "f5_partition" {
  type    = string
  default = "Common"
}

variable "nodes" {
  type = map(object({
    address           = optional(string)
    description       = optional(string)
    fqdn_autopopulate = optional(string, "enabled")
    fqdn_interval     = optional(number, 300)
    monitor           = optional(string, "default")
  }))
  default = {}
}

variable "monitors" {
  type = map(object({
    adaptive    = optional(string, "disabled")
    destination = optional(string, "*:*")
    interval    = optional(string, "5")
    parent      = optional(string, "http")
    password    = optional(string)
    receive     = string
    send        = string
    ssl_profile = optional(string)
    timeout     = optional(string, "15")
    username    = optional(string)
  }))
  default = {}
}

variable "pools" {
  type = map(object({
    allow_nat              = optional(string, "yes")
    allow_snat             = optional(string, "yes")
    description            = optional(string)
    load_balancing_mode    = optional(string, "round-robin")
    minimum_active_members = optional(number, 0)
    monitors               = optional(list(string), ["ttp_generic"])
    node_attachments       = optional(map(object({
      fqdn_autopopulate = optional(string, "enabled")
      priority_group    = optional(number, 0)
      ratio             = optional(number, 1)
      state             = optional(string, "enabled")
    })))
  }))
  default = {}
}

node.tf

resource "bigip_ltm_node" "node" {
  for_each    = var.nodes
  name        = "/${var.f5_partition}/${each.key}"
  address     = each.value.address == null ? each.key : each.value.address
  description = each.value.description
  monitor     = each.value.monitor

  // Consider entries without an address as an FQDN node
  dynamic "fqdn" {
    for_each = each.value.address == null ? [true] : []
    content {
      autopopulate = each.value.fqdn_autopopulate
      interval     = each.value.fqdn_interval
    }
  }
}

pool.tf

locals {
  pool_attachments = flatten([
    for pool_key, pool_values in var.pools : [
      for attachment_key, attachment_values in pool_values.node_attachments : {
        pool_name              = pool_key
        node_name              = attachment_key
        node_fqdn_autopopulate = attachment_values.fqdn_autopopulate
        node_priority_group    = attachment_values.priority_group
        node_ratio             = attachment_values.ratio
        node_state             = attachment_values.state
      }
    ]
  ])
}

resource "bigip_ltm_pool" "pool" {
  for_each               = var.pools
  name                   = "/${var.f5_partition}/${each.key}"
  description            = each.value.description
  load_balancing_mode    = each.value.load_balancing_mode
  minimum_active_members = each.value.minimum_active_members
  monitors               = formatlist("/%s/%s", var.f5_partition, each.value.monitors)
  allow_nat              = each.value.allow_nat
  allow_snat             = each.value.allow_snat
  depends_on             = [bigip_ltm_monitor.monitor]
}

resource "bigip_ltm_pool_attachment" "attach_node" {
  for_each = {
    for entry in local.pool_attachments : "${entry.pool_name} : ${entry.node_name}" => entry
  }
  node              = "/${var.f5_partition}/${each.value.node_name}"
  pool              = "/${var.f5_partition}/${each.value.pool_name}"
  state             = each.value.node_state
  ratio             = each.value.node_ratio
  priority_group    = each.value.node_priority_group
  fqdn_autopopulate = each.value.node_fqdn_autopopulate
  depends_on        = [bigip_ltm_pool.pool, bigip_ltm_node.node]
}

misc.auto.tfvars

nodes = {
  "node01" = {
    description = "Test node #1"
    address     = "172.20.1.10"
  }
  "node02" = {
    description = "Test node #2"
    address     = "172.20.1.11"
  }
}

pools = {
  "pool01" = {
    description = "Test pool #1"
    node_attachments = {
      "node01:5000" = {}
      "node02:5000" = {}
    }
    monitors = [
      "tcp"
    ]
  }
}

Terraform plan

Initializing the backend...
Initializing modules...
Initializing provider plugins...
- Reusing previous version of hashicorp/vault from the dependency lock file
- Reusing previous version of f5networks/bigip from the dependency lock file
- Using previously-installed hashicorp/vault v5.2.1
- Using previously-installed f5networks/bigip v1.24.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Success! The configuration is valid.

...

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.F5_automation.bigip_ltm_node.node["node01"] will be created
  + resource "bigip_ltm_node" "node" {
      + address          = "172.20.1.10"
      + connection_limit = (known after apply)
      + description      = "Test node #1"
      + dynamic_ratio    = (known after apply)
      + id               = (known after apply)
      + monitor          = "default"
      + name             = "/Common/node01"
      + rate_limit       = (known after apply)
      + ratio            = (known after apply)
      + session          = (known after apply)
      + state            = (known after apply)
    }

  # module.F5_automation.bigip_ltm_node.node["node02"] will be created
  + resource "bigip_ltm_node" "node" {
      + address          = "172.20.1.11"
      + connection_limit = (known after apply)
      + description      = "Test node #2"
      + dynamic_ratio    = (known after apply)
      + id               = (known after apply)
      + monitor          = "default"
      + name             = "/Common/node02"
      + rate_limit       = (known after apply)
      + ratio            = (known after apply)
      + session          = (known after apply)
      + state            = (known after apply)
    }

  # module.F5_automation.bigip_ltm_pool.pool["pool01"] will be created
  + resource "bigip_ltm_pool" "pool" {
      + allow_nat              = "yes"
      + allow_snat             = "yes"
      + description            = "Test pool #1"
      + id                     = (known after apply)
      + load_balancing_mode    = "round-robin"
      + minimum_active_members = 0
      + monitors               = [
          + "/Common/tcp",
        ]
      + name                   = "/Common/pool01"
      + reselect_tries         = (known after apply)
      + service_down_action    = (known after apply)
      + slow_ramp_time         = (known after apply)
    }

  # module.F5_automation.bigip_ltm_pool_attachment.attach_node["pool01 : node01:5000"] will be created
  + resource "bigip_ltm_pool_attachment" "attach_node" {
      + connection_limit      = (known after apply)
      + connection_rate_limit = (known after apply)
      + dynamic_ratio         = (known after apply)
      + fqdn_autopopulate     = "enabled"
      + id                    = (known after apply)
      + monitor               = (known after apply)
      + node                  = "/Common/node01:5000"
      + pool                  = "/Common/pool01"
      + priority_group        = 0
      + ratio                 = 1
      + state                 = "enabled"
    }

  # module.F5_automation.bigip_ltm_pool_attachment.attach_node["pool01 : node02:5000"] will be created
  + resource "bigip_ltm_pool_attachment" "attach_node" {
      + connection_limit      = (known after apply)
      + connection_rate_limit = (known after apply)
      + dynamic_ratio         = (known after apply)
      + fqdn_autopopulate     = "enabled"
      + id                    = (known after apply)
      + monitor               = (known after apply)
      + node                  = "/Common/node02:5000"
      + pool                  = "/Common/pool01"
      + priority_group        = 0
      + ratio                 = 1
      + state                 = "enabled"
    }

Plan: 5 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  ~ bigip_nodes          = {
      + node01                                                  = "/Common/node01"
      + node02                                                  = "/Common/node02"
        # (89 unchanged attributes hidden)
    }
  ~ pool_attachments     = {
      + "pool01 : node01:5000"                                                                = "/Common/pool01"
      + "pool01 : node02:5000"                                                                = "/Common/pool01"
        # (106 unchanged attributes hidden)
    }
  ~ pool_names           = {
      + pool01                     = "/Common/pool01"
        # (61 unchanged attributes hidden)
    }

Terraform apply

module.F5_automation.bigip_ltm_pool.pool["pool01"]: Creating...
module.F5_automation.bigip_ltm_node.node["node01"]: Creating...
module.F5_automation.bigip_ltm_node.node["node02"]: Creating...
module.F5_automation.bigip_ltm_node.node["node02"]: Creation complete after 0s [id=/Common/node02]
module.F5_automation.bigip_ltm_node.node["node01"]: Creation complete after 0s [id=/Common/node01]
module.F5_automation.bigip_ltm_pool.pool["pool01"]: Creation complete after 1s [id=/Common/pool01]
module.F5_automation.bigip_ltm_pool_attachment.attach_node["pool01 : node02:5000"]: Creating...
module.F5_automation.bigip_ltm_pool_attachment.attach_node["pool01 : node01:5000"]: Creating...
module.F5_automation.bigip_ltm_pool_attachment.attach_node["pool01 : node02:5000"]: Creation complete after 0s [id=/Common/pool01-/Common/node02:5000]
module.F5_automation.bigip_ltm_pool_attachment.attach_node["pool01 : node01:5000"]: Creation complete after 0s [id=/Common/pool01-/Common/node01:5000]

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Manual modification of pool member state via UI

Follow-up Terraform plan

...
No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Expected Behavior

Terraform F5 module detects the real infrastructure state change in the pool member state and includes the correction in the plan.

Actual Behavior

Terraform plan claims / states that the real infrastructure matches our configuration.

[sunitharonan]

Hi @asmajlovic, thank you for reporting this bug. A manual change to a pool member’s state (e.g., from enabled to disabled or forced offline in the BIG-IP UI) should be detected during a Terraform state refresh, causing terraform plan to show a diff that would revert the state to match the configuration. I created a backlog item INFRAANO-2033 for our team to work on this bug.

[sunitharonan]

Hi @asmajlovic , In order for us to prioritize this issue please reach out to us at infraano_devs@f5.com