Skip to content

IPv6 Addresses Missing from Allocation Status with Macvlan/DHCP CNI plugins #27108

@arodd

Description

@arodd
  • Currently Nomad captures the addresses returned by CNI in cniNetworkConfigurator.cniToAllocNet (client/allocrunner/networking_cni.go:438-515). It walks res.Interfaces[name].IPConfigs and copies the first IPv4 into AllocNetworkStatus.Address and the first IPv6 into AllocNetworkStatus.AddressIPv6. Because we never re-read the network namespace, any addresses that are not explicitly reported by the CNI result are invisible to Nomad.

  • In a macvlan+dhcp chain the DHCP plugin emits only the IPv4 lease. The IPv6 addresses inside the allocation’s namespace are being configured later by kernel SLAAC/DHCPv6 and never appear in res.Interfaces, so netStatus.AddressIPv6 is left empty. Multiple IPv6 addresses would also be collapsed to just the first even if they were reported, because AllocNetworkStatus only has a single AddressIPv6 field (nomad/structs/structs.go:12263-12286).

  • The allocation network status captured above is handed to the service-registration layer by groupServiceHook (client/allocrunner/group_service_hook.go:285-311). All registration providers—Nomad service discovery, Consul, etc.—use the common helper serviceregistration.GetAddress to turn that status into an advertised address (client/serviceregistration/address.go:14-187). When address_mode="alloc" we always return netStatus.Address (IPv4); when address_mode="alloc_ipv6" we return netStatus.AddressIPv6.

  • Nomad service discovery calls GetAddress before it publishes each service (client/serviceregistration/nsd/nsd.go:372), and the Consul client does the same (command/agent/consul/service_client.go:1320,1505). Because AddressIPv6 is empty, both providers register only the IPv4 address and alloc_ipv6 registrations end up blank.

Implications

  • As long as we rely solely on the CNI result, any IPv6 addresses that are configured by the kernel (RA/SLAAC) or by a plugin that doesn’t populate IPConfigs will never reach AllocNetworkStatus, so Nomad cannot advertise them in either its own service discovery or Consul.

  • Separately, once this is resolved, we currently only store a single IPv4 and IPv6 string per allocation, so additional IPv6 addresses cannot be represented without extending AllocNetworkStatus.

Next steps

  1. Enhance cniToAllocNet (or a post-Setup hook) to inspect the allocation’s namespace (e.g. via netlink.AddrList on netStatus.InterfaceName) so we can populate AddressIPv6 even when the CNI plugin doesn’t report it. Once that field is filled, Nomad service discovery and Consul registrations will pick it up automatically through GetAddress.

  2. Continue discussion around whether Nomad should expose multiple IPv6 addresses, which can be a relatively common configuration when using both SLAAC and DHCPv6, or when enabling privacy extensions. Privacy extensions are less relevant unless using a fixed MAC address, which isn't the case by default with CNI unless otherwise configured.

Nomad version

1.11

Operating system and Environment details

Linux amd64/arm64

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    Needs Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions