-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
-
Currently Nomad captures the addresses returned by CNI in
cniNetworkConfigurator.cniToAllocNet(client/allocrunner/networking_cni.go:438-515). It walksres.Interfaces[name].IPConfigsand copies the first IPv4 intoAllocNetworkStatus.Addressand the first IPv6 intoAllocNetworkStatus.AddressIPv6. Because we never re-read the network namespace, any addresses that are not explicitly reported by the CNI result are invisible to Nomad. -
In a macvlan+dhcp chain the DHCP plugin emits only the IPv4 lease. The IPv6 addresses inside the allocation’s namespace are being configured later by kernel SLAAC/DHCPv6 and never appear in
res.Interfaces, sonetStatus.AddressIPv6is left empty. Multiple IPv6 addresses would also be collapsed to just the first even if they were reported, becauseAllocNetworkStatusonly has a singleAddressIPv6field (nomad/structs/structs.go:12263-12286). -
The allocation network status captured above is handed to the service-registration layer by
groupServiceHook(client/allocrunner/group_service_hook.go:285-311). All registration providers—Nomad service discovery, Consul, etc.—use the common helperserviceregistration.GetAddressto turn that status into an advertised address (client/serviceregistration/address.go:14-187). Whenaddress_mode="alloc"we always returnnetStatus.Address(IPv4); whenaddress_mode="alloc_ipv6"we returnnetStatus.AddressIPv6. -
Nomad service discovery calls
GetAddressbefore it publishes each service (client/serviceregistration/nsd/nsd.go:372), and the Consul client does the same (command/agent/consul/service_client.go:1320,1505). BecauseAddressIPv6is empty, both providers register only the IPv4 address andalloc_ipv6registrations end up blank.
Implications
-
As long as we rely solely on the CNI result, any IPv6 addresses that are configured by the kernel (RA/SLAAC) or by a plugin that doesn’t populate
IPConfigswill never reachAllocNetworkStatus, so Nomad cannot advertise them in either its own service discovery or Consul. -
Separately, once this is resolved, we currently only store a single IPv4 and IPv6 string per allocation, so additional IPv6 addresses cannot be represented without extending
AllocNetworkStatus.
Next steps
-
Enhance
cniToAllocNet(or a post-Setup hook) to inspect the allocation’s namespace (e.g. vianetlink.AddrListonnetStatus.InterfaceName) so we can populateAddressIPv6even when the CNI plugin doesn’t report it. Once that field is filled, Nomad service discovery and Consul registrations will pick it up automatically throughGetAddress. -
Continue discussion around whether Nomad should expose multiple IPv6 addresses, which can be a relatively common configuration when using both SLAAC and DHCPv6, or when enabling privacy extensions. Privacy extensions are less relevant unless using a fixed MAC address, which isn't the case by default with CNI unless otherwise configured.
Nomad version
1.11
Operating system and Environment details
Linux amd64/arm64
Metadata
Metadata
Assignees
Labels
Type
Projects
Status