Install of Talos on OVHCloud dedicated server with private(vrack) and public interface

[morind]

Hi,
I've been struggling with a problem for several days.
I want to install Talos onto one OVH dedicated server with the default public interface and one interface linked to the "vrack" offer proposed by OVH (Level 2 routing).
I want to use a dedicated VLAN (VLAN ID: 3 in my case)
After lot of investigations and watching plenty of issues on github... I'm a bit lost ;(
I use metal platform.
The first boot is ok. I enforce the bootLoaderPath with OVHCloud API to boot my server with talos and I generate the image with a custom "schematic"
I've tried a basic one, with just the platform "metal" and a more complicated schematic conf based on Metal Network Configuration

I've tried with Talos 1.9.6 and 1.11.1. And with 1.11.1 with the default CNI (Flannel) and now (my target) with Cilium.
At the end, Talos is well bootstrapped. Cilium is OK.
Connectivity is ok with the "public" world from the default interface but I can't reach anything on my private network.
Here after traces in a "netshoot" Pod (arping and tcpdump performed from within a container in the Pod)

arping -c 1 10.3.1.241
ARPING 10.3.1.241 from 10.3.1.240 enp1303449.3
Sent 1 probes (1 broadcast(s))
Received 0 response(s)

18:08:51.262905 xx:xx:xx:c5:cd > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 3, p 0, ethertype ARP (0x0806), Request who-has 10.3.1.241 (ff:ff:ff:ff:ff:ff) tell 10.3.1.240, length 28

The VLAN ID 3 seems well defined in the request and sent to the right interface but not response nonetheless.
The IP 10.3.1.241 is the other dedicated server on Ubuntu and 10.3.1.240 is the IP of my Talos server I've enforced.
Subnet is 10.3.0.0/16
I don't want/have to use the gateway 10.3.0.1 for now.
List of (relevant) interfaces:

8: ens3f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether xx:xx:xx:xx:2d:9a brd ff:ff:ff:ff:ff:ff
    altname enp193s0f0np0
    altname enxb8e9241c2d9a
    inet XX:XX:XX.235/32 metric 1024 brd XX:XX:XX.235 scope global ens3f0np0
       valid_lft forever preferred_lft forever
    inet6 fe80::XXXX:2d9a/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
9: ens3f1np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether xx:xx:xx:xx:2d:9b brd ff:ff:ff:ff:ff:ff
    altname enp193s0f1np1
    altname enxb8e9241c2d9b
    inet6 fe80::XXXX:2d9b/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
10: enp129s0f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether xx:xx:xx:xx:c5:cc brd ff:ff:ff:ff:ff:ff
    altname enxe09d7311c5cc
    inet6 fe80::XXX:XXX:fe11:c5cc/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
11: enp129s0f1np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether xx:xx:xx:xx:c5:cd brd ff:ff:ff:ff:ff:ff
    altname enxe09d7311c5cd
    inet 10.3.1.240/32 brd 10.3.1.240 scope global enp129s0f1np1
       valid_lft forever preferred_lft forever
    inet6 fe80::XXX:XXX:fe11:c5cd/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
12: enp1303449.3@enp129s0f1np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether xx:xx:xx:xx:c5:cd brd ff:ff:ff:ff:ff:ff
    inet 10.3.1.240/16 brd 10.3.255.255 scope global enp1303449.3
       valid_lft forever preferred_lft forever
    inet6 fe80::XXX:XXX:fe11:c5cd/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever

ens3f0np0: the public interface with DHCP
ens3f1np1: not used
enp129s0f0np0: not used
enp129s0f1np1: the interface linked to my OVH private network (Vrack)
enp1303449.3: The Vlan interface I've configured based on enp129s0f1np1

Any ideas ? Any suggestions ? I take everything ;)

Note: with Ubuntu on same type of dedicated server it works. I just had to follow this doc

[morind]

HI @dedene
You've already successfully dealt with OVH and Talos according to your post here
Any tips for me to configure my dedicated server with 2 interfaces (Vrack + Public) / Talos / Cilium ?

[BeArchiTek]

Hello @morind

Why did you let 10.3.1.240/32 inenp129s0f1np1 ?
As you are using VLAN (3), I would expect to have the IP only on the sub-interface.

[morind]

Thanks @BeArchiTek !
Yes, good catch. I've made so many tries and Im not sure to have tested with IP only defined on VLAN interface and not on the physical one.
I'm going to test that with some couple of new tests
If I found the solution, I will post it here..