How to specify underline interface for vxlan.calico?

[shaqtsui]

By default vxlan.calico is built on top of default route's interface.
But sometimes default route corresponding interface is not used for node communication.

[tomastigera]

I don't think it is the default route interface. I needs to be the interface that has the IP that is used as the node IP within the cluster

https://github.com/projectcalico/calico/blob/release-v3.30/felix/dataplane/linux/vxlan_mgr.go#L650

This depends on your mode ip auto detection.

If you need something more complex like using the vxlan overlay over multiple interfaces, that is currently not possible, but will be with 3.31 and eBPF dataplane.

[shaqtsui]

I needs to be the interface that has the IP that is used as the node IP within the cluster
Seems not the case.

Check my config:
kg nodes -o give:

NAME       STATUS   ROLES           AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                         KERNEL-VERSION   CONTAINER-RUNTIME
server01   Ready    control-plane   50d   v1.32.3   192.168.1.88   <none>        Debian GNU/Linux 12 (bookworm)   6.1.0-33-amd64   containerd://1.7.27
server02   Ready    <none>          50d   v1.32.3   192.168.1.89   <none>        Debian GNU/Linux 12 (bookworm)   6.1.0-33-amd64   containerd://1.7.27
server03   Ready    <none>          50d   v1.32.3   192.168.1.90   <none>        Debian GNU/Linux 12 (bookworm)   6.1.0-33-amd64   containerd://1.7.27

ip -d link show vxlan.calico in server 01(192.168.1.88) gives:

17: vxlan.calico: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 66:36:5c:54:4c:62 brd ff:ff:ff:ff:ff:ff promiscuity 0  allmulti 0 minmtu 68 maxmtu 65535 
    vxlan id 4096 local 112.64.52.140 dev ppp0 srcport 0 0 dstport 4789 nolearning ttl auto ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536

ip route gives :

default via 112.64.52.1 dev ppp0 proto static metric 102 
10.244.6.0/26 via 10.244.6.16 dev vxlan.calico onlink 
10.244.40.192/26 via 10.244.40.200 dev vxlan.calico onlink 
blackhole 10.244.188.0/26 proto 80 
10.244.188.32 dev cali7d125b1d2c0 scope link 
10.244.188.33 dev cali46e4017f36f scope link 
10.244.188.34 dev cali638360e8a11 scope link 
10.244.188.35 dev cali95c7f7ff2bc scope link 
10.244.188.37 dev calia4887f73c9b scope link 
10.244.188.40 dev cali1edd383dc22 scope link 
112.64.52.1 dev ppp0 proto kernel scope link src 112.64.52.140 
112.64.52.1 dev ppp0 proto kernel scope link src 112.64.52.140 metric 102 
192.168.1.0/24 dev br0 proto kernel scope link src 192.168.1.88 metric 425 

From above, I can see vxlan.calico is buit ontop of default rout's interface, not node ip's interface

[mazdakn]

What you are saying is correct in your env. Clearly, VXLAN tunnels are using the default route. But it does not mean the behavior is baked into Calico. Baiscally, this is a factor of what address should be used for routing, and it can be configured. This is what we call IP Autodection: https://docs.tigera.io/calico/latest/reference/configure-calico-node#ip-autodetection-methods

[shaqtsui]

Any more lights from community?