Littera Nova

I write things sometimes.

Using Calico with multiple interfaces

Word count: 222 (~2 minutes), Last modified: Sun, 10 Jul 2022 00:31:57 GMT

If you're using Calico and your nodes have multiple network interfaces, you may have issues where your DNS strangely times out on some nodes and some pods sometimes. This might manifest, as it did for me, like this:

headnode01:~$ sudo calicoctl node status
Calico process is running.

IPv4 BGP status
+--------------+-------------------+-------+----------+---------+
| PEER ADDRESS |     PEER TYPE     | STATE |  SINCE   |  INFO   |
+--------------+-------------------+-------+----------+---------+
| 10.142.0.11  | node-to-node mesh | start | 00:02:10 | Connect |
+--------------+-------------------+-------+----------+---------+

IPv6 BGP status
No IPv6 peers found.

If it stays stuck in start, try checking whether your nodes have multiple IPs (for example via something like pdsh -g node ip addr | fgrep brd | cut -f 4 -d ' ' | sort | uniq -c). If they do, you may want to set the IP_AUTODETECTION_METHOD to that CIDR (for example kubectl -n calico-system set env ds calico-node IP_AUTODETECTION_METHOD=cidr=10.141.0.0/24; kubectl -n calico-system rollout restart ds calico-node). If your node status now looks like this:

headnode01:~$ sudo calicoctl node status
Calico process is running.

IPv4 BGP status
+--------------+-------------------+-------+----------+-------------+
| PEER ADDRESS |     PEER TYPE     | STATE |  SINCE   |  INFO       |
+--------------+-------------------+-------+----------+-------------+
| 10.141.0.11  | node-to-node mesh | up    | 00:13:22 | Established |
+--------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

then you've successfully changed the address. Happy kubeing!