Multus and IPAM plugins

Multus CNI is a CNI plugin that enables attaching multiple network interfaces to pods. Multus does not replace CNI plugins, instead it acts as a CNI plugin multiplexer. Multus is useful in certain use cases, especially when pods are network intensive and require extra network interfaces that support dataplane acceleration techniques such as SR-IOV.

Multus can not be deployed standalone. It always requires at least one conventional CNI plugin that fulfills the Kubernetes cluster network requirements. That CNI plugin becomes the default for Multus, and will be used to provide the primary interface for all pods. When deploying K3s with default options, that CNI plugin is Flannel.

Add the Helm Repository

To deploy Multus, we recommend using the following helm repo:

helm repo add rke2-charts https://rke2-charts.rancher.io
helm repo update

Configure the IPAM plugin

An IPAM plugin is required to assign IP addresses on the extra interfaces created by Multus.

host-local

The host-local IPAM plugin allocates ip addresses out of a set of address ranges. It stores the state locally on the host filesystem, hence ensuring uniqueness of IP addresses on a single host. Therefore, we don't recommend it for multi-node clusters. This IPAM plugin does not require any extra deployment. For more information: https://www.cni.dev/plugins/current/ipam/host-local/.

To use the host-local plugin, please create a file called multus-values.yaml with the following content:

config:
  cni_conf:
    confDir: /var/lib/rancher/k3s/agent/etc/cni/net.d
    binDir: /var/lib/rancher/k3s/data/current/bin/
    kubeconfig: /var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig

Whereabouts

Whereabouts is an IP Address Management (IPAM) CNI plugin that assigns IP addresses cluster-wide.

To use the Whereabouts IPAM plugin, please create a file called multus-values.yaml with the following content:

config:
  cni_conf:
    confDir: /var/lib/rancher/k3s/agent/etc/cni/net.d
    binDir: /var/lib/rancher/k3s/data/current/bin/
    kubeconfig: /var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig
rke2-whereabouts:
  fullnameOverride: whereabouts
  enabled: true
  cniConf:
    confDir: /var/lib/rancher/k3s/agent/etc/cni/net.d
    binDir: /var/lib/rancher/k3s/data/current/bin/

Multus DHCP daemon

The dhcp IPAM plugin can be deployed when there is already a DHCP server running on the network. This daemonset takes care of periodically renewing the DHCP lease. For more information please check the official docs of DHCP IPAM plugin.

To use the DHCP plugin, please create a file called multus-values.yaml with the following content:

config:
  cni_conf:
    confDir: /var/lib/rancher/k3s/agent/etc/cni/net.d
    binDir: /var/lib/rancher/k3s/data/current/bin/
    kubeconfig: /var/lib/rancher/k3s/agent/etc/cni/net.d/multus.d/multus.kubeconfig
manifests:
  dhcpDaemonSet: true

Deploy Multus

After creating the multus-values.yaml file, everything is ready to install Multus:

helm install multus rke2-charts/rke2-multus -n kube-system --kubeconfig /etc/rancher/k3s/k3s.yaml --values multus-values.yaml

The helm chart install will deploy a DaemonSet to create Multus pods for installing the required CNI binaries in /var/lib/rancher/k3s/data/current/ and Multus CNI config in /var/lib/rancher/k3s/agent/etc/cni/net.d.

For more information about Multus, refer to the multus-cni documentation.