Advanced Options / Configuration
This section contains advanced information describing the different ways you can run and manage K3s, as well as steps necessary to prepare the host OS for K3s use.
Certificate Management
Certificate Authority Certificates
K3s generates self-signed Certificate Authority (CA) Certificates during startup of the first server node. These CA certificates are valid for 10 years, and are not automatically renewed.
For information on using custom CA certificates, or renewing the self-signed CA certificates, see the k3s certificate rotate-ca
command documentation.
Client and Server certificates
K3s client and server certificates are valid for 365 days from their date of issuance. Any certificates that are expired, or within 90 days of expiring, are automatically renewed every time K3s starts.
For information on manually rotating client and server certificates, see the k3s certificate rotate
command documentation.
Token Management
By default, K3s uses a single static token for both servers and agents. With care, this token can be rotated once the cluster has been created.
It is also possible to enable a second static token that can only be used to join agents, or to create temporary kubeadm
style join tokens that expire automatically.
For more information, see the k3s token
command documentation.
Configuring an HTTP proxy
If you are running K3s in an environment, which only has external connectivity through an HTTP proxy, you can configure your proxy settings on the K3s systemd service. These proxy settings will then be used in K3s and passed down to the embedded containerd and kubelet. Note that proxy configuration and other environment variables from the host are NOT passed into Pods.
The K3s installation script will automatically take the HTTP_PROXY
, HTTPS_PROXY
and NO_PROXY
, as well as the CONTAINERD_HTTP_PROXY
, CONTAINERD_HTTPS_PROXY
and CONTAINERD_NO_PROXY
variables from the current shell, if they are present, and write them to the environment file of your systemd service, usually:
/etc/systemd/system/k3s.service.env
/etc/systemd/system/k3s-agent.service.env
Of course, you can also configure the proxy by editing these files.
K3s will automatically add the cluster internal Pod and Service IP ranges and cluster DNS domain to the list of NO_PROXY
entries. You should ensure that the IP address ranges used by the Kubernetes nodes themselves (i.e. the public and private IPs of the nodes) are included in the NO_PROXY
list, or that the nodes can be reached through the proxy.
HTTP_PROXY=http://your-proxy.example.com:8888
HTTPS_PROXY=http://your-proxy.example.com:8888
NO_PROXY=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
If you want to configure the proxy settings for containerd without affecting K3s and the Kubelet, you can prefix the variables with CONTAINERD_
:
CONTAINERD_HTTP_PROXY=http://your-proxy.example.com:8888
CONTAINERD_HTTPS_PROXY=http://your-proxy.example.com:8888
CONTAINERD_NO_PROXY=127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
Using Docker as the Container Runtime
K3s includes and defaults to containerd, an industry-standard container runtime. As of Kubernetes 1.24, the Kubelet no longer includes dockershim, the component that allows the kubelet to communicate with dockerd. K3s 1.24 and higher include cri-dockerd, which allows seamless upgrade from prior releases of K3s while continuing to use the Docker container runtime.
To use Docker instead of containerd:
-
Install Docker on the K3s node. One of Rancher's Docker installation scripts can be used to install Docker:
curl https://releases.rancher.com/install-docker/20.10.sh | sh
-
Install K3s using the
--docker
option:curl -sfL https://get.k3s.io | sh -s - --docker
-
Confirm that the cluster is available:
$ sudo k3s kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system local-path-provisioner-6d59f47c7-lncxn 1/1 Running 0 51s
kube-system metrics-server-7566d596c8-9tnck 1/1 Running 0 51s
kube-system helm-install-traefik-mbkn9 0/1 Completed 1 51s
kube-system coredns-8655855d6-rtbnb 1/1 Running 0 51s
kube-system svclb-traefik-jbmvl 2/2 Running 0 43s
kube-system traefik-758cd5fc85-2wz97 1/1 Running 0 43s -
Confirm that the Docker containers are running:
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3e4d34729602 897ce3c5fc8f "entry" About a minute ago Up About a minute k8s_lb-port-443_svclb-traefik-jbmvl_kube-system_d46f10c6-073f-4c7e-8d7a-8e7ac18f9cb0_0
bffdc9d7a65f rancher/klipper-lb "entry" About a minute ago Up About a minute k8s_lb-port-80_svclb-traefik-jbmvl_kube-system_d46f10c6-073f-4c7e-8d7a-8e7ac18f9cb0_0
436b85c5e38d rancher/library-traefik "/traefik --configfi…" About a minute ago Up About a minute k8s_traefik_traefik-758cd5fc85-2wz97_kube-system_07abe831-ffd6-4206-bfa1-7c9ca4fb39e7_0
de8fded06188 rancher/pause:3.1 "/pause" About a minute ago Up About a minute k8s_POD_svclb-traefik-jbmvl_kube-system_d46f10c6-073f-4c7e-8d7a-8e7ac18f9cb0_0
7c6a30aeeb2f rancher/pause:3.1 "/pause" About a minute ago Up About a minute k8s_POD_traefik-758cd5fc85-2wz97_kube-system_07abe831-ffd6-4206-bfa1-7c9ca4fb39e7_0
ae6c58cab4a7 9d12f9848b99 "local-path-provisio…" About a minute ago Up About a minute k8s_local-path-provisioner_local-path-provisioner-6d59f47c7-lncxn_kube-system_2dbd22bf-6ad9-4bea-a73d-620c90a6c1c1_0
be1450e1a11e 9dd718864ce6 "/metrics-server" About a minute ago Up About a minute k8s_metrics-server_metrics-server-7566d596c8-9tnck_kube-system_031e74b5-e9ef-47ef-a88d-fbf3f726cbc6_0
4454d14e4d3f c4d3d16fe508 "/coredns -conf /etc…" About a minute ago Up About a minute k8s_coredns_coredns-8655855d6-rtbnb_kube-system_d05725df-4fb1-410a-8e82-2b1c8278a6a1_0
c3675b87f96c rancher/pause:3.1 "/pause" About a minute ago Up About a minute k8s_POD_coredns-8655855d6-rtbnb_kube-system_d05725df-4fb1-410a-8e82-2b1c8278a6a1_0
4b1fddbe6ca6 rancher/pause:3.1 "/pause" About a minute ago Up About a minute k8s_POD_local-path-provisioner-6d59f47c7-lncxn_kube-system_2dbd22bf-6ad9-4bea-a73d-620c90a6c1c1_0
64d3517d4a95 rancher/pause:3.1 "/pause"
Using etcdctl
etcdctl provides a CLI for interacting with etcd servers. K3s does not bundle etcdctl.
If you would like to use etcdctl to interact with K3s's embedded etcd, install etcdctl using the official documentation.
ETCD_VERSION="v3.5.5"
ETCD_URL="https://github.com/etcd-io/etcd/releases/download/${ETCD_VERSION}/etcd-${ETCD_VERSION}-linux-amd64.tar.gz"
curl -sL ${ETCD_URL} | sudo tar -zxv --strip-components=1 -C /usr/local/bin
You may then use etcdctl by configuring it to use the K3s-managed certificates and keys for authentication:
sudo etcdctl version \
--cacert=/var/lib/rancher/k3s/server/tls/etcd/server-ca.crt \
--cert=/var/lib/rancher/k3s/server/tls/etcd/client.crt \
--key=/var/lib/rancher/k3s/server/tls/etcd/client.key
Configuring containerd
K3s includes containerd 2.0 as of the February 2025 releases: v1.31.6+k3s1 and v1.32.2+k3s1.
Be aware that containerd 2.0 prefers config version 3, while containerd 1.7 prefers config version 2.
K3s will generate a configuration file for containerd at /var/lib/rancher/k3s/agent/etc/containerd/config.toml
, using values specific to the current cluster and node configuration.
For advanced customization, you can create a containerd config template in the same directory:
- For containerd 2.0, place a version 3 configuration template in
config-v3.toml.tmpl
See the containerd 2.0 documentation for more information. - For containerd 1.7 and earlier, place a version 2 configuration template in
config.toml.tmpl
See the containerd 1.7 documentation for more information.
Containerd 2.0 is backwards compatible with prior config versions, and k3s will continue to render legacy version 2 configuration from config.toml.tmpl
if config-v3.toml.tmpl
is not found.
The template file is rendered into the containerd config using the text/template
library.
See ContainerdConfigTemplateV3
and ContainerdConfigTemplate
in templates.go
for the default template content.
The template is executed with a ContainerdConfig
struct as its dot value (data argument).
Base template
You can extend the K3s base template instead of copy-pasting the complete stock template out of the K3s source code. This is useful if you only need to build on the existing configuration by adding a few extra lines before or after the defaults.
#/var/lib/rancher/k3s/agent/etc/containerd/config-v3.toml.tmpl
{{ template "base" . }}
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.'custom']
runtime_type = "io.containerd.runc.v2"
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.'custom'.options]
BinaryName = "/usr/bin/custom-container-runtime"
SystemdCgroup = true
For best results, do NOT simply copy a prerendered config.toml
into the template and make your desired changes. Use the base template, or provide a full template based on the k3s defaults linked above.