v1.28.X
Upgrade Notice
Before upgrading from earlier releases, be sure to read the Kubernetes Urgent Upgrade Notes.
Release v1.28.7+k3s1
This release updates Kubernetes to v1.28.7, and fixes a number of issues.
For more details on what's new, see the Kubernetes release notes.
Changes since v1.28.6+k3s2:
- Chore: bump Local Path Provisioner version (#9426)
- Bump cri-dockerd to fix compat with Docker Engine 25 (#9293)
- Auto Dependency Bump (#9419)
- Runtimes refactor using exec.LookPath (#9431)
- Directories containing runtimes need to be included in the $PATH environment variable for effective runtime detection.
- Changed how lastHeartBeatTime works in the etcd condition (#9424)
- Bump Flannel v0.24.2 + remove multiclustercidr (#9401)
- Allow executors to define containerd and docker behavior (#9254)
- Update Kube-router to v2.0.1 (#9404)
- Backports for 2024-02 release cycle (#9462)
- Enable longer http timeout requests (#9444)
- Test_UnitApplyContainerdQoSClassConfigFileIfPresent (#9440)
- Support PR testing installs (#9469)
- Update Kubernetes to v1.28.7 (#9492)
- Fix drone publish for arm (#9508)
- Remove failing Drone step (#9516)
- Restore original order of agent startup functions (#9545)
- Fix netpol startup when flannel is disabled (#9578)
Release v1.28.6+k3s2
This release updates Kubernetes to v1.28.6, and fixes a number of issues.
For more details on what's new, see the Kubernetes release notes.
Important Notes
Addresses the runc CVE: CVE-2024-21626 by updating runc to v1.1.12.
Changes since v1.28.5+k3s1:
- Add a retry around updating a secrets-encrypt node annotations (#9125)
- Wait for taint to be gone in the node before starting the netpol controller (#9175)
- Etcd condition (#9181)
- Backports for 2024-01 (#9203)
- Pin opa version for missing dependency chain (#9216)
- Added support for env *_PROXY variables for agent loadbalancer (#9206)
- Etcd node is nil (#9228)
- Update to v1.28.6 and Go 1.20.13 (#9260)
- Use
ipFamilyPolicy: RequireDualStackfor dual-stack kube-dns (#9269) - Backports for 2024-01 k3s2 (#9336)
- Bump runc to v1.1.12 and helm-controller to v0.15.7
- Fix handling of bare hostname or IP as endpoint address in registries.yaml
- Bump helm-controller to fix issue with ChartContent (#9346)
Release v1.28.5+k3s1
This release updates Kubernetes to v1.28.5, and fixes a number of issues.
For more details on what's new, see the Kubernetes release notes.
Changes since v1.28.4+k3s1:
- Remove s390x steps temporarily since runners are disabled (#8983)
- Remove s390x from manifest (#8998)
- Fix overlapping address range (#8913)
- Modify CONTRIBUTING.md guide (#8954)
- Nov 2023 stable channel update (#9022)
- Default runtime and runtime classes for wasm/nvidia/crun (#8936)
- Added runtime classes for wasm/nvidia/crun
- Added default runtime flag for containerd
- Bump containerd/runc to v1.7.10-k3s1/v1.1.10 (#8962)
- Allow setting default-runtime on servers (#9027)
- Bump containerd to v1.7.11 (#9040)
- Update to v1.28.5-k3s1 (#9081)
Release v1.28.4+k3s2
This release updates Kubernetes to v1.28.4, and fixes a number of issues.
For more details on what's new, see the Kubernetes release notes.
Changes since v1.28.3+k3s2:
- Update channels latest to v1.27.7+k3s2 (#8799)
- Add etcd status condition (#8724)
- Now the user can see the etcd status from each node in a simple way
- ADR for etcd status (#8355)
- Wasm shims detection (#8751)
- Automatic discovery of WebAssembly runtimes
- Add warning for removal of multiclustercidr flag (#8758)
- Improve dualStack log (#8798)
- Optimize: Simplify and clean up Dockerfile (#8244)
- Add: timezone info in image (#8764)
-
- New timezone info in Docker image allows the use of
spec.timeZonein CronJobs
- New timezone info in Docker image allows the use of
-
- Bump kine to fix nats, postgres, and watch issues (#8778)
- Bumped kine to v0.11.0 to resolve issues with postgres and NATS, fix performance of watch channels under heavy load, and improve compatibility with the reference implementation.
- QoS-class resource configuration (#8726)
- Containerd may now be configured to use rdt or blockio configuration by defining
rdt_config.yamlorblockio_config.yamlfiles.
- Containerd may now be configured to use rdt or blockio configuration by defining
- Add agent flag disable-apiserver-lb (#8717)
- Add agent flag disable-apiserver-lb, agent will not start load balance proxy.
- Force umount for NFS mount (like with longhorn) (#8521)
- General updates to README (#8786)
- Fix wrong warning from restorecon in install script (#8871)
- Fix issue with snapshot metadata configmap (#8835)
- Omit snapshot list configmap entries for snapshots without extra metadata
- Skip initial datastore reconcile during cluster-reset (#8861)
- Tweaked order of ingress IPs in ServiceLB (#8711)
- Improved ingress IP ordering from ServiceLB
- Disable helm CRD installation for disable-helm-controller (#8702)
- More improves for K3s patch release docs (#8800)
- Update install.sh sha256sum (#8885)
- Add jitter to client config retry to avoid hammering servers when they are starting up (#8863)
- Handle nil pointer when runtime core is not ready in etcd (#8886)
- Bump dynamiclistener; reduce snapshot controller log spew (#8894)
- Bumped dynamiclistener to address a race condition that could cause a server to fail to sync its certificates into the Kubernetes secret
- Reduced etcd snapshot log spam during initial cluster startup
- Remove depends_on for e2e step; fix cert rotate e2e (#8906)
- Fix etcd snapshot S3 issues (#8926)
- Don't apply S3 retention if S3 client failed to initialize
- Don't request metadata when listing S3 snapshots
- Print key instead of file path in snapshot metadata log message
- Update to v1.28.4 and Go to v1.20.11 (#8920)
- Remove s390x steps temporarily since runners are disabled (#8983)
- Remove s390x from manifest (#8998)
Release v1.28.3+k3s2
This release updates Kubernetes to v1.28.3, and fixes a number of issues.
For more details on what's new, see the Kubernetes release notes.
Changes since v1.28.3+k3s1:
- Restore selinux context systemd unit file (#8593)
- Update channel to v1.27.7+k3s1 (#8753)
- Bump Sonobuoy version (#8710)
- Bump Trivy version (#8739)
- Fix: Access outer scope .SystemdCgroup (#8761)
- Fixed failing to start with nvidia-container-runtime
- Upgrade traefik chart to v25.0.0 (#8771)
- Update traefik to fix registry value (#8792)
- Don't use iptables-save/iptables-restore if it will corrupt rules (#8795)
Release v1.28.3+k3s1
This release updates Kubernetes to v1.28.3, and fixes a number of issues.
For more details on what's new, see the Kubernetes release notes.
Changes since v1.28.2+k3s1:
- Fix error reporting (#8250)
- Add context to flannel errors (#8284)
- Update channel, September patch release (#8397)
- Add missing link to drone in documentation (#8295)
- Include the interface name in the error message (#8346)
- Add extraArgs to vpn provider (#8354)
- Allow to pass extra args to the vpn provider
- Disable HTTP on main etcd client port (#8402)
- Embedded etcd no longer serves http requests on the client port, only grpc. This addresses a performance issue that could cause watch stream starvation under load. For more information, see https://github.com/etcd-io/etcd/issues/15402
- Server token rotation (#8215)
- Fix issues with etcd member removal after reset (#8392)
- Fixed an issue that could cause k3s to attempt to remove members from the etcd cluster immediately following a cluster-reset/restore, if they were queued for removal at the time the snapshot was taken.
- Fix gofmt error (#8439)
- Added advertise address integration test (#8344)
- Added cluster reset from non bootstrap nodes on snapshot restore e2e test (#8292)
- Fix .github regex to skip drone runs on gh action bumps (#8433)
- Added error when cluster reset while using server flag (#8385)
- The user will receive a error when --cluster-reset with the --server flag
- Update kube-router (#8423)
- Update kube-router to v2.0.0-rc7 to fix performance issues
- Add SHA256 signatures of the install script (#8312)
-
- Add SHA256 signatures of the install script.
-
- Add --image-service-endpoint flag (#8279)
- Add
--image-service-endpointflag to specify an external image service socket.
- Add
- Don't ignore assets in home dir if system assets exist (#8458)
- Pass SystemdCgroup setting through to nvidia runtime options (#8470)
- Fixed issue that would cause pods using nvidia container runtime to be killed after a few seconds, when using newer versions of nvidia-container-toolkit.
- Improve release docs - updated (#8414)
- Take IPFamily precedence based on order (#8460)
- Fix spellcheck problem (#8507)
- Network defaults are duplicated, remove one (#8523)
- Fix slemicro check for selinux (#8526)
- Update install.sh.sha256sum (#8566)
- System agent push tags fix (#8568)
- Fixed tailscale node IP dualstack mode in case of IPv4 only node (#8524)
- Server Token Rotation (#8265)
- Users can now rotate the server token using
k3s token rotate -t <OLD_TOKEN> --new-token <NEW_TOKEN>. After command succeeds, all server nodes must be restarted with the new token.
- Users can now rotate the server token using
- E2E Domain Drone Cleanup (#8579)
- Bump containerd to v1.7.7-k3s1 (#8604)
- Bump busybox to v1.36.1 (#8602)
- Migrate to using custom resource to store etcd snapshot metadata (#8064)
- Switch build target from main.go to a package. (#8342)
- Use IPv6 in case is the first configured IP with dualstack (#8581)
- Bump traefik, golang.org/x/net, google.golang.org/grpc (#8624)
- Update kube-router package in build script (#8630)
- Add etcd-only/control-plane-only server test and fix control-plane-only server crash (#8638)
- Use
version.Programnot K3s in token rotate logs (#8653) - [Windows Port (#7259)
- Fix CloudDualStackNodeIPs feature-gate inconsistency (#8667)
- Re-enable etcd endpoint auto-sync (#8675)
- Manually requeue configmap reconcile when no nodes have reconciled snapshots (#8683)
- Update to v1.28.3 and Go to v1.20.10 (#8682)
- Fix s3 snapshot restore (#8729)
Release v1.28.2+k3s1
This release updates Kubernetes to v1.28.2, and fixes a number of issues.
For more details on what's new, see the Kubernetes release notes.
Changes since v1.28.1+k3s1:
- Update channel for version v1.28 (#8305)
- Bump kine to v0.10.3 (#8323)
- Update to v1.28.2 and go v1.20.8 (#8364)
- Bump embedded containerd to v1.7.6
- Bump embedded stargz-snapshotter plugin to latest
- Fixed intermittent drone CI failures due to race conditions in test environment setup scripts
- Fixed CI failures due to changes to api discovery changes in Kubernetes 1.28
Release v1.28.1+k3s1
This release is K3S's first in the v1.28 line. This release updates Kubernetes to v1.28.1.
Important
This release includes remediation for CVE-2023-32187, a potential Denial of Service attack vector on K3s servers. See https://github.com/k3s-io/k3s/security/advisories/GHSA-m4hf-6vgr-75r2 for more information, including documentation on changes in behavior that harden clusters against this vulnerability.
Critical Regression
Kubernetes v1.28 contains a critical regression (kubernetes/kubernetes#120247) that causes init containers to run at the same time as app containers following a restart of the node. This issue will be fixed in v1.28.2. We do not recommend using K3s v1.28 at this time if your application depends on init containers.
For more details on what's new, see the Kubernetes release notes.
Changes since v1.27.5+k3s1:
- Update to v1.28.1 (#8239)
- CLI Removal for v1.28.0 (#8203)
- Secrets Encryption V3 (#8111)
- Add new CLI flag to disable TLS SAN CN filtering (#8252)
- Added a new
--tls-san-securityoption.
- Added a new
- Add RWMutex to address controller (#8268)