What you will set up
k3s is a small, certified Kubernetes in one binary, from the team behind Rancher. It comes with what a cluster needs to be useful at once: Traefik to route web traffic to your apps, CoreDNS, a metrics server, and storage on the server's own disk.
Here it runs in k3s's rootless mode, under a user of its own called k3s, so that a container breaking out of Kubernetes lands as that unprivileged user, not as root. Web traffic comes in through Caddy from the Podman guide, which gets the certificates, as for the other apps on the server.
Rootless mode is marked experimental by k3s, and has limits worth knowing before you start:
- It runs a single node. A cluster of several servers needs k3s's normal installation, as root, on servers of their own.
- Only services of type
LoadBalancerget a port on the server, and ports below 1024 move up by 10000: Traefik's 80 and 443 become 10080 and 10443.
Every step below was run on a Melonslab server with Debian 13, next to the apps from our other guides:
- k3s v1.36.4 started rootless, with CoreDNS, Traefik, the metrics server and the storage provisioner running.
- A demo app with two replicas answered through Caddy over HTTPS, and received the visitor's real address.
- With ufw, the Kubernetes API and Traefik's ports could not be reached from the internet.
- A persistent volume kept its data across a reboot, and everything came back by itself.
k3s used about 740 MB of memory with the demo app. k3s asks for at least 2 GB on the server.
Before you start
You need:
- a server set up as in the Podman guide, with Caddy running, and ufw from the security guide;
- an A record and an AAAA record for
demo.example.compointing at your server, for the demo app.
Replace demo.example.com with your own name throughout.
1. Prepare the server
As root, install the tools rootless k3s uses for its network and storage. The Podman guide has most of them already:
apt install -y slirp4netns fuse-overlayfs uidmap
Let users' services control CPU, memory and disk use, which the rootless kubelet requires:
mkdir -p /etc/systemd/system/user@.service.d
printf '[Service]\nDelegate=cpu cpuset io memory pids\n' > /etc/systemd/system/user@.service.d/delegate.conf
systemctl daemon-reload
And turn on IP forwarding, without which k3s refuses to start:
printf 'net.ipv4.ip_forward=1\n' > /etc/sysctl.d/90-k3s.conf
sysctl --system
ufw still drops traffic forwarded through the server, unless a rule allows it, as the WireGuard guide does for its own.
2. Install k3s
curl -sfL https://get.k3s.io | INSTALL_K3S_SKIP_ENABLE=true INSTALL_K3S_SKIP_START=true sh -
This is k3s's own installer, and it installs the newest stable release. The two settings keep it from starting k3s as root: it installs /usr/local/bin/k3s, and kubectl alongside it, and a root service, k3s.service, which stays off. k3s --version shows the version.
3. Create the user
useradd -m -s /bin/bash k3s
loginctl enable-linger k3s
machinectl shell k3s@
Everything up to step 6 runs as k3s.
4. Start k3s
Install k3s's own service file for rootless mode, and start it:
mkdir -p ~/.config/systemd/user
curl -sfL -o ~/.config/systemd/user/k3s-rootless.service https://raw.githubusercontent.com/k3s-io/k3s/main/k3s-rootless.service
systemctl --user daemon-reload
systemctl --user enable --now k3s-rootless
After a minute or two, check it:
kubectl get nodes
kubectl get pods -A
kubectl finds its login for the rootless cluster, ~/.kube/k3s.yaml, by itself. The node is Ready, and the pods for CoreDNS, Traefik, the metrics server and the storage provisioner are Running. The two helm-install pods show Completed: they installed Traefik.
5. Keep the ports closed
k3s opens three ports on the server: 6443 for the Kubernetes API, and Traefik's 10080 and 10443. ufw blocks all three from the internet, and this guide never opens them: Caddy reaches Traefik on the server itself, and you use kubectl as k3s on the server.
6. Deploy an app
As an example, deploy whoami, a tiny web server that shows the request it received. Create ~/whoami.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: whoami
spec:
replicas: 2
selector:
matchLabels:
app: whoami
template:
metadata:
labels:
app: whoami
spec:
containers:
- name: whoami
image: docker.io/traefik/whoami:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: whoami
spec:
selector:
app: whoami
ports:
- port: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: whoami
spec:
rules:
- host: demo.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: whoami
port:
number: 80
The Deployment runs two copies of the app, the Service gives them one address inside the cluster, and the Ingress tells Traefik to send requests for demo.example.com to it. Apply it:
kubectl apply -f ~/whoami.yaml
kubectl rollout status deploy/whoami
7. Put Caddy in front
Traefik replaces the visitor's address, which Caddy sends along, unless it trusts where the connection comes from. In rootless mode, Caddy's connections reach Traefik from 10.42.0.1. Create ~/traefik-config.yaml:
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
ports:
web:
forwardedHeaders:
# Caddy's connections reach Traefik from this address.
trustedIPs:
- 10.42.0.1/32
kubectl apply -f ~/traefik-config.yaml
k3s reinstalls Traefik with the change within a minute. Then go back to root with exit, switch to machinectl shell caddy@, and add this block at the end of ~/Caddyfile:
demo.example.com {
reverse_proxy 127.0.0.1:10080
}
Restart Caddy with systemctl --user restart caddy, and open https://demo.example.com. whoami shows the request, with your own address first in X-Forwarded-For, and X-Forwarded-Proto: https. Each new app gets its own Ingress with its own name, and its own block in the Caddyfile, pointing at the same 127.0.0.1:10080.
8. Keep data
Apps that keep data ask for a persistent volume. k3s creates them with its storage provisioner, as folders under /home/k3s/.rancher/k3s/storage. For example:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data
spec:
accessModes: [ReadWriteOnce]
resources:
requests:
storage: 1Gi
A pod then mounts it under volumes with persistentVolumeClaim: {claimName: data}. The data stays when the pod is replaced, and when the server restarts.
9. Keep it up to date
k3s does not update itself. To update, run the installer from step 2 again as root, which installs the newest stable release, and then restart k3s as k3s:
systemctl --user restart k3s-rootless
Traefik, CoreDNS and the other parts that come with k3s are updated with it. Read the release notes on k3s's GitHub page before a new minor version, such as 1.37.
10. Back up
As k3s:
mkdir -p ~/backup
systemctl --user stop k3s-rootless
tar -czf ~/backup/k3s.tar.gz -C ~/.rancher/k3s server/db server/token server/cred server/tls storage
systemctl --user start k3s-rootless
That saves the cluster's database, with everything you applied, its keys and certificates, and the persistent volumes. Keep your YAML files too, so that you can apply them again anywhere. Copy ~/backup to another machine, and keep it private.
Troubleshooting
k3s stops at once, and journalctl --user -u k3s-rootless says expected sysctl value "net.ipv4.ip_forward" to be "1". Do the last part of step 1.
Your app sees 10.42.0.1 as the visitor's address, or http instead of https. Apply the Traefik change from step 7, and wait for the Traefik pod to be replaced.