Day 8: K3d and Lightweight Kubernetes — Running Multi-Node Clusters on 16 GB of RAM

Week 2 moves the stack onto Kubernetes. The first obstacle is the cluster itself. Standard distributions such as kubeadm or OpenShift can take 4 to 8 GB of RAM for the control plane before a single workload runs, which is most of a 16 GB laptop.

K3s and k3d

K3s is a CNCF-certified lightweight Kubernetes. It removes legacy cloud-provider drivers, in-tree storage plugins and alpha feature gates, and for a single server it can use an embedded SQLite datastore instead of etcd. It keeps full Kubernetes API compatibility, so the manifests you write here work on bigger clusters later.

k3d runs K3s inside Docker containers: one container per node.

Describing the cluster

A config file makes the cluster reproducible.

# k3d-config.yaml
apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
  name: ai-cluster
servers: 1
agents: 2
image: rancher/k3s:v1.30.0-k3s1
ports:
  - port: 80:80
    nodeFilters: [loadbalancer]
  - port: 443:443
    nodeFilters: [loadbalancer]
options:
  k3s:
    extraArgs:
      - arg: --disable=traefik          # we install our own ingress later (Envoy or Cilium)
        nodeFilters: [server:0]
      - arg: --disable=servicelb
        nodeFilters: [server:0]
  kubeconfig:
    updateDefaultKubeconfig: true
    switchCurrentContext: true

k3d cluster create --config k3d-config.yaml
kubectl get nodes -o wide
docker stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}"

The first command creates one server and two agents, and the cluster is usually ready in well under a minute. The second shows the three nodes as Ready, and the third shows how much memory each node container uses, so you can budget the rest of the machine for workloads. Remove everything with k3d cluster delete ai-cluster.

Disabling Traefik and the service load balancer is a choice for this series, because later lessons install Envoy and Cilium. If you just want an ingress working quickly, leave Traefik enabled, as the Week 2 capstone does. Also note that a plain k3d cluster does not expose a GPU to its nodes. GPU workloads need extra setup, so on a laptop it is normal to use CPU-friendly engines here.

Why real nodes matter

Because k3d creates actual multiple nodes, you get true multi-node scheduling on one machine: taints, tolerations and anti-affinity all behave as they would in production. You can keep stateful vector databases and stateless inference pods on separate nodes, so a noisy model pod cannot starve the database.

A cluster that boots in seconds and disappears with one command makes experiments cheap, which is exactly what you want while learning.

Run it yourself

This is the exact example from the video. This really creates a K3d cluster from the file and deletes it again. I used host port 18080 to avoid clashes on my laptop.

apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
  name: ai-demo
servers: 1
agents: 1
image: rancher/k3s:v1.35.5-k3s1
ports:
  - port: 18080:80
    nodeFilters: [loadbalancer]
options:
  k3s:
    extraArgs:
      - arg: --disable=traefik
        nodeFilters: [server:0]

Run it:

k3d cluster create --config k3d-config.yaml

Output I got while recording the video:

ai-demo 1/1 1/1 true
k3d-ai-demo-agent-0 Ready
k3d-ai-demo-server-0 Ready
containers: 3
80/tcp -> 0.0.0.0:18080
traefik pods: 0

Coming Up Next

Day 9: Kubernetes core for machine learning: pods, Deployments and headless services.

#Kubernetes #K3d #K3s #DevOps #CloudNative #PlatformEngineering #LocalDev #MLOps