Course outline▾
Week 1 · Bare-Metal Infrastructure & Advanced Containerization
- Day 1The AI Infrastructure Shift — Why the Model is Only 10% of the Production System
- Day 2Advanced Docker for AI — Multi-Stage Builds for Heavy Python Runtimes
- Day 3Hardware Acceleration in Docker — GPU Passthrough and Limits for Constrained Hardware
- Day 4Optimizing Model Weights — Quantization Basics (GGUF, AWQ) for Efficient Inference
- Day 5Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution
- Day 6Containerizing AI Microservices — Building and Packaging APIs with FastAPI
- Day 7Docker Compose — Architecting a Multi-Container Database and Inference Stack Locally
Week 2 · Kubernetes Foundation for MLOps
- Day 8K3d and Lightweight Kubernetes — Running Multi-Node Clusters on 16 GB of RAM
- Day 9Kubernetes Core for Machine Learning — Pods, Deployments and Headless Services
- Day 10Persistent Volumes (PV/PVC) — Managing Heavy Model Weights and Vector Stores
- Day 11StatefulSets for Vector Databases — Deploying Milvus or Qdrant in Kubernetes
- Day 12Auto-Scaling AI Inference — Using Horizontal Pod Autoscalers for Spiky API Traffic
- Day 13Helm Charts — Packaging, Versioning and Updating AI Deployments
- Day 14Week 2 Capstone — Deploying a Highly Available Local AI Application on K3d
Week 3 · High-Performance Networking & Security Mesh
- Day 15AI Network Bottlenecks — Transitioning from iptables to eBPF
- Day 16Cilium & eBPF: Sub-Millisecond Routing for Heavy AI Microservices
- Day 17Service Mesh 101: Installing Istio over Your Local Cluster
- Day 18Envoy Proxy as an AI Gateway: Load Balancing and Terminating LLM Prompts
- Day 19Zero-Trust mTLS: Encrypting Traffic Between Autonomous Agents
- Day 20API Rate Limiting & Egress Control: Preventing Agents from Exfiltrating Data
- Day 21Distributed Tracing: Debugging Agent Workflows with Jaeger and OpenTelemetry
Week 4 · Enterprise Agents & CI/CD Workflows
- Day 22Introduction to MLOps: Bridging AI Development with Continuous Integration
- Day 23GitOps with ArgoCD: Automated, Self-Healing AI Deployments
- Day 24Building Ephemeral AI Sandboxes: Secure Execution Environments for Agent Code
- Day 25Multi-Agent Orchestration Frameworks: Exploring Mastra, LangGraph, and CrewAI
- Day 26Tool-Calling in Production: Connecting Agents Securely to Internal Databases
- Day 27MLOps Guardrails: Evaluating Agents and Preventing Hallucinations in Real-Time
- Day 28Continuous Training Pipelines: Connecting Data Ingestion to Automated Model Retraining
- Day 29Managing Technical Debt and Infrastructure Drift in Enterprise AI
- Day 30The AI-Native Platform: Scaling from a Sandbox Environment to Global Cloud Architecture
Day 15: AI Network Bottlenecks — Transitioning from iptables to eBPF
2026-11-21 · 8 min read
Week 3 turns to the network and security layer. Most teams tune the model and the GPU, then discover that the network path inside the cluster is eating CPU that inference needed. This lesson explains why, and what replaces it.
How a Service IP reaches a pod
When one pod calls a Kubernetes Service, kube-proxy translates the Service IP to a pod IP using Linux iptables rules. Rules are appended to chains, and the kernel checks a packet against them in sequence. That is an O(N) lookup. In a cluster with thousands of Services the evaluation adds measurable latency to every new connection, and burns host CPU. (kube-proxy also has an IPVS mode that uses hash tables, but many clusters still run the default iptables mode.)
Why AI makes it worse
AI traffic is heavy. A request can carry a long context, and a RAG pipeline can stream batches of embeddings. A single 1,536-dimension float32 embedding is about 6 KB, a 128,000-token context is roughly half a megabyte of text, and a batch of 1,000 embeddings is around 6 MB. Multiply that by thousands of concurrent requests across replicas and vector databases, and packet handling competes with inference for the same CPUs.
eBPF: programs in the kernel
eBPF lets you load small, verified programs into the Linux kernel's execution path without writing a kernel module or rebooting. Instead of walking an iptables chain, an eBPF program on a kernel hook looks up the destination in a hash map, a constant-time operation that does not depend on how many Services exist, and sends the packet straight to the target pod. It also avoids extra trips through user space.
The safety comes from the verifier: before an eBPF program runs, the kernel checks that it cannot crash the kernel, access memory it should not, or loop forever. That is what makes it acceptable to run programs from a networking plugin inside the kernel data path.
What to take away
- The iptables model is O(N) in the number of services and can become a CPU tax for chatty AI workloads. Whether it hurts you depends on your cluster size and traffic, so measure before you migrate, for example by watching host CPU softirq time and per-connection latency.
- eBPF replaces the rule chain with a constant-time lookup, which scales with traffic instead of with service count.
- You do not write the eBPF programs yourself. A networking plugin such as Cilium ships them, and that is the next lesson.
You can see how many iptables rules a node carries with sudo iptables-save | wc -l. Compare that count on a small cluster and on a large one, and the growth with the number of Services becomes concrete.
Run it yourself
This is the exact example from the video. This counts steps for a rule chain and a hash lookup. It models the algorithms, it is not a benchmark of iptables or eBPF.
def chain(rules, key): # iptables: walk the rules in order
for n, r in enumerate(rules, 1):
if r == key:
return n
for n in (100, 1000, 10000):
rules = [f"svc-{i}" for i in range(n)]
table = {r: i for i, r in enumerate(rules)} # eBPF: a hash map
last = rules[-1]
table[last] # one lookup
print(f"{n:5d} svcs: chain {chain(rules, last):5d}, map 1")
Run it:
python scan.py
Output I got while recording the video:
100 svcs: chain 100, map 1
1000 svcs: chain 1000, map 1
10000 svcs: chain 10000, map 1
Coming Up Next
Day 16: Cilium and eBPF: sub-millisecond routing for heavy AI microservices.
#eBPF #KubernetesNetworking #MLOps #LinuxKernel #PlatformEngineering #TechInfrastructure