← All posts

Course · 30 lessons

AI Advanced: Supercomputing, Kernels & Frontier Systems

How the biggest AI systems really work

A 30-day deep dive for specialists: GPU cluster architecture and interconnects, custom kernels and compilers, distributed training and reasoning models, then agent swarms, MCP and sovereign AI infrastructure.

Starting soon: 30 lessons, one a day.

Week 1 · High-Performance GPU Cluster Architecture & Interconnect Fabrics

  1. Day 1Beyond Kubernetes: The Architecture of Modern AI SuperclustersComing soon
  2. Day 2GPU Memory Hierarchy & The Bandwidth WallComing soon
  3. Day 3Interconnect Topologies: NVLink, NVSwitch, and PCIe Gen 5/6Coming soon
  4. Day 4AI Network Fabrics: InfiniBand vs. RoCE v2Coming soon
  5. Day 5NCCL Deep Dive: Optimizing Collective CommunicationsComing soon
  6. Day 6Diagnosing Physical Cluster DegradationComing soon
  7. Day 7Slurm vs. Kubernetes for SupercomputingComing soon

Week 2 · Low-Level Performance Engineering & Custom AI Compilers

  1. Day 8GPU Hardware Architecture from the InsideComing soon
  2. Day 9Writing Custom GPU Kernels with OpenAI TritonComing soon
  3. Day 10The Mechanics of FlashAttentionComing soon
  4. Day 11Modern PyTorch Compilers: TorchDynamo and TorchInductorComing soon
  5. Day 12Next-Gen Microscaling Precision: FP8, FP4, and MX FormatsComing soon
  6. Day 13Speculative Decoding & Medusa ArchitectureComing soon
  7. Day 14Advanced KV-Cache EngineeringComing soon

Week 3 · Distributed Scaling & Advanced Post-Training (RL & Reasoning)

  1. Day 15The 4D Parallelism ParadigmComing soon
  2. Day 16DeepSpeed ZeRO & PyTorch FSDP-2Coming soon
  3. Day 17Mixture of Experts (MoE) Architecture at ScaleComing soon
  4. Day 18Preference Alignment Beyond SFT: DPO, KTO, and PPOComing soon
  5. Day 19Reasoning Models & Reinforcement Learning via GRPOComing soon
  6. Day 20Synthetic Data Flywheels & Automated Curriculum DesignComing soon
  7. Day 21Model Distillation & Extreme QuantizationComing soon

Week 4 · Autonomous Swarms, Model Context Protocol (MCP) & Enterprise Sovereignty

  1. Day 22Model Context Protocol (MCP) ArchitectureComing soon
  2. Day 23Decentralized Multi-Agent SwarmsComing soon
  3. Day 24Self-Healing and Self-Compiling Code RuntimesComing soon
  4. Day 25Frontier AI Red-Teaming & Adversarial DefenseComing soon
  5. Day 26Confidential Computing for AI InfrastructureComing soon
  6. Day 27Sovereign AI Datacenter ArchitectureComing soon
  7. Day 28Real-Time Multimodal Streaming PipelinesComing soon
  8. Day 29AI Infrastructure FinOps & Compute Unit EconomicsComing soon
  9. Day 30The Post-Transformer HorizonComing soon