Goal
I want to understand how large compute infrastructure is run, and reading is not getting me there. I’ll be documenting my journey to break this into a series of frameworks and skills in my learning journey. The initial study will be done virtually until I reach a point where I’m blocked by hardware.
The lab
Three Debian 12 virtual machines (2 vCPUs each) on a single Proxmox host, built from scripts and configured with Ansible. There are no GPUs in it.
That is where the series starts, and the lab grows as it goes. Part 1 puts a Slurm scheduler on the three machines, and Part 2 adds a Kubernetes cluster on the same three. Part 3 is reading on how industry uses the two schedulers, and Part 4 puts a model server, object storage, Ray and Terraform on top of it, and Part 5 adds the tools a shared cluster needs: job queues, event-driven scaling, delivery from git, policy, a gateway and self-service provisioning. In Part 6 the lab gets a few extra short-lived machines around it, for emulated management controllers, an inventory, a network boot server, RAID drills and a separate management network. Part 7 adds an emulated cloud account and emulated GPUs, and a virtual NUMA machine. Part 8 is reading only, on the GPU hardware the lab cannot run. The three machines stay the core of it.
3 Industry usedone · reading only
6 Bare metaldone
Emulated BMCs·Rack power plan·Drift and burn-in·NetBox inventory·Hardening and Secure Boot·Network boot with PXE·RAID and management network
7 Cloud + GPUemulated done · real run pending
AWS autoscaling·GPU fleet monitoring·GPU scheduling·NUMA topology
Diagram: the nine parts and where each stands today. Each part and each post is a link.
What I want to learn in each part
The map above is for finding posts. This table says what I was trying to learn from each part and what it costs.
| # | Part | What I want to learn | Cost |
|---|---|---|---|
| 0 | Foundations | How the lab is put together, why LXC containers and Tailscale fit, and how Go and Python differ for tooling | free |
| 1 | Slurm | How a scheduler places work, tracks nodes and keeps accounting | free |
| 2 | Kubernetes | The cluster lifecycle: upgrades, adding and removing nodes, autoscaling, capacity and monitoring | free |
| 3 | Slurm and Kubernetes in industry | Which scheduler large GPU sites run, why many run both, and what changes for rack-scale NVLink systems, from public evidence | free, reading only |
| 4 | ML platform on Kubernetes | What it takes to serve a model and move data to it, with Ray, infrastructure as code and a command line tool | free |
| 5 | Platform layer on Kubernetes | How a shared cluster is governed: delivery from git, queues and quotas, scaling, policy, traffic and self-service | free |
| 6 | Bare-metal fleet, emulated | How a fleet of servers is brought up and kept healthy without real hardware: management controllers, power, inventory, hardening, provisioning, storage and networks | free |
| 7 | Cloud and GPU | How cloud and GPU capacity is requested, watched and scheduled, tried on emulators first so nothing is spent, plus how memory topology affects placement | the real run spends money |
| 8 | References | How GPUs connect inside a server and between servers, what the platform names mean, and what GPU racks need for power and cooling | free, reading only |
Part 7 holds the only real run that costs money. Part 8 has no lab: it is reading and arithmetic on hardware I do not have, so each post says that at the start.
Challenges
Some of what I want to learn can’t be done in this lab. I have no real GPUs or rack, no out-of-band management controllers, no high-speed network fabric or power distribution gear, and no production traffic or other people depending on it.
Where a part touches one of them, I’ll use an emulator or a worked exercise on paper and say which in the post. In the future I’ll detail the hardware kit I’ll buy to help practice further.
Next
Take the quiz to test your understanding: final quiz for Parts 1 to 8
Part 0: the foundations, Part 1: building a three-node Slurm cluster, Part 2: Kubernetes on the same three VMs, Part 3: Slurm and Kubernetes in industry and Part 4: a model server on Kubernetes under load.
Comments