Kubernetes on Bare Metal: Why It's Harder Than You Think (And Why It's Worth It)

TL;DR

Running Kubernetes on bare metal means you are the cloud provider. There is no LoadBalancer API, no magic CSI volume provisioning, and no managed control plane.

• The Stack: kubeadm (Bootstrap), Cilium (eBPF Networking), MetalLB (Load Balancing), Longhorn (Storage), Traefik + cert-manager (Ingress/TLS).
• The Verdict: It is operationally painful to set up, but the ROI in cost savings, raw performance, and deep systems knowledge is unmatched.

The Cloud Made Us Soft

Let me start with a confession: the first time I tried running Kubernetes on bare metal, I spent three hours staring at my terminal, wondering why my newly deployed LoadBalancer service was stuck in <pending>.

If you've spent your entire career on AWS (EKS), Azure (AKS), or GCP (GKE), you've been spoiled. You type kubectl expose, and a cloud controller silently spins up an Application Load Balancer, attaches target groups, and configures security groups. It feels like magic.

On bare metal? Nothing happens. No API is listening. No hardware is being provisioned. You are the cloud now.

And that exact moment of frustration is where the real engineering begins.

Why Repatriate? (The Case for Bare Metal)

Before we get into the architecture, we have to answer the obvious question: Why subject yourself to this?

Cloud repatriation is becoming a massive trend for mature engineering organizations. Here is why we do it:

1. The Cost Multiplier — Cloud bills compound. A high-memory, compute-optimized 3-node cluster on EC2 can easily cost more in six months than buying the physical hardware outright.
2. Raw I/O Performance — No hypervisor overhead. No noisy neighbors. No IOPS throttling or burst-credit exhaustion. When you run a database on a bare-metal NVMe drive, you get the full speed of the PCIe bus.
3. Absolute Control — You own the kernel, the network path, and the storage layer. No surprise vendor deprecations or forced version upgrades.
4. Edge Computing — Factories, military installations, and remote data centers don't have low-latency fiber to us-east-1.

The cloud abstracts away the hardware, but on bare metal, you must architect for the physical reality of failures.

The trade-off? Every abstraction the cloud provided must now be designed, deployed, and maintained by you.

The Architecture: Rebuilding the Cloud

To make bare metal work, you have to replace managed services with open-source equivalents. Here is what that mapping looks like in modern production environments:

Every row in the right column is a critical architectural decision. Let's look at the hardest ones.

Deep Dive 1: Networking Without a VPC

On bare metal, your Container Network Interface (CNI) isn't just a plugin; it is your network. Choose poorly, and you will be debugging dropped packets using tcpdump at 2 AM.

My absolute go-to for modern bare metal is Cilium.

By leveraging eBPF (Extended Berkeley Packet Filter) instead of legacy iptables, Cilium routes packets directly in the Linux kernel.

• Performance: It bypasses the massive CPU overhead of evaluating thousands of iptables rules.
• Observability: Hubble (Cilium's UI) gives you a live, visual service map of every dropped packet.
• Kube-Proxy Replacement: Cilium completely replaces the aging kube-proxy component, dramatically speeding up service routing.

Solving the LoadBalancer Crisis: MetalLB

To fix the <pending> LoadBalancer issue, we use MetalLB. It hooks into your network and assigns real IPs to your services using either Layer 2 ARP/NDP or BGP routing.

Here is the exact configuration that brings your bare-metal LoadBalancers to life:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: production-pool
  namespace: metallb-system
spec:
  addresses:
    - 192.168.10.200-192.168.10.250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default-advertisement
  namespace: metallb-system

With this applied, when you type kubectl expose, MetalLB grabs an IP from the pool, sends an ARP broadcast to your physical switch ("Hey, I own this IP!"), and traffic instantly starts flowing to your pods. It's incredibly elegant.

Deep Dive 2: The Storage Dilemma

Cloud storage is invisible—you request a PersistentVolumeClaim (PVC) and a network disk appears. On bare metal, stateful workloads (like PostgreSQL or Redis) are terrifying without a solid storage backend.

While Rook-Ceph is the industry standard for massive scale, it requires a dedicated PhD in storage engineering to operate safely.

For 90% of teams, Longhorn (built by SUSE/Rancher) is the pragmatic choice. It's a CNCF project that provides highly available distributed block storage. It automatically replicates your volumes across multiple physical nodes. If a hard drive catches fire, Longhorn seamlessly fails over to a replica on another node.

The 3 AM Operational Reality

Deploying bare-metal Kubernetes is fun. Operating it is where the battle is fought. If you're going down this path, you must own these responsibilities:

1. etcd Backups are Life and Death: If your control plane crashes and your etcd database is corrupted, your cluster is gone. Forever. You must automate etcdctl snapshot save to an off-cluster location (like an S3 bucket or a NAS) via a CronJob.
2. Hardware Dies: Hard drives fail. Network interface cards drop packets. RAM goes bad. You must architect your deployments with podAntiAffinity so you never have all replicas of a critical microservice running on the same physical blade.
3. Kernel Panics: A bad Linux kernel update can break your eBPF network stack. Always cordon and drain nodes one by one, verify the upgrade, and then move to the next.

The Payoff

Yes, bare-metal Kubernetes is significantly harder.

But engineers who build and operate bare-metal clusters understand distributed systems at a fundamentally deeper level. They know exactly how BGP routing works. They understand Linux kernel namespaces, cgroups, and block storage replication. They can architect for catastrophic failure because they've experienced failure without an AWS support ticket to fall back on.

Managed Kubernetes is a product. Bare-metal Kubernetes is an education.

When production inevitably goes down, the engineer who learned K8s on bare metal is exactly who you want in the war room.

Stop fighting your own infrastructure. Bare metal Kubernetes is a minefield if you don't know the landscape. If your clusters are unstable, your network is dropping packets, or you're terrified of upgrading your control plane, you are losing money to operational friction.

I build and stabilize production-grade bare metal infrastructure that actually works.

Let's fix your cluster today. Book a Free Infrastructure Audit.