Certified Kubernetes Administrator (CKA) #2 Cluster Architecture 1: Control plane (apiserver/etcd/scheduler/controller-manager)

In #1 The Exam Environment we nailed down the setup and time management. Now for the main event. CKA is an exam that asks “how does a cluster run, and how do you fix it when it breaks,” and the starting point of that answer is the control plane. The control plane is the brain of the cluster. It decides which Pod goes on which node, ceaselessly closes the gap between the desired state and the actual state, and receives and stores every change in one place.

In this post we’ll work through, from an operator’s perspective, what the four core components and the cloud-controller-manager that make up the control plane each do, how they run as static Pods, and what symptoms appear when each one dies. The intuition you build here becomes the foundation for #24 Control plane Troubleshooting and #7 etcd Backup and Restore. If the basic concepts of Kubernetes feel unfamiliar, reading Kubernetes Basics Part 1 first will make this post land much better.

A cluster splits into the control plane and the nodes #

A Kubernetes cluster splits broadly into two kinds of machines: the control plane nodes that govern the cluster, and the worker nodes where the actual workloads (Pods) run. This post covers only the control plane; the worker node’s kubelet, kube-proxy, and CRI continue in #3.

What the control plane does, in one sentence: it receives and stores the desired state the user declared, and ceaselessly reflects that state onto the actual cluster. Five components split this work among themselves.

Once you understand how these five interlock, you can immediately pinpoint “which component is this a symptom of” when a failure hits. This is exactly why the Troubleshooting domain is 30% of the exam.

kube-apiserver: the gateway for all communication #

kube-apiserver is the front door of the control plane. Every request against the cluster passes through the apiserver, without exception. kubectl commands, the other control plane components, the nodes’ kubelets, and the controllers all talk to the apiserver’s REST API. In other words, the apiserver is also the only component in the cluster that can access etcd directly. Everything else reads and writes state only through the apiserver.

When a request comes in, the apiserver takes it through three gates in order.

1. Authentication. It confirms who the requesting subject is. It determines identity using certificates, tokens, ServiceAccounts, and the like.
2. Authorization. It checks whether that subject has permission to perform the action. In CKA this is mostly decided by RBAC, covered in depth in #9.
3. Admission control. Before a request that has passed authentication and authorization is actually written to etcd, it is validated or mutated. Default injection, policy enforcement, and the like happen here.

Only a request that passes all three gates is recorded in etcd. You can confirm the apiserver’s behavior and run-time arguments in its static Pod manifest.

# apiserver static Pod manifest
cat /etc/kubernetes/manifests/kube-apiserver.yaml

The command section of the manifest lists key flags such as --etcd-servers, --client-ca-file, and --authorization-mode. In CKA, a problem where the apiserver won’t come up most often stems from a typo in one of these flags or a certificate path.

etcd: the single source of truth for cluster state #

etcd is a distributed key-value store and the one and only place where all of the cluster’s state is stored. Which Deployments exist, which Pod is placed on which node, which ConfigMaps and Secrets exist — all of it is held in etcd. etcd is the final destination of every object we create through the apiserver.

A key operating principle follows from this. Backing up etcd means backing up the entire cluster state. Conversely, if etcd is corrupted, the cluster’s memory disappears wholesale. That’s why saving and restoring etcd snapshots is a heavily weighted topic in CKA, and #7 is dedicated to it.

# Check etcd member status (find the cert paths in the apiserver manifest)
ETCDCTL_API=3 etcdctl member list \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key

In a cluster installed with kubeadm, etcd runs stacked together on top of the control plane node by default. To raise availability, you use an external etcd cluster configuration that splits etcd off onto separate machines, which is covered in #5 HA Cluster.

kube-scheduler: deciding Pod placement #

kube-scheduler is the component that looks at a new Pod that doesn’t yet have a node assigned and decides which node it goes on. One important misconception to clear up: the scheduler does not run Pods on nodes directly. All it does is decide. Picking a suitable node and writing the Pod’s nodeName field is where the scheduler’s role ends; actually running the container is the kubelet’s job.

The placement decision happens in two phases.

1. Filtering. Nodes that can’t satisfy the resources and constraints the Pod requires are excluded from the candidates. Nodes without enough resources to meet the requests, nodes that can’t tolerate a taint, or nodes that don’t match a nodeSelector are dropped.
2. Scoring. The remaining candidate nodes are scored and the most suitable one is chosen.

The nodeSelector, affinity, and taints/tolerations that influence this decision are covered in earnest in #13 and #14. When the scheduler stops, new Pods don’t get a node assigned and stay in Pending. Pods that were already running are unaffected.

kube-controller-manager: a collection of reconciliation loops #

kube-controller-manager is the component that bundles several controllers into a single process and runs them. Each controller takes on one responsibility and runs a loop that ceaselessly compares the desired state with the actual state and closes the gap when there is one. This loop is called a reconciliation loop.

The representative controllers are as follows.

For example, if one Pod dies in a Deployment with 3 replicas, the ReplicaSet controller detects the gap — desired 3, actual 2 — creates a new Pod, and brings the count back to 3. This self-healing is the core value of the controller-manager. When the controller-manager stops, all of these loops halt, so dead Pods aren’t recreated and Deployment changes aren’t reflected either.

cloud-controller-manager: separating out cloud integration #

cloud-controller-manager is the cloud-specific logic that connects Kubernetes to a particular cloud provider, extracted into a separate component. It handles verifying that a node exists as an actual cloud instance, creating a cloud load balancer for a LoadBalancer-type Service, and setting up routing on nodes.

On-premises, bare metal, and local clusters for CKA practice often don’t have a cloud-controller-manager. That’s because if you don’t need cloud integration, you don’t need this component either. For the exam, understanding this separation concept is enough.

The control plane that runs as static Pods #

In a cluster installed with kubeadm, the control plane components run as static Pods. Static Pods are not managed by the apiserver or the scheduler. The node’s kubelet reads the manifest files in a specific directory directly and runs and manages those Pods. That directory is /etc/kubernetes/manifests/.

# List of control plane static Pod manifests
ls /etc/kubernetes/manifests/
# etcd.yaml
# kube-apiserver.yaml
# kube-controller-manager.yaml
# kube-scheduler.yaml

There’s a chicken-and-egg problem here. The apiserver has to be up before you can create a Pod, but if the apiserver itself is a Pod, who brings it up? The answer is the kubelet. The kubelet bootstraps the control plane Pods simply by reading the files in this directory — no apiserver required. That’s why the control plane is placed as static Pods.

From an operations perspective, the most important property is this. The kubelet watches this directory, and when a file changes, it automatically restarts the corresponding Pod. So to fix a single flag of the apiserver, you don’t use kubectl edit — you edit this manifest file directly. The moment you save, the kubelet detects the change and brings the Pod back up.

# Example: kubelet auto-restarts after an apiserver flag edit
vim /etc/kubernetes/manifests/kube-apiserver.yaml
# Right after saving, the kubelet detects the change and recreates the apiserver Pod

A common mistake arises here. If you introduce a YAML indentation error or a typo in a flag, the kubelet tries to bring up the Pod with the broken manifest, fails, and the apiserver itself can’t come up. At that point kubectl doesn’t work, so you have to get onto the node and trace the cause with journalctl and the container runtime logs. This scenario is exactly the heart of #24.

Static Pods have one more marker. A static Pod also appears in the apiserver as a “mirror Pod,” with the node name appended to its name. For example, it shows up as something like kube-apiserver-controlplane.

The declarative model and reconciliation #

Tie together how the components cooperate into a single principle, and the core philosophy of Kubernetes comes into view. Kubernetes operates on a declarative model. The user doesn’t dictate a procedure like “create a Pod, then place it on a node.” Instead, the user declares only the desired state: “I want this Deployment with 3 replicas to exist.” That declaration passes through the apiserver and is stored in etcd. Then the scheduler and the various controllers run their own reconciliation loops, continuously converging the actual state toward that declaration. When a node dies and a Pod disappears, a controller recreates it and the scheduler places it on a new node — no user intervention needed. Operating the control plane ultimately means keeping this convergence loop running healthily.

The failure impact of control plane components #

You need to know exactly the symptoms that appear in the cluster when each component dies, so troubleshooting goes faster. The key is distinguishing workloads that were already running from new operations.

The key takeaway from the table is that even when the control plane dies, already-running Pods keep running. That’s because the node’s kubelet and container runtime operate independently of the control plane. A control plane failure means losing the ability to receive and reflect new changes — it does not immediately cut off current traffic. That said, if the apiserver or etcd stay dead for long, self-healing and all operations eventually halt, so a fast recovery is needed.

Commands to check control plane status #

The commands to check the health of the control plane in the CKA exam and on the job are worth getting into your hands. The control plane of a kubeadm cluster runs as static Pods in the kube-system namespace, so you see it at a glance with the following.

# Status of control plane component Pods
k get pods -n kube-system

# Narrow it down to just the control plane components
k get pods -n kube-system -l tier=control-plane

There’s also a command that asks the components’ health directly. componentstatuses is an old API, but you still run into it in the exam.

# scheduler/controller-manager/etcd health (old API, shortened to cs)
k get componentstatuses
k get cs

When tracing why a specific component won’t come up, look at the Pod logs and the manifest together.

# apiserver Pod logs
k logs -n kube-system kube-apiserver-controlplane

# When the apiserver is dead and kubectl doesn't work, go directly on the node
sudo crictl ps -a | grep apiserver
sudo journalctl -u kubelet -f

# Open the problematic static Pod manifest directly
cat /etc/kubernetes/manifests/kube-apiserver.yaml

The last pattern matters. When the apiserver dies, even k logs doesn’t work, so the flow goes: get onto the node, look at the containers with crictl, trace the kubelet logs with journalctl, then fix the manifest directly.

Exam points #

• The apiserver is the single gateway for all communication. Every request passes through authentication, authorization, and admission and is recorded in etcd, and the apiserver is the only component that accesses etcd directly.
• etcd is the single source of truth for cluster state. An etcd backup is a cluster state backup, and etcd restore is a key exam area in #7.
• The scheduler only decides; the kubelet runs. When the scheduler dies, new Pods stay in Pending.
• The controller-manager is a collection of reconciliation loops. When it dies, self-healing stops and dead Pods aren’t recovered.
• The control plane is static Pods in /etc/kubernetes/manifests/. When you edit a file, the kubelet automatically restarts the Pod, so you edit this manifest directly to change a flag.
• Even when a component dies, already-running Pods keep running. The intuition to back-trace which component the problem is from the symptom is the starting point of troubleshooting.
• Get k get pods -n kube-system, k get cs, crictl ps -a, and journalctl -u kubelet into your hands.

Wrap-up #

What this post locked in:

• The control plane is the brain of the cluster. apiserver, etcd, scheduler, controller-manager, and cloud-controller-manager split the roles among themselves.
• The apiserver is the gateway, etcd is the store, the scheduler is the placement decision, the controller-manager is the convergence loop, and the cloud-controller-manager is the cloud integration.
• The control plane of a kubeadm cluster runs as static Pods in /etc/kubernetes/manifests/, managed directly by the kubelet and auto-restarted when a file changes.
• Knowing the per-component failure symptoms lets you quickly pinpoint “where it broke.” Already-running Pods keep running even through a control plane failure.
• Kubernetes operates on a declarative model, and the reconciliation that converges the actual state onto the desired state is the essence of operations.

Next: Cluster Architecture 2 #

Now that we’ve seen the brain that is the control plane, we go down to the side that actually moves the hands and feet. In #3 Cluster Architecture 2, we’ll look directly at what the worker node’s kubelet (the agent that actually runs Pods on the node), kube-proxy (which routes Service traffic), and CRI (the container runtime interface) each do, and at the Pod networking model of how Pods communicate with one another.