Namespace and Labels

This is the last chapter of Part 1. As we’ve followed along through this book, one fact slipped by quietly — the Pod, Deployment, Service, ConfigMap, and Secret we’ve made so far all went into the single default namespace. Labels too, which we’ve kept meeting since the selector in Chapter 4 Deployment and ReplicaSet, have not yet been organized in one place. In this chapter we cover how to organize a cluster into a human-readable shape with those two tools — Namespace and labels. At the end, we briefly preview the next part (Part 2 Workloads and Operations).

By the end of this chapter you’ll have a cluster cleanly split by environment · by team. The same manifest works in its own place without conflict whether you put it in dev / staging / prod, and it becomes the foundation for splitting permissions · resources · and network policy by that compartment too.

The limits of the default namespace #

If you look at all the cluster’s objects once with the -A option, you can see a lot of things up that we didn’t make.

kubectl get all -A

NAMESPACE     NAME                                   READY   STATUS    RESTARTS   AGE
kube-system   pod/coredns-5d78c9869d-2xvlw           1/1     Running   0          3d
kube-system   pod/etcd-minikube                      1/1     Running   0          3d
kube-system   pod/kube-apiserver-minikube            1/1     Running   0          3d
kube-system   pod/kube-controller-manager-minikube   1/1     Running   0          3d
kube-system   pod/kube-proxy-7gbdn                   1/1     Running   0          3d
kube-system   pod/kube-scheduler-minikube            1/1     Running   0          3d
default       pod/web-7f8b6c8f7d-abcde               1/1     Running   0          1h

kube-system is the namespace where the control plane components K8s uses to run itself gather. coredns (cluster DNS), etcd, apiserver, controller-manager, scheduler, kube-proxy — the components we saw as a diagram in Chapter 1 What Kubernetes Is all live here. It’s the isolated space we’ve met since Chapter 2 Local Environment but never named.

kubectl get ns

NAME              STATUS   AGE
default           Active   3d
kube-node-lease   Active   3d
kube-public       Active   3d
kube-system       Active   3d

The four system namespaces created by default — default, kube-system, kube-public, kube-node-lease — are always up. Let’s pin them one at a time.

• default — where objects created without the -n option go. All objects of Chapters 1 ~ 6 gathered here.
• kube-system — where the K8s control plane components live. A regular user doesn’t touch it directly.
• kube-public — for objects readable even without authentication. Rarely used, for exposing cluster info.
• kube-node-lease — separated out from 1.13 to manage node heartbeats efficiently. An operator never touches it directly.

At this point there is no immediate problem because the system runs on its own, but putting everything in one default breaks down once you run dev / staging / prod together in one cluster, teams A · B share one cluster, or you want to keep some apps together and some in another group. The same-named Service must exist separately per environment, permissions must split per environment, and an incident in one environment must not leak into another.

What a Namespace solves #

A Namespace, summed up in one line, is a virtual cluster inside one cluster. It’s a separation layer similar to Linux user accounts or git branches — a tool that splits logical compartments on top of the same physical resources. The things it solves are these four at the core.

• Name-space separation — you can keep same-named objects separately in different namespaces. A Service named web can exist as separate things in both dev and prod and not conflict.
• The unit of RBAC — you can divide permissions per namespace. It’s the base unit of policies like “team A can read and write only within the team-a namespace, and can’t see beyond it.”
• The unit of resource quotas — with ResourceQuota and LimitRange objects you can set ceilings on CPU · memory · object counts per namespace. A tool to stop dev from eating prod’s resources.
• The unit of NetworkPolicy — you can write policies that block or allow traffic between namespaces. By default all namespaces can pass traffic to each other, and to narrow that you have to use NetworkPolicy.

Here it’s worth pinning one thing clearly — a Namespace itself is not a security boundary. It’s merely a logical compartment that splits object names. Real isolation is done separately by the latter three of the four items above (RBAC, resource quota, NetworkPolicy). If you only made a Namespace and wrote neither RBAC nor NetworkPolicy, an authorized user can see and touch objects of any namespace, and Pods communicate with each other. In this chapter we cover only the manifest shape, and the depth of RBAC / NetworkPolicy / ResourceQuota is covered in Chapter 14 RBAC / NetworkPolicy / ResourceQuota.

Cluster scope vs namespace scope #

Objects come in two branches. A namespace-scoped (namespaced) object must belong to some namespace, and a cluster-scoped object exists as one thing cluster-wide without a namespace. Splitting the objects we saw in Chapters 1 ~ 6 gives this.

It’s intuitive that a Node doesn’t belong to a namespace — a node is a physical · virtual machine and a resource of the whole cluster, not belonging to any environment. You can also confirm which side an object belongs to with a command.

kubectl api-resources --namespaced=true

kubectl api-resources --namespaced=false

When you’re operating and get confused about “do I need to add -n to this object?”, running this once gives the answer right away.

Creating a Namespace #

You can make it in one line imperatively.

kubectl create namespace dev

namespace/dev created

If you want to leave the intent in git, write it as a manifest.

apiVersion: v1
kind: Namespace
metadata:
  name: dev
  labels:
    env: dev

kubectl apply -f dev-ns.yaml

apiVersion, like ConfigMap · Secret, is the core group’s v1. A Namespace itself is a cluster-scoped object, so you don’t write a namespace: field inside its metadata (you can’t).

There are two paths to put an object into a specific namespace.

• Write it in the manifest — write the one line metadata.namespace: dev directly in the object’s metadata.
• As a command option — specify with -n, like kubectl apply -f web.yaml -n dev.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: dev
  labels:
    app: web
spec:
  # ... the rest is the same as [Chapter 4](./deployment-and-replicaset/)

When both are present, the manifest’s value takes precedence (the command option applies only when the manifest has no namespace). To reduce confusion, it’s better to use only one of the two. From the standpoint of leaving the intent in git, writing it in the manifest is kinder to the next person — a command option remains only in shell history, and the next time someone sees the same manifest, “where does this go?” becomes a question again.

Looking around per NS #

With the -n option you see only one namespace’s objects.

kubectl get pods -n dev

kubectl get all -n kube-system

The -A option sees all namespaces at once.

kubectl get pods -A

Writing -n each time is tedious. If you change the default namespace of the current context, from then on it goes to that namespace even without the option.

kubectl config set-context --current --namespace=dev

kubectl config view --minify | grep namespace:

This command writes the default namespace into the current context of ~/.kube/config. After that, just typing kubectl get pods shows dev’s Pods.

kubens — switch namespace in one line #

Because the command above is long, the tool almost all operators use is kubens (the partner of the kubectx package). You can hop namespaces in one line.

kubens                  # print current namespace + candidate list
kubens dev              # switch to dev
kubens -                # go back to the previous namespace

If kubectx is for switching clusters (contexts), kubens is for switching namespaces. Both come from one package — install kubectx on Homebrew, apt, or scoop and it comes along. For someone who touches K8s daily it’s a convenience tool close to essential.

How objects in an NS call each other #

It’s time to pin once more the flow we saw in the DNS section of Chapter 5 Service. Pods living in the same namespace could call each other by just the Service name.

http://web/api          # the Service called web, by short name

To call a Service in another namespace, append the namespace after the name.

http://web.prod/api                     # short
http://web.prod.svc.cluster.local/api   # FQDN

Every Service inside a K8s cluster resolves to the FQDN <service>.<namespace>.svc.cluster.local. Even if you shorten it and write just web.prod, the cluster DNS fills the rest on its own. Organizing in one line — DNS is the bridge between namespaces. If a namespace is the compartment that splits object names, DNS plays the role of the passage that lets you call an object across that compartment.

Labels vs annotations #

Here, shifting perspective, we move to the other axis of cluster organization — labels. Labels have actually been with us all along since the selector of Chapter 4 — a Deployment’s spec.selector.matchLabels, a Pod template’s metadata.labels, Chapter 5’s Service’s spec.selector are all mechanisms that bundle and pick out objects by label.

What’s often confused with a label is an annotation. Both are written as key-value under metadata in the same shape, but their uses split clearly.

The one-line difference — a label is a search key, an annotation is a memo pad. What a K8s controller (Deployment, Service, NetworkPolicy, etc.) looks at when choosing which object to handle is the label, and what an external tool (Prometheus, Helm, ArgoCD, the Ingress controller, etc.) or an operator uses to attach metadata to an object is the annotation.

The constraints on keys and values differ slightly too. Because labels are used in selectors, their form is narrow — keys are roughly ASCII alphanumerics plus -, _, ., and values are similarly short strings (both within tens of chars). Annotations have that constraint relaxed, so arbitrary text · JSON can go in. The kubectl.kubernetes.io/last-applied-configuration annotation holding a whole manifest’s JSON is an example of that.

metadata:
  name: web
  labels:
    app: web
    env: prod
    tier: backend
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "8080"
    kubernetes.io/change-cause: "bump nginx 1.27 -> 1.28"

Standard label convention — app.kubernetes.io/* #

You can freely attach your own keys like app=web to labels, but there’s a standard label set the K8s community recommends. They’re the six whose key prefix starts with app.kubernetes.io/.

The reason to use these keys is because operational tools · dashboards recognize them as a standard. Tools like Lens, k9s, Datadog, and Helm look at these keys and group objects to show them. Compatibility is clearly better than using only your own keys. It’s fine to use your own keys (e.g., env, team) together, and writing the standard label set alongside your own labels is the typical shape.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: prod
  labels:
    app.kubernetes.io/name: nginx
    app.kubernetes.io/instance: web-prod
    app.kubernetes.io/version: "1.27"
    app.kubernetes.io/component: frontend
    app.kubernetes.io/part-of: shop-platform
    app.kubernetes.io/managed-by: kubectl
    env: prod
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx
      app.kubernetes.io/instance: web-prod
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx
        app.kubernetes.io/instance: web-prod
        app.kubernetes.io/version: "1.27"
        app.kubernetes.io/component: frontend
        env: prod
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          ports:
            - containerPort: 80

It’s safer to put only the unchanging ones among the standard labels into the selector’s matchLabels and the Pod template’s labels. Putting a key like version into the selector means you have to change the selector too when you bump the version, which easily catches you in the selector immutability pitfall we saw in Chapter 4.

Picking out by label — kubectl -l #

If labels are attached, you can pick out objects with kubectl’s -l option. The selector syntax has a few forms, from a simple equals to set expressions.

kubectl get pods -l app=web
kubectl get pods -l env=prod,tier=backend         # AND

Joining with commas is AND. OR is written with the set expressions below.

kubectl get pods -l 'env in (dev,staging)'
kubectl get pods -l 'env notin (prod)'
kubectl get pods -l 'tier'                         # ones that have the tier label
kubectl get pods -l '!debug'                       # ones without the debug label

You can also use it across several object kinds at once.

kubectl get deploy,svc,cm -l app.kubernetes.io/instance=web-prod

kubectl delete pods -l env=dev

The last command is powerful — it deletes all matching Pods at once. In an operational cluster, incidents of putting in a wrong label and deleting more objects than intended happen from time to time. Before a bulk delete, it’s safer to first run a get with the same selector to confirm the targets are the ones you intended. The finished version of the diagnostic tree is organized in Chapter 27 kubectl debugging patterns.

The reason the -l selector syntax matters is — the same syntax is used for nearly every object matching in K8s, including a Service’s spec.selector, a Deployment’s spec.selector.matchLabels, a NetworkPolicy’s podSelector, and a ResourceQuota’s scopeSelector. Learn labels once and the selectors of the objects layered on top read naturally.

A hypothetical operational frame — putting it all together #

Fitting the tools so far into one manifest bundle reveals the basic shape of an operational cluster. Three namespaces dev / staging / prod, inside them a same-named Deployment · Service · ConfigMap · Secret, with environment and version attached as labels.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: prod
  labels:
    app.kubernetes.io/name: nginx
    app.kubernetes.io/instance: web-prod
    app.kubernetes.io/version: "1.27"
    app.kubernetes.io/component: frontend
    app.kubernetes.io/part-of: shop-platform
    env: prod
spec:
  replicas: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx
      app.kubernetes.io/instance: web-prod
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx
        app.kubernetes.io/instance: web-prod
        app.kubernetes.io/component: frontend
        env: prod
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          ports:
            - containerPort: 80
          envFrom:
            - configMapRef:
                name: web-config
            - secretRef:
                name: db-secret
---
apiVersion: v1
kind: Service
metadata:
  name: web
  namespace: prod
  labels:
    app.kubernetes.io/name: nginx
    app.kubernetes.io/instance: web-prod
    env: prod
spec:
  selector:
    app.kubernetes.io/name: nginx
    app.kubernetes.io/instance: web-prod
  ports:
    - port: 80
      targetPort: 80

dev and staging are nearly the same manifest — only the metadata.namespace, instance, version, env labels differ per environment, and the rest stays the same. This shape is the basis of an operational cluster where objects split cleanly per environment within one cluster.

In operations, when you have to apply the same manifest with slight per-environment changes, swapping by hand each time invites frequent mistakes. So the tools that come out are Helm (template + values separation) and Kustomize (base + overlay). They’re outside this chapter’s scope, and covered together with real deployment on EKS in Chapter 22 app deployment skeleton and Chapter 20 GitOps.

Cleanup and teardown #

Clean up the dev namespace we made in this chapter.

kubectl delete ns dev

namespace "dev" deleted

The reason this one line is scary is that all objects inside that namespace disappear with it. Deployment, Service, Pod, ConfigMap, Secret, PVC are all deleted asynchronously. In an operational cluster, deleting a namespace should be regarded as nearly the last card — press it once and the data inside can fly away too, and depending on the reclaimPolicy setting of the persistent volume we’ll cover in Chapter 9 PV / PVC / StorageClass, the disk itself can disappear along with it.

kubectl config set-context --current --namespace=default

If you’ve installed kubens, switching back is one line: kubens default.

Part 1 retrospective — what this gave you over seven chapters #

Since this is the last chapter of Part 1, let’s organize once.

• Chapter 1 What Kubernetes Is — why K8s. The limits of a container on one host, what an orchestrator solves, the picture of the control plane and workers.
• Chapter 2 Local Environment — a local cluster. The uses and differences of the three (minikube / kind / Docker Desktop), the kubectl context.
• Chapter 3 kubectl and Your First Pod — your first Pod. The spine of a manifest (apiVersion / kind / metadata / spec), kubectl apply / get / describe / logs / exec.
• Chapter 4 Deployment and ReplicaSet — Deployment and ReplicaSet. Declarative deployment, rolling update, kubectl rollout, selector immutability.
• Chapter 5 Service — Service. The three stages ClusterIP / NodePort / LoadBalancer, cluster-internal DNS.
• Chapter 6 ConfigMap and Secret — ConfigMap and Secret. 12-factor’s “store config in the environment,” the three injection methods env / envFrom / volume.
• Chapter 7 Namespace and labels (this chapter) — how to organize a cluster. The namespace as the unit of RBAC · quota · NetworkPolicy, the standard label convention, the kubectl -l selector.

At this point you can read and write what a single K8s manifest means even on first sight. Open a company cluster’s manifest directory and the object kinds and field names inside it won’t feel unfamiliar. The deeper topics layered on top are the storyline of Part 2 (Workloads and Operations).

Exercises #

1. Create a new namespace with kubectl create namespace dev, and move the web Deployment + ConfigMap + Secret manifests of Chapter 6 wholesale to metadata.namespace: dev and kubectl apply. Write down side by side the kubectl get all -n dev and kubectl get all -n default outputs, and confirm that same-named objects can exist in two namespaces at once. Then, inside dev’s web Pod, compare what curl http://web.default/ and curl http://web/ give.
2. Attach all six app.kubernetes.io/* keys from §“Standard label convention” to the web Deployment of Chapter 4. Note in a paragraph, connected to §“the selector immutability pitfall,” why you put only name and instance into the selector’s matchLabels and keep version as a label only. Then confirm which objects kubectl get pods,svc,cm -l app.kubernetes.io/instance=<...> bundles to show at once.
3. Bring up two Pods with the labels team=a and team=b in the same namespace, and record in turn the outputs of the three commands kubectl get pods -l 'team in (a)', kubectl get pods -l 'team notin (a)', kubectl get pods -l '!team'. Infer in one line how the same selector syntax applies to the podSelector of Chapter 14 RBAC / NetworkPolicy / ResourceQuota.

In one line: a Namespace is a virtual cluster inside one cluster, splitting the name space and becoming the unit of RBAC · ResourceQuota · NetworkPolicy. A label is a search key by which a selector bundles and picks out objects, and an annotation is a memo pad for tools · operators. Keep the manifest body as one set regardless of environment, and separate the parts that differ per environment with namespaces and labels · ConfigMap · Secret.

Next chapter #

Part 1 is done. In Chapter 8 StatefulSet / DaemonSet / Job / CronJob, the first chapter of Part 2 Workloads and Operations, we cover the controllers other than Deployment. We organize the shape and selection criteria of workloads where each instance must have its own name and own disk, like a database (StatefulSet), workloads that must run one per node, like a log collector · node monitor (DaemonSet), and one-off batch jobs (Job · CronJob).

The whole storyline of Part 2 is as follows.

Finishing Part 2 brings you to the level where you can stably operate a variety of workloads on a cluster. After that, Part 3 (Depth) covers topics that expand into the operator’s view, like CNI · Admission Controller · CRD · observability · GitOps.