K8s Intermediate #3: Ingress and Ingress Controller — The External Entry Point

The third post in the K8s Intermediate series. While #2 covered the model for how data inside the cluster survives, this post shifts the viewpoint outside the cluster — to the model of how external traffic enters the Services inside. The LoadBalancer of Basics #5 is the standard for external entry, but if a single cluster has dozens of Services to expose externally, spinning up that many LoadBalancers becomes a burden in both cost and management. This post follows the two layers — the Ingress object that gathers that burden in one place, and the Ingress Controller that resolves those manifests into actual routing.

This series is K8s Intermediate, 7 posts.

• #1 StatefulSet / DaemonSet / Job / CronJob — Controllers beyond Deployment
• #2 PV / PVC / StorageClass — the persistent data model
• #3 Ingress and Ingress Controller — the external entry point ← this post
• #4 resources.requests / limits — Pod resource requests and limits
• #5 Health checks — liveness / readiness / startup probes
• #6 Autoscaling — HPA / VPA / Cluster Autoscaler
• #7 RBAC / NetworkPolicy / ResourceQuota — security and resource policy

What a single LoadBalancer can’t solve — the starting point #

The LoadBalancer Service shape from Basics #5 reduces to this: with type: LoadBalancer, the cloud provider automatically creates an external LB, and traffic flows through that LB’s external IP into the backend Pods of one Service. Simple and powerful enough.

The problem starts once the number of Services grows past one. A common shape of an operational cluster looks like this:

• web Service — user-facing web pages
• api Service — REST API backend
• admin Service — administrative pages
• static Service — dedicated static asset delivery

If you slap type: LoadBalancer on all four to expose them externally, you get four cloud LBs. Whether AWS NLB/ALB, GCP LB, or Azure LB, every LB has its own hourly cost, and that cost piles up in proportion to the number of Services. With dozens of Services, the LB cost becomes non-trivial.

Cost isn’t the only burden. The operational overhead grows in lockstep.

• Manage as many domains as LBs — each external-facing domain pins to its own LB, so DNS records have to be managed at that count.
• TLS certificates per LB — certificate issuance and renewal must be managed per LB.
• Host/path-based routing isn’t possible — even if you want example.com/api to go to the api Service and example.com/static to go to the static Service, a single LoadBalancer Service points at exactly one Service’s backend. To resolve this routing, someone has to add another layer.

The abstraction that gathers all three in one place is Ingress. One Ingress Controller stands behind one cloud LB, and that Controller looks at domain and path to split traffic across many Services in the cluster. TLS termination is also handled in one place. From the outside it looks like a single LB, but inside it branches according to the routing rules in the Ingress manifests.

Ingress’s two layers — object and controller #

A frequent trap with K8s lies right here. Writing an Ingress manifest alone doesn’t move any traffic. Ingress requires both layers to be present.

Objects like Deployment and Service are handled directly by the K8s control plane via its own controllers, but Ingress is different. The Ingress controller is not built into K8s — it’s an external component that must be installed separately into the cluster. If you apply an Ingress manifest to a cluster without a controller installed, the object is created but no traffic flows anywhere — the apply succeeds, but external requests never get a response.

The mental picture is best held like this:

[external client]
       │
       ▼
[ cloud LB ]  ←── one LoadBalancer Service exposed by the Ingress Controller
       │
       ▼
[ Ingress Controller Pod ]  ←── nginx / Traefik / ALB, etc.
       │
       │  reads the routing rules in Ingress objects and branches
       ├──────────┬──────────┐
       ▼          ▼          ▼
   [Service A] [Service B] [Service C]
       │          │          │
     Pods       Pods       Pods

The key — there’s just one external cloud LB, and the Ingress Controller behind it merges the rules of all Ingress objects to perform actual routing. Even with 100 Ingress objects in the cluster, the cloud LB is usually still one.

Kinds of Ingress Controllers #

K8s standardizes the manifest format for the Ingress object, but how that’s interpreted into routing differs by controller implementation. The controllers commonly seen in operation:

Splitting them roughly into two camps makes it easier to grasp.

• In-cluster Pods route — controllers like ingress-nginx, Traefik, and HAProxy run their own nginx/Traefik Pods in the cluster, and those Pods receive and process traffic. The cloud LB stands in front of those Pods as a simple L4 distributor.
• Cloud resources route — the AWS ALB Controller, GKE Ingress, and AKS AGIC have the cloud’s managed LB itself (ALB/CLB/Application Gateway) do host/path routing. Inside the cluster there’s just a small controller Pod translating Ingress objects into cloud LB rules.

Even with the same Ingress manifest, the annotation keys and behavior differ depending on which controller processes it. So when adding controller-specific annotations to a manifest, the first thing to check is which controller is in use.

Installing an Ingress Controller #

In an operational cluster, you usually install one kind of controller once and route all Ingresses through it. The shape of installation per environment, briefly:

minikube #

A single addons command installs ingress-nginx.

minikube addons enable ingress

kubectl get pods -n ingress-nginx
NAME                                       READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-6d4b7f6c8-abc12   1/1     Running   0          30s

kind #

kind applies ingress-nginx via a separate manifest.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml

You also need a cluster-creation config that maps host ports 80/443. Follow the official ingress-nginx guide.

EKS / GKE / AKS #

On managed clusters, installing the controller via Helm is the standard pattern.

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace

If you use the AWS Load Balancer Controller on EKS, additional IAM and service account setup follows.

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  --namespace kube-system \
  --set clusterName=my-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

GKE has the Google-managed GKE Ingress enabled by default, so the experience of just applying an Ingress manifest and getting a GCLB created automatically is possible without separate installation. AKS has a similar managed option.

The installed controller usually sits in its own namespace (ingress-nginx, kube-system, etc.) as a bundle of Pods, Services, Deployments, and ConfigMaps. Having one kind of controller installed on a cluster is usually enough.

First Ingress manifest — simple domain routing #

Starting from the simplest shape — an Ingress that sends all traffic for example.com to the web Service on port 80.

Assume the backend Service is up.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  type: ClusterIP
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 80

Note that the Service type is ClusterIP. The Service behind an Ingress only needs to be cluster-internal. External exposure is handled by the Ingress Controller’s own LoadBalancer Service. There’s no reason to put type: LoadBalancer on this Service.

Now the Ingress manifest.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
spec:
  ingressClassName: nginx
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 80

Walking the fields one by one:

• apiVersion: networking.k8s.io/v1 — Ingress’s stable version. Reached stable in 1.19; the older extensions/v1beta1 and networking.k8s.io/v1beta1 are no longer used.
• spec.ingressClassName: nginx — points to which Ingress Controller should handle this object. If only one controller is in the cluster and it’s the default IngressClass you can omit this, but specifying it is safer. More in the IngressClass section below.
• spec.rules[].host: example.com — expresses which host the traffic belongs to. Matched against the HTTP Host header or HTTPS SNI.
• spec.rules[].http.paths[].path: / — the path to match. / means every path on this host.
• spec.rules[].http.paths[].pathType: Prefix — how the path is matched. The most common value.
• spec.rules[].http.paths[].backend.service — Service name and port to send matched traffic to.

After applying this manifest, the state looks like:

kubectl apply -f ingress-web.yaml
kubectl get ingress

NAME   CLASS   HOSTS         ADDRESS          PORTS   AGE
web    nginx   example.com   34.120.10.20     80      30s

The ADDRESS column fills with the IP or hostname of the external entry point. This IP is the same as the EXTERNAL-IP of the LoadBalancer Service exposing the Ingress Controller. Even with 100 Ingress objects in the cluster, this ADDRESS is usually the same single IP.

DNS mapping #

For example.com to actually point at the ADDRESS above in production, an A record (or CNAME) at that IP must be set in external DNS. This isn’t something K8s does automatically. With a tool like ExternalDNS, there’s an operational pattern where the host fields of Ingress objects are watched and DNS records are created automatically in cloud DNS.

Path-based routing #

When you want to send different paths of the same host to different Services. example.com/api goes to the backend-api Service, and example.com/static goes to the cdn Service.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
spec:
  ingressClassName: nginx
  rules:
    - host: example.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: backend-api
                port:
                  number: 8080
          - path: /static
            pathType: Prefix
            backend:
              service:
                name: cdn
                port:
                  number: 80
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 80

The behavior of this manifest:

• example.com/api/users → backend-api:8080
• example.com/static/logo.png → cdn:80
• example.com/ or example.com/about → web:80

Paths aren’t evaluated top-to-bottom — the longer match wins. Because /api matches longer than /, a request to /api/users goes to backend-api. This priority rule can subtly differ across controller implementations, so in operation it’s safer to write manifests so paths don’t overlap as much as possible.

The three pathType values #

pathType defines how path matching works. Three values exist.

The most common value used in operation is Prefix. Exact is for exposing a single endpoint, and ImplementationSpecific is for using controller-specific features (e.g., ingress-nginx’s regex). For manifest portability, sticking with Prefix is the safe choice.

The slash-segmented matching of Prefix is a subtle point worth flagging — writing /api matches /api/users but not /api2. K8s interprets Prefix as a directory prefix. This distinction occasionally matters for security — it prevents an unintended path from flowing to the same backend.

host + path combination #

Handling different domains together in one Ingress. Splitting api.example.com and app.example.com is a common configuration.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi
spec:
  ingressClassName: nginx
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: backend-api
                port:
                  number: 8080
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 80

Just put two entries with different hosts in the rules array. One Ingress can hold multiple hosts together, or you can create a separate Ingress object per host. In practice, splitting Ingresses by domain group is a common pattern — one per namespace, one per team, aligning the object boundary with the ownership boundary.

Hosts also support wildcards. Writing *.example.com matches one-level subdomains like foo.example.com and bar.example.com. However, *.example.com does not match two-level subdomains like foo.bar.example.com — it follows the same rule as DNS wildcards.

TLS termination — Ingress + Secret #

External entry in operation is almost always HTTPS. By pairing the tls field of Ingress with a K8s Secret, certificate termination can be handled in one place.

First, a Secret holding the certificate and key is required.

kubectl create secret tls example-com-tls \
  --cert=fullchain.pem \
  --key=privkey.pem

The Secret type is kubernetes.io/tls, and inside it has two keys: tls.crt (certificate chain) and tls.key (private key). The same result is possible via manifest, but that requires putting the base64-encoded private key directly into the YAML, so the imperative command above is usually cleaner.

Connect this Secret to the Ingress.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - example.com
      secretName: example-com-tls
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 80

Traffic for the domains listed in spec.tls[].hosts is TLS-terminated when received by the Ingress Controller, and forwarded to the inner Service as plain HTTP. Thanks to this shape, the Service- and Pod-side manifests don’t need to think about HTTPS, and certificate renewal is just swapping the Secret.

cert-manager — automatic issuance and renewal #

The standard tool for automatically issuing and renewing certificates from free CAs like Let’s Encrypt is cert-manager. After installing cert-manager into the cluster and creating a ClusterIssuer object once, certificate issuance proceeds automatically just by adding a single annotation to the Ingress manifest.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - example.com
      secretName: example-com-tls
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 80

Seeing the cert-manager.io/cluster-issuer annotation, cert-manager automatically progresses through this cycle:

1. Verify domain ownership via an ACME challenge (HTTP-01 or DNS-01).
2. Issue the certificate from Let’s Encrypt.
3. Place the certificate and key into the Secret named in secretName.
4. Auto-renew about 30 days before expiry.

cert-manager’s installation and ClusterIssuer setup involve quite a few steps, so this post just sketches the shape and defers the details to the K8s practice track. cert-manager is effectively the operational standard, so knowing its name and where it fits in the stack is enough for now.

IngressClass — when there are two or more controllers #

Most clusters keep one kind of controller, but sometimes two kinds need to coexist in the same cluster. For example, public traffic via ingress-nginx, internal admin traffic via the AWS ALB Controller. The object that expresses which controller should process which Ingress object is IngressClass.

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: nginx
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true"
spec:
  controller: k8s.io/ingress-nginx

Two key fields:

• spec.controller — an identifier pointing to which controller implementation handles this IngressClass. ingress-nginx is k8s.io/ingress-nginx, the AWS ALB Controller is ingress.k8s.aws/alb, and so on per implementation.
• ingressclass.kubernetes.io/is-default-class: "true" annotation — marks this IngressClass as the cluster default. If an Ingress manifest omits ingressClassName, the controller of the default IngressClass handles it.

In the Ingress manifest, spec.ingressClassName declares which class it belongs to.

spec:
  ingressClassName: alb   # processed by AWS ALB Controller
  rules:
    - host: admin.example.com
      ...

If another Ingress in the same cluster says ingressClassName: nginx, that one is processed by ingress-nginx. The two controllers don’t fight over the same object.

kubectl get ingressclass

NAME            CONTROLLER             PARAMETERS   AGE
nginx (default) k8s.io/ingress-nginx   <none>       30d
alb             ingress.k8s.aws/alb    <none>       10d

The class with the (default) mark is the default. Keep only one default IngressClass per cluster as the standard. If two are marked default, it becomes ambiguous which one will process a new Ingress object.

Gateway API — the successor standard to Ingress #

One last note. The Ingress object is enough for simple host/path routing, but for more expressive routing (header-based, query-parameter-based, weighted traffic distribution, etc.), it ends up depending on annotations, which then differ per controller. The successor standard SIG-Network created to resolve this limit is Gateway API.

Gateway API splits objects into three:

Many controllers — ingress-nginx, Traefik, Cilium, Istio — support Gateway API, and an operator starting a new cluster is at the stage of considering Gateway API. That said, Ingress is still the dominant standard, and what an operator encounters in existing clusters is mostly Ingress. The deeper model of Gateway API is deferred to the K8s advanced track; this post just notes its name and place.

Operational pattern — one LoadBalancer + many Ingresses #

Locking in this post’s mental picture in one line: the external entry of an operational cluster usually consolidates into one cloud LoadBalancer. That LoadBalancer is attached to the single Service (type: LoadBalancer) the Ingress Controller spins up for its own exposure, and all external traffic passes through that one LB.

[ cloud LoadBalancer ]   ←── one. cost in one place
            │
            ▼
[ Ingress Controller (nginx Pod set) ]   ←── ingress-nginx namespace
            │
            │  routing rules from Ingress objects
            ├──────────┬──────────┬──────────┐
            ▼          ▼          ▼          ▼
       [web]      [api]       [admin]    [static]   ←── each a ClusterIP Service

The effect of this structure, in one line each:

• Cloud LB cost in one place — even as Service count grows, the LB stays at one.
• TLS termination in one place — certificate and cert-manager configuration also gathers in one place.
• Domain/path routing expressed in manifests — exposing a new Service is adding one Ingress object.
• DNS records simple — pinning *.example.com to the LB’s IP/name once routes every host through that LB.

The cloud LB is usually one regardless of Service count, but when traffic isolation is needed (public vs internal, etc.), spinning up two controller bundles to have two LBs is also a common pattern. The shape in the IngressClass section above is an example of that operational pattern.

Availability of the Ingress Controller itself #

One risk of this structure — the Ingress Controller can become a single point of failure. All external traffic passes through that Controller, so if the Controller Pods fail simultaneously, all external traffic is severed. In operation, the standard is to have:

• Multiple Controller Pods — set the Deployment’s replicas to 2 or more. Distribute via PodDisruptionBudget and anti-affinity so at least one Pod survives node failure.
• Resource headroom — set the Controller Pod’s CPU and memory limits with enough headroom. The topic of the next post #4.
• Monitoring — alarm on the Prometheus metrics the Controller exposes (request count, 5xx rate, response time).

Summary #

The flow held in this post:

• Limits of one LoadBalancer per Service — as exposed Services grow, cloud LB cost and management burden grows in proportion. Host/path routing and TLS termination also need to be gathered in one place.
• Ingress’s two layers — Ingress object (manifest, intent) and Ingress Controller (runtime). Without the controller, writing only the object does nothing.
• Kinds of controllers — ingress-nginx (most common), Traefik, HAProxy, AWS ALB Controller, GKE Ingress, Cilium. Split into branches where in-cluster Pods route and where cloud resources route.
• Host routing — spec.rules[].host branches by host. Wildcards (*.example.com) are also supported.
• Path routing and pathType — Prefix (most common, slash-segmented prefix), Exact (exact match), ImplementationSpecific (per controller).
• TLS termination — connect host and Secret in spec.tls. The Secret type is kubernetes.io/tls. Using cert-manager for Let’s Encrypt automatic issuance and renewal is the standard pattern.
• IngressClass — when two or more controllers are in the same cluster, expresses which controller an Ingress object belongs to. Default mark on only one.
• Gateway API — the successor standard to Ingress. More expressive. Ingress is still the standard, but worth considering on a new cluster.
• Operational pattern — one cloud LB → one Ingress Controller bundle → routing per Ingress object → ClusterIP Services. LB cost, TLS, and routing gather in one place. The next operational concern is the controller’s own availability and resource headroom.

Once this model is in hand, whenever you encounter an Ingress object in a cluster’s manifest directory, you can read at a glance which controller is handling it and how it connects to the external LB.

Next — resources.requests / limits #

One thing briefly noted at the end of this post is the starting point of the next post — resource headroom for the Ingress Controller Pod. Since the Controller handles all external traffic, if its Pod runs short on CPU or memory, the traffic itself is choked. Yet how a Pod expresses the resources it needs in K8s, and how that expression affects scheduler decisions and node OOM behavior, hasn’t been covered yet.

#4 resources.requests / limits — Pod resource requests and limits walks through the model of the two fields resources.requests and resources.limits in a Pod manifest, the behavior difference between CPU and memory (CPU throttles, memory OOMKills), QoS classes (Guaranteed / Burstable / BestEffort), and the operational pattern of enforcing namespace defaults via LimitRange — all in one cycle.