K8s Practice #2: App Deployment Skeleton — Deployment / Service / Ingress / Helm
The second post in the K8s Practice series. We brought up an EKS cluster in #1, but the workload still hasn’t been placed on top of it. This post is one cycle of putting the imaginary service myshop-api on that empty cluster. We write the bundle from Deployment to Ingress as manifests, auto-create an ALB with AWS Load Balancer Controller to establish the external entry point, and abstract the whole bundle as a Helm chart so the same chart deploys to dev and prod with different values.
This series is K8s Practice, 6 posts.
- #1 EKS Cluster Setup — Terraform / eksctl / IRSA / Addons
- #2 App deployment skeleton — Deployment / Service / Ingress / Helm ← this post
- #3 DB integration — RDS / Secrets Manager / External Secrets / connection pool
- #4 CI/CD pipeline — GitHub Actions / ECR / ArgoCD
- #5 Monitoring/alarming — Prometheus / CloudWatch / Alertmanager
- #6 Operations checklist — upgrades / backup,recovery / cost / security
Looking at myshop-api’s manifests as one bundle #
To put myshop-api on EKS, the following objects are needed.
| Object | Role |
|---|---|
Namespace | Isolation unit binding workloads |
ServiceAccount | Pod’s ID + IRSA attachment point (DB connection in #3) |
ConfigMap | Environment config (log level, feature flags, etc.) |
Secret | DB password, etc. (actual values filled via External Secrets in #3) |
Deployment | Actual workload — Pod replicas |
Service | Fixed virtual IP for inside-cluster access |
Ingress | External entry point (resolved as ALB) |
HorizontalPodAutoscaler | CPU-based Pod count auto-adjustment |
PodDisruptionBudget | Minimum availability guarantee during node upgrades |
These 9 objects form the standard bundle for one operational workload. The sections below walk through each one.
Namespace and ServiceAccount #
apiVersion: v1
kind: Namespace
metadata:
name: myshop
labels:
name: myshop
team: platform
env: prod
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: myshop-api
namespace: myshop
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/myshop-prod-apiThe Namespace’s labels are the labels covered in Basics #7. Consistently attaching labels like team / env / cost-center lets #5’s Prometheus reuse them as is when grouping workloads.
The ServiceAccount’s IRSA annotation is the annotation covered in Advanced #2. The myshop-api Pod automatically receives the permissions of the IAM Role written here while running with this ServiceAccount. The Role’s definition itself is managed in Terraform.
module "myshop_api_irsa" {
source = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
version = "~> 5.0"
role_name = "myshop-prod-api"
oidc_providers = {
main = {
provider_arn = var.oidc_provider_arn
namespace_service_accounts = ["myshop:myshop-api"]
}
}
role_policy_arns = {
secrets = aws_iam_policy.myshop_secrets_read.arn
}
}The ARN of the Role created by this Terraform module is written in the ServiceAccount annotation. As a result, the AWS SDK inside the Pod can call Secrets Manager, S3, and CloudWatch using this Role’s permissions.
ConfigMap and Secret #
apiVersion: v1
kind: ConfigMap
metadata:
name: myshop-api
namespace: myshop
data:
LOG_LEVEL: "info"
FEATURE_NEW_CHECKOUT: "true"
CACHE_TTL_SECONDS: "300"
AWS_REGION: "ap-northeast-2"
---
apiVersion: v1
kind: Secret
metadata:
name: myshop-api
namespace: myshop
type: Opaque
stringData:
DATABASE_URL: "postgresql://will-be-replaced-by-external-secrets"The Secret’s actual value isn’t placed in the manifest. #3 covers the flow where External Secrets Operator auto-syncs AWS Secrets Manager secrets into K8s Secrets. This post locks in only the skeleton, and the secret management model covered in Advanced #6 GitOps comes back in #4.
Deployment — Pod replicas #
apiVersion: apps/v1
kind: Deployment
metadata:
name: myshop-api
namespace: myshop
labels:
app.kubernetes.io/name: myshop-api
app.kubernetes.io/component: backend
spec:
replicas: 3
selector:
matchLabels:
app.kubernetes.io/name: myshop-api
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
template:
metadata:
labels:
app.kubernetes.io/name: myshop-api
app.kubernetes.io/component: backend
spec:
serviceAccountName: myshop-api
containers:
- name: api
image: 123456789012.dkr.ecr.ap-northeast-2.amazonaws.com/myshop-api:1.4.2
ports:
- containerPort: 8000
name: http
envFrom:
- configMapRef:
name: myshop-api
- secretRef:
name: myshop-api
resources:
requests:
cpu: "200m"
memory: "256Mi"
limits:
cpu: "1"
memory: "512Mi"
readinessProbe:
httpGet:
path: /health/ready
port: http
initialDelaySeconds: 5
periodSeconds: 5
livenessProbe:
httpGet:
path: /health/live
port: http
initialDelaySeconds: 30
periodSeconds: 10
startupProbe:
httpGet:
path: /health/live
port: http
failureThreshold: 30
periodSeconds: 10
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app.kubernetes.io/name: myshop-apiStandard elements of operational manifests gathered:
strategy.rollingUpdate— withmaxUnavailable: 0, the new version must come up sufficiently before the old is brought down. The default protection of zero downtime.envFrom— inject ConfigMap and Secret as environment variables at once. No need to write each key individually.- Three probes — the three covered in Intermediate #5. liveness conservatively (start after 30s), readiness immediately (5s), startup as a grace period for long initialization.
resources— requests / limits covered in Intermediate #4. This workload is Burstable QoS (requests < limits).topologySpreadConstraints— distribute Pods across multiple AZs. Even if one AZ dies, Pods in the other AZ survive.- app.kubernetes.io/ standard labels — the standard from Basics #7. Selector, monitoring, logging are all grouped by these labels.
Service — fixed entry point inside cluster #
apiVersion: v1
kind: Service
metadata:
name: myshop-api
namespace: myshop
labels:
app.kubernetes.io/name: myshop-api
spec:
type: ClusterIP
selector:
app.kubernetes.io/name: myshop-api
ports:
- name: http
port: 80
targetPort: http
protocol: TCPtype: ClusterIP is the key. The Service itself is a virtual IP only accessible inside the cluster, and external exposure is the responsibility of the next Ingress to be made. The same model covered in Basics #5.
AWS Load Balancer Controller — Ingress to ALB mapping #
EKS uses the standard K8s Ingress object, but to actually resolve that object into an ALB (AWS Application Load Balancer), a component called AWS Load Balancer Controller must be installed in the cluster.
helm repo add eks https://aws.github.io/eks-charts
helm repo update
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
-n kube-system \
--set clusterName=myshop-prod \
--set serviceAccount.create=false \
--set serviceAccount.name=aws-load-balancer-controllerThe ServiceAccount is pre-created with IRSA in Terraform. The controller watches Ingress objects in the cluster, and whenever an Ingress is created, it provisions a matching ALB via the AWS API.
Ingress manifest #
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myshop-api
namespace: myshop
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
alb.ingress.kubernetes.io/ssl-redirect: '443'
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:ap-northeast-2:123456789012:certificate/abc-123
alb.ingress.kubernetes.io/healthcheck-path: /health/ready
external-dns.alpha.kubernetes.io/hostname: api.myshop.example.com
spec:
rules:
- host: api.myshop.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myshop-api
port:
number: 80Each annotation has meaning.
scheme: internet-facing— internet-exposed ALB. Internal-only isinternal.target-type: ip— registers Pod IPs directly to the ALB target group. ALB watches Pod IP changes.listen-ports / ssl-redirect— accept HTTPS only and auto-redirect HTTP.certificate-arn— ACM certificate. Pre-issue certificate for the domain bound with Route53 via ACM.healthcheck-path— health check path ALB sends to each Pod.external-dns.alpha.kubernetes.io/hostname— the external-dns component auto-registers this ALB’s DNS as an A record in Route53.
About 1–2 minutes after applying this manifest, a new ALB appears in the AWS console and the api.myshop.example.com domain is pointed at it.
HPA — Pod auto-adjustment per traffic #
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: myshop-api
namespace: myshop
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myshop-api
minReplicas: 3
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
behavior:
scaleUp:
stabilizationWindowSeconds: 30
scaleDown:
stabilizationWindowSeconds: 300When average CPU utilization exceeds 60%, scale up Pod count; when it drops, scale down. The same model covered in Intermediate #6. The behavior stabilization window is the key — scale-up at 30s for fast response, scale-down at 5min for conservatism. This setting provides fast response to traffic spikes while preventing Pods from oscillating on transient dips.
PodDisruptionBudget — the floor of availability #
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: myshop-api
namespace: myshop
spec:
minAvailable: 2
selector:
matchLabels:
app.kubernetes.io/name: myshop-apiWhat this single manifest prevents: during voluntary disruptions like node upgrades, node drain, or Cluster Autoscaler’s node reclaim, it guarantees that myshop-api never drops below 2 running Pods simultaneously. This blocks the classic K8s operational incident — “all Pods came down at once during an EKS upgrade and caused downtime.”
Bundling as a Helm chart #
The 7–9 manifests written so far work on their own, but deploying the same workload to dev / prod with different values means duplicating manifests per environment. Helm enters at this point.
Chart structure #
charts/myshop-api/
├── Chart.yaml
├── values.yaml # defaults
├── values-dev.yaml # dev override
├── values-prod.yaml # prod override
└── templates/
├── _helpers.tpl
├── namespace.yaml
├── serviceaccount.yaml
├── configmap.yaml
├── secret.yaml
├── deployment.yaml
├── service.yaml
├── ingress.yaml
├── hpa.yaml
└── pdb.yamlChart.yaml #
apiVersion: v2
name: myshop-api
description: myshop API service
type: application
version: 0.1.0
appVersion: "1.4.2"
maintainers:
- name: platform-team
email: platform@myshop.example.comvalues.yaml — defaults #
image:
repository: 123456789012.dkr.ecr.ap-northeast-2.amazonaws.com/myshop-api
tag: "1.4.2"
pullPolicy: IfNotPresent
replicaCount: 3
resources:
requests:
cpu: 200m
memory: 256Mi
limits:
cpu: 1
memory: 512Mi
serviceAccount:
create: true
name: myshop-api
irsaRoleArn: ""
config:
LOG_LEVEL: info
FEATURE_NEW_CHECKOUT: "true"
CACHE_TTL_SECONDS: "300"
ingress:
enabled: true
host: api.myshop.example.com
certificateArn: ""
autoscaling:
enabled: true
minReplicas: 3
maxReplicas: 20
targetCPUUtilizationPercentage: 60
pdb:
enabled: true
minAvailable: 2values-prod.yaml — environment override #
replicaCount: 5
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 2
memory: 1Gi
serviceAccount:
irsaRoleArn: arn:aws:iam::123456789012:role/myshop-prod-api
config:
LOG_LEVEL: warn
ingress:
host: api.myshop.example.com
certificateArn: arn:aws:acm:ap-northeast-2:123456789012:certificate/abc-123
autoscaling:
minReplicas: 5
maxReplicas: 50Template — Deployment example #
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "myshop-api.fullname" . }}
labels:
{{- include "myshop-api.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
{{- include "myshop-api.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{- include "myshop-api.selectorLabels" . | nindent 8 }}
spec:
serviceAccountName: {{ .Values.serviceAccount.name }}
containers:
- name: api
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
envFrom:
- configMapRef:
name: {{ include "myshop-api.fullname" . }}
- secretRef:
name: {{ include "myshop-api.fullname" . }}
resources:
{{- toYaml .Values.resources | nindent 12 }}Helpers like {{ include "myshop-api.fullname" . }} are defined in _helpers.tpl. This is a standard Helm chart pattern; running helm create to generate the skeleton includes them automatically.
Deployment #
helm upgrade --install myshop-api charts/myshop-api \
-n myshop --create-namespace \
-f charts/myshop-api/values.yaml \
-f charts/myshop-api/values-dev.yamlhelm upgrade --install myshop-api charts/myshop-api \
-n myshop --create-namespace \
-f charts/myshop-api/values.yaml \
-f charts/myshop-api/values-prod.yamlThe same chart renders differently per environment. This command is replaced by ArgoCD Application sync at the #4 CI/CD stage, but it remains valid for manual deployments during development.
cert-manager and external-dns — auxiliary components #
To automatically obtain and renew Let’s Encrypt certificates inside the cluster instead of managing ACM certificates directly, adopt cert-manager. To auto-manage Route53 DNS records from manifests, adopt external-dns alongside it.
helm repo add jetstack https://charts.jetstack.io
helm install cert-manager jetstack/cert-manager \
-n cert-manager --create-namespace \
--set installCRDs=truehelm install external-dns external-dns/external-dns \
-n external-dns --create-namespace \
--set provider=aws \
--set aws.region=ap-northeast-2 \
--set "domainFilters[0]=myshop.example.com"This is the standard setup for operational clusters. Whenever a domain is added, a single annotation in the manifest is all that’s needed — no opening the Route53 console. cert-manager is unnecessary when using ACM certificates, but it is more consistent across multi-cluster / multi-cloud environments.
Checks after first deploy #
Items to check right after deploy.
kubectl get all -n myshop
kubectl get hpa -n myshop
kubectl get pdb -n myshop
kubectl get ingress -n myshopkubectl describe ingress myshop-api -n myshop | grep "Address"curl -i https://api.myshop.example.com/health/readyThese four commands verify that Pods, HPA, Ingress, and the external entry point are all operational. When 200 responses start coming back, the cluster’s first workload is live.
Closing #
We put myshop-api on the empty EKS cluster as a bundle of manifests, auto-provisioned an ALB via AWS Load Balancer Controller to make the external entry point, and abstracted all manifests as a Helm chart so the same chart unfolds with different values across dev / prod. At this point, myshop-api is accessible from outside but has no data store — it’s an empty shell that only returns 200 on /health/ready. The next post fills that void — connection with RDS PostgreSQL, the path of safely injecting passwords via Secrets Manager, the flow of syncing K8s Secret with cloud secret store via External Secrets Operator, and connection pool operational principles — all in one cycle.