Docker Advanced #1: BuildKit and buildx — What the Builder Actually Is

The Intermediate series treated BuildKit as “something that’s already turned on.” Advanced opens up the lid. This post lays out the structure of the builder itself, and the role of buildx, the CLI sitting on top of BuildKit.

This post in the Docker Advanced series:

• #1 BuildKit and buildx ← this post
• #2 Multi-architecture images (linux/amd64 + arm64)
• #3 Image security — non-root, distroless, scan (Trivy)
• #4 SBOM and signing (cosign)
• #5 Resource limits and cgroups
• #6 Production operations — restart policy, healthcheck, graceful shutdown

A short history of two builders #

Intermediate #2 noted Docker has two generations of build engines. One step further:

The legacy builder was the original docker build. Simple model — read the Dockerfile top-to-bottom, run each line in a temporary container, commit it, repeat. Intuitive but limited. No parallelism, simple cache model, easy to bake secrets into images.

BuildKit started outside Docker (the Moby project) and was later integrated. Two core ideas:

1. Compile the Dockerfile into a graph (LLB) — independent nodes run in parallel
2. Separate frontend from backend — extensible enough to accept formats other than Dockerfile

   Dockerfile        bake.hcl       other frontend
       │               │                │
       ▼               ▼                ▼
   ┌──────────────────────────────────────────┐
   │   Frontend (Dockerfile parser, etc.)      │
   └────────────────┬─────────────────────────┘
                    │ produces
                    ▼
   ┌──────────────────────────────────────────┐
   │   LLB — Low-Level Build graph             │
   │   (dependency graph, cache keys, parallel)│
   └────────────────┬─────────────────────────┘
                    │ executed by
                    ▼
   ┌──────────────────────────────────────────┐
   │   BuildKit backend                        │
   │   (snapshotter, cache, executor)          │
   └────────────────┬─────────────────────────┘
                    │ outputs
                    ▼
        image / tar / local directory / etc.

You almost never touch LLB directly, but it’s a useful mental model for understanding why “this line really does run alongside that one.”

What that # syntax= line really is #

The line touched on in Intermediate #2:

# syntax=docker/dockerfile:1.7
FROM ...

That’s a frontend pin. It tells BuildKit which version of the Dockerfile parser to use. BuildKit fetches that frontend as an OCI image and uses it for compilation. Whenever new Dockerfile features land (RUN --mount, COPY --link, --exclude, …), the frontend version moves up.

Sensible pins:

• docker/dockerfile:1 — latest 1.x. The common default.
• docker/dockerfile:1.7 — pinned to the 1.7 series.
• docker/dockerfile:1.7.0 — exact version.

For production / reproducibility, pinning the minor version is safer.

buildx — the CLI on top of BuildKit #

The difference between docker build and docker buildx build:

In short, buildx is the CLI that exposes BuildKit’s full feature set. Recent Docker increasingly merges the two — docker build calls into buildx behind the scenes. New code? Just write buildx.

Builder instances — where the build runs #

The central concept in buildx is the builder instance. You choose where the BuildKit daemon that actually performs the build lives.

docker buildx ls
# NAME/NODE          DRIVER/ENDPOINT             STATUS    PLATFORMS
# default *          docker                                
#  \_ default        \_ unix:///var/run/...      running   linux/amd64, linux/arm64

By default it’s a docker-driver builder named default. Builds run on the BuildKit baked into the Docker daemon itself.

Drivers #

The biggest limit of the docker driver is no multi-platform builds. So creating a docker-container builder is usually the first step.

Create a new builder #

docker buildx create --name multi --driver docker-container --use
docker buildx ls
# NAME/NODE          DRIVER/ENDPOINT             STATUS    PLATFORMS
# multi *            docker-container                      
#  \_ multi0         \_ unix:///var/run/...      inactive
# default            docker                                
#  \_ default        \_ unix:///var/run/...      running   linux/amd64, linux/arm64

• --name multi — builder name
• --driver docker-container — BuildKit in its own container
• --use — make it the active builder right away

inactive just means no build has run yet. The first build automatically starts a BuildKit container (moby/buildkit:...).

docker buildx inspect multi --bootstrap
# Brings the BuildKit container up immediately

Switching / removing:

docker buildx use default     # back to default
docker buildx use multi       # to multi
docker buildx rm multi        # delete
docker buildx prune --all     # clean caches across all builders

--output — where the build result goes #

This is where buildx’s expressive range shows. The build result doesn’t have to land as an image.

# 1) As a Docker image (load) — into the local image cache
docker buildx build --output type=docker -t myapp .
# Or (shortcut)
docker buildx build --load -t myapp .

# 2) Push directly to a registry
docker buildx build --output type=registry -t ghcr.io/me/myapp:1.0 .
# Or (shortcut)
docker buildx build --push -t ghcr.io/me/myapp:1.0 .

# 3) As a tar file
docker buildx build --output type=tar,dest=myapp.tar .

# 4) As an OCI image layout (directory)
docker buildx build --output type=oci,dest=myapp-oci/ .

# 5) Extract the build's filesystem locally (no image at all)
docker buildx build --output type=local,dest=./out --target builder .

type=local is interesting — it doesn’t make an image at all, just extracts the files produced by the build. Useful when CI wants to hand a build artifact (e.g., a compiled binary) to another step that has nothing to do with Docker.

docker buildx build --target builder --output type=local,dest=./bin .
ls bin/
# myapp

Multi-platform builds — a quick taste #

On the docker-container driver, one command can build for two architectures simultaneously.

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t ghcr.io/me/myapp:1.0 \
  --push .

What this produces is a manifest list (OCI image index) that wraps the per-arch images. A consumer pulls the right image for their architecture automatically. Depth in #2.

One conflict between --load and --platform: the local Docker image cache only supports a single platform — you can’t --load multiple platforms at once. Either --push, or pick one platform to --load.

Cache export — deeper --cache-to/from #

External caches from Intermediate #2. A few more options:

docker buildx build \
  --cache-to=type=registry,ref=ghcr.io/me/myapp:cache,mode=max,compression=zstd \
  --cache-from=type=registry,ref=ghcr.io/me/myapp:cache \
  --push -t ghcr.io/me/myapp:1.0 .

For deep multi-stage builds, mode=max makes a real difference. Builder / dependencies / test stage caches all stay alive, so small changes finish near-instantly. The trade-off is a larger cache.

docker buildx bake — declarative multi-target builds #

When you need to build several images at once (a monorepo: web + worker + admin), shell-scripting buildx build calls gets messy fast. bake solves that declaratively.

group "default" {
  targets = ["web", "worker", "admin"]
}

target "web" {
  context = "./web"
  tags    = ["ghcr.io/me/web:1.0", "ghcr.io/me/web:latest"]
  platforms = ["linux/amd64", "linux/arm64"]
  cache-from = ["type=registry,ref=ghcr.io/me/web:cache"]
  cache-to   = ["type=registry,ref=ghcr.io/me/web:cache,mode=max"]
}

target "worker" {
  context = "./worker"
  tags    = ["ghcr.io/me/worker:1.0"]
  platforms = ["linux/amd64", "linux/arm64"]
}

target "admin" {
  inherits = ["web"]
  context  = "./admin"
  tags     = ["ghcr.io/me/admin:1.0"]
}

docker buildx bake --push
# Builds all three at once, multi-platform, with shared caches

What bake gives you:

• Parallel builds — web, worker, admin build simultaneously when independent
• inherits for shared config
• Variables / functions — HCL variable, function for environment forks
• JSON / HCL / compose as inputs — yes, your compose.yaml’s build: is a valid input

docker buildx bake -f compose.yaml --push

You don’t repeat the same definition twice — CI runs the same builds that compose.yaml defines.

Common bake pattern — environment forks #

variable "TAG" { default = "dev" }
variable "REGISTRY" { default = "ghcr.io/me" }

target "_common" {
  platforms = ["linux/amd64", "linux/arm64"]
}

target "web" {
  inherits = ["_common"]
  context  = "./web"
  tags     = ["${REGISTRY}/web:${TAG}"]
}

docker buildx bake web                              # dev tag
TAG=1.2.3 REGISTRY=ghcr.io/me docker buildx bake web --push   # production

CI build scripts get a lot lighter.

Other useful buildx options #

--progress=plain         # full log (instead of BuildKit's one-line summary)
--progress=quiet         # minimal output
--no-cache-filter=test   # ignore cache for a specific stage only
--build-context other=./other-dir   # name an additional build context
--allow=network.host     # allow host network inside the build (security caveat)

--build-context is useful in monorepos where you want to bring another directory in as a context. The Dockerfile pulls from it like COPY --from=other ./shared/types /app/types.

Wrap-up #

The picture from this post:

• BuildKit = LLB (graph) + frontend (Dockerfile parser) + backend. # syntax= pins the frontend.
• buildx is the CLI to BuildKit’s full feature set. Use it for new code.
• Builder instances are central — the docker driver can’t build multi-platform, so creating a docker-container builder is usually step one.
• --output sends results to images / tar / OCI / local dir / registry.
• For external cache, mode=max + compression=zstd shines in multi-stage builds.
• docker buildx bake declaratively orchestrates monorepo / multi-target builds. compose.yaml works as input too.

In the next post (#2 Multi-architecture images) we go behind the --platform option — manifest lists, QEMU emulation, native multi-arch builders, and how to avoid the common “image built on Apple Silicon won’t run on the production server” incident.