When terraform plan Succeeds but apply Fails: Causes and Partial-Apply Recovery
Few moments in Terraform operations are as disorienting as a clean terraform plan followed by an apply that dies halfway through. The conclusion up front: plan is a simulation and apply makes real API calls, so their validation scopes differ by design, and the first thing to do after a failure is not fixing the error but establishing how much got created. Apply is not a transaction: everything built before the failure stays, and mishandling that state produces orphaned resources and state drift. How plan computes changes is covered in Basics #5, what state really is.
What plan cannot catch: the validation asymmetry #
Plan validates configuration syntax, types, references between resources, and the diff against current state, plus whatever schema checks the provider can run locally. Everything that only AWS can judge is validated for the first time when apply calls the real API. Five typical causes live in that gap.
| Type | Typical error | Why plan misses it |
|---|---|---|
| Missing permissions | AccessDenied, UnauthorizedOperation | Plan needs only read access, so write-permission holes stay hidden |
| IAM propagation delay | InvalidParameterValue (role not found variants) | A just-created role or policy has not propagated to other services yet |
| Service quotas | LimitExceeded, AddressLimitExceeded | Limits are judged by the API at creation time |
| Global name collisions | BucketAlreadyExists, already in use | Uniqueness across all accounts can only be checked at creation |
| Parameter combinations | Unsupported, InvalidParameterCombination | Server-side rules like instance types unavailable in an AZ |
The first type is the most common in practice. The standard CI setup splits a read-only role for plan and a write role for apply, and forgetting to extend the apply role’s policy when a new resource type lands produces exactly this pattern. The second type appears when a role is created and referenced within the same apply. The provider absorbs most of it with retries, but unusually slow propagation surfaces as a failure, and for that one the answer is simply to run apply again.
Right after the failure: assess the state first #
If apply died on resource 6 of 10, resources 1–5 exist and are recorded in state. Terraform updates state after each successful resource creation, so most failures leave a clean split: what succeeded is in state, what failed onward is not. Recovery is then straightforward.
- Fix the cause the error names: add the permission, request the quota increase, change the name — whatever matches the type.
- Run plan again: already-created resources show as no changes, and only the failed resource and subsequent resources appear as pending creation. Confirm this plan matches expectations.
- Run apply again: the remainder gets created and the stack reaches the original target state. Nothing needs rebuilding from scratch, and nothing already built needs deleting.
The thing to resist is doing anything else in between. Hand-creating the failed resource in the console produces a name collision on the next apply; deleting already-created resources in the console splits state from reality and compounds the confusion at refresh time. The default recovery is fix-and-rerun; console intervention is the last resort.
The “already exists” error: reality ahead of state #
If the rerun hits BucketAlreadyExists or another already-exists error, you have the more annoying case: the API call succeeded and the resource exists, but the process died before the response was recorded, so the state entry is missing. This follows network drops or force-killed CI jobs.
There are two options. To adopt the real resource into code, attach it to state with an import block. If the resource looks incompletely created, delete it via console or CLI and let apply rebuild it. Either way the goal is reconciling state with reality, and the deciding question is whether the resource has accumulated data: an empty resource is simpler to delete and recreate, one with data is safer to import. The broader state-surgery toolbox is Ops #2, drift and state surgery.
Applying with -target gets recommended often at this point, but it is also a tool that manufactures one more partial-apply state. A stack where only part of the dependency graph has been applied is hard for the next person to reason about, so recover toward a fully clean plan and reserve -target for genuine isolation needs.
Prevention #
- Make apply-role permissions part of the release process: a PR introducing a new resource type reviews the CI apply role’s policy alongside it. Make “plan passing is not a permission check” shared team knowledge.
- Check quotas ahead of bulk creation: VPCs, EIPs, and EC2 vCPUs are the usual suspects; confirm in Service Quotas and file increases before, not after.
- Suffix globally unique names: bake an account ID or random suffix into S3 bucket names so collisions cannot happen.
- Reduce force-killed CI jobs: set job timeouts comfortably above apply duration, and prefer letting a run finish then reverting over cancelling mid-apply — that is what prevents missing state entries.
Summary #
- Plan validates schema, references, and state diffs; permissions, quotas, name uniqueness, and parameter combinations are judged only by apply’s real API calls.
- Apply is not a transaction, but successful resources are recorded in state, so most failures recover with fix-and-rerun alone.
- Touching the console right after a failure makes things worse. The center of recovery is confirming the rerun plan looks exactly as expected.
- An already-exists error signals a missing state entry. Import if the resource holds data; delete and recreate if it is empty.
- Not mistaking plan for a permission check, plus quota and naming discipline, prevents most of these incidents outright.