When Secrets End Up in tfstate in Plaintext: Rotation First, Then Prevention

Infrastructure Terraform IaC State
4 min read

There is a moment when someone managing RDS passwords or API keys with Terraform opens the tfstate file and freezes: every value marked sensitive is sitting there in plaintext. The conclusion up front: plaintext state is design, not a bug, and when exposure is plausible, the order of operations starts with key rotation, not state cleanup. Once rotation is done, the leaked values are dead values, and only then does prevention get designed. The response ordering is the same principle as committing a secret to Git; this post covers the Terraform-specific parts.

Why plaintext: sensitive is a blindfold, not a lock #

To decide changes on the next apply, Terraform must know the current values. Comparing whether a password argument changed requires the previous password to be in state. So state records every resource attribute as its literal value. What sensitive = true does is mask the value in plan output and display — it has no effect on the file contents.

One conclusion follows immediately: access to the state file is access to the secrets. The list of people, roles, and pipelines that can read state is the list of secret readers, and operating on that premise is the starting point of the prevention section below.

Incident response: rotate → history → state, in that order #

1. Judge the exposure scope. Where did the state live and who could read it? Was a local tfstate committed to an internal repository, or a public one? Was the S3 backend bucket’s policy open too wide? Public exposure counts as a leak immediately; internal-only follows your organization’s bar, with rotation as the safe default.

2. Rotate the secrets. Changing the DB password, reissuing API keys, replacing IAM access keys — invalidating the exposed values is the substance of the response. No amount of scrubbing state removes the possibility that the values were already copied, so cleanup without rotation is not a response.

3. Purge git history. If tfstate was ever committed, remove it from history with filter-repo and follow the force-push procedure — the same steps and cautions as the committed-secret post — and add *.tfstate* to .gitignore to block recommits. The structural fix is not keeping local state at all: move to a remote backend, the subject of Basics #9.

4. State values update once rotation lands. Update the code and variables to the new secret and apply; state swaps to the new values. If the old values lingering in backend version history (S3 versioning) bothers you, deleting those object versions is legitimate tidying — after rotation, never as a substitute for it.

Prevention, part 1: use the paths that keep values out of state #

Terraform 1.10–1.11 introduced the structural fix: ephemeral values and write-only arguments. A secret read through an ephemeral resource exists only during the run and is recorded in neither state nor plan. A write-only argument (such as aws_db_instance’s password_wo) writes the value once to the provider without storing it in state, with replacement signaled through a version trigger like password_wo_version.

For new code, then, instead of the old path of “take the password as a variable, put it in the argument, let it land in state,” the default can be reading from Secrets Manager or SSM ephemerally and flowing it through a write-only argument. The concrete code and the SSM versus Secrets Manager choice are covered in Practice #6, secrets management. The simpler dodge remains valid too: keep Terraform away from the secret entirely and let the application read it from SSM at runtime. Terraform then handles only the reference (the ARN), and only the reference lands in state.

Prevention, part 2: treat state itself as a secret store #

Some values will always land in state (attributes of created resources, for instance), so the state side gets hardened in parallel.

  • Encryption at rest: enable SSE-KMS on the S3 backend and narrow who can decrypt via the KMS key policy — one more layer of control than default SSE-S3.
  • Minimal access: cut the state bucket’s IAM and bucket policies down to “the Terraform execution role only.” A state bucket with broad read access is the same as publishing the secrets company-wide.
  • Versioning and logs: versioning is both a recovery tool and an investigation tool. With S3 access logs and CloudTrail data events on, “who read it” has an answer when it matters.
  • Minimal outputs: outputting a secret propagates it into parent-stack state and CI logs. Hold the line that outputs carry references only.

Summary #

  • tfstate stores every value in plaintext by design. Sensitive only masks display, so treat state access as secret access.
  • The substance of incident response is rotation. State cleanup and history purges are tidying that follows rotation, never substitutes for it.
  • If tfstate was ever committed, the set is filter-repo history removal, .gitignore blocking, and migration to a remote backend.
  • New code defaults to ephemeral plus write-only arguments so values never land in state; app-reads-at-runtime designs remain valid.
  • State-side hardening is four basics: SSE-KMS, minimal access, versioning with access logs, minimal outputs.
X