Terraform Associate Exam Cram - Part 7

Implement and maintain state

This is Part 7 of the Terraform Associate Exam Cram. It covers the following Terraform Associate Certification exam objectives:

7	Implement and maintain state	Documentation	Tutorial
7a	Describe default `local` backend	Backends Backend Type: `local`	Initialize Terraform Configuration
7b	Describe state locking	State Locking
7c	Handle backend and cloud integration authentication methods	Command: login	Collaborate using HCP Terraform
7d	Differentiate remote state backend options	Backend Types	Store Remote State
7e	Manage resource drift and Terraform state	Refresh-Only Mode	Manage Resource Drift Use Refresh-Only Mode to Sync Terraform State
7f	Describe backend block and cloud integration in configuration	HCP Terraform Configuration Backend Configuration	Store Remote State
7g	Understand secret management in state files	Sensitive Data in State	Protect Sensitive Input Variables
	Practice Questions

< Prev: Objective 6 - Use the core Terraform workflow
Next >: Objective 8 - Read, generate, and modify configuration

7a. Describe default `local` backend

The primary purpose of Terraform state is to store bindings between objects in a remote system and resource instances declared in your configuration. When Terraform creates a remote object it will associate the identity of that remote object with a particular resource instance.

Backend defines where Terraform's state snapshots are stored. A given Terraform configuration can either specify a backend, integrate with HCP Terraform, or do neither and default to storing state locally.

If a configuration includes no backend or cloud block, Terraform defaults to using the local backend, which stores state in a file named terraform.tfstate in the current working directory.

Example local Backend Configuration:

terraform { backend "local" { path = "relative/path/terraform.tfstate" # The path to the tfstate file (Optional) } }

7b. Describe state locking

State locking happens automatically on all operations that could update Terraform state. This prevents potential corruptions caused by simultaneous write operations.

Locking is supported by most remote backends (e.g., S3 with DynamoDB, GCS, Terraform Cloud, Consul, AzureRM).

If state locking fails, Terraform will not continue and display an error.

The terraform force-unlock LOCK_ID command allows you to manually remove a state lock, for example, when automatic unlocking fails. Use the -force option to proceed without a confirmation prompt.

7c. Handle backend and cloud integration authentication methods

Remote backends, cloud providers, and HCP Terraform generally require access credentials and some form of authentication.

Commonly used methods for providing credentials include:

Environment variables.
Credentials files.
CLI authentication (az login, gcloud auth login).

Hard-coding credentials directly in Terraform configuration (inside required_providers or backend blocks) is not recommended.

Backend Credentials

Use partial backend configuration when supplying backend access credentials. That is, leave credential-related arguments unset in the backend block and provide them using other methods, such as:

Credentials files (vary by backend), e.g., ~/.aws/credentials.
Environment variables (vary by backend), for example: AWS_ACCESS_KEY_ID/ AWS_SECRET_ACCESS_KEY, GOOGLE_APPLICATION_CREDENTIALS, GOOGLE_BACKEND_CREDENTIALS.
A configuration file with key/value assignments (same format as terraform.tfvars) specified via the terraform init -backend-config=PATH command line. The values from the file are merged with what is in the configuration's backend block. Note that Terraform will include these values in both the backend configuration file .terraform/terraform.tfstate and plan files.
Command-line key/value pairs specified via the terraform init -backend-config="KEY=VALUE" command. The -backend-config="KEY=VALUE" flag can be specified multiple times. Keep in mind that many shells retain entered commands in a history file.
Interactively - Terraform will prompt you for the required values, unless interactive input is disabled with the -input=false flag.

Note:
The local backend configuration file .terraform/terraform.tfstate is different and entirely separate from the Terraform state file terraform.tfstate. Terraform stores the state file (terraform.tfstate) in the location defined by the backend configuration.

Provider Credentials

Terraform provider's documentation in the public Terraform Registry describes how to configure credentials in the required_providers block for any given provider.

Typically, resource providers can obtain necessary credentials from several sources, including:

Parameters in the provider configuration (hard-coding credentials in any Terraform configuration is not recommended).
Environment variables.
Shared credentials files.
Shared configuration files.
Container / instance / service credentials.
External credentials / federation.

HCP Terraform

The terraform login command obtains an API token for HCP Terraform, Terraform Enterprise, or other host that offers Terraform services. Use this command only in interactive scenarios.

Usage: terraform login [hostname]

If no hostname is provided, the default hostname is app.terraform.io (HCP Terraform).

If not overridden by credentials helper settings in the CLI configuration, the terraform login command writes credentials to local file .terraform.d/credentials.tfrc.json.

The terraform logout [hostname] command removes locally-stored credentials for specified hostname.

For unattended automated deployments configure credentials manually in the CLI configuration file.

Alternatively, use a host-specific environment variable to provide an API token. Environment variable names should have the prefix TF_TOKEN_ added to the domain name, with periods encoded as underscores. For example, the value of a variable named TF_TOKEN_app_terraform_io will be used when the CLI makes service requests to the hostname app.terraform.io.

CLI Configuration File

The CLI configuration file configures per-user settings customizing Terraform CLI behaviors, which apply across all Terraform working directories.

The following credential settings can be set in the CLI configuration file (.terraformrc or terraform.rc):

credentials - configures credentials for use with HCP Terraform or Terraform Enterprise.
credentials_helper - configures an external helper program for the storage and retrieval of credentials for HCP Terraform or Terraform Enterprise.

7d. Differentiate remote state backend options

Backends define where Terraform's state snapshots are stored. A given Terraform configuration can either specify a backend, integrate with HCP Terraform, or default to storing state locally.

Use remote backend when multiple individuals or teams need access to the infrastructure state data.

Backends

Terraform includes a selection of built-in backends:

local - Stores state on the local filesystem, locks that state using system APIs.
remote - Supports state for HCP Terraform and Terraform Enterprise. Not recommenced.
azurerm - Azure Blob Storage. Supports state locking and consistency checking.
consul - Consul KV store. Supports state locking.
cos - Tencent Cloud Object Storage (COS). Supports state locking.
gcs - Google Cloud Storage (GCS). Supports state locking.
http - Stores the state using a simple REST client. Optionally supports state locking with LOCK and UNLOCK requests.
kubernetes - Stores the state in a Kubernetes secret. Supports state locking, with locking done using a Lease resource.
oci - Oracle Cloud Infrastructure (OCI) Object Storage. Supports state locking (Terraform v1.12 or later).
oss - Alibaba Cloud Object Storage Service (OSS). Supports state locking and consistency checking via Alibaba Cloud Table Store.
pg - Postgres database. Supports state locking.
s3 - Amazon S3. Support for native state locking was added in Terraform v1.10. Prior versions require Dynamo DB.

Terraform v1.3 removed support for the following backends:

artifactory - Stores state as an artifact in a given repository in Artifactory. Does not support state locking.
etcd - etcd 2.x; does not support state locking.
etcdv3 - etcd KV store. Supports state locking.
manta - Stores state as an artifact in Manta, HTTP-based object store. Supports state locking, with locking within Manta.
swift - Stores state as an artifact in Swift. Supports state locking.

Additional backends cannot be added as plugins.

Backends are configured with a backend block nested within the top-level terraform block:

terraform { backend "s3" { # backend type - S3 bucket = "mytfbucket" # Name of the S3 Bucket. key = "path/to/state" # Path to the state file inside the S3 Bucket. region = "us-east-1" } # ... }

The block label of the backend block indicates which backend type to use (s3). The arguments used in the block's body are specific to the chosen backend type (bucket, key, region). They configure where and how the backend will store the configuration's state, and in some cases configure other behavior. A configuration can only include one backend block.

A backend block cannot refer to named values (like input variables, locals, or data source attributes).

Terraform Cloud / HCP Terraform

HCP Terraform offers secure remote state storage and makes it easier to collaborate on infrastructure development.

To use HCP Terraform as a backend, include a cloud block within the top-level terraform block:

terraform { cloud { organization = "MY-ORGANIZATION-NAME" # The name of the organization workspaces { # Specifies which remote HCP Terraform workspaces to use name = "my_workspace_name" } } # ... }

A configuration can only include one cloud block and the cloud block cannot refer to named values like input variables, locals, or data source attributes.

7e. Manage resource drift and Terraform state

Drift occurs when real infrastructure gets out of sync with Terraform state, usually due to manual changes, external updates, etc.

By default, Terraform compares the state file to real infrastructure every time terraform plan or terraform apply is invoked. First, Terraform performs in-memory state refresh to reflect the actual configuration of the infrastructure. This ensures that Terraform determines the correct changes to include in the plan. When the plan is applied (with terraform apply), Terraform will update both, the infrastructure and the state file.

The -refresh-only and -refresh plan customization options alow to control Terraform refresh behavior.

The -refresh-only options instructs Terraform to create a plan that updates the Terraform state to match changes made to remote infrastructure objects outside of Terraform (i.e., a plan to bring the changes into state). This allows you to review the proposed changed before applying.

Run terraform plan -refresh-only to determine the drift between the current state file and actual infrastructure.

Update the state with terraform apply -refresh-only. Applying a refresh-only plan does not result in changes to the infrastructure.

The -refresh=false options disables the default behavior of refreshing state while creating the plan. This can potentially make planning faster, but with the risk of planning against an outdated state.

Related Command:

The terraform refresh command reads the current settings from all managed remote objects and updates the Terraform state to match. It automatically overwrite the state file without giving you the option to review the modifications first. This command is deprecated. Instead, use the -refresh-only flag with terraform apply and terraform plan commands.

7f. Describe `backend` block and cloud integration in configuration

Backends

Backends are configured with a backend block nested within the top-level terraform block:

terraform { backend "s3" { # Backend type - S3 bucket = "mytfstatebucket" # Name of the S3 Bucket. key = "path/to/state" # Path to the state file inside the S3 Bucket. region = "us-east-1" } # ... }

The block label of the backend block indicates which backend type to use. The arguments used in the block's body are specific to the chosen backend type. They configure where and how the backend will store the configuration's state, access credentials, and other backend-specific parameters.

The following should be taken into account when configuring backends:

A configuration can only provide one backend block.
A backend block cannot refer to named values (e.g., variables, locals, or data source attributes).
Changing backend requires state migration (terraform init -migrate-state).
Hard-coding credentials directly inside backend blocks is not recommended. Use partial backend configuration and supply the sensitive values through environment variables, credential files, etc.

Terraform Cloud / HCP Terraform

The cloud block is a nested block within the top-level terraform block. It specifies which HCP Terraform workspaces to use for the current working directory.

terraform { cloud { organization = "<organization-name>" # The name of the organization to connect to hostname = "app.terraform.io" # The hostname of the Terraform Enterprise deployment; defaults to app.terraform.io token = "<token>" # A token for authenticating with HCP Terraform. workspaces { # Specifies which remote HCP Terraform workspaces to use tags = [ "<workspace-tag>" ] # A map of key-value tags or a list key-only tags. Mutually exclusive with 'name' # name = "<workspace-name>" # Mutually exclusive with 'tags' project = "<project-name>" # The name of the HCP Terraform project to use. } } # ... }

The following should be taken into account when configuring cloud blocks:

A configuration can only provide one cloud block.
A cloud block cannot be used with state backends. A configuration can use one or the other, but not both.
A cloud block cannot refer to named values (e.g., input variables, locals, data source attributes).

Workspaces

The Terraform CLI workspaces are different from workspaces in HCP Terraform. Terraform CLI workspaces allow multiple state files to exist within a single directory, letting you use one configuration for multiple environments. HCP Terraform workspaces contain everything needed to manage a given set of infrastructure, and function like separate working directories.

Terraform CLI workspaces

Each Terraform configuration is associated with a backend (backend must support multiple named workspaces).
A configuration can have one or more workspaces configured.
By default, Terraform uses a single workspace named default.
Each workspace has an associated state file.
All state files for a given configuration are stored in the same backend, typically differentiated by using a prefix or key attribute.
The name of the current workspace can be referenced in configuration using ${terraform.workspace}.
Workspaces are managed with the terraform workspace commands.

terraform workspace commands:

terraform workspace new NAME - Create a new workspace. Use the -state=path flag to copy an existing state file into the new workspace.
terraform workspace list - Show all workspaces.
terraform workspace show - Display the name of the current workspace.
terraform workspace select NAME - Switch to an existing workspace.
terraform workspace delete NAME - Delete a workspace (except default).

HCP Terraform workspaces

Each workspace has its own dedicated state file, managed by HCP Terraform.
Workspaces can integrate with version control systems to automatically trigger runs on commit.
They include workspace-specific variables (environment and Terraform variables), credentials, and secrets.
Manage Terraform runs and retain detailed history and logs.
Support role-based access control (RBAC) for fine-grained team permissions.
Workspaces can be organized into projects for better grouping and management.

7g. Understand secret management in state files

Terraform stores the state as plain text, including variable values, even if you have flagged them as sensitive.

Recommendations for handling sensitive data in state:

Use remote backends with encryption at rest.
Use access controls (IAM roles, ACLs) to limit who has access to the state files or backend.
Use audit logs to track state access.
Do not commit state files to a version control system.

Note, Terraform masks the outputs with sensitive = true, but still stores the values in state in plain text.

Practice Questions

What is the purpose of the Terraform state file?

What is the default name of the state file in Terraform?

What is the purpose of the terraform workspace command?

What is the purpose of a backend in Terraform?

Which command can be used to remove a lock on the state file manually?

What is the effect of running terraform workspace new <name>?

Which Terraform command removes a resource from the state file without deleting it from the infrastructure?

Which feature prevents multiple administrators from concurrently modifying the Terraform state, thus avoiding potential corruption or conflicts?

What happens to the Terraform state file when changes are made directly to managed resources via the Azure Portal?

Which backend does the Terraform CLI use by default for storing state?

What is the recommended method for storing secrets required to connect to a Terraform remote backend?

What is the recommended method to protect sensitive data stored in Terraform state files?

What is the purpose of the terraform workspace select command?

What does the terraform state show command do?

What is the primary function of a backend in Terraform?

Which command can be used to list the workspaces in the current directory?

Which Terraform command can be used to update the state file with the real infrastructure?

What is the purpose of the terraform state mv command?

What is a key benefit of using a remote backend for Terraform state management?

Which command is used to display the current state of all resources in the configuration?

Which command can be used to remove a resource from the state file so it can be recreated during the next apply?

Which command is used to create a new Terraform workspace?

What does the terraform state pull command do?

You deployed a VM on a cloud provider using Terraform but didn't define any output values. How can you quickly find its IP address?