Terraform Associate Exam Cram - Part 6

This is the Part 6 of the Terraform Associate 004 Exam Cram. It covers the following Terraform Associate Certification exam objectives:

< Prev: Objective 5 - Terraform modules
Next >: Objective 7 - Maintain infrastructure with Terraform

The primary purpose of Terraform state is to store bindings between objects in a remote system and resource instances declared in your configuration. When Terraform creates a remote object it will associate the identity of that remote object with a particular resource instance.

Backend defines where Terraform's state snapshots are stored. A given Terraform configuration can either specify a backend, integrate with HCP Terraform, or do neither and default to storing state locally.

If a configuration includes no backend or cloud block, Terraform defaults to using the local backend, which stores state in a file named terraform.tfstate in the current working directory.

Example local Backend Configuration:

Note:
The local backend configuration file .terraform/terraform.tfstate is different and entirely separate from the Terraform state file terraform.tfstate. Terraform stores the state file (terraform.tfstate) in the location defined by the backend configuration.

Back to Top

State locking happens automatically on all operations that could update Terraform state. This prevents potential corruptions caused by simultaneous write operations.

Locking is supported by most remote backends (e.g., S3 with DynamoDB, GCS, Terraform Cloud, Consul, AzureRM).

If state locking fails, Terraform will not continue and display an error.

The terraform force-unlock LOCK_ID command allows you to manually remove a state lock, for example, when automatic unlocking fails. Use the -force option to proceed without a confirmation prompt.

Back to Top

Backends

Terraform includes a selection of built-in backends:

• local - Stores state on the local filesystem, locks that state using system APIs.
• remote - Supports state for HCP Terraform and Terraform Enterprise. Not recommenced.
• azurerm - Azure Blob Storage. Supports state locking and consistency checking.
• consul - Consul KV store. Supports state locking.
• cos - Tencent Cloud Object Storage (COS). Supports state locking.
• gcs - Google Cloud Storage (GCS). Supports state locking.
• http - Stores the state using a simple REST client. Optionally supports state locking with LOCK and UNLOCK requests.
• kubernetes - Stores the state in a Kubernetes secret. Supports state locking, with locking done using a Lease resource.
• oci - Oracle Cloud Infrastructure (OCI) Object Storage. Supports state locking (Terraform v1.12 or later).
• oss - Alibaba Cloud Object Storage Service (OSS). Supports state locking and consistency checking via Alibaba Cloud Table Store.
• pg - Postgres database. Supports state locking.
• s3 - Amazon S3. Support for native state locking was added in Terraform v1.10. Prior versions require Dynamo DB.

Additional backends cannot be added as plugins.

Backends are configured with a backend block nested within the top-level terraform block:

The block label of the backend block indicates which backend type to use. The arguments used in the block's body are specific to the chosen backend type. They configure where and how the backend will store the configuration's state, access credentials, and other backend-specific parameters.

The following should be taken into account when configuring backends:

• A configuration can only provide one backend block.
• A backend block cannot refer to named values (e.g., variables, locals, or data source attributes).
• Changing backend requires state migration (terraform init -migrate-state).
• Hard-coding credentials directly inside backend blocks is not recommended. Use partial backend configuration and supply the sensitive values through environment variables, credential files, etc.

Authentication

Remote backends, cloud providers, and HCP Terraform generally require access credentials and some form of authentication.

Commonly used methods for providing credentials include:

• Environment variables.
• Credentials files.
• CLI authentication (az login, gcloud auth login).

Hard-coding credentials directly in Terraform configuration (inside required_providers or backend blocks) is not recommended.

Backend Credentials

Use partial backend configuration when supplying backend access credentials. That is, leave credential-related arguments unset in the backend block and provide them using other methods, such as:

• Credentials files (vary by backend), e.g., ~/.aws/credentials.
• Environment variables (vary by backend), for example: AWS_ACCESS_KEY_ID/ AWS_SECRET_ACCESS_KEY, GOOGLE_APPLICATION_CREDENTIALS, GOOGLE_BACKEND_CREDENTIALS.
• A configuration file with key/value assignments (same format as terraform.tfvars) specified via the terraform init -backend-config=PATH command line. The values from the file are merged with what is in the configuration's backend block. Note that Terraform will include these values in both the backend configuration file .terraform/terraform.tfstate and plan files.
• Command-line key/value pairs specified via the terraform init -backend-config="KEY=VALUE" command. The -backend-config="KEY=VALUE" flag can be specified multiple times. Keep in mind that many shells retain entered commands in a history file.
• Interactively - Terraform will prompt you for the required values, unless interactive input is disabled with the -input=false flag.

Note:
The local backend configuration file .terraform/terraform.tfstate is different and entirely separate from the Terraform state file terraform.tfstate. Terraform stores the state file (terraform.tfstate) in the location defined by the backend configuration.

Provider Credentials

Terraform provider's documentation in the public Terraform Registry describes how to configure credentials in the required_providers block for any given provider.

Typically, resource providers can obtain necessary credentials from several sources, including:

• Parameters in the provider configuration (hard-coding credentials in any Terraform configuration is not recommended).
• Environment variables.
• Shared credentials files.
• Shared configuration files.
• Container / instance / service credentials.
• External credentials / federation.

CLI Configuration File

The CLI configuration file configures per-user settings customizing Terraform CLI behaviors, which apply across all Terraform working directories.

The following credential settings can be set in the CLI configuration file (.terraformrc or terraform.rc):

• credentials - configures credentials for use with HCP Terraform or Terraform Enterprise.
• credentials_helper - configures an external helper program for the storage and retrieval of credentials for HCP Terraform or Terraform Enterprise.

Workspaces

The Terraform CLI workspaces are different from workspaces in HCP Terraform. Terraform CLI workspaces allow multiple state files to exist within a single directory, letting you use one configuration for multiple environments. HCP Terraform workspaces contain everything needed to manage a given set of infrastructure, and function like separate working directories.

Terraform CLI workspaces

• Each Terraform configuration is associated with a backend (backend must support multiple named workspaces).
• A configuration can have one or more workspaces configured.
• By default, Terraform uses a single workspace named default.
• Each workspace has an associated state file.
• All state files for a given configuration are stored in the same backend, typically differentiated by using a prefix or key attribute.
• The name of the current workspace can be referenced in configuration using ${terraform.workspace}.
• Workspaces are managed with the terraform workspace commands.

terraform workspace commands:

• terraform workspace new NAME - Create a new workspace. Use the -state=path flag to copy an existing state file into the new workspace.
• terraform workspace list - Show all workspaces.
• terraform workspace show - Display the name of the current workspace.
• terraform workspace select NAME - Switch to an existing workspace.
• terraform workspace delete NAME - Delete a workspace (except default).

Back to Top

Drift

Drift occurs when real infrastructure gets out of sync with Terraform state, usually due to manual changes, external updates, etc.

By default, Terraform compares the state file to real infrastructure every time terraform plan or terraform apply is invoked. First, Terraform performs in-memory state refresh to reflect the actual configuration of the infrastructure. This ensures that Terraform determines the correct changes to include in the plan. When the plan is applied (with terraform apply), Terraform will update both, the infrastructure and the state file.

Refresh-Only Mode

Refresh-only mode instructs Terraform to create a plan that updates the Terraform state to match changes made to remote objects outside of Terraform. This is useful if state drift has occurred and you want to reconcile your state file to match the drifted remote objects. Applying a refresh-only run does not result in further changes to remote objects.

The -refresh-only and -refresh plan customization options alow to control Terraform refresh behavior.

The -refresh-only options instructs Terraform to create a plan that updates the Terraform state to match changes made to remote infrastructure objects outside of Terraform (i.e., a plan to bring the changes into state). This allows you to review the proposed changed before applying.

Run terraform plan -refresh-only to determine the drift between the current state file and actual infrastructure.

Update the state with terraform apply -refresh-only. Applying a refresh-only plan does not result in changes to the infrastructure.

The -refresh=false options disables the default behavior of refreshing state while creating the plan. This can potentially make planning faster, but with the risk of planning against an outdated state.

Related Command:

The terraform refresh command reads the current settings from all managed remote objects and updates the Terraform state to match. It automatically overwrite the state file without giving you the option to review the modifications first. This command is deprecated. Instead, use the -refresh-only flag with terraform apply and terraform plan commands.

State Refactoring

State refactoring - reorganization of managed resources to improve maintainability and performance by splitting your Terraform state across multiple configurations.

The following are some common opportunities for refactoring:

• Long Terraform applies.
• Changes to management lifecycles.
• Changes to resource ownership.
• Reusable module opportunity.

Consider the following resource properties when deciding how to group resources together:

• Volatility and rate of change.
• Stateful vs stateless.
• Access and team responsibility.

Use the following dynamic approaches to reference resources in another state files:

• Resource-specific data source.
• tfe_outputs data source for HCP Terraform or Terraform Enterprise.
• terraform_remote_state data source for remote backend, or the local backend.

Use the terraform graph command to help visualize the relationships between the resources in your configuration and identify any dependencies.

Resource Migration

There are two common approaches to migrating resources between two Terraform state files:

• Remove and import - Use the removed and import blocks to keep a record of the configuration history.
• Move resources directly to a new state file - Use the terraform state mv command to move resources into a different state file.

Remove and Import

1. In the source configuration, retrieve the latest Terraform state and save the output as a backup:

   terraform state pull > terraform.tfstate.backup

2. Record the configuration details and attributes of the resource you are planning to move:

   terraform state show resource_type.example

3. For each resource you want to migrate, replace the resource block with a removed block:

   # resource "resource_type" "example" {
   #  ... 
   # }
   
   removed {
     from = resource_type.example
     lifecycle {
       destroy = false
     }
   }

4. Run terraform plan to ensure that Terraform will not destroy the resources.
5. Run terraform apply to remove the resources from state.
6. In the destination configuration, add a resource block and an import block for each added resource:

   resource "resource_type" "example" {
     # ... 
   }
   
   import {
     id = "provider-specific-id"  
     to = resource_type.example
   }

7. Run terraform plan to ensure that Terraform will properly import the resources.
8. Run terraform apply to complete the import.

After you complete the migration you can optionally remove the removed and import blocks, or choose to keep them as a record of the resource's lifecycle.

Direct Move

1. In the directory with your source configuration, save the state file locally:

   terraform state pull > source.tfstate

2. In the directory with your destination configuration, save the state file locally:

   terraform state pull > destination.tfstate

3. For each resource you want to migrate, move the resource from the source state to the destination state:

   terraform state mv -state source/source.tfstate \
       -state-out destination/destination.tfstate \
       resource_type.example resource_type.example

4. In the directory with your source configuration, update the remote state:

   terraform state push source.tfstate

5. In the directory with your destination configuration, update the remote state:

   terraform state push destination.tfstate

6. Remove the migrated resources from the source configuration.
7. Run terraform plan on the source configuration and ensure that Terraform will not make any changes to the infrastructure.
8. Add the migrated resources from the destination configuration.
9. Run terraform plan on the destination configuration and ensure that Terraform will not make any changes to the infrastructure.

removed Block

moved Block

Use the moved block to programmatically change the address of (rename) a resource.

Note that removing a moved block is a breaking change because any configurations that refer to the old address will plan to delete the existing object instead of move it.

Back to Top

Back to Top

< Prev: Objective 5 - Terraform modules
Next >: Objective 7 - Maintain infrastructure with Terraform

More Terraform Tutorials:
Terraform Associate Exam Cram
Understanding Terraform Variable Precedence
Terraform Value Types Tutorial
Terraform count Explained with Practical Examples
Terraform for_each Tutorial with Practical Examples
Exploring Terraform dynamic Blocks with GCP Examples
Working with External Data in Terraform
Terraform Modules FAQ