Terraform Modules FAQ (Part 2)

Terraform modules are very effective in structuring and managing infrastructure as code, enabling reusability, consistency, and scalability across projects. In this FAQ, we continue the discussion started in the Part 1 and cover the most common questions about versioning of Terraform modules, troubleshooting, and best practices.

Using Repositories to Store Terraform Modules:

Where should I store and version my Terraform modules?
Can I store and access Terraform modules from S3, GCS, or a web server?
How do I manage module versions for locally stored modules?

Using External and Community Modules:

What are the pros and cons of using external modules vs. writing your own?
Are there risks to using external or community modules?

Common Pitfalls and Troubleshooting Terraform Modules

What are the common mistakes to avoid when using Terraform modules?
How do I debug issues in modules?
How do I enable verbose output to debug Terraform issues?
How do I handle module upgrades and breaking changes?

Using Repositories to Store Terraform Modules

Where should I store and version my Terraform modules?

It is a best practice to store Terraform configurations and modules in version-controlled repositories. This enables teams to track changes, roll back to previous versions if needed, and manage infrastructure in a predictable, consistent manner.

There are two main options for storing and sharing modules: locally stored modules and modules in external registries.

Locally stored modules reside in a local directory within the project structure. Such approach is commonly used during development or for tightly coupled configurations.

Example usage:

tf-project/ │── main.tf │── variables.tf │── outputs.tf ├── modules/ │ ├── network/ │ │ ├── main.tf

# main.tf module "network" { source = "./modules/network" }

Pros:

Easy to develop and test quickly.
No external dependencies.
Useful for simple or isolated projects.

Cons:

Harder to reuse across multiple repositories or teams.
No built-in version control (relying on Git branch or file system).
Possibility of code duplication or divergence in larger projects.

Modules in external registries, as the name suggests, are hosted in central Git-like repositories or Terraform module registry (public or private).

Example with GitHub repository:

module "mirror" { source = "git::https://github.com/example-repo/tf-pub-modules.git//mirror?ref=v1.01" message = "mirror mirror on the wall" }

Where //mirror indicates the location of the module within the repository and ref=v1.0 references the required version or revision of the module.

Example with Terraform Registry:

module "network" { source = "terraform-google-modules/network/google" version = "11.0.0" # insert the 3 required variables here }

Modules on the public Terraform Registry can be referenced using a registry source address of the form <NAMESPACE>/<NAME>/<PROVIDER>. Refer to the module's information page for the exact address to use.

When using modules from a module registry, pin exact versions (version = "11.0.0") or use version constraints (version = "~> 11.0") to avoid breaking changes.

Pros:

Easy version management via Git tags or Registry versions.
Promotes reuse across teams and environments.
Enables centralized governance and review processes

Cons:

Create external dependency, if using GitHub, Terraform Registry, etc.
Adds some overhead in managing access, releases, and versioning.
More complex to set up, develop, and debug.

In summary:
Use locally stored modules during initial development or for small, self-contained use cases. For production, collaboration, and reusability, opt for external registries or Git-based modules with proper versioning and documentation.

For more details see Terraform - Module Sources

Can I store and access Terraform modules from S3, GCS, or a web server?

Yes, Terraform supports retrieving modules from remote sources such as Amazon S3 buckets, Google Cloud Storage (GCS) buckets, or even directly from HTTP(S) URLs. In these cases, the module is packaged as a .zip, .bz2, or .gz archive, commonly referred to as a Terraform module package.

A Terraform package is a compressed archive of a module directory that can be fetched from a remote location and used as a module source. This package must contain a valid Terraform module directory structure (e.g., main.tf, variables.tf, outputs.tf, etc.), and it is extracted during the terraform init process.

Example with a S3 bucket:

module "vpc" { # Use the S3 object URL source = "s3::https://my-tf-modules.s3.us-east-1.amazonaws.com/vpc.zip” }

Example with a GCS bucket:

module "vpc" { # Use the GCS API endpoint URL + bucket name + object path source = "gcs::https://www.googleapis.com/storage/v1/my-tf-modules/vpc.zip }

Example with a web server (HTTP):

module "vpc" { source = "https://example.com/my-tf-modules/vpc.zip//modules/vpc" }

If the module is located in a subdirectory relative to the root of the package, add the module path to the URL, preceded by a double slash (//). For example: //modules/vpc.

How do I manage module versions for locally stored modules?

Terraform does not support version constraints for local modules. To manage versions manually, consider using versioned directory structures - store different versions in separate folders (../modules/vpc/v1.1).

Example using a local module:

modules/ ├── vpc/ │ ├── v1.0/ │ ├── v1.1/ │ ├── latest/

Then reference the specific version in the Terraform configuration:

module "vpc" { source = "../modules/vpc/v1.1" }

Using External and Community Modules

What are the pros and cons of using external modules vs. writing your own?

External or third-party modules are best for common infrastructure patterns and rapid deployment.

Pros:

Save time
Leverage community expertise
Often well-tested and documented

Cons:

May not fit your exact use case
Potential for bloated or overly generic code
Dependency on external maintenance

Self-developed modules are better suited for custom workflows, internal tools, or organization-specific deployments.

Pros:

Fully customizable
Tailored to internal requirements and standards
Easier to audit and control
No external dependencies

Cons:

Higher development and maintenance effort
Requires internal documentation and testing

Are there risks to using external or community modules?

Yes, there are some risks:

Security risks if modules contain unsafe defaults or unreviewed code.
Maintenance risks if the module is abandoned or frequently changes.
Compatibility risks with newer versions of Terraform or cloud providers.

To mitigate these, always:

Review the module code and documentation before using it.
Ensure it aligns with your security and coding standards.
Pin specific versions to avoid surprises.
Consider forking and managing critical modules in your own repositories if needed.

Common Pitfalls and Troubleshooting Terraform Modules

What are the common mistakes to avoid when using Terraform modules?

Some frequent mistakes when working with Terraform modules include:

Overloading modules with too many responsibilities, making them hard to maintain or reuse.
Not providing default values for variables, resulting in confusing errors during usage.
Hardcoding values instead of using input variables.
Not specifying module versions, leading to unexpected changes when upstream modules are updated.
Ignoring outputs, which can make it difficult to integrate modules with other parts of your infrastructure.
Improper directory structure, which can cause issues with file references and readability.

How do I debug issues in modules?

Debugging issues in Terraform modules can involve a variety of tools and techniques. The most common problems include incorrect variable types, missing required inputs, dependency order issues, or syntax errors. Below are practical steps and tools to help troubleshoot these problems effectively:

Use terraform fmt to check for syntax and formatting issues.

This command reformats your configuration files to follow Terraform's standard style. While it's primarily used for formatting, it also helps catch simple syntax errors (e.g., missing brackets, incorrect indentation) that could impact parsing.

Use terraform validate to catch static errors.

This command performs a static analysis of the configuration to ensure it is syntactically valid and internally consistent, including:

Type mismatches (e.g., passing a string instead of a list)
Referencing undeclared variables
Using invalid combinations of arguments

Use terraform console for interactive debugging.

This opens an interactive shell where you can inspect variables, expressions, and outputs. It's useful for understanding complex data structures, outputs, or expression evaluations.

Example:

> var.ports [80, 443] > length(var.ports) 2

Check module variable definitions and type constraints.

Ensure your module's variables.tf file declares the correct types, and that consuming configurations provide matching values.

Check resource dependencies and ordering.

Use depends_on when Terraform can't automatically infer dependencies - especially for modules or resources that reference outputs from others.

Expose module variables via outputs for debugging.

If you want to inspect or debug a module’s variable in the console, expose it using an output in that module:

Example:

output "debug_cidr" { value = var.cidr_block }

Simplify or isolate the module to narrow down the issue.

Sometimes testing with a minimal set of inputs helps identify the root cause.

How do I enable verbose output to debug Terraform issues?

Terraform logging can be enabled by setting the TF_LOG environment variable to a desired level:

export TF_LOG=DEBUG terraform plan

Levels include: TRACE, DEBUG, INFO, WARN, and ERROR. TRACE provides the highest level of logging. Setting TF_LOG to JSON outputs logs at the TRACE level or higher, and uses a parsable JSON encoding as the formatting.

To disable logging, either unset TF_LOG, or set it to off.

You can also generate provider logs by setting the TF_LOG_PROVIDER environment variable.

Use TF_LOG_PATH to redirect logs to a file:

export TF_LOG_PATH=terraform-debug.log

Verbose logs can help diagnose issues like variable resolution, provider behavior, and module initialization.

How do I handle module upgrades and breaking changes?

Upgrading a module can introduce breaking changes if:

Input/output variables are renamed or removed.
Resource names or behaviors change.

To safely upgrade:

Specify module version constraints in your configuration..
Review the changelog or release notes for the new module version.
Test upgrades in a non-production environment first.
Use terraform plan to preview changes before applying them.

Conclusion

Understanding how to effectively use, debug, and manage Terraform modules is key to building scalable and maintainable infrastructure as code. By following best practices, leveraging tools like Terraform console, validate, and fmt, and being mindful of common pitfalls, you can reduce errors and improve collaboration across teams.