Carl Christian Sautter

Terraform is one of the top IaC (Infrastructure as Code) tools. With Terraform, developers can create, change and version infrastructure securely and efficiently. The desired state of a cloud-based or local infrastructure is described in the declarative configuration language HCL (HashiCorp Configuration Language). Terraform can then use this to create a plan to provide the infrastructure. Terraform has many advantages but has limitations in certain areas. Terragrunt extends Terraform by making it easier to reuse and organize Terraform configurations. It provides a unified structure for projects with multiple environments and improves the handling of remote states and modules. Terragrunt can also automatically resolve dependencies between multiple modules.

├── environments
│   ├── dev
│   │   ├── main.tf
│   │   ├── outputs.tf
│   │   ├── provider.tf
│   │   └── variables.tf
│   └── prod
│       ├── main.tf
│       ├── outputs.tf
│       ├── provider.tf
│       └── variables.tf
└── modules
   ├── ec2
   │   ├── main.tf
   │   ├── outputs.tf
   │   └── variables.tf
   └── network
       ├── main.tf
       ├── outputs.tf
       └── variables.tf

|── live
|    ├── terragrunt.hcl
|    ├── _env
|    │   ├── ec2.hcl
|    │   ├── network.hcl
|    │   └── provider.hcl
|    ├── prod
|    │   ├── env.hcl
|    │   ├── app
|    │   │   └── terragrunt.hcl
|    │   └── mysql
|    │       └── terragrunt.hcl
|    └── dev
|        ├── env.hcl
|        ├── ec2
|        │   └── terragrunt.hcl
|        └── network
|            └── terragrunt.hcl
└── modules
   ├── ec2
   │   ├── main.tf
   │   ├── outputs.tf
   │   └── variables.tf
   └── network
       ├── main.tf
       ├── outputs.tf
       └── variables.tf

Disadvantages of Terraform

reusability and modularity

Terraform offers the possibility to organize code in modules for reuse. However, managing and using many different modules in complex infrastructure environments with multiple stages, such as Dev, Test and Prod, becomes challenging.

State File Management

Terraform stores the current status of the entire infrastructure in a common state file. In environments with multiple modules and teams, a central state file can cause problems. If multiple team members are working on different modules, race conditions can easily arise in which the state file is temporarily locked.

Code duplication and configuration of different environments

Terraform does not have native support for automatic configuration across environments (e.g. dev, staging, production). The developer has to build complex solutions with variables or other mechanisms themselves. When working in multiple environments, one tends to duplicate configurations because Terraform does not provide an out-of-the-box solution for dealing with multiple environments. This leads to redundant configurations and makes maintenance difficult.

Advantages of Terragrunt as a wrapper

Terragrunt is a wrapper around Terraform and addresses some of these weaknesses.

Management of environments and DRY principle

Terragrunt supports better structuring of Terraform code by promoting the DRY (“Don’t Repeat Yourself”) principle. Instead of duplicating configurations for different environments, Terragrunt provides mechanisms to efficiently manage variables and infrastructure for different environments. It simplifies the handling of shared configurations by using a single configuration file and only overriding environment-specific values.

Docs: https://terragrunt.gruntwork.io/docs/features/keep-your-terraform-code-dry/

Dependency Management

In complex Terraform setups, there are often dependencies between different modules (e.g. a network needs to be deployed before a database). In Terragrunt, dependencies between modules can be explicitly defined to automatically execute them in the correct order.

Docs: https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#dependency

Automated state management and provider configuration

Instead of duplicating the same state backend and provider configuration in each module, Terragrunt can inject this configuration automatically. This reduces redundant code in Terraform configuration files and simplifies administration.

State Backend Configuration

Terragrunt supports automatic management of state files for different modules and environments. Terragrunt can isolate state files per environment and module, for example by automatically storing them in different S3 buckets (for AWS) or integrating them into suitable storage solutions such as Gitlab Remote State Backend.

The following example defines a template for the Gitlab HTTP backend.

locals {
  // read the env vars
  env_locals        = read_terragrunt_config(find_in_parent_folders("${get_original_terragrunt_dir()}/../env.hcl"))
  // evaluate the gitlab url
  gitlab_url_eval   = can(local.env_locals.locals.gitlab_url) ? local.env_locals.locals.gitlab_url : "https://gitlab.com"
  gitlab_project_id = local.env_locals.locals.gitlab_project_id
}

remote_state {
  backend = "http"
  generate = {
    path      = "backend_http_gitlab.tf"
    if_exists = "overwrite_terragrunt"
  }
  config = {
    address        = "${local.gitlab_url_eval}/api/v4/projects/${local.env_locals.locals.gitlab_project_id}/terraform/state/${local.env_locals.locals.env}_${basename(get_original_terragrunt_dir())}"
    lock_address   = "${local.gitlab_url_eval}/api/v4/projects/${local.env_locals.locals.gitlab_project_id}/terraform/state/${local.env_locals.locals.env}_${basename(get_original_terragrunt_dir())}/lock"
    unlock_address = "${local.gitlab_url_eval}/api/v4/projects/${local.env_locals.locals.gitlab_project_id}/terraform/state/${local.env_locals.locals.env}_${basename(get_original_terragrunt_dir())}/lock"
    lock_method    = "POST"
    unlock_method  = "DELETE"
    retry_wait_min = 5
    username       = "${get_env("TF_HTTP_USERNAME")}"
    // the env variable TF_HTTP_PASSWORD is evaluated at runtime, i do not recommend to save it for security reasons
  }
}

Code: https://github.com/csautter/terragrunt-blueprint/blob/main/deployments/terraform/backend_http_gitlab.hcl

Docs: https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#remote_state

The root terragrunt.hcl file loads the Terraform backend defined in env.hcl.

locals {
  env_locals                = read_terragrunt_config(find_in_parent_folders("env.hcl"))
  tflint_hook_enabled       = get_env("DISABLE_TFLINT_HOOK", "false") == "true" ? false : true
  trivy_hook_enabled        = get_env("DISABLE_TRIVY_HOOK", "false") == "true" ? false : true
  backend_target_evaluation = can(local.env_locals.locals.state_backend) ? local.env_locals.locals.state_backend : "local"
  backend_target            = contains(["http_gitlab", "local"], local.backend_target_evaluation) ? local.backend_target_evaluation : "local"
  backend_config            = read_terragrunt_config(find_in_parent_folders("backend_${local.backend_target}.hcl"))
}

remote_state {
  backend  = local.backend_config.remote_state.backend
  config   = local.backend_config.remote_state.config
  generate = local.backend_config.remote_state.generate
}

Code: https://github.com/csautter/terragrunt-blueprint/blob/main/deployments/terraform/terragrunt.hcl

provider configuration

Terragrunt enables simplified provider configurations in different environments and modules. The provider configurations can be generated automatically depending on the current environment.
In the example of provider_aws_config.hcl, an AWS provider configuration is generated. All parameters are loaded from the env.hcl file of the respective environment (dev, staging, prod).

locals {
  env = read_terragrunt_config(find_in_parent_folders("env.hcl"))
}

generate "aws" {
  path      = "provider_aws.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<-EOF
     provider "aws" {
        default_tags {
          tags = {
            Environment = "${local.env.locals.env}"
          }
        }
        region  = "${local.env.locals.region}"
        profile = "${get_env("AWS_PROFILE", "${local.env.locals.aws_profile}")}"
        allowed_account_ids = ["${get_env("AWS_ACCOUNT_ID", "${local.env.locals.aws_account_id}")}"]
     }
  EOF
}

Code: https://github.com/csautter/terragrunt-blueprint/blob/main/deployments/terraform/env/_env/provider_aws_config.hcl

The AWS provider configuration can then be loaded into the modules using an include block.

include "provider_vault_config" {
  path   = "${get_terragrunt_dir()}/../../_env/provider_aws_config.hcl"
  expose = true
}

Code: https://github.com/csautter/terragrunt-blueprint/blob/main/deployments/terraform/env/dev/aws_dummy/terragrunt.hcl

Docs: https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#include

hooks

In Terragrunt, hooks can be defined that trigger trivy, tflint or terraform fmt every time terraform plan/apply is executed. The immediate feedback during development can significantly increase code quality.

terraform {
  before_hook "terraform_fmt" {
    commands = ["apply", "plan"]
    execute  = ["terraform", "fmt", "-recursive"]
  }
  before_hook "terragrunt_hclfmt" {
    commands = ["apply", "plan"]
    execute  = ["terragrunt", "hclfmt"]
  }
  before_hook "tflint" {
    commands = local.tflint_hook_enabled ? ["apply", "plan"] : []
    execute  = ["tflint"]
  }
  before_hook "trivy" {
    commands = local.trivy_hook_enabled ? ["apply", "plan"] : []
    execute  = ["trivy", "config", "."]
  }
}

Code: https://github.com/csautter/terragrunt-blueprint/blob/main/deployments/terraform/terragrunt.hcl

Docs: https://terragrunt.gruntwork.io/docs/features/hooks

Conclusion

Terraform is a very powerful tool for managing infrastructure, but it has limitations when it comes to handling state files, modularity, and managing environments. Terragrunt was developed to address these weaknesses, particularly through improvements in reusability, state management, and dependency handling. By using Terragrunt in my projects, I was able to significantly increase code quality and reduce complexity.