Simplify Terraform By Generating Configurations

Terraform is an awesome tool. To make it more awesome though we have wrapped it with some custom Ruby ERB templating to generate our terraform configurations from Yaml configurations.

Terraform uses a declarative language. You describe the state you want and it figures out how to get there. The declarative nature of Terraform does not afford us the same control that a language like Ruby can provide, which is fine, but I have found that I end up managing _massive_ Terraform configurations. It's easy to make mistakes. Generating our Terraform configurations allows us to create more robust services in a shorter amount of time. It's much easier to edit an 80 line yaml file than a 5000 line Terraform configuration.

We have a script, let's call it "Terraform Generator", or `tg`. We pass to `tg` an environment configuration like `develop`.

tg develop

This will generate the terraform configuration `terraform/develop/main.yml` from a yaml file at `configs/develop.yml`

The code to generate the file is pretty simple. By running the generated plan through `terraform fmt` we can also do some validation to ensure it's not broken! Simply put we read in the environments yaml file, parse an erb template with the content from that yaml file, output a terraform configuration file, then validate that file.

`tg` does basically the following:

def output_terraform
    renderer = ERB.new(File.open("templates/#{@tf_template}").read)
    rendered = renderer.result(@context)

    out_file = File.open("#{@output_directory}/main.tf", 'w')
    out_file.puts(rendered)

    format_terraform
  end

  def format_terraform
    system "terraform fmt #{@output_directory}"
  end

  def create_output_directory
    directory = "terraform/#{@config['config_src']}"

    unless File.directory?(directory)
      FileUtils.mkdir_p(directory)
    end

    directory
  end

  def main(*argv)
    config_src = argv.last

    @config           = YAML.load_file("configs/#{config_src}.yaml")
    @tf_template      = 'main.tf.erb'
    @output_directory = create_output_directory
    @context          = binding

    output_terraform
  end

  main(*ARGV)

The yaml file may look something like:

version: 1

region: us-west-1

tfstate: develop

cluster:
  name: DEVELOP

load_balancers:
  frontend:
    ssl_certs:
      - example.com
  backend:
    ssl_certs:
      - example.com

ssl_certs:
  default:
    domain_name: 'example.com'
    subject_alternative_names:
      - '*.example.com'

services:

  my_service:
    task_count: 2
    cpu: 256
    memory: 512
    https:
      load_balancer: frontend
      url: my_service.example.com
      health_check:
        path: /
    autoscaling:
      min: 2
      max: 8
      cpu: 75 

  my_other_service:
    task_count: 2
    cpu: 256
    memory: 512
    https:
      load_balancer: backend
      url: my_other_service.example.com
      health_check:
        path: /
    autoscaling:
      min: 2
      max: 4
      cpu: 60

The ERB template:

terraform {
  required_version = ">= 0.14.5"

  backend "s3" {
    bucket = "terraform-state"
    key    = "<%= @config['tfstate'] %>/terraform.tfstate"
    region = "<%= @config['region'] %>"

    # Force encryption
    encrypt = true
  }

  required_providers {
    aws = {
      version = "~> 3.24.1"
    }
  }
}

provider "aws" {
  region = "<%= @config['region'] %>"
}
<% @config['services'].each do |service, values| %>
   ...
<% end %>

... etc ...

Related Posts