Terraform HashiCorp certification - Infrastructure as Code

The content here is under the Attribution 4.0 International (CC BY 4.0) license

TLDR

Infrastructure as code (IaC) is a growing trend as DevOps became the standard for delivery applications. This content uses the Hashicorp preparation course available on Udemy as a guide to share the takeaways and the learnings for you to ace the certification.

Introduction

Infrastructure as code (IaC) is a growing trend as DevOps became the standard for delivery applications and measurement through a maturity model to scale the impact that technical practices have on business outcomes [1]. As such, cloud providers have their way of offering infrastructure, in the past, administrators had to manually execute commands on the servers and manually provide resources (which it’s the case until today).

Terraform provides a solution through code for managing infrastructure. It is a single place to talk to different providers and keep the changes repeatable and versioned, giving the confidence to change the infrastructure of any application. The idea is to automate as much as possible and you do not manage your infrastructure clicking around the interface, but through code [2].

This post is a companion for my studies on the terraform certification course [3]. The code samples created on this post can be found on GitHub.

Overview before start

From the start, the course goes through terraform basics (how to install and execute commands) to a more advanced style on how terraform handles state and how to manage different resources. The course focuses on the certification and not so much on the small details (such as navigating the official documentation). The instructor shows that he knows what he is presenting daily, which I see as positive.

Besides that, the course focuses on virtual machines (VMs) across different cloud providers, most of the examples are around VMs and not that much about different services for each vendor. This strategy is understandable as it would take much longer for a course to go over different services across vendors. On the other hand, VMs are the basis for any vendor and they give the chance to go over subjects like networking, authentication, resource tiers, etc.

Points of attention

The course claims to be 2021-ready, therefore, the version presented is outdated. Currently (at the time of writing this post), terraform is on version 1.0.* and the course uses version 0.13.. Such a difference makes commands like the *refresh not the same in both versions (As I noted later in the section about state).

Lastly, the sound quality varies depending on which lecture you are in. Sometimes too loud, sometimes too low and the background noise is something that took my attention out of the content being presented.

What is terraform

• Terraform is open-source
• Terraform is a human readable format, no need to be a programmer (Hashicorp Configuration Language or JSON)
• Terraform dilemma: One syntax, One workflow any infrastructure
• Terraform is an infrastructure provisioning tool and not a configuration management tool

your *.tf/*.tf.json       -->       desired configuration      -->    Cloud providers
files                                    terraform                  (aws, azure, google)

Terraform uses a plan to manage execution flow, it gives the admin a repeatable flow and confidence to manage infrastructure. Following this section another way of consuming the basics of terraform is to watch this YouTube video that gives an introduction in fifteen minutes.

Benefits of terraform

• Terraform is declarative. You declare the desired state and Terraform takes what is needed to reach the goal.
• Terraform is immutable
• Terraform is idempotent

Getting started with Terraform

It is recommended to create a user just for Terraform with access to manage resources on the cloud providers. For example, in AWS, the terraform user can be an administrator with programmatic access only.

Terraform under the hood

• Terraform is plugin architecture based (Micro kernel as per [4])
• Terraform uses RPC to call plugins (providers) and then each provider calls upstream APIs

Core

Terraform core manages the workflow for terraform (plan and apply), it’s a single binary written in go language and it’s available for the major operating systems.

               STATE

              ________
TF CONFIG --> | CORE | -->  PLAN/APPLY
              |______|

               GRAPH

Plugin

Manages communication with infrastructure providers, for creating, deleting, updating or deleting a resource.

Syntax

# this is a comment

block_type "label" "label" {
  "key" = "value" # value can be a hard coded value, or a expression (reference to some resource)
}

Nested blocks

block_type "label" "label" {
  "key" = "value"

  block_type "label" "label" {
    "key" = "value"
  }
}

• Terraform uses arguments and attributes to create a configuration for resources.
• Arguments assigns a value to a name
• Attributes are piece of data associated with a resource such as: name, address, it depends on the provider

Expressions

Expressions are used to compute a value that will be available in the future.

resource "resource_name" "my_custom_label_for_the_resource" {
  "key" = "value"
}

Resources are made available via terraform itself or contributed by the community. Those resources can be found under hashicorp providers page, each cloud vendor will provide different sets of resources.

Providers

Terraform providers are unique, the following code is not valid:

provider "aws" {
  region = "us-east-1"
}

provider "aws" {
  region = "us-east-2"
}

Defining multiple providers, can be achieved using alias:

provider "aws" {
  region = "us-east-1"
}

provider "aws" {
  alias = "myprovider"  # <-- now it is valid to be used
  region = "us-east-2"
}

State

Terraform uses a state file (terraform.tfstate) to compare the real world state and what the desired state is. This links back to the idempotent, having such a file allows terraform to not destroy and recreate all the infrastructure all the time.

• state and backup are automatically created in json format
• there is a parameter to pass in a custom state using the flag -state
• by default state file is stored locally, but it can be saved remotely
• state file is not encrypted at rest

Terraform offers a command to evaluate the real world state and update the local state to reflect that.

terraform apply -refresh-only

Note: previously terraform had the command terraform refresh, but as the documentation states, it has been deprecated since version v0.15.4 [5].

Takeaways to consider:

1. Do not edit terraform state manually
2. State file does not provision anything, it is used to analyse the real world state
3. terraform show can be used to see the current state (a better json visualization)

Resources

Resources are one of the most important concepts in terraform, resource configuration block are your services that you want to use in the cloud. For example, in AWS it could be a EC2 instance, in azure it could be an App Services or in Google it could be Virtual Machine instances.

Resource syntax:

resource_type "local_name" {
  key = "value" # or an expression
}

Example of AWS resource:

resource "aws_instance" "myvirtualmachine" {
  ami = "ami-*******"
  instance_type= "t2.small"
}

Resources also have meta-arguments that can be used when provisioning the resource, a few of them are listed below.

• provider
• count
• depends_on
• for_each
• life cycle
• provisioner and connection

Provisioners

Provisioners adds the possibility of performing some actions after a resource has been created, for example, executing scripts or any command. Terraform advises to use provisioners as a last resort.

• local-exec (executes commands on the local system)
• remote-exec (executes commands on the remote target after the resource is created by terraform)
• file

There is no rollback in terraform, if something goes wrong when provisioning a resource, it might be on the cloud provider but the state will be not updated, terraform marks the resource as taint (taint = mark it for deletion on the next apply). It also happens if you add things manually in the infrastructure and then run terraform.

local-exec

resource "aws_instance" "my_vm" {
  provisioner "local-exec" {
    command = "echo ${self.id} >> myvmid"
  }
}

remote-exec

Two types of connection supported:

• ssh
• winrm

resource "aws_instance" "my_vm" {
  provisioner "remove-exec" {
    command = ""
  }
}

Import infrastructure to terraform

Importing infrastructure allow you to fetch resources already created in the cloud under terraform control.

terraform import resource_type.name id

The workflow to import a resource from the cloud to terraform requires extra steps in order to version the resources, the flow is as follows:

1. Create the resource on terraform file
2. Go to the created resource in the cloud provider
3. Update the terraform file with the id’s found for each resource

For a virtual machine for example in AWS, a valid import terraform file would be as follows:

resource "aws_instance" "imported_vm" {
  ami = "ami-xxxxxxxx"
  instance_type = "t2.small"
  tags {
    Name = "imported VM"
  }
}

Importing such resource, would require to run the following command:

terraform import aws_instance.imported_vm id-xxxxxxxx

Terraform console

Terraform console is a interactive shell to interact with the infrastructure.

terraform console

Terraform workflow

• How terraform manages life cycle of a resource
• All the steps from code to resource creation

write code --> plan --> apply --> destroy -> repeat

Init

Terraform init download plugins and save to .terraform/plugins.

Validate

Terraform validate verifies your current files.

Plan

Terraform plan creates a plan to execute the changes based on the real world state and the desired configuration

• -out save the plan on a file, good for auditing.

Variables

Terraform offer variables to be used on the execution flow, there are three possible places to define them:

1. Command line
2. File (terraform.tfvars)
3. Environment variables (it must start with TF_VAR), this kind of variable fits to be used with secrets

The precedence that terraform reads the file is:

• Terraform scan for a file called terraform.tfvars and tries to match the variables in the file
• If none is found terraform uses the env variable
• If no value is provided id uses the default

Variables can’t have the following names:

• Source
• Count
• For_each
• Version
• Provider
• Life cycles
• Locals
• depends_on

outputs

Output values for any data related to a resource provisioned, it creates a implicit dependency.

As a best practice: define the output.tf file to declare the desired outputs from the resources.

Resources:

• Output Values

Data source

Data sources are accessed via resources by terraform. It is not recommended to:

• Refer to data in the terraform block that has not created yet
• Configure an explicit dependency with terraform data with depends_on

Functions

Terraform has built-in functions developed by terraform, it offers functionality to read a file, replace something in a string, operate on lists and many more.

Note: terraform does not support custom functions

resource "resource_type" "string_replace" {
  prop1 = replace("my value", "my", "your")
}

Condition

Terraform also supports conditional values as expressions to be evaluated. Therefore, it must be in line, otherwise it won’t work.

variable type {
  default = "free-tier"
}

resource "resource_type" "string_replace" {
  machine_type = var.type == "free-tier" ? "t2.small" : "t5.large" # ternary operator
}

Dependencies

Terraform has two different types of dependencies, the first is known as implicit. For example, one resource depends on the other and it is not declared.

Logging

Logging with Terraform is based on an environment variable named TF_LOG, it sets the desired log, the available options are:

• TRACE
• INFO
• WARN
• DEBUG
• ERROR

Note: INFO, WARN, DEBUG and ERROR are not reliable, terraform recommends to use TRACE instead.

Running terraform with log level enabled, is as follows:

export TF_LOG=TRACE
terraform apply

Another option would be:

TF_LOG=TRACE terraform apply

Note 2: if any unsupported log level is provided, terraform will default to TRACE

References

1. [1]M. Zarour, N. Alhammad, M. Alenezi, and K. Alsarayrah, “A research on DevOps maturity models,” Int. J. Recent Technol. Eng, vol. 8, no. 3, pp. 4854–4862, 2019.
2. [2]Y. Brikman, Terraform Up & Running - Writing Infrastructure as Code. O’Reilly Media, Inc, 2019.
3. [3]D. Houari, “Terraform Master Course | HashiCorp Certified Terraform 2021,” 2021 [Online]. Available at: . [Accessed: 07-Aug-2021]
4. [4]R. Mark, Software Architecture Patterns. O’Reilly Media, Inc, 2015.
5. [5]Terraform.io, “Command:refresh,” 2021 [Online]. Available at: https://www.terraform.io/docs/cli/commands/refresh.html. [Accessed: 11-Aug-2021]

Appendix

This section holds extra info while following the terraform course.

Terraform 1.0 announcement

Terraform announced its version 1.0 in Jun 2021 as recorded on a YouTube video, it depicts the product maturity and its journey along the ten years of history. Terraform also is focusing on the testing side of things, it was announced the official test research for Terraform.

• Official Hashicorp - terraform sample questions
• Testing HashiCorp Terraform
• Configure Terraform for Multiple Azure Subscriptions

Related subjects

• How We Use Terraform At Slack by Archie Gunasekara - last accessed Jan 05, 2023
• The Terraform AWS Provider – Authenticate Terraform to AWS
• terraform-playground

Changelog

• Nov 21, 2023 - Added “The Terraform AWS Provider to the related subjects”