Skip directly to search

Skip directly to content

 

11 Things I wish I knew before working with Terraform – part 2

 
 

Architecture | Julian Alarcon |
23 July 2019

This post has been created for people who plan to start using Terraform on a project, in the hope it may help to save some time by sharing some of my lessons learned. And yes, the title is true – I wish I had known most of these lessons before starting to work with Terraform. I have split 11 lessons across two posts – here is part 2.

5. DEFINING OUTPUTS


Outputs show the information needed after Terraform templates are deployed. They are also used within modules to export information.

output "instance_id" {
value = "${aws_instance.instance_ws.id}"
}


When used within modules, two outputs must be defined, one in the module and a similar one in the configuration files. These outputs need to be explicitly defined. Output information is stored in a Terraform state file and can be queried by other terraform templates.

It's recommended to define Outputs for resources even if you are not using them at the time. Check the resource and the outputs provided by the resource and choose wisely which information will be useful for your infrastructure when you are using this Terraform resource. By doing so, you will decrease the need to go back and edit your module and your resource because an output is required by a new resource that you are defining.

Also, as you may want to organize your files, you can save the outputs files in a specific file called outputs.tf.

6. DEFINE EVEN THE SMALLEST COMPONENTS


When you use Terraform there is a tendency to start by focusing on the larger components, meaning that sometimes you miss smaller components that can cause frustration and technical debt. During the creation of components, Terraform will use the default options of the provider that you are using, if these are not predefined. It's important to acknowledge the default components in use and define them in Terraform, as it's possible that you need them in the future and default options may be modified by your chosen providers with no notice, resulting in two different component sets or changes in the properties of the components.

Examples of these include the Route Tables, which are sometimes not a focus area at the beginning of a project, or Elastic Container Repositories, which are easy to define but not always top of mind.

resource "aws_ecr_repository" "repository" {
name = "name_of_repo"
}

7. TERRAFORM INTERPOLATIONS


The interpolation syntax is a powerful feature which allows you to reference variables, resource attributes, call functions, etc.

String variables -> ${var.foo}
Map variables -> ${var.amis["us-east-1"]}
List variables -> ${var.subnets[idx]}


When you need to retrieve data from a module output or from the state of particular resources, you can use the module. or the data. syntax to call the desired attributes.

# Getting information from a module
output "my_module_bar_value_from_module" {
value = ${module.my_module.bar}
}

# Getting information from a data source
resource "aws_instance" "web" {
ami = "${data.aws_ami.my_amy.id}"
instance_type = "t3.micro"
}


You can also use some arithmetic or logical operations with interpolation. In this snip of code, if the evaluation of var.something is true (1, true) the VPN resource will be included.:

resource "aws_instance" "vpn" {
count = "${var.something ? 1 : 0}"
}


You can find more information about the supported Interpolations in the Terraform documentation

8. ENVIRONMENT MANAGEMENT


Workspaces in Terraform


Initially, these were known in Terraform version 0.9 as Environments, but since version 0.10 onwards, Terraform has renamed this feature to Workspaces.

It is possible to define new workspaces, change workspaces or delete workspaces using the terraform workspace command.

$ terraform workspace -h
Usage: terraform workspace


Create, change and delete Terraform workspaces.

Subcommands:
delete Delete a workspace
list List Workspaces
new Create a new workspace
select Select a workspace
show Show the name of the current workspace

There are a couple of advantages to using Workspaces:

1. They are defined by Hashicorp, so it's possible that improved features could be developed in the future
2. They reduce the usage of code

However, Workspaces also present a couple of challenges:

1. They are still an early implementation
2. They are not yet supported by all backends
3. It is not clear at the time of deployment (terraform apply) which workspace will be used (terraform workspace show)

Folders structure


One simple and useful option is to define components inside folders by environment.

project-01
├── dev
├── clusters
└── ecs_cluster
├── service01
├── main.tf
├── outputs.tf
├── variables.tf
├── service02
├── service03
├── service04
├── database
├── main.tf
├── outputs.tf
└── variables.tf
├── elasticsearch
├── main.tf
├── outputs.tf
└── variables.tf
└── vpc
├── main.tf
├── outputs.tf
└── variables.tf
├── global
└── web_login
├── main.tf
├── outputs.tf
└── variables.tf
└── terraform_state
├── output.tf
├── variables.tf
└── main.tf
├── prod
├── clusters
├── …
├── qa
├── clusters
├── …
└── README.md

Advantages:

1. Clear definition of the environment being deployed (in the folder path)
2. Most commonly used and fail-proof option for public deployments
3. Terraform States can be defined for each environment folder with no issues
4. Specify the name of the outputs for each environment

Issues:

1. Duplicated code
2. An overwhelming number of folders for larger projects
3. Copying of code and replacing of core values is always needed

So, which one should you choose? The folder structure by environments is easier and simple to use and it's a current recommended way to split the different components of your infrastructure.

9. RECOMMENDED WORKFLOW FOR COMMANDS


The terraform command has multiple options, so I wanted to share this as a recommended workflow at the moment of deployment:

1. Download the modules and force the update

The command terraform init initialises the workspace, downloading the providers, modules and setting up the terraform state backend. But if a module is already downloaded, Terraform won't recognise that a new version of a module is available. With terraform get it is possible to download the modules, but it's recommended to use the -update option to force an update.

terraform get -update

2. Once that you have the latest modules, is necessary to initialise the Terraform workspace to start downloading the providers and modules (already completed through the first command) and initialise the terraform state backend

terraform init
It's possible to also use the -upgrade option to force the update of providers, plugins and modules.

3. Prior deployment, Terraform is able to define a plan for deployment (what will be created, modified or destroyed). This plan is useful to use in a Pipeline to check the changes before initialising the real deployment.

It's important to note that the plan option is not always 1:1 with the deployment. If a component is already deployed, or if there are not enough permissions provided, the plan will pass but the deployment could fail.

terraform plan

To simplify things, it's possible to run all in one bash line: terraform get -update && terraform init && terraform plan

4. When you reach the final deployment step, Terraform will create a plan and provide the option to respond with yes or no to deploy the desired architecture.

terraform apply

Terraform can't rollback after deployment. So, if an error appears in the deployment, the issue should be solved in that moment Also, is possible to destroy the deployment (terraform destroy), but it will destroy everything and not rollback the changes.

It's possible to specify the application of a specific change or to destroy a specific resource with the option -target.

10. CALLING EXTERNAL DATA


The use of data sources allows a Terraform configuration to build on information defined outside of Terraform, or defined by another separate Terraform configuration, including:

■ Data from a remote state, this is useful to call states from another terraform deployments>
data "terraform_remote_state" "vpc_state" {
backend = "s3"
config {
bucket = "ci-cd-terraform-state"
key = "vpc/terraform.tfstate"
region = "us-east-1"
}
}


■ Data from AWS or external systems>
data "aws_ami" "linux" {
most_recent = true

filter {
name = "name"
values = ["amzn2-ami-hvm-2.0.20180810-x86_64-gp2*"]
}

filter {
name = "virtualization-type"
values = ["hvm"]
}

owners = ["137112412989"]
}

11. NEW AND INTERESTING INFORMATION


Terratest


To test your code, you can use a great tool called Terratest. This tool uses Go's unit testing framework.

Terraform version 0.12


The next Terraform version was announced a few months ago and will improve some interpolation and bring some changes to the HCL language.

Julian Alarcon

DevOps Engineer

Julian is a DevOps engineer who loves open source software culture, sharing his knowledge and working on weird and wonderful projects that get him to think outside the box. He also enjoys learning from new cultures and tries to experience one new thing every day. With a technical background spanning almost 10 years, Julian and his team help bring big ideas to life. He is also a coffee lover from a coffee country, an amateur photographer and a great conversationalist, especially when beer is involved.

 

Categories

 

Related Articles

  • 23 July 2019

    11 Things I wish I knew before working with Terraform – part 2

  • 25 June 2019

    11 Things I wish I knew before working with Terraform – part 1

  • 30 May 2019

    Microservices and Serverless Computing

  • 14 May 2019

    Edge Services

  • 28 January 2019

    Internet Scale Architecture

 

From This Author

  • 25 June 2019

    11 Things I wish I knew before working with Terraform – part 1

Most Popular Articles

Kubernetes Design Principles Part 1
 

Architecture | Armen Kojekians | 30 April 2019

Kubernetes Design Principles Part 1

11 Things I wish I knew before working with Terraform – part 2
 

Architecture | Julian Alarcon | 23 July 2019

11 Things I wish I knew before working with Terraform – part 2

Creating A Visual Culture
 

Agile | Madalin Ilie | 03 September 2019

Creating A Visual Culture

Extracting Data from Images in Presentations
 

AI | Alexandru Mortan | 20 August 2019

Extracting Data from Images in Presentations

Cognitive Computing Using Cloud-Based Resources
 

AI | Radu Orghidan | 17 September 2019

Cognitive Computing Using Cloud-Based Resources

11 Things I wish I knew before working with Terraform – part 1
 

Architecture | Julian Alarcon | 25 June 2019

11 Things I wish I knew before working with Terraform – part 1

The Twisted Concept of Securing Kubernetes Clusters
 

Architecture | Vlad Calmic | 05 November 2019

The Twisted Concept of Securing Kubernetes Clusters

Microservices and Serverless Computing
 

Architecture | Radu Vunvulea | 30 May 2019

Microservices and Serverless Computing

Cognitive Computing Using Cloud-Based Resources II
 

AI | Radu Orghidan | 01 October 2019

Cognitive Computing Using Cloud-Based Resources II

 

Archive

  • 05 November 2019

    The Twisted Concept of Securing Kubernetes Clusters

  • 01 October 2019

    Cognitive Computing Using Cloud-Based Resources II

  • 17 September 2019

    Cognitive Computing Using Cloud-Based Resources

  • 03 September 2019

    Creating A Visual Culture

  • 20 August 2019

    Extracting Data from Images in Presentations

  • 06 August 2019

    Evaluating the current testing trends

  • 23 July 2019

    11 Things I wish I knew before working with Terraform – part 2

  • 12 July 2019

    The Rising Cost of Poor Software Security

  • 09 July 2019

    Developing your Product Owner mindset

  • 25 June 2019

    11 Things I wish I knew before working with Terraform – part 1

  • 30 May 2019

    Microservices and Serverless Computing

  • 14 May 2019

    Edge Services

  • 30 April 2019

    Kubernetes Design Principles Part 1

  • 09 April 2019

    Keeping Up With The Norm In An Era Of Software Defined Everything

  • 25 February 2019

    Infrastructure As Code With Terraform

  • 11 February 2019

    Distributed Agile – Closing the Gap Between the Product Owner and the Team

  • 28 January 2019

    Internet Scale Architecture

We are listening

How would you rate your experience with Endava so far?

We would appreciate talking to you about your feedback. Could you share with us your contact details?

 

By using this site you agree to the use of cookies for analytics, personalized content and ads. Learn More