Terraform

This article is about utilising a tool from HashiCorp called Terraform to build infrastructure as a code. Meaning to spin up AWS instances, setup security groups, VPC and any other cloud based infrastructure component.

Install terraform

wget https://releases.hashicorp.com/terraform/0.9.1/terraform_0.9.1_linux_amd64.zip
unzip terraform_0.9.1_linux_amd64.zip
sudo mv ./terraform /usr/local/bin

When terraform is run it looks for .tf file where configuration is stored. The look up process is limited to a flat directory and never leaves the directory that runs from. Therefore if you wish to address a common file a symbolic-link needs to be created within the directory you have .tf file.

vi example.tf 
provider "aws" {
	access_key = "AK01234567890OGD6WGA" 
	secret_key = "N8012345678905acCY6XIc1bYjsvvlXHUXMaxOzN"
	region = "eu-west-1"
}
resource "aws_instance" "webserver" {
	ami = "ami-405f7226"
	instance_type = "t2.nano"
}

The example above will create an Ubuntu instance.

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
<...>

+ aws_instance.webserver
   ami:                         "ami-405f7226"
   associate_public_ip_address: "<computed>"
   availability_zone:           "<computed>"
   ebs_block_device.#:          "<computed>"
   ephemeral_block_device.#:    "<computed>"
   instance_state:              "<computed>"
   instance_type:               "t2.nano"
   ipv6_addresses.#:            "<computed>"
   key_name:                    "<computed>"
   network_interface_id:        "<computed>"
   placement_group:             "<computed>"
   private_dns:                 "<computed>"
   private_ip:                  "<computed>"
   public_dns:                  "<computed>"
   public_ip:                   "<computed>"
   root_block_device.#:         "<computed>"
   security_groups.#:           "<computed>"
   source_dest_check:           "true"
   subnet_id:                   "<computed>"
   tenancy:                     "<computed>"
   vpc_security_group_ids.#:    "<computed>"

Apply stage, if runs first time will create terraform.tfstate after all changes are done. This file should not be modified manually. It's used to compare what is out in cloud already so the next time APPLY stage runs it will look at the file and execute only necessary changes.

$ terraform apply
aws_instance.webserver: Creating...
 ami:                         "" => "ami-405f7226"
 associate_public_ip_address: "" => "<computed>"
 availability_zone:           "" => "<computed>"
 ebs_block_device.#:          "" => "<computed>"
 ephemeral_block_device.#:    "" => "<computed>"
 instance_state:              "" => "<computed>"
 instance_type:               "" => "t2.nano"
 ipv6_addresses.#:            "" => "<computed>"
 key_name:                    "" => "<computed>"
 network_interface_id:        "" => "<computed>"
 placement_group:             "" => "<computed>"
 private_dns:                 "" => "<computed>"
 private_ip:                  "" => "<computed>"
 public_dns:                  "" => "<computed>"
 public_ip:                   "" => "<computed>"
 root_block_device.#:         "" => "<computed>"
 security_groups.#:           "" => "<computed>"
 source_dest_check:           "" => "true"
 subnet_id:                   "" => "<computed>"
 tenancy:                     "" => "<computed>"
 vpc_security_group_ids.#:    "" => "<computed>"
aws_instance.webserver: Still creating... (10s elapsed)
aws_instance.webserver: Creation complete (ID: i-0eb33af34b94d1a78)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: 

$ terraform show
aws_instance.webserver:
 id = i-0eb33af34b94d1a78
 ami = ami-405f7226
 associate_public_ip_address = true
 availability_zone = eu-west-1c
 disable_api_termination = false
 ebs_block_device.# = 0
 ebs_optimized = false
 ephemeral_block_device.# = 0
 iam_instance_profile = 
 instance_state = running
 instance_type = t2.nano
 ipv6_address_count = 0
 ipv6_addresses.# = 0
 key_name = 
 monitoring = false
 network_interface_id = eni-2c6b1553
 private_dns = ip-172-31-7-109.eu-west-1.compute.internal
 private_ip = 172.31.7.109
 public_dns = ec2-34-249-29-115.eu-west-1.compute.amazonaws.com
 public_ip = 34.249.29.115
 root_block_device.# = 1
 root_block_device.0.delete_on_termination = true
 root_block_device.0.iops = 100
 root_block_device.0.volume_size = 8
 root_block_device.0.volume_type = gp2
 security_groups.# = 0
 source_dest_check = true
 subnet_id = subnet-92a4bbf6
 tags.% = 0
 tenancy = default
 vpc_security_group_ids.# = 1
 vpc_security_group_ids.1039819662 = sg-5201fb2b

$ terraform destroy
Do you really want to destroy?
 Terraform will delete all your managed infrastructure.
 There is no undo. Only 'yes' will be accepted to confirm.
 Enter a value: yes
aws_instance.webserver: Refreshing state... (ID: i-0eb33af34b94d1a78)
aws_instance.webserver: Destroying... (ID: i-0eb33af34b94d1a78)
aws_instance.webserver: Still destroying... (ID: i-0eb33af34b94d1a78, 10s elapsed)
aws_instance.webserver: Still destroying... (ID: i-0eb33af34b94d1a78, 20s elapsed)
aws_instance.webserver: Still destroying... (ID: i-0eb33af34b94d1a78, 30s elapsed)
aws_instance.webserver: Destruction complete

Destroy complete! Resources: 1 destroyed.

After the instance has been terminated the terraform.tfstate looks like below:

vi terraform.tfstate
{
    "version": 3,
    "terraform_version": "0.9.1",
    "serial": 1,
    "lineage": "c22ccad7-ff26-4b8a-bf19-819477b45202",
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {},
            "depends_on": []
        }
    ]
}

AWS credentials profiles and variable files

Instead to reference secret_access keys within .tf file directly we can use AWS profile file. This file will be look at for the profile variable we specify in variables.tf file.

$ vi ~/.aws/credentials    #create AWS profile file
[terraform-profile1]       #profile name
aws_access_key_id = AK01234567890OGD6WGA
aws_secret_access_key = N8012345678905acCY6XIc1bYjsvvlXHUXMaxOzN

Then we can now remove the secret_access keys from the main .tf file (example.tf) and amend as follows:

provider "aws" {                provider "aws" { 
  access_key = "AK012345678 =>    profile = "${var.profile}"
  secret_key = "N8012345678       region  = "${var.region }"
  region = "eu-west-1"

and create a variable file to reference it

$ vi variables.tf
variable "region" {
        default = "eu-west-1"
}
variable "profile" {}     #variable without a default value will prompt to type in the value. And that should be 'terraform-profile1'

Run terraform

$ terraform   #it it will prompt for a value
$ terraform plan -var 'profile=terraform-profile1'  #this way value can be set

Example

Prerequisites are:

• ~/.aws/credential file exists
• variables.tf exist, with context below:

$ vi variables.tf
variable "region"          { default = "eu-west-1"                                   }
variable "profile"         { description = "AWS credentials profile you want to use" }
variable "key_name"        { description = "Name of the AWS key pair"                }
variable "public_key_path" { description = <<DESCRIPTION
Path to the SSH public keys for authentication.
Example: ~./ssh/terraform.pub
DESCRIPTION }

Terraform .tf file

$ vi example.tf
provider "aws" {
  region = "${var.region}"
  profile = "${var.profile}"
}
resource "aws_vpc" "vpc" {
  cidr_block = "10.0.0.0/16"
}
# Create an internet gateway to give our subnet access to the open internet
resource "aws_internet_gateway" "internet-gateway" {
  vpc_id = "${aws_vpc.vpc.id}"
}
# Give the VPC internet access on its main route table
resource "aws_route" "internet_access" {
  route_table_id         = "${aws_vpc.vpc.main_route_table_id}"
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = "${aws_internet_gateway.internet-gateway.id}"
}
# Create a subnet to launch our instances into
resource "aws_subnet" "default" {
  vpc_id                  = "${aws_vpc.vpc.id}"
  cidr_block              = "10.0.1.0/24"
  map_public_ip_on_launch = true

  tags {
    Name = "Public"
  }
}
# Our default security group to access
# instances over SSH and HTTP
resource "aws_security_group" "default" {
  name        = "terraform_securitygroup"
  description = "Used for public instances"
  vpc_id      = "${aws_vpc.vpc.id}"

  # SSH access from anywhere
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  # HTTP access from the VPC
  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/16"]
  }
  # outbound internet access
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1" # all protocols
    cidr_blocks = ["0.0.0.0/0"]
  }
}
resource "aws_key_pair" "auth" {
  key_name   = "${var.key_name}"
  public_key = "${file(var.public_key_path)}"
}
resource "aws_instance" "webserver" {
  ami = "ami-405f7226"
  instance_type = "t2.nano"

  key_name = "${aws_key_pair.auth.id}"
  vpc_security_group_ids = ["${aws_security_group.default.id}"]

  # We're going to launch into the public subnet for this.
  # Normally, in production environments, webservers would be in
  # private subnets.
  subnet_id = "${aws_subnet.default.id}"

  # The connection block tells our provisioner how to
  # communicate with the instance
  connection {
    user = "ubuntu"
  }
  # We run a remote provisioner on the instance after creating it 
  # to install Nginx. By default, this should be on port 80
  provisioner "remote-exec" {
    inline = [
      "sudo apt-get -y update",
      "sudo apt-get -y install nginx",
      "sudo service nginx start"
    ]
  }
}

Run a plan

$ terraform plan
var.key_name
  Name of the AWS key pair

  Enter a value: id_rsa        #name of the key_pair

var.profile
  AWS credentials profile you want to use

  Enter a value: terraform-profile   #aws profile in ~/.aws/credentials file

var.public_key_path
  Path to the SSH public keys for authentication.
  Example: ~./ssh/terraform.pub

  Enter a value: ~/.ssh/id_rsa.pub  #path to the matching public key

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed. Cyan entries are data sources to be read.

+ aws_instance.webserver
    ami:                         "ami-405f7226"
    associate_public_ip_address: "<computed>"
    availability_zone:           "<computed>"
    ebs_block_device.#:          "<computed>"
    ephemeral_block_device.#:    "<computed>"
    instance_state:              "<computed>"
    instance_type:               "t2.nano"
    ipv6_addresses.#:            "<computed>"
    key_name:                    "${aws_key_pair.auth.id}"
    network_interface_id:        "<computed>"
    placement_group:             "<computed>"
    private_dns:                 "<computed>"
    private_ip:                  "<computed>"
    public_dns:                  "<computed>"
    public_ip:                   "<computed>"
    root_block_device.#:         "<computed>"
    security_groups.#:           "<computed>"
    source_dest_check:           "true"
    subnet_id:                   "${aws_subnet.default.id}"
    tenancy:                     "<computed>"
    vpc_security_group_ids.#:    "<computed>"

+ aws_internet_gateway.internet-gateway
    vpc_id: "${aws_vpc.vpc.id}"

+ aws_key_pair.auth
    fingerprint: "<computed>"
    key_name:    "id_rsa"
    public_key:  "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfc piotr@ubuntu"

+ aws_route.internet_access
    destination_cidr_block:     "0.0.0.0/0"
    destination_prefix_list_id: "<computed>"
    egress_only_gateway_id:     "<computed>"
    gateway_id:                 "${aws_internet_gateway.internet-gateway.id}"
    instance_id:                "<computed>"
    instance_owner_id:          "<computed>"
    nat_gateway_id:             "<computed>"
    network_interface_id:       "<computed>"
    origin:                     "<computed>"
    route_table_id:             "${aws_vpc.vpc.main_route_table_id}"
    state:                      "<computed>"

+ aws_security_group.default
    description:                           "Used for public instances"
    egress.#:                              "1"
    egress.482069346.cidr_blocks.#:        "1"
    egress.482069346.cidr_blocks.0:        "0.0.0.0/0"
    egress.482069346.from_port:            "0"
    egress.482069346.ipv6_cidr_blocks.#:   "0"
    egress.482069346.prefix_list_ids.#:    "0"
    egress.482069346.protocol:             "-1"
    egress.482069346.security_groups.#:    "0"
    egress.482069346.self:                 "false"
    egress.482069346.to_port:              "0"
    ingress.#:                             "2"
    ingress.2165049311.cidr_blocks.#:      "1"
    ingress.2165049311.cidr_blocks.0:      "10.0.0.0/16"
    ingress.2165049311.from_port:          "80"
    ingress.2165049311.ipv6_cidr_blocks.#: "0"
    ingress.2165049311.protocol:           "tcp"
    ingress.2165049311.security_groups.#:  "0"
    ingress.2165049311.self:               "false"
    ingress.2165049311.to_port:            "80"
    ingress.2541437006.cidr_blocks.#:      "1"
    ingress.2541437006.cidr_blocks.0:      "0.0.0.0/0"
    ingress.2541437006.from_port:          "22"
    ingress.2541437006.ipv6_cidr_blocks.#: "0"
    ingress.2541437006.protocol:           "tcp"
    ingress.2541437006.security_groups.#:  "0"
    ingress.2541437006.self:               "false"
    ingress.2541437006.to_port:            "22"
    name:                                  "terraform_securitygroup"
    owner_id:                              "<computed>"
    vpc_id:                                "${aws_vpc.vpc.id}"

+ aws_subnet.default
    assign_ipv6_address_on_creation: "false"
    availability_zone:               "<computed>"
    cidr_block:                      "10.0.1.0/24"
    ipv6_cidr_block_association_id:  "<computed>"
    map_public_ip_on_launch:         "true"
    tags.%:                          "1"
    tags.Name:                       "Public"
    vpc_id:                          "${aws_vpc.vpc.id}"

+ aws_vpc.vpc
    assign_generated_ipv6_cidr_block: "false"
    cidr_block:                       "10.0.0.0/16"
    default_network_acl_id:           "<computed>"
    default_route_table_id:           "<computed>"
    default_security_group_id:        "<computed>"
    dhcp_options_id:                  "<computed>"
    enable_classiclink:               "<computed>"
    enable_dns_hostnames:             "<computed>"
    enable_dns_support:               "true"
    instance_tenancy:                 "<computed>"
    ipv6_association_id:              "<computed>"
    ipv6_cidr_block:                  "<computed>"
    main_route_table_id:              "<computed>"


Plan: 7 to add, 0 to change, 0 to destroy.