AWS/CLI

From Ever changing code
Jump to navigation Jump to search

Install AWS cli (command line)

curl -O https://bootstrap.pypa.io/get-pip.py
python get-pip.py
pip install awscli

or

sudo apt-get install awscli  #it will update a lot of packages to Python3 but will leave 2.7 as default

Configure AWS cli credentials

When you run aws configure and enter credentials, they are stored in a file at ~/.aws/credentials. Additionally, some configuration settings—such as the default region—are stored at ~/.aws/config.

~/.aws/credentials vs ~/.aws/config
  • config - contains config, profile name is prefixed with word profile, eg: [profile <profile-name>]
  • credentials - contains credentials for a profile, the profile name is not prefixed, eg: [<profile-name>]. Note credentials can be stored in the config file as well.


$ aws configure                 #sets up default profile
$ aws configure --profile dev   #configure named 'dev' profile
$ aws configure --profile piotr #configure named 'piotr' profile
$ aws ec2 describe-regions      #to get a list all available regions

The /.aws/config profile sections must have the format of [profile profile-name], except for the default profile. For example:


Example ~/.aws/credentials file

[default]
aws_access_key_id=111xxxxxxxxxxxxxxxxx
aws_secret_access_key=111xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[profile dev]
aws_access_key_id=222xxxxxxxxxxxxxxxxx
aws_secret_access_key=222xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[profile piotr]
aws_access_key_id=333xxxxxxxxxxxxxxxxx
aws_secret_access_key=333xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

You can also use environment variables described here

Configure AWS cli with Cross-Account Roles & MFA

Let’s assume that we have 4 AWS accounts:

  • One identity-account (acc_no: 000xxxxxxxxx) where we provision IAM users, create their API Credentials and assign MFA -
    • user 'piotr' mfa_arn: arn:aws:iam::000xxxxxxxxx:mfa/user.name
  • 3 for our different environments
    • development (acc_no: 111xxxxxxxxx)
      • role: CrossAccountSignin_Admin: role_arn:aws:iam::111xxxxxxxxx:role/CrossAccountSignin_Admin
      • Trusted entities: acc_no: 000xxxxxxxxx (identity-account) and Conditions: aws:MultiFactorAuthPresent
    • staging (acc_no: 222xxxxxxxxx)
      • role: CrossAccountSignin_Admin: role_arn:aws:iam::222xxxxxxxxx:role/CrossAccountSignin_Admin
      • Trusted entities: acc_no: 000xxxxxxxxx (identity-account) and Conditions: aws:MultiFactorAuthPresent
    • production (acc_no: 333xxxxxxxxx)
      • role: CrossAccountSignin_Admin: role_arn:aws:iam::333xxxxxxxxx:role/CrossAccountSignin_Admin
      • Trusted entities: acc_no: 000xxxxxxxxx (identity-account) and Conditions: aws:MultiFactorAuthPresent

Let’s also assume that the proper roles and policies have already been put in place, allowing us to switch between accounts using the management console. We can create CLI profiles for each of these accounts.


Credentials file ~/.aws/credentials

[piotr-identity-account]
aws_access_key_id=333xxxxxxxxxxxxxxxxx
aws_secret_access_key=333xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


Config file ~/.aws/config

[profile piotr-identity-account]
region = eu-west-1

[profile development]
region = eu-west-1
source_profile = piotr-identity-account
role_arn   = arn:aws:iam::111xxxxxxxxx:role/CrossAccountSignin_Admin
mfa_serial = arn:aws:iam::000xxxxxxxxx:mfa/piotr

[profile staging]
region = eu-west-1
source_profile = piotr-identity-account
role_arn   = arn:aws:iam::222xxxxxxxxx:role/CrossAccountSignin_Admin
mfa_serial = arn:aws:iam::000xxxxxxxxx:mfa/piotr

[profile production]
region = eu-west-1
source_profile = piotr-identity-account
role_arn   = arn:aws:iam::333xxxxxxxxx:role/CrossAccountSignin_Admin
mfa_serial = arn:aws:iam::000xxxxxxxxx:mfa/piotr
Test
$ aws ec2 describe-instances --profile development
Enter MFA code for arn:aws:iam::000xxxxxxxxx:mfa/piotr: ***

By default the temporary API token is valid for 1 hour, once expired you will be prompted for MFA again.

Working with MFA

If you’re using the aws CLI, using MFA with a Credentials Profile is fairly straight forward. You just need to add the mfa_serial to your Config File in ~/.aws/config:

[profile with-mfa]
mfa_serial = arn:aws:iam::123456789012:mfa/jon-doe

The mfa_serial parameter should be set to the ARN of your MFA device, which you can get from the IAM User page of the AWS Web Console. It should be of the format:

arn:aws:iam::<ACCOUNT_ID>:mfa/<USERNAME>

Once you’ve added the mfa_serial parameter, the next time you run the aws CLI with that Named Profile, it will prompt you for an MFA token:

$ aws s3 ls --profile with-mfa
Enter MFA code:

But what if you’re running something that isn’t aws, such as terraform or packer. Most other tools will not automatically prompt for an MFA token. Instead, you have to do the MFA authentication process outside of that tool. First, you configure your Credentials File with your normal (permanent) AWS Access Keys (e.g. by running aws configure). Next, you run the aws sts get-session-token command, passing it the ARN of your MFA device and an MFA token from the Google Authenticator App or your key fob:

aws sts get-session-token \
  --serial-number    arn:aws:iam::123456789012:mfa/jon-doe \
  --token-code       123456 \
  --duration-seconds 43200

This will return a blob of JSON that contains Temporary Access Keys (note the --duration-seconds argument in the earlier command, which specifies when these Temporary Access Keys will expire):

{
  "Credentials": {
    "SecretAccessKey": "secret-access-key",
    "SessionToken": "temporary-session-token",
    "Expiration": "expiration-date-time",
    "AccessKeyId": "access-key-id"
  }
}

You will need to take these Temporary Access Keys and copy them into a Named Profile in your Credentials File in ~/.aws/credentials:

[profile-with-mfa]
aws_access_key_id     = <Access-key-as-in-returned-output>
aws_secret_access_key = <Secret-access-key-as-in-returned-output>
aws_session_token     = <Session-Token-as-in-returned-output>

Note that with Temporary Access Keys, you set not only aws_access_key_id and aws_secret_access_key in your Credentials File, but also aws_session_token.

Now you can run those other CLI tools with this Named Profile:

export AWS_PROFILE=profile-with-mfa
terraform apply

Note that the Temporary Access Keys expire after a certain period of time (typically 12 hours), so you’ll have to do this over again.


Other related AWS cli commands

aws sts get-caller-identity --profile dev          #info about you, this will create temp credentials
aws sts get-session-token --duration-seconds 28800 #get a session token by default 15 min, here 8h

https://github.com/605data/aws-scripts/tree/master/awscli-mfa https://random.ac/cess/2017/10/28/aws-cli-key-rotation-script-v2/ https://github.com/sweharris/aws-cli-mfa

Assuming role

aws sts assume-role --role-arn "arn:aws:iam::11111111111:role/CrossAccountAgent" --role-session-name "terraform" --profile agent-terraform

Tools making work with sts easier

aws-vault
Securely store your AWS credentials in your operating system’s keystore (e.g., Keychain, KWallet), Automatically set those credentials as environment variables when executing a command, Handle all the aws sts commands for you when using IAM Roles or MFA.
aws-auth
bash script Gruntwork subscribers only

Multi account aws cli execution loop

Helper programme to execute awscli commands across multiple AWS accounts (profiles) and regions.

#Create temp creds for all accounts protected by MFA tokens, run this without timeout command
$ # ./awscli-crossaccounts.sh aws sts get-caller-identity

#Run a command across accounts in different regions
$ # ./awscli-crossaccounts.sh aws ses get-identity-verification-attributes --identities "example.com"

#!/bin/bash
##Usage
# ./awscli-crossaccounts.sh <full aws cli command> it will be looped through all AWS profiles and regions
green_bold="\e[1;32m"; reset="\e[0m"
awsprofiles=("default" "dev" "prod")
awsregions=("us-east-1" "eu-west-1" "ca-central-1")

for profile in "${awsprofiles[@]}"; do
  for region in "${awsregions[@]}"; do
    echo -e "${green_bold}$@ --profile $profile --region $region${reset}" #display command to execute
    timeout 5 "$@" --profile "$profile" --region "$region"; sleep 0.5     #execute, timeout does not allow for interactive MFA session
  done
done

References

Examples

Create a reusable delegation set with a unique string '20170409'

aws route53 create-reusable-delegation-set --caller-reference 20170409

List the reusable-delegation-set created in ~/.aws/credentials profile

aws route53 list-reusable-delegation-sets --profile terraform-profile

IAM server certificates

List IAM server certificates, delete a certificate

aws iam list-server-certificates  --output text  --query 'ServerCertificateMetadataList[*].[Expiration,ServerCertificateName]'  | sort
aws iam list-server-certificates | grep <ServerCertificateName>
aws iam delete-server-certificate --server-certificate-name <ServerCertificateName>


Upload a certificate to IAM

aws iam upload-server-certificate --server-certificate-name cert_name --certificate-body file://cert_name.crt \
                                  --certificate-chain file://cert_name.pem --private-key file://cert_name.key


Check expiry date

certificate_name=<ServerCertificateName>
aws iam get-server-certificate --server-certificate-name $certificate_name --output text --query 'ServerCertificate.CertificateBody' | openssl x509 -text | less

List all instances and their status

aws ec2 describe-instances --query 'Reservations[*].Instances[*].[State.Name,InstanceId,Tags[?Key==`Name`].Value]' --output text

Get account number and name (alias)

ACCOUNTID=$(timeout 3 aws sts get-caller-identity  --output text | cut -f1 2>/dev/null)
ACCO_NAME=$(timeout 3 aws iam list-account-aliases --output text | cut -f2 2>/dev/null)

References