Install `awscli`

<syntaxhighlightjs lang="bash"> sudo apt-get install awscli # via apt-get pip install awscli # via pip </syntaxhighlightjs>

Install `awscli v2`

This is a method of running aws-v1 (installed eg via pip) along with aws-v2. Both installed in local user directories. <syntaxhighlightjs lang="bash"> curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip INSTALL_DIR=~/.local/usr/local/aws-cli-v2 # /usr/local/aws-cli

   BIN_DIR=~/.local/bin/aws-cli-v2       # /usr/local/bin/aws-cli-v2

mkdir -p {$INSTALL_DIR,$BIN_DIR} ./aws/install --install-dir $INSTALL_DIR --bin-dir $BIN_DIR You can now run: /home/vagrant/.local/bin/aws-cli-v2/aws --version

| -i, --install-dir <path> defaults to /usr/local/aws-cli
| -b, --bin-dir <path> The directory to store symlinks to executables, defaults to /usr/local/bin

ln -s $BIN_DIR/aws ~/.local/bin/aws2

Show version

/usr/local/bin/aws-cli-v2/aws --version aws-cli/2.2.11 Python/3.8.8 Linux/4.14.67-66.56.amzn1.x86_64 exe/x86_64.amzn.2018 prompt/off

Uninstall

which aws ls -l $(which aws) rm $BIN_DIR/{aws,aws_completer} # Delete the two symlinks in the --bin-dir directory rm -rf $INSTALL_DIR # Delete the --install-dir directory </syntaxhighlightjs>

Configure

Environment variables

AWS_ACCESS_KEY_ID=1111 # access keyID for your user or keyID retrieved when using 'sts' service
AWS_SECRET_ACCESS_KEY=2222
AWS_DEFAULT_REGION=eu-west-1
AWS_SESSION_TOKEN=3333 # token retrieved when using 'sts' service

# in 2015 AWS decided to rename AWS_SECURITY_TOKEN(deprecated) into AWS_SESSION_TOKEN. There 
# are still tools eg. Ansible, Terraform and Boto that support the deprecated variable name. 
AWS_SECURITY_TOKEN=$AWS_SESSION_TOKEN

CLI credentials file

When you run aws configure and enter credentials, they are stored in a file at ~/.aws/credentials. Additionally, some configuration settings—such as the default region—are stored at ~/.aws/config.

~/.aws/credentials vs ~/.aws/config

config - contains config, profile name is prefixed with word profile, eg: [profile <profile-name>]
credentials - contains credentials for a profile, the profile name is not prefixed, eg: [<profile-name>]. Note credentials can be stored in the config file as well.

$ aws configure                 #sets up default profile
$ aws configure --profile dev   #configure named 'dev' profile
$ aws configure --profile piotr #configure named 'piotr' profile
$ aws ec2 describe-regions      #to get a list all available regions

Note: config file at least with awscliv1 does not allow for trailing spaces in the configuration

The ~/.aws/config profile sections must have the format of [profile profile-name], except for the default profile, i.e:

[default]
aws_access_key_id     = 111xxxxxxxxxxxxxxxxx
aws_secret_access_key = 111xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[profile dev]
aws_access_key_id     = 222xxxxxxxxxxxxxxxxx
aws_secret_access_key = 222xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[profile piotr]
aws_access_key_id     = 333xxxxxxxxxxxxxxxxx
aws_secret_access_key = 333xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

You can also use environment variables described here

Configure AWS cli with Cross-Account Roles & MFA

Let’s assume that we have 4 AWS accounts:

One identity-account (acc_no: 000xxxxxxxxx) where we provision IAM users, create their API Credentials and assign MFA -
- user 'piotr' mfa_arn: arn:aws:iam::000xxxxxxxxx:mfa/user.name
3 for our different environments
- development (acc_no: 111xxxxxxxxx)
  - role: CrossAccountSignin_Admin: role_arn:aws:iam::111xxxxxxxxx:role/CrossAccountSignin_Admin
  - Trusted entities: acc_no: 000xxxxxxxxx (identity-account) and Conditions: aws:MultiFactorAuthPresent
- staging (acc_no: 222xxxxxxxxx)
  - role: CrossAccountSignin_Admin: role_arn:aws:iam::222xxxxxxxxx:role/CrossAccountSignin_Admin
  - Trusted entities: acc_no: 000xxxxxxxxx (identity-account) and Conditions: aws:MultiFactorAuthPresent
- production (acc_no: 333xxxxxxxxx)
  - role: CrossAccountSignin_Admin: role_arn:aws:iam::333xxxxxxxxx:role/CrossAccountSignin_Admin
  - Trusted entities: acc_no: 000xxxxxxxxx (identity-account) and Conditions: aws:MultiFactorAuthPresent

Let’s also assume that the proper roles and policies have already been put in place, allowing us to switch between accounts using the management console. We can create CLI profiles for each of these accounts.

Credentials file ~/.aws/credentials

[piotr-identity-account]
aws_access_key_id    = 333xxxxxxxxxxxxxxxxx
aws_secret_access_key= 333xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Config file ~/.aws/config

[profile piotr-identity-account]
region = eu-west-1

[profile development]
region         = eu-west-1
role_arn       = arn:aws:iam::111xxxxxxxxx:role/CrossAccountSignin_Admin
mfa_serial     = arn:aws:iam::000xxxxxxxxx:mfa/piotr
source_profile = piotr-identity-account

[profile staging]
region         = eu-west-1
role_arn       = arn:aws:iam::222xxxxxxxxx:role/CrossAccountSignin_Admin
mfa_serial     = arn:aws:iam::000xxxxxxxxx:mfa/piotr
source_profile = piotr-identity-account

[profile production]
region         = eu-west-1
role_arn       = arn:aws:iam::333xxxxxxxxx:role/CrossAccountSignin_Admin
mfa_serial     = arn:aws:iam::000xxxxxxxxx:mfa/piotr
source_profile = piotr-identity-account

Test

$ aws ec2 describe-instances --profile development
Enter MFA code for arn:aws:iam::000xxxxxxxxx:mfa/piotr: ***

By default the temporary API token is valid for 1 hour, once expired you will be prompted for MFA again.

Working with MFA

If you’re using the aws CLI, using MFA with a Credentials Profile is fairly straight forward. You just need to add the mfa_serial to your Config File in ~/.aws/config:

[profile with-mfa]
mfa_serial = arn:aws:iam::123456789012:mfa/jon-doe

The mfa_serial parameter should be set to the ARN of your MFA device, which you can get from the IAM User page of the AWS Web Console.

It should be of the format:

arn:aws:iam::<ACCOUNT_ID>:mfa/<USERNAME>

Once you’ve added the mfa_serial parameter, the next time you run the aws CLI with that Named Profile, it will prompt you for an MFA token:

$ aws s3 ls --profile with-mfa
Enter MFA code:

But what if you’re running something that isn’t aws, such as terraform or packer. Most other tools will not automatically prompt for an MFA token. Instead, you have to do the MFA authentication process outside of that tool. First, you configure your Credentials File with your normal (permanent) AWS Access Keys (e.g. by running aws configure). Next, you run the aws sts get-session-token command, passing it the ARN of your MFA device and an MFA token from the Google Authenticator App or your key fob:

aws sts get-session-token \
  --serial-number    arn:aws:iam::123456789012:mfa/jon-doe \
  --token-code       123456 \
  --duration-seconds 43200

This will return a blob of JSON that contains Temporary Access Keys (note the --duration-seconds argument in the earlier command, which specifies when these Temporary Access Keys will expire):

{
  "Credentials": {
    "SecretAccessKey": "secret-access-key",
    "SessionToken": "temporary-session-token",
    "Expiration": "expiration-date-time",
    "AccessKeyId": "access-key-id"
  }
}

You will need to take these Temporary Access Keys and copy them into a Named Profile in your Credentials File in ~/.aws/credentials:

[profile-with-mfa]
aws_access_key_id     = <Access-key-as-in-returned-output>
aws_secret_access_key = <Secret-access-key-as-in-returned-output>
aws_session_token     = <Session-Token-as-in-returned-output>

Note that with Temporary Access Keys, you set not only aws_access_key_id and aws_secret_access_key in your Credentials File, but also aws_session_token.

Now you can run those other CLI tools with this Named Profile:

export AWS_PROFILE=profile-with-mfa
terraform apply

Note that the Temporary Access Keys expire after a certain period of time (typically 12 hours), so you’ll have to do this over again.

Other related AWS cli commands

aws sts get-caller-identity --profile dev          #info about you, this will create temp credentials
aws sts get-session-token --duration-seconds 28800 #get a session token by default 15 min, here 8h

https://github.com/605data/aws-scripts/tree/master/awscli-mfa https://random.ac/cess/2017/10/28/aws-cli-key-rotation-script-v2/ https://github.com/sweharris/aws-cli-mfa

Assuming role

aws sts assume-role --role-arn "arn:aws:iam::11111111111:role/CrossAccountAgent" --role-session-name "terraform" --profile agent-terraform

Tools making work with sts easier

aws-vault: Securely store your AWS credentials in your operating system’s keystore (e.g., Keychain, KWallet), Automatically set those credentials as environment variables when executing a command, Handle all the aws sts commands for you when using IAM Roles or MFA.
aws-auth: bash script Gruntwork subscribers only

`aws-google-auth`

Acquire AWS STS (temporary) credentials via Google Apps SAML Single Sign On

pip install aws-google-auth

Config

$ aws-google-auth 
Google username: user.name@acme.com
Google IDP ID: C00aaabbb
Google SP ID: 977777777777
Google Password: *****
Please visit the following URL to view your CAPTCHA: https://accounts.google.com/Captcha?v=2&ctoken=AAAAAAAAA-AAAAAAAAAAAAAAA-BBB(...)-(...)_AAAAAA_AAAAAAA
Captcha (case insensitive): fafafafafa
Enter SMS token: G-111111
[  1] arn:aws:iam::111111111111:role/saml-administrator
[  2] arn:aws:iam::222222222222:role/saml-administrator
Type the number (1 - 2) of the role to assume: 2           
Assuming arn:aws:iam::222222222222:role/saml-administrator
Credentials Expiration: 2020-02-12 17:21:51+00:00

Ones setup is completed ~/.aws credentials files will be updated

cat ~/.aws/config # the following profile will be added
[profile sts]
region = eu-west-1
google_config.ask_role = False
google_config.keyring = False
google_config.duration = 43200
google_config.google_idp_id = C00aaabbb
google_config.role_arn = arn:aws:iam::222222222222:role/saml-administrator
google_config.google_sp_id = 977777777777
google_config.u2f_disabled = False
google_config.google_username = user.name@acme.com
google_config.bg_response = None


cat ~/.aws/credentials
[sts]
aws_access_key_id = AAAAAAAAAAAAAAAAAAA
aws_secret_access_key = bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
aws_security_token = FwoAAAAAAAAAAAAA//////////(...)M0eH
aws_session_expiration = 2020-02-12T17:21:51+0000
aws_session_token = FwoAAAAAAAAAAAAA//////////(...)M0eH

# The SAML2 response file wll be created
cat ~/.aws/saml_cache_C00aaabbb.xml
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<saml2p:Response xmlns:saml2p="urn:oasis:names:tc:SAML:2.0:protocol" Destination="https://signin.aws.amazon.com/saml" ...
    </saml2:Assertion>
</saml2p:Response>

okta-aws-cli-assume-role

Install. It requires Java JRE, sucessfully tested with OpenJDK v11 on Ubuntu 20.04 by installing default-jre apt package.

$ PREFIX=~/.okta bash <(curl -fsSL https://raw.githubusercontent.com/oktadeveloper/okta-aws-cli-assume-role/master/bin/install.sh) -i
Warning: Java is not installed. Make sure to install that
Installing into ~/.okta
Latest release JAR file: https://github.com/oktadeveloper/okta-aws-cli-assume-role/releases/download/v2.0.4/okta-aws-cli-2.0.4.jar
Fetching JAR file → ~/.okta/okta-aws-cli-2.0.4.jar
Symlinking ~/.okta/okta-aws-cli.jar → okta-aws-cli-2.0.4.jar
Creating example ~/.okta/config.properties

# Add the following to ~/.bash_profile or ~/.profile:

# OktaAWSCLI
[ -f "$HOME/.okta/bash_functions" ]                            && . "$HOME/.okta/bash_functions"
[ -d "$HOME/.okta/bin" && ":$PATH:" != *":$HOME/.okta/bin:"* ] && PATH="$HOME/.okta/bin:$PATH"

Add $HOME/.okta/bin at minumum

echo '[ -d "$HOME/.okta/bin" ] && export PATH="$PATH:$HOME/.okta/bin"' >> ~/.profile

Create a configuration file, read more.... Note this file does not contain your Okta password.

# vi ~/.okta/config.properties
#OktaAWSCLI
OKTA_ORG=acmecorp.oktapreview.com.changeme.local
OKTA_AWS_APP_URL=https://acmecorp.oktapreview.com.changeme.local/home/amazon_aws/0oa5zrwfs815KJmVF0h7/137
OKTA_USERNAME=piotr
OKTA_BROWSER_AUTH=false
OKTA_MFA_CHOICE=OKTA.push
OKTA_PASSWORD_CMD=$(echo "yourpassword") # command to fetch your password, instead prompting you

Available commands after adding PATH=$PATH:$HOME/.okta/bin

$ ls -1a ~/.okta/bin/
okta-credential_process # [1] init Okta login + MFA, returns security tokens in json to STDOUT
okta-listroles          # [2] init Okta login + MFA, outputs IAM roles arns, that a user can assume
awscli   # bash wrapper
withokta # bash wrapper

Populate list of roles you can assume (unsure if this creates temporary file), but then you will be able to assume role using --profile described in the next step.

$ okta-listroles
Username: piotr
Password: ***

JSON response: [{"accountName":"Account: aws-111-us (111111111111)","roleOptions":[{"roleName":"saml-111-Administrator","roleArn":"arn:aws:iam::111111111111:role/saml-111-Administrator"}]},{"accountName":"Account: aws-222-eu (222222222222)","roleOptions":...]

Therefore, you can use 'credential_process' syntax in your ~/.aws/config file to get creds. These creds are not saved anywhere, or exported by default so considered safe option.

# vi ~/.aws/config
[profile okta-dev]
credential_process = okta-credential_process arn:aws:iam::111111111111:role/saml-dev-Administrator
region = us-east-1

References

[1] okta-credential_process readme
[2] okta-listroles readme

Auth to Kubernetes with okta-kubectl-auth plugin

securing-aws-access-with-okta Tubi

Multi account aws cli execution loop

Helper programme to execute awscli commands across multiple AWS accounts (profiles) and regions.

#Create temp creds for all accounts protected by MFA tokens, run this without timeout command
$ # ./awscli-crossaccounts.sh aws sts get-caller-identity

#Run a command across accounts in different regions
$ # ./awscli-crossaccounts.sh aws ses get-identity-verification-attributes --identities "example.com"

#!/bin/bash
##Usage
# ./awscli-crossaccounts.sh <full aws cli command> it will be looped through all AWS profiles and regions
green_bold="\e[1;32m"; reset="\e[0m"
awsprofiles=("default" "dev" "prod")
awsregions=("us-east-1" "eu-west-1" "ca-central-1")

for profile in "${awsprofiles[@]}"; do
  for region in "${awsregions[@]}"; do
    echo -e "${green_bold}$@ --profile $profile --region $region${reset}" #display command to execute
    timeout 5 "$@" --profile "$profile" --region "$region"; sleep 0.5     #execute, timeout does not allow for interactive MFA session
  done
done

References

Cli Assume Roles Aws docs
Roles Terms and Concepts Aws docs!
Configure AWS cli with Cross-Account Roles & MFA

Examples

Create a reusable delegation set with a unique string '20170409'

aws route53 create-reusable-delegation-set --caller-reference 20170409

List the reusable-delegation-set created in ~/.aws/credentials profile

aws route53 list-reusable-delegation-sets --profile terraform-profile

IAM server certificates

List IAM server certificates, delete a certificate

aws iam list-server-certificates  --output text  --query 'ServerCertificateMetadataList[*].[Expiration,ServerCertificateName]'  | sort
aws iam list-server-certificates | grep <ServerCertificateName>
aws iam delete-server-certificate --server-certificate-name <ServerCertificateName>

Upload a certificate to IAM

aws iam upload-server-certificate --server-certificate-name cert_name --certificate-body file://cert_name.crt \
                                  --certificate-chain file://cert_name.pem --private-key file://cert_name.key

Check expiry date

certificate_name=<ServerCertificateName>
aws iam get-server-certificate --server-certificate-name $certificate_name --output text --query 'ServerCertificate.CertificateBody' | openssl x509 -text | less

List all instances and their status

aws ec2 describe-instances --query 'Reservations[*].Instances[*].[State.Name,InstanceId,Tags[?Key==`Name`].Value]' --output text

Get account number and name (alias)

ACCOUNTID=$(timeout 3 aws sts get-caller-identity  --output text | cut -f1 2>/dev/null)
ACCO_NAME=$(timeout 3 aws iam list-account-aliases --output text | cut -f2 2>/dev/null)

Tag name in PS1 Bash prompt

Add to your .bashrc, or create a file in /etc/profile.d

INSTANCE_ID=$(wget -qO- http://instance-data/latest/meta-data/instance-id)
REGION=$(wget -qO- http://instance-data/latest/meta-data/placement/availability-zone | sed 's/.$//')
export PS1="$(aws ec2 describe-tags --region $REGION --filter "Name=resource-id,Values=$INSTANCE_ID" --output=text | cut -f5)$ "

Other variations of export PS1

export PS1="$(aws ec2 describe-tags \
  --filters "Name=resource-id,Values=$INSTANCE_ID" 'Name=key,Values=Name' \
  --query 'Tags[].Value' --output text)$ "

export PS1="$(aws ec2 describe-instances --instance-id $INSTANCE_ID \
  --query 'Reservations[].Instances[].Tags[?Key==`Name`].Value' --output text)$ "

Your instance need to have following IAM Profile permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1409309287000",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeTags"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

AWS Feature request on https://forums.aws.amazon.com - Any plans to make tags available through instance metadata?

References

AWScli config files
create-reusable-delegation-set aws . route53 cli
Configuring White Label Name Servers AWS docs

AWS/CLI

Contents

Install `awscli`

Install `awscli v2`

Configure

Configure AWS cli with Cross-Account Roles & MFA

Working with MFA

Assuming role

Tools making work with sts easier

`aws-google-auth`

okta-aws-cli-assume-role

Auth to Kubernetes with okta-kubectl-auth plugin

Multi account aws cli execution loop

References

Examples

IAM server certificates

List all instances and their status

Get account number and name (alias)

Tag name in PS1 Bash prompt

References

Navigation menu

AWS/CLI

Install awscli

Install awscli v2

Configure

Configure AWS cli with Cross-Account Roles & MFA

Working with MFA

Assuming role

Tools making work with sts easier

aws-google-auth

okta-aws-cli-assume-role

Auth to Kubernetes with okta-kubectl-auth plugin

Multi account aws cli execution loop

References

Examples

IAM server certificates

List all instances and their status

Get account number and name (alias)

Tag name in PS1 Bash prompt

References

Navigation menu

Search

Install `awscli`

Install `awscli v2`

`aws-google-auth`