HashiCorp/Vault

It's a tool for Managing Secrets. With following features:

• secures, stores, and tightly controls access to tokens, passwords, certificates, API keys and more... Access control policies provide strict control over who can access what secrets.
• Vault handles leasing, key revocation, key rolling, key versioning and auditing
• Through a unified API, users can access an encrypted Key/Value store and network encryption-as-a-service
• generate AWS IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH credentials and more...

Usage

Initialising

While initializing, you can configure the seal behavior of Vault. Here, with 1 unseal key for simplicity. Vault prints out several keys here. Don't clear your terminal.

$ vault operator init -key-shares=1 -key-threshold=1
Unseal Key 1: UipPveVsmScOb/s/lk9C7OBZTspGC7SHGptDwDxbjbs= 

Initial Root Token: d227493b-63ee-c807-9cda-a8cfcaaef98b 

Vault initialized with 1 key shares and a key threshold of 1. Please securely distribute the key shares printed above. 
When the Vault is re-sealed, restarted, or stopped, you must supply at least 1 of these keys to unseal it before it can start 
servicing requests. 

Vault does not store the generated master key. Without at least 1 key to reconstruct the master key, Vault will remain permanently
sealed!

It is possible to generate new unseal keys, provided you have a quorum of existing unseal keys shares. See "vault rekey" 
for more information.

$ vault status | grep Sealed
Sealed true

Unseal and login/connect

$ vault operator unseal <key 1> #the 'Unseal Key 1' UipPveVsmScOb/s/lk9C7OBZTspGC7SHGptDwDxbjbs= is created during init process
# vault unseal --address=$VAULT_CLIENT_ADDR "$VAULT_UNSEAL_KEY" #deprecated command and removed in 0.12
$ vault status | grep Sealed
Sealed false

# Connect your client before performing any operations using a token
$ vault login <root token> #token is 'Initial Root Token: d227493b-63ee-c807-9cda-a8cfcaaef98b'

List available secret engines

$ vault secrets list

Read and write secrets

$ vault kv put secret/apikey key="password1234"
$ vault kv get secret/apikey

Update the secret data

$ vault kv put secret/apikey key="password8888" owner="dev" #multi-key entry, it will delete any previous key/values
$ vault kv put secret/apikey owner="ops" #update key named 'owner'
$ vault kv get secret/apikey #show keys values

Update the data without overwriting

$ vault kv patch secret/apikey year="2018" #will add key/value to the data

Work with different data versions

$ vault kv metadata get secret/apikey         #retrieves the key metadata
$ vault kv get      -version=1 secret/apikey  #retrieves version 1 of the data
$ vault kv delete   -versions=1 secret/apikey #delete the data, when retrieved again only metadata will get displayed
$ vault kv undelete -versions=1 secret/apikey #recover deleted data
$ vault kv destroy  -versions=1 secret/apikey #permanently delete data
$ vault kv metadata delete secret/apikey      #deletes all versions and metadata
$ vault kv list secret/                       #verify whether a key exists eg. after deletion
No value found at secret/metadata

Seal vault

There is also an API to seal the Vault. This will throw away the encryption key and require another unseal process to restore it. Sealing only requires a single operator with root privileges. This is typically part of a rare "break glass procedure". This way, if there is a detected intrusion, the Vault data can be locked quickly to try to minimize damages. It can't be accessed again without access to the master key shards.

$ vault operator seal

Practical use

Environmental variables

Create environmental variables

VAULT_UNSEAL_KEY=123....789
VAULT_CLIENT_XXX_TOKEN=12345678-abcd-1234-abcd-123456789abc
VAULT_CLIENT_ADDR=http://127.0.0.1:8200

List, read and write with help of jq

List branches and leafs (here users), where secret is always at root of the tree

$ vault list -address=$VAULT_CLIENT_ADDR secret/users
Keys
bob
john

Read values

$ vault read -address=$VAULT_CLIENT_ADDR secret/users/bob_user
Key                     Value
---                     -----
refresh_interval        360h0m0s
password                pass1234
user                    bob@example.com

Export data to a file

# export data (keys/values pairs) to a file
vault read -address=$VAULT_CLIENT_ADDR secret/users/bob_user -format=json | jq .data > bob_credentials.json 
cat bob_credentials.json
{
  "password": "pass1234",
  "user": "bob@example.com"
}

# or more systematically
KEY=secret/users/bob_user; vault read --address=$VAULT_CLIENT_ADDR --format=json $KEY | jq .data > ${KEY////__}

$ ll secret__users__bob_user
-rw-rw-rw- 1 piotr piotr 58 Mar 14 07:43 secret__users__bob_user

# underscore here us a unique character(token) that should not match any part of branch/key, will be used to recreate the key
# Explain 4x'////'
# // -: all occurrences, part of bash parameter expansion option 
# /  -: delimiter
# /  -: string to search, followed by delimiter '/' again

Write data these could be any secrets or configuration into Vault

vault write -address=$VAULT_CLIENT_ADDR secret/users/bob_user @bob_credentials.json

# or more systematically
FILE=secret__users__bob_user; vault write -address=$VAULT_CLIENT_ADDR ${FILE//__//} @${FILE}

Read and write at mass

## Read-out script
#!/bin/bash
declare -a keys
keys=(
secret/dev/db/conStr/database
secret/dev/db/conStr/database/test
secret/dev/redis/credentials
)
idx=1
for key in ${keys[@]}; do
  vault write --format=json --address=$VAULT_CLIENT_ADDR $key @"$idx.json"
  (( idx++ ))
done

## Write-in script
#!/bin/bash
declare -a keys
keys=(
secret/dev/db/conStr/database
secret/dev/db/conStr/database/test
secret/dev/redis/credentials
)
idx=1
for key in ${keys[@]}; do
vault write --format=json --address=$VAULT_CLIENT_ADDR $key @"$idx.json"
(( idx++ ))

Helper functions

a=secret_loc-a_branch1_branch2_key1
echo ${a//__//}
secret/loc-a/branch1/branch2/key1
# /  :- substitutes 1st match
# // :- substitutes all matches

References

• vaultproject Official HashiCorp