Ansible

Ansible - management and configuration system

Ansible install

Python requiremnts for Ansible

Management host

Ansible 2.5 and above have support for Python 3. Previous to 2.5, the Python 3 support was considered a tech preview.

Target hosts

Most Ansible modules that execute under a POSIX environment require a Python interpreter on the target host. Interpereter Discovery

Pip installation and python virtual environments

Assuming you are in Ubuntu 18.04 LTS, using default Python2 thus use of pip. Do not upgrade pip package using pip as it can break your system. Eg. in Ubuntu use only apt packet manager to managage pip binary.

sudo apt-get install python-pip # optional: python3-pip
sudo apt install virtualenv virtualenvwrapper

#install all using pip itself if your OS is configured to do it
# pip install --upgrade pip virtualenv virtualenvwrapper

Python virtual environment will have pip installed and when an environment is active the correct symlinking is in place. So, you should not need to worry about what pip or pip3 version to use.

Ansible 1.9.4 & Python2

virtualenv ~/ansibleenv/ansible194 # ~/ansibleenv it's a convenience path for organization purposes; not required
source ansible194/bin/activate
(ansible194) vagrant@u18cli-1 $ pip install ansible==1.9.4
(ansible194) vagrant@u18cli-1 $ pip install awscli boto boto3
(ansible194) vagrant@u18cli-1 $ deactivate # if needed

Ansible 2.8.1 & Python2

virtualenv ~/ansibleenv/ansible281
source ansible281/bin/activate
(ansible281) vagrant@u18cli-1 $ pip install ansible==2.8.1
(ansible281) vagrant@u18cli-1 $ pip install awscli boto boto3
(ansible281) vagrant@u18cli-1 $ ansible --version # verify version
ansible 2.8.1
  config file = None
  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/vagrant/ansibleenv/ansible281/local/lib/python2.7/site-packages/ansible
  executable location = /home/vagrant/ansibleenv/ansible281/bin/ansible
  python version = 2.7.15+ (default, Nov 27 2018, 23:36:35) [GCC 7.3.0]   # <- python2

Ansible 2.8.7 & Python3

python3 -m venv ~/ansibleenv/ansible287
source ~/ansibleenv/ansible281/bin/activate
(ansible287) vagrant@ubuntu-bionic $ ansible --version
(ansible281) vagrant@u18cli-1 $ pip install awscli boto boto3
ansible 2.8.7
  config file = None
  configured module search path = ['/home/vagrant/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/vagrant/ansibleenv/ansible287/lib/python3.6/site-packages/ansible
  executable location = /home/vagrant/ansibleenv/ansible287/bin/ansible
  python version = 3.6.9 (default, Nov  7 2019, 10:44:02) [GCC 8.3.0]        # <- python3

Pip ansible related packages

pip install ansible-lint yq requests openshift
# openshift - for Kubernetes module, eg. version 0.10.1 uses apply patching, not just delete/create

Resources

• pip Openshift openshift-restclient-python

Apt-get packet management

apt-cache policy ansible | grep -A1 Installed # check version it will install
sudo apt-get install ansible

Install dependencies manually

sudo apt-get install python python-setuptools python-crypto python-jinja2 python-paramiko python-pkg-resources python-yaml python-httplib2 python-netaddr 
sudo apt-get install build-essential libssl-dev libffi-dev python-dev </span> #Ubuntu 16.04 LTS

Zip binary

Download a version from Ansible git repository you need

wget https://releases.ansible.com/ansible/ansible-1.9.4.tar.gz
tar -xzvf ansible-1.9.4.tar.gz 
cd ansible-1.9.4/
sudo  make      #requires pre-reqs
sudo python setup.py install

Compile source code from github

git clone git://github.com/ansible/ansible.git --recursive
cd ansible
make
sudo make install

Optional packets

Python Module Management - Pip install

wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py

Install boto python package, required to run ec2 modules

Boto is Amazon Web Services Library. The complete list of supported AWS services can be found on Python.org

sudo pip install boto     #install
sudo pip install boto3    #certain Ansible ec2-modules require newer version
pip list                  #check version
python -c "import boto; print boto.Version"    #check version

Configure Boto3 AWS credentials

On startup, Boto3 library looks for configuration files in the following locations (in this order):

1. Passing credentials as parameters in the boto.client() method
2. Passing credentials as parameters when creating a Session object
3. Environment variables
   • export AWS_ACCESS_KEY_ID=***
   • export AWS_SECRET_ACCESS_KEY=***
4. Shared AWS credential file (~/.aws/credentials), also used by Terraform and Ansible
5. AWS config file (~/.aws/config), used by AWScli
6. Assume Role provider
7. Boto2 system-wide config file (/etc/boto.cfg and ~/.boto) for user-specific settings
8. Instance metadata service on an Amazon EC2 instance that has an IAM role configured.

The credentials and config files are discussed in more detail here. Configure your profiles:

$ aws configure --profile piotr
AWS Access Key ID [None]: AKIAJ123456789123456
AWS Secret Access Key [None]: 1234e7TlsSCzw1YOUym3tlyU123456789123456
Default region name [None]: eu-west-1
Default output format [None]: json

Then following files will be created in ~/.aws/{config,credentials}

$ cat ~/.aws/config
[default]
region=us-east-1

$ cat ~/.aws/credentials
[default]
aws_access_key_id = AKIAJ123456789123456
aws_secret_access_key = 1234e7TlsSCzw1YOUym3tlyU123456789123456

Install awscli

sudo pip install awscli

Install Ansible mysqldb module

Lack of it can cause error message: msg: the python mysqldb module is required

Ubuntu 14, Ubuntu 16, Debian 8.6 (jessie), Ubuntu 18.04

sudo apt-get install python-pip python-dev libmysqlclient-dev python-mysqldb
# libmysqlclient-dev - required prerequsit for 'MySQL-python' pip package
# python-mysqldb     - required on Ubuntu 18.04 LTS , with Ansible 1.9.4 and Python 2.7.15+

Fedora 24

sudo dnf install python python-devel mysql-devel redhat-rpm-config gcc

Install Python pip package, note sudo -H option used

vagrant@ubuntu-bionic $ sudo -H pip install MySQL-python
(ansible194) vagrant@ubuntu-bionic $ pip install MySQL-python # when using virtual environements

Install python modules

sudo pip install requests

Manage different versions of Ansible

Install AVM

git clone https://github.com/k33nice/avm.git

Adhoc ansible <host-pattern> [options] commands

Copy a user ssh public key to remote server, if you do not specify a username, the current user will be used

# Run ansible against localhost
ubuntu@host $> cd ~/.ssh
ubuntu@host $> ssh-keygen # press Enter 3x
ubuntu@host $> cat id_rsa.pub >> authorized_key # option1 add ssh key to localhost current user authorized keys
ubuntu@host $> ssh-copy-id localhost            # option2 add ssh key to localhost current user authorized keys
ubuntu@host $> ssh-copy-id username@server.com  # option3 add ssh key to remote server
ubuntu@host $> cat > hostfile <<EOF
localhost
EOF
ubuntu@host $> ansible -i hostfile localhost -m ping
localhost | success >> {
    "changed": false,
    "ping": "pong"
}

Once you can ssh to a remote server following ad hoc commands will work

ansible -i hostfile host -m setup       -a "filter=ans*ipv4*" --become --ask-become-pass
#                        -m modulename  -a "module arguments" --options

 -i hostfile            # optional if default /etc/ansible/hosts exists, or is specified in ansible.cfg 
 -b, --become           # run operations with become (nopasswd implied)
 --become-method=METHOD # privilege escalation method to use (default=sudo),
                        # valid choices: [ sudo | su | pbrun | pfexec | runas ]
 -K, --ask-become-pass  # will prompt you if sudo requires password
 -s, --sudo             # run operations with sudo (nopasswd) (deprecated, use become)
 -S, --su               # run operations with su              (deprecated, use become)

Ping a host, specify a user that Ansible should connect as and the private key

ansible            centos-1 -m ping -u username --private-key=~/.ssh/id_rsa
ansible -i ./hosts ubuntu-1 -m ping

Execute a module against localhost, --connection local

ansible all -i "localhost," -c local -m shell -a "apt-get remove git -y"   --become
ansible all -i "localhost," -c local -m apt   -a "pkg=git state=installed" --become

# ssh connection: fails as publickey is not in authorized keys of ubuntu user
ubuntu@host $> ansible -i hostfile all -m ping --connection ssh # ssh is default connection
localhost | FAILED => SSH Error: Permission denied (publickey).
    while connecting to 127.0.0.1:22

# local connection: works though
ubuntu@host $> ansible -i hostfile all -m ping --connection local # -c short version
localhost | success >> {
    "changed": false,
    "ping": "pong"
}

Adhoc commands

ansible aws   -m setup -a "filter=ansible_all_ipv4_addresses" -u user     # all ip addresses
ansible local -m setup -a "filter=ans*ipv4*"                              # filter facts
ansible app -m shell   -a "apt-get -y install lynx" -b --ask-become-pass  # -s deprecated replaced by -b "become"
ansible app -m apt     -a "pkg=lynx state=installed update_cache=true" -b -K
ansible app -m file    -a "path=/tmp/etc state=directory mode=0700 owner=root" # create directory
ansible app -m copy    -a "src=/etc/fstab dest=/tmp/etc/fstab"                 # copy a file to a remote system
ansible app -m command -a "rm -rf /tmp/etc/fstab"                              # delete a file
ansible app -m service -a "name=apache2 state=stopped" -u user -b -K           # stop Ubuntu apache
ansible app -m apt     -a "name=apache2 state=absent"  -b --ask-become-pass    # removes package
ansible aws -m cron    -a "name=List minute=0 hour=12 job='ls -al /var > /root/var.log'" -u user --become
ansible aws -m command -a "crontab -l" -u user -b
ansible aws -m user    -a "user=piotr state=present uid=5001 shell=/bin/bash"
ansible aws -m user    -a "user=piotr state=absent remove=yes"  # remove=yes removes also HOME and any emails

Capture Ansible output into JSON and save on the local control_server in directory called eg. install_resuts

ansible aws -m apt -a "name=lynx state=installed" -t install_resuts

Dry run:

• run ansible comamnd with --check | -C parameter to validate playbook/ad-hoc commands, no modification will be made on remote nodes. No all modules support the check mode.

Modules

shell is not interactive, therefore 'apt-get install' requires -y flag. STDOUT is displayed on your terminal. The pipe and all redirections do work. Executes commands on a remote node.

copy - copies files from a local control server to remote node

fetch - copies files from remote node to the local box

Get facts

Examples of the most common facts. It requires Python to be installed on a remote node

ansible awsweb -m setup -a 'filter=ansible_distr*' -u user --become --ask-become-pass
ansible awsweb -m setup -a 'filter=ansible_fqdn'
ansible awsweb -m setup -a 'filter=ansible_interfaces'
ansible awsweb -m setup -a 'filter=ansible_kernel'
ansible awsweb -m setup -a 'filter=ansible_mem*'
ansible awsweb -m setup -a 'filter=ansible_proc*'

Config file lookup process

Nearly all parameters can be overridden in ansible-playbook or with command line flags. ansible will read ANSIBLE_CONFIG, ansible.cfg in the current working directory, .ansible.cfg in the home directory or /etc/ansible/ansible.cfg, whichever it finds first.

1. export ANSIBLE_CONFIG=/home/test/config/ansible.cfg environment variable
2. ansible.cfg in the working current directory a command is run
3. ~/.ansible.cfg in a user home directory
4. /etc/ansible/ansible.cfg Ansible system config file

AWS credentials

In order to allow Ansible to interact with AWS Resources it requires secret keys to call APIs. These can be hard coded within a playbook or exported within your environment. Below you can find useful script that you can source each time you need Ansible to manage your AWS infrastructure.

vi credentials.sh
export AWS_ACCOUNT_ID='777666555444'              # not required but handy to have
export AWS_DEFAULT_REGION='eu-west-1'             # not required
export AWS_ACCESS_KEY_ID='AAABBBCCCDDDEEEFFF11'        # Ansible environment variable
export AWS_SECRET_ACCESS_KEY='ttttppooooozz'  # Ansible environment variable

Provision using terraform

https://github.com/dzeban/c10k/blob/master/infrastructure/main.tf

Prepare environment for automation

Ansible Control server - place where you run playbooks from, it connects to all other hosts therefore it requires:

• ansible user
• ssh key pair eg. ansible.pem and ansible.pub
• ssh config with disabled strict check or all remote hosts IPs in known_hosts file
• python 2.7 installed

Configuration nodes (servers you with to manage)

• ansible user with sudo privileges that does not require a password
• ssh ansible.pub key content in ~/.ssh/authorized_keys file
• python 2.7 installed

Note: When working with AWS each instance already has public key inserted into it's default user. Therefore to connect from Ansible control server to Ubuntu AMI you connect with ubuntu@aws.amazon.com. To connect to Amazon AMI Linux you'd connect using ec2-user@aws.amazon.com with your VPC private key.

Add remote nodes PUB keys to known_hosts

ssh-keyscan hos1 host2 host3 >> ./.ssh/known_hosts

Add remote nodes fingerprints to known_hosts file on Ansible Control server using Ansible playbook (not tested)

---
# ansible playbook that adds ssh fingerprints to known_hosts
- hosts: all
  connection: local
  gather_facts: no
  tasks:
  - command: /usr/bin/ssh-keyscan -T 10 {{ ansible_host }}
    register: keyscan
  - lineinfile: name=~/.ssh/known_hosts create=yes line={{ item }}
    with_items: '{{ keyscan.stdout_lines }}'

Add Ansible's Control Server user public key onto remote nodes

Install the control_node's PUB key onto remote nodes to enable password-less ssh connection. Do not use sudo elevated privileges as this would add our PUB key to a remote note root user.

$ ansible-playbook ssh-addkey.yml -u vagrant --ask-pass

The playbook ssh-addkey.yml looks like

--- #Default vagrant user password is 'vagrant'
- hosts: all
  gather_facts: no
  remote_user: vagrant
  tasks:
  - name: install ssh key
    authorized_key: user=vagrant key="{{ lookup('file', '/home/vagrant/.ssh/id_rsa.pub') }}" state=present

Allow an ansible_service user to run sudo without password

sudo visudo
ansible ALL=(ALL)    NOPASSWD: ALL    #user can run as root without password
sudo -l    #check your rules, last rule take precedence

Stop Ansible to require sudo password at each run

sudo vi /etc/ansible/ansible.cfg
#ask_sudo_pass = True         #needs to be commented out, otherwise works like --ask-become-pass

Install ansible_service user ssh public key on a localhost

ssh-copy-id localhost
ssh-copy-id localhost.localdomain

Basic playbook

<syntaxhighlightjs lang="yaml"> --- - hosts : localhost

 become: true
 tasks :
   - name: Install packer
     apt :
       name : packer
       state: latest

</syntaxhighlightjs>

executable playbooks

#!/usr/bin/ansible-playbook --inventory=../hosts
---
- hosts: ...

This is especially useful when running from a config/ops repo checkout which includes your hosts, hostvars and groupvars.

• remember to make the playbook executable
• if you run ansible-playbook playbook.yml, the shebang gets ignored
• in Linux use long options without spaces, as shebang allows for a program name and one option. Notice the use of the long-form switch, which does not have a space between --inventory and ../hosts

Variables used within playbooks - TARGET SECTION

Specific to a playbook by adding a section:

- hosts: awsweb
  vars: 
    controls_server: localhost
    web_root: /var/wwwroot
  tasks:
    - name: Task1

Include variables from files

- hosts: awsweb                                                #Example of variables file content:
  vars_files:                 cat ./vars.yml	
   - vars.yml   --------->    --- # YAML file
  tasks:                      controls_server: localhost
   - name: Task1              web_root: /var/wwwroot

Prompt a user to provide a value to the variable

- hosts: awsweb
  vars_prompt: 
    - name: controls_server     #variable name
      prompt: Provide Controls Serve name
  tasks:
    - name: Task1

Handlers section

In the example below handler 'Restart Apache' will be called only on change status of 'Install apache web server' task

tasks:
     - name: Install apache web server
       action: apt pkg=apache2 state=installed
       tags:
         - packages
       notify: Restart Apache         #notification matches the name of the handler
handlers:
     - name: Restart Apache
       action: service name=apache2 state=restarted

Execution order

Handlers are run in the order they are listed in a handlers file, not in the order that they are notified.

These notify actions are triggered at the end of each block of tasks in a playbook, and will only be triggered once even if notified by multiple different tasks. They are triggered after all tasks completed successfully.

Variables passed at command line

Any variable can be passed at command line

ansible-playbook variable_fromcommandline.yml --extra-vars "hosts=awsweb user=user pkg=telnet"

Tags section

Run a specific part of the configuration without running the whole playbook. See the playbook example above tags section

ansible-playbook example.yml --tags "configuration,packages"  #will run only tags sections
ansible-playbook example.yml --skip-tags "notification" #skips tagged sections

There is a special always tag that will always run a task, unless specifically skipped (–skip-tags always)

Start at task and step

ansible-playbook example.yml --start-at-task="Name of task"  #execute from this task to the EOF
ansible-playbook example.yml --step #will step you through tasks asking y/n/c 

You can use a wildcard * within the task name.

Conditional statements and loops

wait_for

- name: Wait for port 80 is listening
   wait_for:
     port  : 80
     state : started

until

The play below will loop until: find a string in shell: output.

- name: Check if HTTPD is running
   shell   : system status httpd
   until   : result.stdout.find("active (running)") #search string
   delay   : 5 #seconds
   retries : 5
   register: result

References

• Untangling Ansible Loops

Roles

Roles are automation around include directives. Therefore directories like tasks, handlers, vars and meta are automatically included as long the 'main.yml' file is there. There is no need to reference them using referenced or absolute paths, they are automatically available and included in plays.

The main.yml file contains all directives relevant to the uppper directory it is in, can contain a list of tasks, handles or vars.

Let's assume you have structure like this

site.yml
webservers.yml  <- eg. master control file recalls 'roles' named as a directory in roles/rolename
fooservers.yml
roles/
   common/           webservers/           
     files/            files/         - files used locally or transferred to a remote node    
     templates/        templates/     - Jinja2 templates
     tasks/            tasks/         - (include) individual tasks that play will do something
     handlers/         handlers/      - (include) eg. server restarts, shared among tasks
     vars/             vars/          - (vars) binary values something equals something else
     defaults/         defaults/      - default settings
     meta/             meta/          - (vars) roles dependencies

In a playbook (eg. webservers.yml), it would look like this:

---
- hosts: webservers
  roles:
     - common
     - webservers

This designates the following behaviors, for each role 'x':

• If roles/x/tasks/main.yml exists, tasks listed therein will be added to the play
• If roles/x/handlers/main.yml exists, handlers listed therein will be added to the play
• If roles/x/vars/main.yml exists, variables listed therein will be added to the play
• If roles/x/meta/main.yml exists, any role dependencies listed therein will be added to the list of roles
  • Any copy, script, template or include tasks (in the role) can reference files in roles/x/{files,templates,tasks}/ (dir depends on task) without having to path them relatively or absolutely

Execution order

In a play any roles always execute before tasks. To manipulate the flow you can use pre_ and post_ directives

--- - hosts: awsweb pre_tasks: - name: When the ROLE start raw: date > role_start-end.log roles: - webservers post_tasks: - name: When the ROLE end raw: date >> role_start-end.log

Roles execution order pre_task role1 role2 ownrole meta ownrole tasks role3 terminus post_tasks

Roles path

You make roles system-wide available by including its path in /etc/ansible/ansible.cfg file

roles_path    = /home/test/playbooks/roles:/etc/ansible/roles

By default, a playbook that calls a role searches relative directories:

./roles/name_of_role/
./name_of_role/

Ansible Vault

ansible-vault create secure.yml   #give a password that will be used as an encryption key
ansible-vault rekey secure.yml    #change a password
ansible-vault decrypt secure.yml  #decrypt the file
ansible-vault encrypt secure.yml  #encrypt existing file, can pass multiple files at once

Use Vault with playbooks

ansible-playbook secure.yml --ask-vault-pass

README.md - markdown format

.md stands for markdown and is generated at the bottom of your github page as html Typical syntax includes:

# H1
## H2
### H3
#### H4
##### H5
###### H6

Alternatively, for H1 and H2, an underline-ish style:

Alt-H1 heading
======

Alt-H2 subheading
------

*This will be Italic*
**This will be Bold**

- Unordered list can also use '-'  or '+'

• Inline `code` has `back-ticks around` it.
• Blocks of code are either fenced by lines with three back-ticks ```, or are indented with four spaces

Code blocks are between triple back-ticks

```
echo "Hello world"
```

References

• Markdown-Cheatsheet

Practical uses

Get ip address from facts

This is an example playbook that assigns a fact variable to locally defined variable or call the fact variable directly in the debug statement.

---
- hosts: localhost
  connection: local
  vars: 
    - IpAddr: "{{ ansible_default_ipv4.address }}"  #assigns a fact variable to local variable, quotes to expand the variable value are necessary
  tasks:
    - debug: var=ansible_default_ipv4.address 
    - debug: var=IpAddr

Example of a task to read delegated node ip address

 - name: Get private IP address of AWS node
      action: setup
      delegate_to: fqdn_or_ip
      register: private_ip
 - debug: var=ansible_default_ipv4.address #IP can be referenced simply by {{ ansible_default_ipv4.address }}

Galaxy roles

This is public repository of available roles. The command can also create a role tree:

ansible-galaxy init myrole-1 --offline # creates roles/myrole-1 role's files tree

• Jenkins-CI

YAML

Yaml array notation

monitoring_files: [
    { dest: /opt/scripts/stackdriver/payloads, group: root }, 
    { dest: /opt/scripts/stackdriver/payloads, group: root }, 
    { dest: /opt/scripts/stackdriver/payloads, group: root }
  ]

Reference

• Ansible installation
• My GitLab repository Linux Academy - Ansible training examples
• Learning Ansible with Vagrant Justin at sysadmincasts.com
• List of boto supported AWS services Python.org
• jinja_functions Buildin functions with examples