Ansible
Ansible - management and configuration system
Install
Using apt-get packet management
apt-cache policy ansible | grep -A1 Installed # check version it will install sudo apt-get install ansible
Install dependencies manually
sudo apt-get install python python-setuptools python-crypto python-jinja2 python-paramiko python-pkg-resources python-yaml python-httplib2 python-netaddr build-essential libssl-dev libffi-dev python-dev #in green for Ubuntu 16.04 LTS
Download zip binary
Download a version from Ansible git repository you need
wget https://releases.ansible.com/ansible/ansible-1.9.4.tar.gz tar -xzvf ansible-1.9.4.tar.gz cd ansible-1.9.4/ sudo make sudo python setup.py install
Download from git hub
git clone git://github.com/ansible/ansible.git --recursive cd ansible make sudo make install
Optional packets
Python Module Management - Pip install
wget https://bootstrap.pypa.io/get-pip.py && sudo python get-pip.py
Install boto python package, required to run ec2 modules
Boto is Amazon Web Services Library. The complete list of supported AWS services can be found on Python.org
sudo pip install boto #install sudo pip install boto3 #certain Ansible ec2-modules require newer version pip list #check version python -c "import boto; print boto.Version" #check version
Configure Boto3 AWS credentials
On startup, Boto3 library looks for configuration files in the following locations (in this order):
- Passing credentials as parameters in the boto.client() method
- Passing credentials as parameters when creating a Session object
- Environment variables
export AWS_ACCESS_KEY_ID=***export AWS_SECRET_ACCESS_KEY=***
- Shared AWS credential file (~/.aws/credentials), also used by Terraform and Ansible
- AWS config file (~/.aws/config), used by AWScli
- Assume Role provider
- Boto2 system-wide config file (/etc/boto.cfg and ~/.boto) for user-specific settings
- Instance metadata service on an Amazon EC2 instance that has an IAM role configured.
Each of those locations is discussed in more detail here.
[default] aws_access_key_id = YOUR_KEY aws_secret_access_key = YOUR_SECRET
Then, set up a default region (in e.g.~/.aws/config):
[default] region=us-east-1
Install awscli
sudo pip install awscli
Install python modules
sudo pip install requests
Build VM with Vagrant
sudo apt-get install virtualbox
Then install Vagrant
Adhoc commands reference
--options ansible* host/-i hostfile -m modulename -a "module arguments" -b (become) --ask-become-pass (-K in short)
-b, --become run operations with become (nopasswd implied)
--become-method=METHOD privilege escalation method to use (default=sudo),
valid choices: [ sudo | su | pbrun | pfexec | runas ]
-s, --sudo run operations with sudo (nopasswd) (deprecated, use become)
-S, --su run operations with su (deprecated, use become)
ansible aws -m setup -a "filter=ansible_all_ipv4_addresses" -u user #all ip addresses
ansible local -m setup -a "filter=ans*ipv4*" #filter facts
ansible appsrv -m shell -a "apt-get -y install lynx" -b --ask-become-pass #-s deprecated replaced by -b "become"
ansible app -m apt -a "pkg=lynx state=installed update_cache=true" -b -K
ansible app -m file -a "path=/tmp/etc state=directory mode=0700 owner=root" #create directory
ansible app -m copy -a "src=/etc/fstab dest=/tmp/etc/fstab" #copy a file to a remote system
ansible app -m command -a "rm -rf /tmp/etc/fstab" #delete a file
ansible app -m service -a "name=apache2 state=stopped" -u user -b -K #stop Ubuntu apache
ansible app -m apt -a "name=apache2 state=absent" -b --ask-become-pass #removes package
ansible aws -m user -a "user=piotr state=present uid=5001 shell=/bin/bash"
ansible aws -m user -a "user=piotr state=absent remove=yes" #remove=yes removes also HOME and any emails
ansible aws -m cron -a "name=List minute=0 hour=12 job='ls -al /var > /root/var.log'" -u user --become
ansible aws -m command -a "crontab -l" -u user -b
Specify a user that ansible control server should connect as, a key also can be specified
$ ansible centos -m ping -u username --private-key=~/.ssh/id_rsa
Copy a user ssh public key to remote server, if you do not specify a username, the current user will be used
ssh-copy-id username@server.com
Execute a module against localhost, --connection local
ansible all -i "localhost," -c local -m shell -a "apt-get remove git -y" --become ansible all -i "localhost," -c local -m apt -a "pkg=git state=installed" --become
Capture Ansible output into JSON and save on the local control_server in directory called eg. install_resuts
ansible aws -m apt -a "name=lynx state=installed" -t install_resuts
Dry run:
- run ansible comamnd with
--checkparameter to validate playbook/ad-hoc commands, no modification will be made on remote nodes
Modules
shellis not interactive, therefore 'apt-get install' requires -y flag. STDOUT is displayed on your terminal. The pipe and all redirections do work. Executes commands on a remote node.copy- copies files from a local control server to remote nodefetch- copies files from remote node to the local box
Get facts
Examples of the most common facts. It requires Python to be installed on a remote node
ansible awsweb -m setup -a 'filter=ansible_distr*' -u user --become --ask-become-pass ansible awsweb -m setup -a 'filter=ansible_fqdn' ansible awsweb -m setup -a 'filter=ansible_interfaces' ansible awsweb -m setup -a 'filter=ansible_kernel' ansible awsweb -m setup -a 'filter=ansible_mem*' ansible awsweb -m setup -a 'filter=ansible_proc*'
Config file lookup process
Nearly all parameters can be overridden in ansible-playbook or with command line flags. ansible will read ANSIBLE_CONFIG, ansible.cfg in the current working directory, .ansible.cfg in the home directory or /etc/ansible/ansible.cfg, whichever it finds first.
export ANSIBLE_CONFIG=/home/test/config/ansible.cfgenvironment variableansible.cfgin the working current directory a command is run~/.ansible.cfgin a user home directory/etc/ansible/ansible.cfgAnsible system config file
AWS credentials
In order to allow Ansible to interact with AWS Resources it requires secret keys to call APIs. These can be hard coded within a playbook or exported within your environment. Below you can find useful script that you can source each time you need Ansible to manage your AWS infrastructure.
vi credentials.sh export AWS_ACCOUNT_ID='777666555444' # not required but handy to have export AWS_DEFAULT_REGION='eu-west-1' # not required export AWS_ACCESS_KEY_ID='AAABBBCCCDDDEEEFFF11' # Ansible environment variable export AWS_SECRET_ACCESS_KEY='tttttssssspppppooooozz' # Ansible environment variable
Prepare environment for automation
Ansible Control server - place where you run playbooks from, it connects to all other hosts therefore it requires:
- ansible user
- ssh key pair eg. ansible.pem and ansible.pub
- ssh config with disabled strict check or all remote hosts IPs in known_hosts file
- python 2.7 installed
Configuration nodes (servers you with to manage)
- ansible user with sudo privileges that does not require a password
- ssh ansible.pub key content in ~/.ssh/authorized_keys file
- python 2.7 installed
Note: When working with AWS each instance already has public key inserted into it's default user. Therefore to connect from Ansible control server to Ubuntu AMI you connect with ubuntu@aws.amazon.com. To connect to Amazon AMI Linux you'd connect using ec2-user@aws.amazon.com with your VPC private key.
Add remote nodes PUB keys to known_hosts
ssh-keyscan hos1 host2 host3 >> ./.ssh/known_hosts
Add remote nodes fingerprints to known_hosts file on Ansible Control server using Ansible playbook (not tested)
---
# ansible playbook that adds ssh fingerprints to known_hosts
- hosts: all
connection: local
gather_facts: no
tasks:
- command: /usr/bin/ssh-keyscan -T 10 {{ ansible_host }}
register: keyscan
- lineinfile: name=~/.ssh/known_hosts create=yes line={{ item }}
with_items: '{{ keyscan.stdout_lines }}'
Add Ansible's Control Server user public key onto remote nodes
Install the control_node's PUB key onto remote nodes to enable password-less ssh connection. Do not use sudo elevated privileges as this would add our PUB key to a remote note root user.
$ ansible-playbook ssh-addkey.yml -u vagrant --ask-pass
The playbook ssh-addkey.yml looks like
--- #Default vagrant user password is 'vagrant'
- hosts: all
gather_facts: no
remote_user: vagrant
tasks:
- name: install ssh key
authorized_key: user=vagrant key="{{ lookup('file', '/home/vagrant/.ssh/id_rsa.pub') }}" state=present
Allow an ansible_service user to run sudo without password
sudo visudo ansible ALL=(ALL) NOPASSWD: ALL #user can run as root without password sudo -l #check your rules, last rule take precedence
Stop Ansible to require sudo password at each run
sudo vi /etc/ansible/ansible.cfg #ask_sudo_pass = True #needs to be commented out, otherwise works like --ask-become-pass
Install ansible_service user ssh public key on a localhost
ssh-copy-id localhost ssh-copy-id localhost.localdomain
Variables used within playbooks - TARGET SECTION
Specific to a playbook by adding a section:
- hosts: awsweb
vars:
controls_server: localhost
web_root: /var/wwwroot
tasks:
- name: Task1
Include variables from files
- hosts: awsweb #Example of variables file content: vars_files: cat ./vars.yml - vars.yml ---------> --- # YAML file tasks: controls_server: localhost - name: Task1 web_root: /var/wwwroot
Prompt a user to provide a value to the variable
- hosts: awsweb vars_prompt: - name: controls_server #variable name prompt: Provide Controls Serve name tasks: - name: Task1
Handlers section
In the example below handler 'Restart Apache' will be called only on change status of 'Install apache web server' task
tasks:
- name: Install apache web server
action: apt pkg=apache2 state=installed
tags:
- packages
notify: Restart Apache #notification matches the name of the handler
handlers:
- name: Restart Apache
action: service name=apache2 state=restarted
- Execution order
Handlers are run in the order they are listed in a handlers file, not in the order that they are notified.
These notify actions are triggered at the end of each block of tasks in a playbook, and will only be triggered once even if notified by multiple different tasks. They are triggered after all tasks completed successfully.
Variables passed at command line
Any variable can be passed at command line
ansible-playbook variable_fromcommandline.yml --extra-vars "hosts=awsweb user=user pkg=telnet"
Tags section
Run a specific part of the configuration without running the whole playbook. See the playbook example above tags section
ansible-playbook example.yml --tags "configuration,packages" #will run only tags sections ansible-playbook example.yml --skip-tags "notification" #skips tagged sections
There is a special always tag that will always run a task, unless specifically skipped (–skip-tags always)
Start at task and step
ansible-playbook example.yml --start-at-task="Name of task" #execute from this task to the EOF ansible-playbook example.yml --step #will step you through tasks asking y/n/c
You can use a wildcard * within the task name.
Conditional statements and loops
- wait_for
- name: Wait for port 80 is listening
wait_for:
port: 80
state: started
- until
The play below will loop until: find a string in shell: output.
- name: Check if HTTPD is running
shell: system status httpd
register: result
until: result.stdout.find("active (running)") #search string
retries: 5 #in seconds
delay: 5
Roles
Roles are automation around include directives. Therefore directories like tasks, handlers, vars and meta are automatically included as long the 'main.yml' file is there. There is no need to reference them using referenced or absolute paths, they are automatically available and included in plays.
The main.yml file contains all directives relevant to the uppper directory it is in, can contain a list of tasks, handles or vars.
Let's assume you have structure like this
site.yml webservers.yml <- eg. master control file recalls 'roles' named as a directory in roles/rolename fooservers.yml roles/ common/ webservers/ files/ files/ - files used locally or transferred to a remote node templates/ templates/ - Jinja2 templates tasks/ tasks/ - (include) individual tasks that play will do something handlers/ handlers/ - (include) eg. server restarts, shared among tasks vars/ vars/ - (vars) binary values something equals something else defaults/ defaults/ - default settings meta/ meta/ - (vars) roles dependencies
In a playbook (eg. webservers.yml), it would look like this:
---
- hosts: webservers
roles:
- common
- webservers
This designates the following behaviors, for each role 'x':
- If roles/x/tasks/main.yml exists, tasks listed therein will be added to the play
- If roles/x/handlers/main.yml exists, handlers listed therein will be added to the play
- If roles/x/vars/main.yml exists, variables listed therein will be added to the play
- If roles/x/meta/main.yml exists, any role dependencies listed therein will be added to the list of roles
- Any copy, script, template or include tasks (in the role) can reference files in roles/x/{files,templates,tasks}/ (dir depends on task) without having to path them relatively or absolutely
Execution order
In a play any roles always execute before tasks. To manipulate the flow you can use pre_ and post_ directives
--- - hosts: awsweb pre_tasks: - name: When the ROLE start raw: date > role_start-end.log roles: - webservers post_tasks: - name: When the ROLE end raw: date >> role_start-end.log
Roles path
You make roles system-wide available by including its path in /etc/ansible/ansible.cfg file
roles_path = /home/test/playbooks/roles:/etc/ansible/roles
By default, a playbook that calls a role searches relative directories:
./roles/name_of_role/ ./name_of_role/
Ansible Vault
ansible-vault create secure.yml #give a password that will be used as an encryption key ansible-vault rekey secure.yml #change a password ansible-vault decrypt secure.yml #decrypt the file ansible-vault encrypt secure.yml #encrypt existing file, can pass multiple files at once
Use Vault with playbooks
ansible-playbook secure.yml --ask-vault-pass
README.md - markdown format
.md stands for markdown and is generated at the bottom of your github page as html Typical syntax includes:
# H1 ## H2 ### H3 #### H4 ##### H5 ###### H6
Alternatively, for H1 and H2, an underline-ish style:
Alt-H1 heading ====== Alt-H2 subheading ------
*This will be Italic* **This will be Bold** - Unordered list can also use '-' or '+'
- Inline `code` has `back-ticks around` it.
- Blocks of code are either fenced by lines with three back-ticks ```, or are indented with four spaces
Code blocks are between triple back-ticks
``` echo "Hello world" ```
References
Practical uses
Get ip address from facts
This is an example playbook that assigns a fact variable to locally defined variable or call the fact variable directly in the debug statement.
---
- hosts: localhost
connection: local
vars:
- IpAddr: "{{ ansible_default_ipv4.address }}" #assigns a fact variable to local variable, quotes to expand the variable value are necessary
tasks:
- debug: var=ansible_default_ipv4.address
- debug: var=IpAddr
Example of a task to read delegated node ip address
- name: Get private IP address of AWS node
action: setup
delegate_to: fqdn_or_ip
register: private_ip
- debug: var=ansible_default_ipv4.address #IP can be referenced simply by {{ ansible_default_ipv4.address }}
Reference
- Ansible installation
- My GitLab repository Linux Academy - Ansible training examples
- Learning Ansible with Vagrant Justin at sysadmincasts.com
- List of boto supported AWS services Python.org