Automate Your Infrastructure with Ansible

http://www.devx.com/opensource/automate-your-infrastructure-with-ansible.html

Ansible is a tool that allows you to control remote servers from the comfort of your laptop. It works over SSH and doesn't require any special software or agent to be installed on the remote machine.

That doesn't sound like much. Any beginning sys admin knows how to run commands or complete shell scripts on remote servers. The power of Ansible comes from its sophisticated conceptual model and from its plugin-based module system. The combination of the two allows you to manage, configure and efficiently orchestrate hundreds, and even thousands, of servers. To top it all off, Ansible is implemented in Python which is my favorite language.

Playing with Ansible on Vagrant

Vagrant is a tool that lets you manage an army of virtual machines right from your laptop. See my First Steps with Vagrant article. It is the perfect companion for Ansible development and testing. In this article, I'll provide examples using a cluster of three Vagrant virtual machines. If you want to follow along, install VirtualBox and Vagrant and use this Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :
hosts = {
  "athos" => "192.168.66.10",
  "porthos" => "192.168.66.11",
  "aramis" => "192.168.66.12"
}
Vagrant.configure("2") do |config|
  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"
 hosts.each do |name, ip|
    config.vm.define name do |machine| 
      machine.vm.network :private_network, ip: ip
      machine.vm.provider "virtualbox" do |v|
        v.name = name
      end
    end
  end
end

To make the cluster easily accessible through SSH type:

vagrant ssh-config >> ~/.ssh-config

This will take care of all the gory details of SSH key-pair generation and user names. Now you should be able to just type:

ssh athos

This will land you in a live SSH session on the athos VM.

Quick Ad-Hoc Commands

Let's start by verifying that all the nodes in our cluster are up and get some information. The first step is to tell Ansible about the machines in our cluster with the inventory file. Ansible uses primarily YAML files, but also supports JSON files. Inventory files are a special format and use the .INI format where the sections are host groups and the items in each group are host names or IP addresses. Here is our inventory file called 'hosts':

[group1]
athos

[group2]
porthos
aramis

To check if all the nodes are up use the 'ping' module:

ansible all -i hosts -m ping

You should see the following: http://i.imgur.com/1SWoMdz.png

The 'all' argument is a host pattern and means run the command on all the hosts. You may also specify individual hosts or host groups. The -i tells Ansible which inventory file to use and the -m is the module to run. There could be many other arguments. In particular, the remote user to run the command and the credentials, which could be a password or a private key file. In this case, Ansible picked up this information from the ~/.ssh/config file.

Here is another example to collect a lot of information using the 'setup' module. Let's run it only on group2:

ansible group2 -i hosts -m setup 

The output is too large, so I created a gist you can explore here.

Playbooks

Ad-hoc commands are all fun and games, but if you are serious then you would want to execute a different series of commands on different host groups. This is where playbooks come in. Playbooks are a list of plays. Each play is a list of tasks to be executed on a group of hosts (could be just one).

The following playbook contains a single play that Installs IPython on every host and then creates a user named gigi and copies the SSH public key to the target host.

-- -
- hosts: all
   tasks:
     - name: Install ipython
      apt: name=ipython state=installed
     - name: Create a user named gigi
     user: name=gigi
     - name: Add public key
     authorized_key: user=gigi key="{{ lookup('file', '~/.ssh/id_rsa.pub') }}" 

Here is how you run the playbook with the ansible-playbook command:

ansible-playbook -i hosts playbook.yml -- sudo 

It is pretty similar to the ansible command, but instead of specifying a module you just provide the name of a playbook. The -- sudo flag executes the playbook commands as sudo, which is often required.

The tasks themselves are actually modules. You can execute each task as an ad-hoc command using the Ansible tool, but it would be very tedious and error-prone. Whenever you have a logical sequence of commands that you want to execute at the same time, consider putting them in a playbook.

The syntax is pretty simple and there are good online reference for the available options for each module here.

Sometimes, you can use more sophisticated approaches with templates and dynamically discovering information during execution. For example, the last task: authroized_key uses the lookup plugin to fetch the content of the local user's id_rsa.pub file and write it to the remote authorized_keys file.

Vars

Playbooks are great and when you combine them with inventories you can execute the same playbooks with different inventory files. You will often need slight variations. For example, in our playbook I created a user called 'gigi' on all the hosts, but maybe I would like to create a different user for some of the hosts and the name of the user should not be hard-coded in the playbook. This is where vars (as in variables) come in.

Roles

Roles are a high-level abstraction that I use very often. I like to think of roles as the atomic unit of deployment. Roles group tasks together, but are not tied to particular hosts. Moreover, they can declare dependencies on other roles. This great because it allows the formation of hierarchies. For example, you may have many types of web applications and services that require all kinds of complicated provisioning and configuration. But, all of them may require a standard nginx installation. You can have a webserver role that other roles depend on and then in your playbook you just specify the top-level role. If at some point you want to change something in your nginx configuration, you just need to change your nginx role. All the other roles that depend on it will automatically get the changes.

Roles have their own directory structure that I will not get into right now, but they provide a lot of fine-grained control and flexibility. Here is an example of two roles: 'common' and 'webserver' where the webserver role depends on the common role:

roles/
   common/
     files/
     templates/
     tasks/
     handlers/
     vars/
     defaults/
     meta/
   webserver/
     files/
     templates/
     tasks/
     handlers/
     vars/
     defaults/
     meta/

Each role has its own directory under the roles parent dir. Then there are multiple directories for each one. The most important directory is 'tasks'. The 'tasks' directory will normally contain a file called main.yml with a list of tasks just like in the playbook. The 'meta' directory contains dependencies.

For example the webserver's 'meta' directory will contain a file called main.yml with the following content:

-- -
dependencies:
- { role: common } 

This indicates that before the webserver's tasks are executed the common role will be executed first.


A playbook for installing the webserver role for a host group called webservers would be:

-- -
- hosts: webservers
 roles:
 - webserver 

The Ansible Vault

Ansible often needs to transfer sensitive data or credentials to remote machines. While the communication is done over SSH, and is thus secure, the Ansible playbooks, roles, and in particular, var files must contain the secret information. These Ansible entities are often stored in source control systems. To protect sensitive data, Ansible provides the vault. It is a mechanism to store encrypted data that is decrypted on the fly and only when needed. The vault password should only be known to a small number of trusted individuals.

Tasks, roles and playbooks can refer to information stored in the vault and it will be available during execution. Remember that when running a playbook that requires vault access you need to provide the vault password.

Conclusion

Ansible is a structured and flexible remote execution framework that provides a coherent model for provisioning, configuring and orchestrating multiple remote machines in a secure way. It is based on abstractions like playbooks, roles, tasks and vars and is extensible via modules/plugins.