[ansible] Ansible: deploy on multiple hosts in the same time

Is it possible to run ansible playbook, which looks like this (it is an example from this site: http://docs.ansible.com/playbooks_roles.html):

- name: this is a play at the top level of a file
  hosts: all
  remote_user: root
  tasks:
  - name: say hi
    tags: foo
    shell: echo "hi..."

- include: load_balancers.yml
- include: webservers.yml
- include: dbservers.yml

in multithread mode?

I want to run three "includes" in the same time (it is deploying to different hosts anyway), like in this diagram:

http://www.gliffy.com/go/publish/5267618

Is it possible?

Ansible supports patterns which can be used for one/multiple hosts,groups and also can exclude/include groups. Here is the link for official document.

As mentioned before: By default Ansible will attempt to run on all hosts in parallel, but task after Task(serial).

If you also want to run Tasks in parallel you have to start different instances of ansible. Here are some ways to to it.

1. Groups

If you already have different groups you can run one ansible instance for each group:

shell-1 #> ansible-playbook site.yml --limit webservers
shell-2 #> ansible-playbook site.yml --limit dbservers
shell-3 #> ansible-playbook site.yml --limit load_balancers

2. Multiple shells

Playbooks

If your playbooks work standalone you can although do this:

shell-1 #> ansible-playbook load_balancers.yml
shell-2 #> ansible-playbook webservers.yml
shell-3 #> ansible-playbook dbservers.yml

Limit

If not, you can let ansible do the fragmentation. When you have 6 hosts and want to run 3 instances with 2 host each, you can do something like this:

shell-1 #> ansible-playbook site.yml --limit all[0-2]
shell-2 #> ansible-playbook site.yml --limit all[2-4]
shell-3 #> ansible-playbook site.yml --limit all[4-6]

3. Background

Of course you can use one shell and put the tasks in background, an simple example would be:

shell-1 #> ansible-playbook site.yml --limit all[0-2] &
shell-1 #> ansible-playbook site.yml --limit all[2-4] &
shell-1 #> ansible-playbook site.yml --limit all[4-6] &

With this method you get all output mixed together in one terminal. To avoid this you can write the output to different files.

ansible-playbook site.yml --limit all[0-2] > log1 &
ansible-playbook site.yml --limit all[2-4] > log2 &
ansible-playbook site.yml --limit all[4-6] > log3 &

4. Better Solutions

Maybe it's better to use a tool like tmux / screen to start the instances in virtual shells.

Or have a look at the "fireball mode": http://jpmens.net/2012/10/01/dramatically-speeding-up-ansible-runs/

If you want to know more about limits, look here: https://docs.ansible.com/playbooks_best_practices.html#what-this-organization-enables-examples

In my case I needed the configuration stage to be blocking as a whole, but execute each role in parallel. I've tackled this issue using the following code:

echo webserver loadbalancer database | tr ' ' '\n' \
| xargs -I % -P 3 bash -c 'ansible-playbook $1.yml' -- %

the -P 3 argument in xargs makes sure that all the commands are ran in parallel, each command executes the respective playbook and the command as a whole blocks until all parts are finished.

As of Ansible 2.0 there seems to be an option called strategy on a playbook. When setting the strategy to free, the playbook plays tasks on each host without waiting to the others. See http://docs.ansible.com/ansible/playbooks_strategies.html.

It looks something like this (taken from the above link):

- hosts: all
  strategy: free
  tasks:
  ...

Please note that I didn't check this and I'm very new to Ansible. I was just curious about doing what you described and happened to come acroess this strategy thing.

EDIT:

It seems like this is not exactly what you're trying to do. Maybe "async tasks" is more appropriate as described here: http://docs.ansible.com/ansible/playbooks_async.html.

This includes specifying async and poll on a task. The following is taken from the 2nd link I mentioned:

- name: simulate long running op, allow to run for 45 sec, fire and forget
  command: /bin/sleep 15
  async: 45
  poll: 0

I guess you can specify longer async times if your task is lengthy. You can probably define your three concurrent task this way.

Check out this POC or MVP of running in parallel one-host-from-every-group (for all hosts) https://github.com/sirkubax/szkolenie3/tree/master/playbooks/playgroups

you may get the inspiration

By default Ansible will attempt to run on all hosts in parallel. See these Ansible docs for details. You can also use the serial parameter to limit the number of parallel hosts you want to be processed at any given time, so if you want to have a playbook run on just one host at a time you can specify serial:1, etc.

Ansible is designed so that each task will be run on all hosts before continuing on to the next task. So if you have 3 tasks it will ensure task 1 runs on all your hosts first, then task 2 is run, then task 3 is run. See this section of the Ansible docs for more details on this.