├── LICENSE ├── README.md ├── ansible.cfg ├── host_vars ├── bar.example.com.yml └── foo.example.com.yml ├── inventory ├── local.yml ├── playbook.yml └── roles └── ansible-pull ├── defaults └── main.yml ├── tasks └── main.yml └── templates ├── ansible-pull.service └── ansible-pull.timer /LICENSE: -------------------------------------------------------------------------------- 1 | Creative Commons Legal Code 2 | 3 | CC0 1.0 Universal 4 | 5 | CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE 6 | LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN 7 | ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS 8 | INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES 9 | REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS 10 | PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM 11 | THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED 12 | HEREUNDER. 13 | 14 | Statement of Purpose 15 | 16 | The laws of most jurisdictions throughout the world automatically confer 17 | exclusive Copyright and Related Rights (defined below) upon the creator 18 | and subsequent owner(s) (each and all, an "owner") of an original work of 19 | authorship and/or a database (each, a "Work"). 20 | 21 | Certain owners wish to permanently relinquish those rights to a Work for 22 | the purpose of contributing to a commons of creative, cultural and 23 | scientific works ("Commons") that the public can reliably and without fear 24 | of later claims of infringement build upon, modify, incorporate in other 25 | works, reuse and redistribute as freely as possible in any form whatsoever 26 | and for any purposes, including without limitation commercial purposes. 27 | These owners may contribute to the Commons to promote the ideal of a free 28 | culture and the further production of creative, cultural and scientific 29 | works, or to gain reputation or greater distribution for their Work in 30 | part through the use and efforts of others. 31 | 32 | For these and/or other purposes and motivations, and without any 33 | expectation of additional consideration or compensation, the person 34 | associating CC0 with a Work (the "Affirmer"), to the extent that he or she 35 | is an owner of Copyright and Related Rights in the Work, voluntarily 36 | elects to apply CC0 to the Work and publicly distribute the Work under its 37 | terms, with knowledge of his or her Copyright and Related Rights in the 38 | Work and the meaning and intended legal effect of CC0 on those rights. 39 | 40 | 1. Copyright and Related Rights. A Work made available under CC0 may be 41 | protected by copyright and related or neighboring rights ("Copyright and 42 | Related Rights"). Copyright and Related Rights include, but are not 43 | limited to, the following: 44 | 45 | i. the right to reproduce, adapt, distribute, perform, display, 46 | communicate, and translate a Work; 47 | ii. moral rights retained by the original author(s) and/or performer(s); 48 | iii. publicity and privacy rights pertaining to a person's image or 49 | likeness depicted in a Work; 50 | iv. rights protecting against unfair competition in regards to a Work, 51 | subject to the limitations in paragraph 4(a), below; 52 | v. rights protecting the extraction, dissemination, use and reuse of data 53 | in a Work; 54 | vi. database rights (such as those arising under Directive 96/9/EC of the 55 | European Parliament and of the Council of 11 March 1996 on the legal 56 | protection of databases, and under any national implementation 57 | thereof, including any amended or successor version of such 58 | directive); and 59 | vii. other similar, equivalent or corresponding rights throughout the 60 | world based on applicable law or treaty, and any national 61 | implementations thereof. 62 | 63 | 2. Waiver. To the greatest extent permitted by, but not in contravention 64 | of, applicable law, Affirmer hereby overtly, fully, permanently, 65 | irrevocably and unconditionally waives, abandons, and surrenders all of 66 | Affirmer's Copyright and Related Rights and associated claims and causes 67 | of action, whether now known or unknown (including existing as well as 68 | future claims and causes of action), in the Work (i) in all territories 69 | worldwide, (ii) for the maximum duration provided by applicable law or 70 | treaty (including future time extensions), (iii) in any current or future 71 | medium and for any number of copies, and (iv) for any purpose whatsoever, 72 | including without limitation commercial, advertising or promotional 73 | purposes (the "Waiver"). Affirmer makes the Waiver for th benefit of each 74 | member of the public at large and to the detriment of Affirmer's heirs and 75 | successors, fully intending that such Waiver shall not be subject to 76 | revocation, rescission, cancellation, termination, or any other legal or 77 | equitable action to disrupt the quiet enjoyment of the Work by the public 78 | as contemplated by Affirmer's express Statement of Purpose. 79 | 80 | 3. Public License Fallback. Should any part of the Waiver for any reason 81 | be judged legally invalid or ineffective under applicable law, then the 82 | Waiver shall be preserved to the maximum extent permitted taking into 83 | account Affirmer's express Statement of Purpose. In addition, to the 84 | extent the Waiver is so judged Affirmer hereby grants to each affected 85 | person a royalty-free, non transferable, non sublicensable, non exclusive, 86 | irrevocable and unconditional license to exercise Affirmer's Copyright and 87 | Related Rights in the Work (i) in all territories worldwide, (ii) for the 88 | maximum duration provided by applicable law or treaty (including future 89 | time extensions), (iii) in any current or future medium and for any number 90 | of copies, and (iv) for any purpose whatsoever, including without 91 | limitation commercial, advertising or promotional purposes (the 92 | "License"). The License shall be deemed effective as of the date CC0 was 93 | applied by Affirmer to the Work. Should any part of the License for any 94 | reason be judged legally invalid or ineffective under applicable law, such 95 | partial invalidity or ineffectiveness shall not invalidate the remainder 96 | of the License, and in such case Affirmer hereby affirms that he or she 97 | will not (i) exercise any of his or her remaining Copyright and Related 98 | Rights in the Work or (ii) assert any associated claims and causes of 99 | action with respect to the Work, in either case contrary to Affirmer's 100 | express Statement of Purpose. 101 | 102 | 4. Limitations and Disclaimers. 103 | 104 | a. No trademark or patent rights held by Affirmer are waived, abandoned, 105 | surrendered, licensed or otherwise affected by this document. 106 | b. Affirmer offers the Work as-is and makes no representations or 107 | warranties of any kind concerning the Work, express, implied, 108 | statutory or otherwise, including without limitation warranties of 109 | title, merchantability, fitness for a particular purpose, non 110 | infringement, or the absence of latent or other defects, accuracy, or 111 | the present or absence of errors, whether or not discoverable, all to 112 | the greatest extent permissible under applicable law. 113 | c. Affirmer disclaims responsibility for clearing rights of other persons 114 | that may apply to the Work or any use thereof, including without 115 | limitation any person's Copyright and Related Rights in the Work. 116 | Further, Affirmer disclaims responsibility for obtaining any necessary 117 | consents, permissions or other rights required for any use of the 118 | Work. 119 | d. Affirmer understands and acknowledges that Creative Commons is not a 120 | party to this document and has no duty or obligation with respect to 121 | this CC0 or use of the Work. 122 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ansible-pull-example 2 | 3 | This repo serves as an example of ansible's pull mode. 4 | 5 | I found pull mode somewhat under-documented, so this repo is 6 | intended to provide a practical example for people wishing to get 7 | started with ansible-pull. In particular, it collects a bit of 8 | utility code that should enable some basic workflows. 9 | 10 | Running ansible in pull mode makes a different trade-off than the 11 | usual centralized ansible workflow. The main benefits are the 12 | implicit scaling to a large number of nodes, a simple 13 | repository-oriented workflow, and avoiding the need for 14 | awx/tower. Drawbacks are mainly that the pull workflow is 15 | somewhat obscure, results in eventually consistent 16 | infrastructure, and has some gotchas detailed below. 17 | 18 | ## invoking ansible 19 | 20 | You'll want to invoke ansible like this if you use this ansible-pull setup: 21 | 22 | ``` 23 | # pull mode (suitable for automation) 24 | $ ansible-pull -U https://git.example.com/ansible.git -i "$(hostname --short)," 25 | 26 | # push mode (development) 27 | $ ansible-playbook -i inventory ./playbook.yml --limit foo.example.com 28 | ``` 29 | 30 | ## practical ansible-pull 31 | 32 | ansible-pull changes the ansible workflow a little. Usually 33 | ansible is run on a central server and targets a set of remote 34 | hosts. In pull mode, each remote host pulls the whole ansible 35 | repository from source control and runs a copy of ansible with 36 | only itself as the sole "remote" host. This results in a few 37 | oddities: 38 | 39 | 1. `inventory_hostname` defaults to `localhost` 40 | 2. groups are unavailable 41 | 3. each play is only applied to the current host (`delegate_to` doesn't work) 42 | 4. pull codebases are usually slower to iterate on when 43 | developing 44 | 5. every host needs a copy of ansible and all modules and their dependencies used by the playbooks installed 45 | 6. hosts must be able to pull the ansible repo 46 | 7. credential management requires a separate solution 47 | 48 | Some approaches for mitigating these oddities follow: 49 | 50 | While the `inventory_hostname` is always localhost by default, it 51 | can be explicitly specified when invoking `ansible-pull`. 52 | 53 | The unavailability of groups is worked around by tagging each 54 | host with their groups in `host_vars` instead of including this 55 | grouping in an inventory. Playbooks can then use this mapping to 56 | synthesize the equivalent push mode groups. 57 | 58 | These sythetic groups are turned into proper groups by the 59 | inventory script that I've provided. This enables push-style 60 | development, which allows iterating on changes more quickly than 61 | solely relying on the pull flow. 62 | 63 | In pull mode, ansible calls a playbook named `local.yml` by 64 | default; the `local.yml` in this repo does the group 65 | synthesis that I described, and then goes on to invoke the 66 | example playbook `playbook.yml`. When developing, you'd invoke 67 | `playbook.yml` in push mode instead, using the inventory script 68 | provided. 69 | 70 | The one-host-per-play limitation doesn't really have a 71 | workaround. If you rely on `host_vars` or facts from other hosts 72 | in a play, you'll need to provide some other data plane for 73 | sharing this information. Some reasonable solutions are static 74 | info in `host_vars`, custom lookup plugins, or something like etcd. 75 | However, consider that pull mode may not be the right solution if 76 | your workflows rely on cross-host communication. 77 | 78 | You'll want to install at least ansible on every host 79 | participating in pull mode. Note that this also applies to the 80 | dependencies listed on each module, and the modules themselves, 81 | too, if they aren't in ansible base. In large sites this can add 82 | up to a considerable amount of total disk space. 83 | 84 | Requiring hosts to track the repository containing your playbooks 85 | also implies a few things. The load on your repository server 86 | scales linearly in the number of hosts using pull mode and 87 | firewalling becomes more difficult. Unless your repository is 88 | anonymously and globally readable, you'll need some way of 89 | provisioning initial credentials on your target hosts to be able 90 | to access it at all. SSH certificates may also be of interest 91 | here. 92 | 93 | This credentials issue also shows up elsewhere. In many setups, 94 | the central server will have some level of access to secrets that 95 | are then pushed to remote hosts. In pull mode, each remote host 96 | is their own central server, so each one requires access to 97 | secrets. This makes several solutions that work well in push 98 | mode, like ansible vault, difficult to deploy securely in pull 99 | mode. Larger setups will probably want to set up something like 100 | Hashicorp's Vault or similar secret management services. 101 | 102 | Finally, I've provided a sample `ansible-pull` role, some example 103 | playbooks, and `host_vars` to help you get started. 104 | -------------------------------------------------------------------------------- /ansible.cfg: -------------------------------------------------------------------------------- 1 | [defaults] 2 | nocows = 1 3 | -------------------------------------------------------------------------------- /host_vars/bar.example.com.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | pull_groups: 4 | - mygroup 5 | - onlybar 6 | -------------------------------------------------------------------------------- /host_vars/foo.example.com.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | pull_groups: 4 | - mygroup 5 | - onlyfoo 6 | -------------------------------------------------------------------------------- /inventory: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # SPDX-License-Identifier: CC0-1.0 4 | 5 | import yaml 6 | import os 7 | import json 8 | import argparse 9 | import collections as coll 10 | 11 | inventory_defaults = { 12 | 'ansible_ssh_port': 22, 13 | 'ansible_ssh_user': 'root', 14 | } 15 | 16 | ansibledir = os.path.dirname(os.path.realpath(__file__)) 17 | host_vars = os.path.join(ansibledir, 'host_vars') 18 | 19 | ansible_groups = coll.defaultdict(list) 20 | ansible_hosts = {} 21 | 22 | for entry in os.scandir(host_vars): 23 | if entry.name.endswith('.yml'): 24 | host = entry.name[:-4] 25 | 26 | yml = {} 27 | with open(entry.path, 'r') as stream: 28 | yml = yaml.safe_load(stream) 29 | 30 | ansible_hosts[host] = { 'ansible_host': host } 31 | for group in yml.get('pull_groups', []): 32 | ansible_groups[group].append(host) 33 | 34 | parser = argparse.ArgumentParser( 35 | description='convert ansible-pull data into push inventory' 36 | ) 37 | pgroup = parser.add_mutually_exclusive_group(required=True) 38 | pgroup.add_argument('-l', '--list', action='store_true', help='list inventory') 39 | pgroup.add_argument('-H', '--host', action='store', help='get a host\'s variables') 40 | args = parser.parse_args() 41 | 42 | if args.host: 43 | print(json.dumps(ansible_hosts[args.host])) 44 | elif args.list: 45 | # format ref: https://docs.ansible.com/ansible/latest/dev_guide/developing_inventory.html 46 | print(json.dumps({ 47 | '_meta': { 48 | 'hostvars': { **ansible_hosts } 49 | }, 50 | 'all': { 51 | 'hosts': list(ansible_hosts.keys()), 52 | 'vars': inventory_defaults, 53 | }, 54 | **{x[0]: {'hosts': x[1]} for x in ansible_groups.items()} 55 | })) 56 | else: 57 | raise NotImplementedError() 58 | -------------------------------------------------------------------------------- /local.yml: -------------------------------------------------------------------------------- 1 | - hosts: all 2 | tasks: 3 | - name: add hosts to inventory groups 4 | add_host: 5 | name: "{{ inventory_hostname }}" 6 | groups: "{{ hostvars[inventory_hostname].pull_groups }}" 7 | changed_when: False 8 | 9 | - import_playbook: playbook.yml 10 | -------------------------------------------------------------------------------- /playbook.yml: -------------------------------------------------------------------------------- 1 | - hosts: mygroup 2 | tasks: 3 | - debug: 4 | msg: "I'm in group 'mygroup'" 5 | 6 | - hosts: onlyfoo 7 | tasks: 8 | - debug: 9 | msg: "I'm in group 'onlyfoo'" 10 | 11 | - hosts: onlybar 12 | tasks: 13 | - debug: 14 | msg: "I'm in group 'onlybar'" 15 | 16 | - name: configure ansible-pull 17 | hosts: all 18 | roles: 19 | - role: ansible-pull 20 | ansible_pull_url: https://git.example.com/ansible.git 21 | -------------------------------------------------------------------------------- /roles/ansible-pull/defaults/main.yml: -------------------------------------------------------------------------------- 1 | #ansible_pull_url: https://git.example.com/ansible.git 2 | ansible_pull_timer: | 3 | OnBootSec=5m 4 | OnCalendar=daily 5 | RandomizedDelaySec=30m 6 | -------------------------------------------------------------------------------- /roles/ansible-pull/tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | - name: install units for ansible-pull 4 | template: 5 | dest: /etc/systemd/system 6 | src: "{{ item }}" 7 | loop: 8 | - ansible-pull.service 9 | - ansible-pull.timer 10 | 11 | - name: enable ansible-pull timer 12 | service: 13 | name: ansible-pull.timer 14 | state: started 15 | enabled: yes 16 | -------------------------------------------------------------------------------- /roles/ansible-pull/templates/ansible-pull.service: -------------------------------------------------------------------------------- 1 | {{ ansible_managed|comment }} 2 | 3 | [Unit] 4 | Description=system configuration upgrade 5 | After=network-online.target 6 | 7 | [Service] 8 | ExecStart=/bin/sh -c 'ansible-pull -U "{{ ansible_pull_url }}" -i "$(uname -n)," --diff' 9 | TimeoutStopSec=600 10 | -------------------------------------------------------------------------------- /roles/ansible-pull/templates/ansible-pull.timer: -------------------------------------------------------------------------------- 1 | {{ ansible_managed|comment }} 2 | 3 | [Unit] 4 | Description=system configuration upgrade 5 | 6 | [Timer] 7 | {{ ansible_pull_timer }} 8 | Unit=ansible-pull.service 9 | Persistent=true 10 | 11 | [Install] 12 | WantedBy=multi-user.target 13 | --------------------------------------------------------------------------------