├── .gitignore ├── .travis.yml ├── LICENSE ├── README.debug ├── README.md ├── TODO ├── defaults └── main.yml ├── files ├── ansible_local.py ├── ansible_run_all.sh ├── bastion_lib.py ├── callback │ ├── lock_run.py │ └── log_sqlite.py ├── checkout_git_repos.sh ├── clean_ssh_public_keys.py ├── generate_ansible_command.py ├── help ├── list ├── update_ansible_config.sh ├── update_external_roles └── update_galaxy.sh ├── meta └── main.yml ├── tasks ├── compat_ansible_admin_group.yml ├── create_repo.yml ├── create_repos.yml ├── git_shell.yml ├── install_ansible_git.yml ├── install_ansible_rpm.yml ├── install_callback_plugins.yml ├── main.yml ├── pre_receive_checks.yml ├── push_remote.yml └── use_tor_proxy.yml └── templates ├── ansible_bastion.yml ├── ansible_git ├── ansible_git_update ├── ansible_local.sudoers ├── clean_ssh_public_keys ├── cmd_config.sudoers ├── file_changed_commit.sh ├── hooks ├── check_branch_master.py ├── check_git_ignore.py ├── push_remote.sh ├── receive-hook.sh ├── trigger_run.sh └── update_checkout.sh ├── push_remote_public.sh ├── push_remote_public.sudoers ├── rerun_last_commit.sh ├── ssh_config └── update_galaxy.sudoers /.gitignore: -------------------------------------------------------------------------------- 1 | .*.swp 2 | *.pyc 3 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | --- 2 | language: python 3 | python: "3.7" 4 | before_install: 5 | - sudo apt-get update -qq 6 | install: 7 | - pip install ansible-lint 8 | - pip install flake8 9 | - ( cd .. ; git clone https://gitlab.com/osas/ansible-role-tor.git tor) 10 | script: 11 | # verify the syntax of the playbook 12 | - set -e ; for i in tasks/*.yml; do ansible-lint $i; done 13 | # verify any shell script 14 | - set -e ; for i in files/*.sh; do bash -n $i ;done 15 | # verify python script with pyflakes and pep8 16 | - set -e ; for i in files/*.py; do flake8 $i ;done 17 | # setup the environment to run ansible 18 | - "echo '---\n- hosts: 127.0.0.1\n remote_user: root\n roles:' > role.yml" 19 | - echo " - ${TRAVIS_REPO_SLUG/*\//}" >> role.yml 20 | - "echo '[defaults]\nroles_path = ../\n' > ansible.cfg" 21 | # run a 2nd syntax check ( not sure if ansible-lint shouldn't already do that ) 22 | - ansible-playbook -i '127.0.0.1,' --syntax-check role.yml 23 | # check that the role can be run 24 | - ansible-playbook -i '127.0.0.1,' -c local --become --become-method sudo -vvvv role.yml 25 | # verify idempotence ( ie, running a 2nd time shouldn't change anything ) 26 | - ansible-playbook -i '127.0.0.1,' -c local --become --become-method sudo -vvvv role.yml | grep -q 'changed=0.*failed=0' 27 | 28 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Open Source and Standards 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.debug: -------------------------------------------------------------------------------- 1 | In order to debug the generate ansible script, you can use this: 2 | 3 | export PYTHONPATH=./files 4 | export GIT_CHECKOUT=../ansible_gluster 5 | python ./files/generate_ansible_command.py --git $GIT_CHECKOUT --path $GIT_CHECKOUT --old 832fcdece904715245af86567463a75255312b3d -n -v 6 | 7 | 8 | This will just show the command to be executed, from a local checkout of the repo (in this case, in ../ansible_gluster). 9 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Ansible module used by OSAS to manage a set of servers using Ansible. 2 | 3 | [![Build Status](https://travis-ci.org/OSAS/ansible-role-ansible_bastion.svg?branch=master)](https://travis-ci.org/OSAS/ansible-role-ansible_bastion) 4 | 5 | Architecture 6 | ------------ 7 | 8 | This role permit to install a trusted server that will deploy configurations 9 | and run playbooks safely without having to distribute ssh keys to every admin. 10 | It use for that 2 git repositories where a authorized user can push, 11 | triggering a automated deployment using a script to deploy only where it is 12 | needed ( ie, if you modify a role, only systems where the role is used will be 13 | targeted ). 14 | 15 | The role use 2 git repositories, one named "public" and the other named 16 | "private". As the name imply, the public repository is meant to be public and 17 | usually will contain everything that is not deemed private such as passwords. The 18 | private repository is used mostly for passwords, and should usually only 19 | contains groups_vars/ or vars/ repository. A post commit hook will extract 20 | the 2 repositories in /etc/ansible, and run ansible if needed. 21 | 22 | For increased security, ansible is run as a non privileged user to reduce 23 | potential attack vectors from the managed system ( see CVE-2014-3498 ). 24 | Others ideas are planned to be implemented. 25 | 26 | Variables 27 | --------- 28 | 29 | A few variables have been added to configure the role, please look at 30 | defaults/main.yml 31 | 32 | Pushing to remote repo 33 | ---------------------- 34 | 35 | It is possible to push automatically the public repository to one or more 36 | distant git repository. To do that, please use the "remotes" variable like this: 37 | 38 | ``` 39 | - hosts: bastion.example.org 40 | roles: 41 | - role: bastion 42 | remotes: 43 | - { name: 'gitlab', url: 'git@gitlab.com:user/repo.git' } 44 | ``` 45 | 46 | You can specify multiple remotes in the list. A separate '_git_pusher' user is created 47 | for that task, and a ssh key is generated for it. 48 | 49 | The playbook do not take care of setting the key on the other side, since that's typically 50 | used for various web services such as Github or Bitbucket, and that requires admin credentials. 51 | 52 | System users and groups 53 | ----------------------- 54 | 55 | Several users will be created, depending on the settings used. 56 | 57 | * ansible_admin, (variable: `ansible_username`). This user is the 58 | one connecting to remote servers. A separate user is used for that to 59 | properly separate the access and reduce the risk of stealing the ssh keys. 60 | 61 | * _git_pusher, (variable `pusher_username`). This user is used to sync the public 62 | repo to a external source. 63 | 64 | * git, (variable `git_username`). When using the git-shell based set, this is the 65 | shared user used by commiters to push modifications. 66 | 67 | Directory layout 68 | ---------------- 69 | 70 | While everything should be configurable later, for now, the layout of the git 71 | repository need to follow a few set of rules: 72 | 73 | - all roles are in roles/ 74 | - requirement.yml can be safely used to update the roles 75 | - all playbooks are in playbooks/ and the one to be used for automated deployment 76 | are named with this pattern: deploy.\*.yml 77 | 78 | Using git snapshot of Ansible 79 | ----------------------------- 80 | 81 | The role can also be used to install ansible right from git, using the HEAD of the 82 | devel branch by default. 83 | 84 | To do that, you can use the `use_ansible_git` flag, along `ansible_git_version` if 85 | you want a different branch and/or version of ansible than the default of 'devel'. This 86 | variable is passed to the 'git' ansible module, so it accept everything the module 87 | accept. 88 | 89 | Using git-shell 90 | --------------- 91 | 92 | In order to not give full shell access to the deployment server, you may opt to use 93 | git-shell and a special git user, with the option `use_git_shell`. 94 | The addition of ssh keys to the git user is not handled by this role, but can be done 95 | rather trivially with the authorized_keys module of ansible. 96 | 97 | The git-shell give access to a few commands to update the externals roles, give help and 98 | list commands. More will be added as pattern of usage will emerge. 99 | 100 | You can also change the `git_username` variable to change the name of the user used to 101 | push. For example and if you want to confuse people, you can enable a cvs user to push 102 | on git repositories with git-shell like this: 103 | 104 | ``` 105 | - hosts bastion.example.org 106 | roles: 107 | - role: bastion 108 | use_git_shell: True 109 | git_username: cvs 110 | ``` 111 | 112 | Groups based ACL 113 | ---------------- 114 | 115 | In order to ease collaboration and increase security, the repositories can be configured 116 | to be writable by a specific group, with proper sudo configuration that avoid the need to use 117 | root user. 118 | 119 | To enable this mode, you need to define the `ansible_commiters_group` variable like this: 120 | 121 | ``` 122 | - hosts: bastion.example.org 123 | roles: 124 | - role: bastion 125 | ansible_committers_group: committers 126 | ``` 127 | 128 | The group will be created if it doesn't already exist. 129 | 130 | By default, committers can only push and trigger the hooks, and this mean everything 131 | they do will be properly audited in git, and run with verification. 132 | 133 | If you want to be able to do more, you can also create a group for admins who will have 134 | more privileges such as running variables ansible commands, which would be equivalent to becoming 135 | root. 136 | 137 | To do that, you can use the `ansible_admins_group` variables like this: 138 | 139 | ``` 140 | - hosts: bastion.example.org 141 | roles: 142 | - role: bastion 143 | ansible_committers_group: committers 144 | ansible_admins_group: admins 145 | ``` 146 | 147 | Like with `ansible_committers_group`, the group will be created if it doesn't exist. Due 148 | to the way group are currently done on Linux, people in the `ansible_admins_group` do 149 | not automatically inherit the access of `ansible_committers_group` for the moment. 150 | 151 | The role will refuse to deploy anything if only `ansible_admins_group` is defined. 152 | 153 | SSH Key type 154 | ------------ 155 | 156 | By default, a RSA key is generated. If you wish to use another type of key, you can pass 157 | the option `ssh_key_type` to use a different type of key. 158 | 159 | This however requires a bugfix on ansible for the automated size selection of the key, 160 | sent as a PR on https://github.com/ansible/ansible-modules-core/pull/4074 161 | 162 | Management of tor hidden services 163 | --------------------------------- 164 | 165 | Since ansible is using ssh and ssh can be used with tor hidden services, you can 166 | choose to enable the support for tor hidden services by using the `enable_onion_support` 167 | option like this: 168 | 169 | ``` 170 | - hosts: bastion.example.org 171 | roles: 172 | - role: bastion 173 | enable_onion_support: True 174 | ``` 175 | 176 | It will configure tor and ssh to use tor for accessing a .onion hostname. 177 | 178 | It can be used like this in hosts file, to connect to a server whose hidden service 179 | is abcdefabcdef.onion: 180 | 181 | ``` 182 | [all] 183 | server ansible_host=abcdefabcdef.onion 184 | 185 | [web] 186 | server 187 | ``` 188 | 189 | Using tor for outgoing connections 190 | ---------------------------------- 191 | 192 | Alternatively, a way to hide the location of the management server is to use tor 193 | for all ssh outgoing connections. This can be done with the option `use_tor_proxy`. 194 | 195 | This will install and start a tor client, and direct outgoing ssh connections in 196 | the tor network. 197 | 198 | Local deployment 199 | ---------------- 200 | 201 | When you have one single server to deploy, you can also use a alternate mode of operation. 202 | 203 | If a playbook named local.yml, or matching the hostname or the fqdn (ie, server.yml or 204 | server.example.org.yml), it will be executed locally with a custom wrapper as root, with 205 | the local connection plugin. 206 | 207 | The naming convention of the playbook mimic the one of ansible-pull. 208 | 209 | Adding a new host to be managed 210 | ------------------------------- 211 | 212 | If a new host is added to the hosts file, the system will automatically try to connect 213 | to it to add its ssh public key to the known_hosts file. This will by default generate 214 | a harmess error message for the time being. 215 | 216 | If the option `run_on_new_host` is set, the playbook listed will also be run. This can 217 | be used to autoimatically add more ssh keys or automatically add the new server to FreeIPA. 218 | 219 | It can be used like this: 220 | 221 | ``` 222 | - hosts: bastion.example.org 223 | roles: 224 | - role: bastion 225 | run_on_new_host: 226 | - deploy_base.yml 227 | - deploy_freeipa.yml 228 | ``` 229 | 230 | Installing collections from rpms 231 | -------------------------------- 232 | 233 | Using the `rpm_collections` variable, the role will install the given collection using rpm, 234 | following the convention used by Fedora. 235 | 236 | ``` 237 | - hosts: bastion.example.org 238 | roles: 239 | - role: bastion 240 | rpm_collections: 241 | - community.general 242 | - ansible.posix 243 | ``` 244 | -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | Ideas to make the bastion more secure 2 | ------------------------------------- 3 | 4 | specific selinux policy for the user 5 | - write the policy 6 | - load/apply it 7 | 8 | use a separate /tmp 9 | 10 | automatically renew the ssh ( add a playbook for that ) 11 | 12 | make sure the firewall run ? 13 | 14 | 15 | Features to add 16 | --------------- 17 | 18 | Permit manual run from user ( sudo ) with git-shell and without, see ansible-rbac, of any playbooks 19 | 20 | Verify tasks with ansible-lint before commit in the repo 21 | 22 | Add a way to specify the ssh key for git with git-shell (look at freeipa and AuthorizedKeysCommand, or manual) 23 | 24 | Playbook for updating ssh keys 25 | - need to have a copy of the old private keys somewhere 26 | 27 | 28 | Development 29 | ----------- 30 | 31 | Add a .travis.yml for tasks/main.yml, check python and shell files. 32 | -------------------------------------------------------------------------------- /defaults/main.yml: -------------------------------------------------------------------------------- 1 | # the username that will run the ansible script 2 | # # user will be created along a pair of ssh key 3 | ansible_username: ansible_admin 4 | # git repositories location, 2 repo will be created as 5 | # private and public 6 | git_repositories_dir: /srv/git_repos 7 | # directory holding the fact cache 8 | # currently using jsonfile, since that's the easiest to deploy 9 | cache_folder: /var/cache/ansible/ 10 | # hour to run ansible run every day, by default at 4am in the morning 11 | # this is fed directly to the cron module 12 | ansible_run_hour: 4 13 | # minute to run ansible. 14 | ansible_run_minute: 0 15 | # use a git snapshot instead of the package (can be used for tracking devel, or 2.X) 16 | use_ansible_git: False 17 | # the branch of ansible to use, if using the git version 18 | ansible_git_version: devel 19 | # install a git-shell on the git user 20 | use_git_shell: False 21 | # the username for remote pushing to git repos 22 | git_username: git 23 | # the username that will run the sync script that push to a remote repo 24 | pusher_username: _git_pusher 25 | # permit to use a specific type of key 26 | ssh_key_type: rsa 27 | # add support for accessing .onion 28 | enable_onion_support: False 29 | # for the ssh connection to go trough tor 30 | use_tor_proxy: False 31 | # list of remote repository to push after a successfull run 32 | remotes: [] 33 | #list of playbook to run when a new host is added 34 | run_on_new_host: [] 35 | # list of collections to install from rpm 36 | rpm_collections: [] 37 | -------------------------------------------------------------------------------- /files/ansible_local.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # 3 | # Copyright (c) 2016 Michael Scherer 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in 13 | # all copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | # 23 | 24 | import sys 25 | import socket 26 | import subprocess 27 | 28 | if len(sys.argv) != 2: 29 | print("Error, not enough arguments") 30 | sys.exit(1) 31 | 32 | path = sys.argv[1] 33 | fqdn = socket.getfqdn() 34 | authorized_path = ['local.yml', fqdn + '.yml', fqdn.split('.')[0] + '.yml'] 35 | authorized_path = ['/etc/ansible/playbooks/' + i for i in authorized_path] 36 | 37 | if path not in authorized_path: 38 | print("Unauthorized path %s" % path) 39 | sys.exit(1) 40 | 41 | subprocess.call('ansible-playbook', '-c', 'local', '-D', path) 42 | -------------------------------------------------------------------------------- /files/ansible_run_all.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Copyright (c) 2014 Michael Scherer 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | # 23 | cd /etc/ansible 24 | 25 | for i in playbooks/deploy*.yml; do 26 | logger "Started run of $i in ansible_run_all.sh" 27 | timeout 1h ansible-playbook -D $i $* 28 | logger "Finished run of $i in ansible_run_all.sh" 29 | done 30 | -------------------------------------------------------------------------------- /files/bastion_lib.py: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright (c) 2014-2016 Michael Scherer 3 | # 4 | # Permission is hereby granted, free of charge, to any person obtaining a copy 5 | # of this software and associated documentation files (the "Software"), to deal 6 | # in the Software without restriction, including without limitation the rights 7 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | # copies of the Software, and to permit persons to whom the Software is 9 | # furnished to do so, subject to the following conditions: 10 | # 11 | # The above copyright notice and this permission notice shall be included in 12 | # all copies or substantial portions of the Software. 13 | # 14 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | # SOFTWARE. 21 | # 22 | # List of function split from generate_ansible_command.py 23 | # to be reused by others scripts 24 | 25 | import subprocess 26 | import tempfile 27 | import socket 28 | import shutil 29 | import os 30 | 31 | from ansible.parsing.dataloader import DataLoader 32 | 33 | from ansible.inventory.manager import InventoryManager 34 | 35 | from ansible.vars.manager import VariableManager 36 | 37 | 38 | class VariableManagerWrapper: 39 | def __init__(self, vm): 40 | self.vm = vm 41 | 42 | def get_vars(self, loader, host): 43 | return self.vm.get_vars(host=host) 44 | 45 | 46 | class InventoryWrapper: 47 | def __init__(self, host_list): 48 | self.loader = DataLoader() 49 | self.im = InventoryManager(loader=self.loader, 50 | sources=[host_list, ]) 51 | self.vm = VariableManager(loader=self.loader, inventory=self.im) 52 | 53 | def get_loader(self): 54 | return self.loader 55 | 56 | def get_variable_manager(self): 57 | return VariableManagerWrapper(self.vm) 58 | 59 | def get_groups(self): 60 | return self.im.groups 61 | 62 | def get_hosts(self, group): 63 | return self.im.get_hosts(pattern=group) 64 | 65 | def get_host(self, host): 66 | return self.im.get_host(host) 67 | 68 | def refresh_inventory(self): 69 | return self.im.refresh_inventory() 70 | 71 | 72 | def get_changed_files(git_repo, old, new): 73 | """ Return a list of files who changed in git_repo between 74 | old and new revision""" 75 | changed_files = set() 76 | if old == '0000000000000000000000000000000000000000': 77 | old = subprocess.check_output(['git', 78 | 'rev-list', 79 | '--max-parents=0', 80 | 'HEAD'], cwd=git_repo).strip() 81 | 82 | cwd = os.getcwd() 83 | with tempfile.TemporaryDirectory(prefix='clone_bastion') as tmpdir: 84 | os.chdir(tmpdir) 85 | subprocess.call(["git", "-c", "safe.directory='*'", "clone", "-q", git_repo, "repo"]) 86 | os.chdir('repo') 87 | diff = subprocess.check_output(["git", '--no-pager', 'diff', 88 | "--name-status", "--diff-filter=ACDMR", 89 | "%s..%s" % (old, new)]) 90 | os.chdir(cwd) 91 | 92 | for line in diff.decode("utf-8").split('\n'): 93 | if len(line) > 0: 94 | splitted = line.split() 95 | if len(splitted) == 2: 96 | (t, filename) = splitted 97 | changed_files.add(filename) 98 | elif len(splitted) == 3: 99 | (t, old_filename, new_filename) = splitted 100 | changed_files.add(old_filename) 101 | changed_files.add(new_filename) 102 | return changed_files 103 | 104 | 105 | def extract_list_hosts_git(revision, path): 106 | """ Extract the hosts list from git, after deduplciating it 107 | and resolving variables """ 108 | result = [] 109 | if revision == '0000000000000000000000000000000000000000': 110 | return result 111 | try: 112 | host_content = subprocess.check_output(['git', 'show', 113 | '%s:hosts' % revision], 114 | cwd=path).decode('utf-8') 115 | # usually, this is done when we can't check the list of hosts 116 | except subprocess.CalledProcessError: 117 | return result 118 | 119 | # beware, not portable on windows 120 | tmp_dir = tempfile.mkdtemp() 121 | tmp_file = tempfile.NamedTemporaryFile('w+', dir=tmp_dir) 122 | tmp_file.write(host_content) 123 | tmp_file.flush() 124 | os.fsync(tmp_file.fileno()) 125 | 126 | inventory = InventoryWrapper(host_list=tmp_file.name) 127 | variable_manager = inventory.get_variable_manager() 128 | loader = inventory.get_loader() 129 | 130 | for group in inventory.get_groups(): 131 | for host in inventory.get_hosts(group): 132 | vars_host = variable_manager.get_vars(loader, host=host) 133 | result.append( 134 | {'name': vars_host.get('ansible_ssh_host', host.name), 135 | 'ssh_args': vars_host.get('ansible_ssh_common_args', ''), 136 | 'connection': vars_host.get('ansible_connection', 'ssh')}) 137 | 138 | # for some reason, there is some kind of global cache that need to be 139 | # cleaned 140 | inventory.refresh_inventory() 141 | # we need to del the temporary file before the directory, or a exception 142 | # is launched and ignored: 143 | # Exception OSError: (2, 'No such file or directory', 144 | # '/tmp/tmpw2IQjN/tmppHjNDO') # in ', 146 | # mode 'w+' at 0x2cfd6f0>> ignored 147 | # 148 | del tmp_file 149 | shutil.rmtree(tmp_dir) 150 | 151 | return result 152 | 153 | 154 | def path_match_local_playbook(playbook_file): 155 | """ Return True if the playbook can be considered to be 156 | run locally (ie, if name match the fqdn, or local)""" 157 | 158 | fqdn = socket.getfqdn() 159 | local_path = ['playbooks/' + i for i in ['local.yml', 160 | fqdn + '.yml', 161 | fqdn.split('.')[0] + '.yml']] 162 | return playbook_file in local_path 163 | -------------------------------------------------------------------------------- /files/callback/lock_run.py: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright (c) 2016 Michael Scherer 3 | # 4 | # Permission is hereby granted, free of charge, to any person obtaining a copy 5 | # of this software and associated documentation files (the "Software"), to deal 6 | # in the Software without restriction, including without limitation the rights 7 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | # copies of the Software, and to permit persons to whom the Software is 9 | # furnished to do so, subject to the following conditions: 10 | # 11 | # The above copyright notice and this permission notice shall be included in 12 | # all copies or substantial portions of the Software. 13 | # 14 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | # SOFTWARE. 21 | # 22 | # 23 | # This callback plugin just ensure that only 1 single instance of ansible is 24 | # running # at a time by default. Since multiple people can push to the same 25 | # repo, I do not want to have issues with multiple concurent runs 26 | # 27 | 28 | from ansible.plugins.callback import CallbackBase 29 | import os 30 | import sys 31 | import atexit 32 | 33 | 34 | class CallbackModule(CallbackBase): 35 | CALLBACK_NAME = 'lock' 36 | 37 | def __init__(self): 38 | super(CallbackModule, self).__init__() 39 | 40 | def v2_playbook_on_start(self, playbook): 41 | lock_file = os.path.expanduser('{}/ansible_run_lock'.format( 42 | os.environ.get('XDG_RUNTIME_DIR', '~'))) 43 | if os.path.exists(lock_file): 44 | # TODO verify if the PID in the file still exist 45 | self._display.display("Deployment already started, exiting") 46 | self._display.display("Remove {} if you want to proceed".format( 47 | lock_file)) 48 | sys.exit(2) 49 | 50 | # TODO try/except, show a better error message 51 | fd = os.open(lock_file, os.O_CREAT | os.O_EXCL | os.O_WRONLY) 52 | os.write(fd, bytes(os.getpid())) 53 | os.close(fd) 54 | atexit.register(os.unlink, lock_file) 55 | -------------------------------------------------------------------------------- /files/callback/log_sqlite.py: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright (c) 2016 Michael Scherer 3 | # 4 | # Permission is hereby granted, free of charge, to any person obtaining a copy 5 | # of this software and associated documentation files (the "Software"), to deal 6 | # in the Software without restriction, including without limitation the rights 7 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | # copies of the Software, and to permit persons to whom the Software is 9 | # furnished to do so, subject to the following conditions: 10 | # 11 | # The above copyright notice and this permission notice shall be included in 12 | # all copies or substantial portions of the Software. 13 | # 14 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 15 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 16 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 17 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 18 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 19 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 20 | # SOFTWARE. 21 | # 22 | # 23 | # This callback plugin log each run in a sqlite database, with the date of 24 | # play, the file used, and the result (number of failure/success) 25 | 26 | from ansible.plugins.callback import CallbackBase 27 | import datetime 28 | import sqlite3 29 | import time 30 | import os 31 | 32 | DEFAULT_DB_NAME = '~/playbook_runs_log.db' 33 | 34 | 35 | class CallbackModule(CallbackBase): 36 | CALLBACK_NAME = 'log_sqlite' 37 | CALLBACK_VERSION = 2.0 38 | CALLBACK_TYPE = 'notification' 39 | 40 | def __init__(self): 41 | super(CallbackModule, self).__init__() 42 | self.init_db() 43 | self._stats = {} 44 | self._stats['ok'] = 0 45 | self._stats['failed'] = 0 46 | self._stats['changed'] = 0 47 | # there is a slight difference between the 2 of the order 48 | # of a the microsecond 49 | # but that's just convenience for displaying 50 | self._stats['date'] = time.time() 51 | self._stats['date_human'] = datetime.datetime.now().strftime("%c") 52 | self._stats['playbook_file'] = 'foo' 53 | 54 | def init_db(self): 55 | self._db_file = os.path.expanduser( 56 | os.environ.get('DB_FILE', DEFAULT_DB_NAME)) 57 | 58 | if not os.path.isfile(self._db_file): 59 | conn = sqlite3.connect(self._db_file) 60 | c = conn.cursor() 61 | c.execute(''' CREATE TABLE runs 62 | (ok INT, 63 | failed INT, 64 | changed INT, 65 | date REAL, 66 | date_human TEXT, 67 | playbook_file TEXT) 68 | ''') 69 | conn.commit() 70 | conn.close() 71 | 72 | def v2_playbook_on_stats(self, stats): 73 | conn = sqlite3.connect(self._db_file) 74 | c = conn.cursor() 75 | c.execute('''INSERT INTO runs VALUES (?,?,?,?,?,?)''', 76 | (self._stats['ok'], self._stats['failed'], 77 | self._stats['changed'], 78 | self._stats['date'], self._stats['date_human'], 79 | self._stats['playbook_file'])) 80 | conn.commit() 81 | conn.close() 82 | 83 | def v2_runner_item_on_ok(self, result): 84 | self._stats['ok'] += 1 85 | if result._result.get('changed', False): 86 | self._stats['changed'] += 1 87 | 88 | def v2_runner_item_on_failed(self, result): 89 | self._stats['failed'] += 1 90 | 91 | # beware, the variable must be named 'playbook', due to some 92 | # hack in ansible for backward compat 93 | def v2_playbook_on_start(self, playbook): 94 | self._stats['playbook_file'] = playbook._file_name 95 | -------------------------------------------------------------------------------- /files/checkout_git_repos.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Copyright (c) 2014-2016 Michael Scherer 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in all 13 | # copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | # 23 | if ! test -d "$1"; then 24 | echo "$1 should be a directory holding all the git repositories, exiting" 25 | exit 1 26 | fi 27 | 28 | cd "$1" 29 | for i in public private; do 30 | GIT_WORK_TREE=/etc/ansible/ GIT_DIR=$i git checkout -q -f 31 | done; 32 | -------------------------------------------------------------------------------- /files/clean_ssh_public_keys.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # 3 | # Copyright (c) 2017 Michael Scherer 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in 13 | # all copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | # 23 | 24 | import sys 25 | import re 26 | import subprocess 27 | 28 | if len(sys.argv) != 2: 29 | print("Usage: %s hostname" % sys.argv[0]) 30 | print("Remove the hostname from .ssh/known_hosts") 31 | sys.exit(1) 32 | 33 | target = sys.argv[1] 34 | 35 | # use a minimal verification on the argument, since we already verify 36 | # with ssh-keygen after. 37 | if not re.match('^[\\w.-]+$', target): 38 | print("Invalid hostname %s" % target) 39 | sys.exit(1) 40 | 41 | if subprocess.call(['ssh-keygen', '-F', target]) != 0: 42 | print("Nothing to clean for %s" % target) 43 | sys.exit(0) 44 | 45 | if subprocess.call(['ssh-keygen', '-R', target]) != 0: 46 | print("Error when cleaning %s" % target) 47 | sys.exit(1) 48 | 49 | print("SSH public key got cleaned for %s" % target) 50 | -------------------------------------------------------------------------------- /files/generate_ansible_command.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # 3 | # Copyright (c) 2014 Michael Scherer 4 | # 5 | # Permission is hereby granted, free of charge, to any person obtaining a copy 6 | # of this software and associated documentation files (the "Software"), to deal 7 | # in the Software without restriction, including without limitation the rights 8 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | # copies of the Software, and to permit persons to whom the Software is 10 | # furnished to do so, subject to the following conditions: 11 | # 12 | # The above copyright notice and this permission notice shall be included in 13 | # all copies or substantial portions of the Software. 14 | # 15 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | # SOFTWARE. 22 | # 23 | 24 | # This script take a git repository, a git checkout, and 2 commits reference 25 | # and run the appropriate ansible-playbook command 26 | # 27 | # It make a few assumptions on the directory tree: 28 | # - that a file /usr/local/bin/update_galaxy.sh can be run by sudo, and update 29 | # the role tree. I want to not run it as root so that's required 30 | # - that playbooks are splitted in $CHECKOUT/playbooks, and that the playbook 31 | # filename to deploy start by "deploy", see get_playbooks_to_run_pattern 32 | # - that you are using a .yml extensions 33 | # 34 | # I keep a separate checkout from the main git repository due to the use of 35 | # a private repository for password and the like 36 | # 37 | # Verbosity setting is still to be be added 38 | # And so does unit tests... 39 | 40 | 41 | import yaml 42 | import os 43 | import glob 44 | import subprocess 45 | import re 46 | import fnmatch 47 | import syslog 48 | import sys 49 | import argparse 50 | import shlex 51 | 52 | # custom library 53 | sys.path.append('/usr/local/lib') 54 | from bastion_lib import extract_list_hosts_git, get_changed_files, \ 55 | path_match_local_playbook # noqa: E402 56 | 57 | parser = argparse.ArgumentParser(description="Run ansible playbooks based " 58 | "on actual changes in git") 59 | parser.add_argument("-v", "--verbose", help="increase output verbosity", 60 | action="store_true") 61 | parser.add_argument("-n", "--dry-run", help="only show what would be done", 62 | action="store_true") 63 | parser.add_argument('--path', help="path of the updated git checkout", 64 | default="/etc/ansible") 65 | parser.add_argument('--config', help="path of the config file", 66 | default="/etc/ansible_bastion.yml") 67 | parser.add_argument('--git', help="git repository path", required=True) 68 | parser.add_argument('--old', help="git commit before the push", required=True) 69 | parser.add_argument('--new', default="HEAD", help="git commit after the push") 70 | 71 | args = parser.parse_args() 72 | 73 | if 'SUDO_USER' in os.environ: 74 | syslog.syslog("Execution by {} with args: {}".format( 75 | os.environ['SUDO_USER'], 76 | ' '.join(sys.argv))) 77 | else: 78 | syslog.syslog("Direct execution with args: {}".format( 79 | ' '.join(sys.argv))) 80 | 81 | 82 | def load_config(config_file): 83 | config = None 84 | if os.path.isfile(config_file): 85 | config = yaml.safe_load(open(config_file, 'r')) 86 | # keep this conditional, since yaml.safe_load return None 87 | # for a empty file 88 | if config is None: 89 | config = {} 90 | 91 | if 'deploy_pattern' not in config: 92 | config['deploy_pattern'] = 'playbooks/deploy*.yml' 93 | if 'run_on_new_host' not in config: 94 | config['run_on_new_host'] = [] 95 | 96 | return config 97 | 98 | 99 | configuration = load_config(args.config) 100 | 101 | cache_role_playbook = {} 102 | 103 | 104 | def get_role(r): 105 | if isinstance(r, str): 106 | return r 107 | elif isinstance(r, dict): 108 | if 'role' in r: 109 | return r['role'] 110 | elif 'name' in r: 111 | return r['name'] 112 | 113 | 114 | def parse_roles_playbook(playbook_file): 115 | if playbook_file in cache_role_playbook: 116 | return cache_role_playbook[playbook_file] 117 | 118 | playbook = yaml.safe_load(open(playbook_file, 'r')) 119 | result = {} 120 | 121 | if playbook is None: 122 | return result 123 | 124 | for doc in playbook: 125 | host = doc['hosts'] 126 | roles = set() 127 | for r in doc.get('roles', []): 128 | roles.add(get_role(r)) 129 | if host in result: 130 | result[host] = result[host].union(roles) 131 | else: 132 | result[host] = roles 133 | cache_role_playbook[playbook_file] = result 134 | return result 135 | 136 | 137 | cache_role_meta = {} 138 | 139 | 140 | def parse_roles_meta(directory): 141 | roles = {} 142 | if directory in cache_role_meta: 143 | return cache_role_meta[directory] 144 | for d, sd, f in os.walk(directory): 145 | if 'main.yml' in f and d.endswith('/meta'): 146 | meta_file = "%s/main.yml" % d 147 | # extract the role name (-5 for /meta) 148 | r = d[len(directory)+1:-5] 149 | meta = yaml.safe_load(open(meta_file, 'r')) 150 | if meta and 'dependencies' in meta: 151 | roles[r] = set() 152 | if meta['dependencies'] is not None: 153 | for dep in meta['dependencies']: 154 | roles[r].add(get_role(dep)) 155 | cache_role_meta[directory] = roles 156 | return roles 157 | 158 | 159 | def get_roles_deps(roles, r): 160 | result = [r] 161 | if r not in roles: 162 | return result 163 | for i in roles[r]: 164 | result.append(i) 165 | if i in roles: 166 | result.extend(get_roles_deps(roles, i)) 167 | return result 168 | 169 | 170 | def get_host_roles_dict(playbook_file, roles_path='/etc/ansible/roles'): 171 | result = {} 172 | roles = parse_roles_meta(roles_path) 173 | hosts = parse_roles_playbook(playbook_file) 174 | 175 | for host in hosts: 176 | result[host] = set() 177 | for r in hosts[host]: 178 | for r2 in get_roles_deps(roles, r): 179 | result[host].add(r2) 180 | return result 181 | 182 | 183 | # return the group 184 | def get_hosts_for_role(role, playbook_file): 185 | """ Return the list of hosts where a role is applied in a specific 186 | playbook """ 187 | result = [] 188 | host_roles = get_host_roles_dict(playbook_file) 189 | for i in host_roles: 190 | if role in host_roles[i]: 191 | result.append(i) 192 | return result 193 | 194 | 195 | def get_playbooks_to_run(checkout_path): 196 | # this list compriehension filter from potentially hostile filename 197 | # since the list is fed to a shell command, I filter to have only safe 198 | # chars. 199 | # do not authorize \s in filename, since this interact badly with shell 200 | # later 201 | return [g for g in glob.glob('%s/%s' % (checkout_path, 202 | configuration['deploy_pattern'])) 203 | if re.search('^[\\w/.-]+$', g)] 204 | 205 | 206 | changed_files = get_changed_files(args.git, args.old, args.new) 207 | 208 | hosts_to_update = set() 209 | playbooks_to_run = set() 210 | local_playbooks_to_run = set() 211 | limits = set() 212 | 213 | commands_to_run = [] 214 | update_requirements = False 215 | for p in get_playbooks_to_run(args.path): 216 | for path in changed_files: 217 | splitted_path = path.split('/') 218 | if path == 'requirements.yml': 219 | update_requirements = True 220 | elif splitted_path[0] == 'roles': 221 | role_name = '/'.join(splitted_path[1:-2]) 222 | if len(get_hosts_for_role(role_name, p)) > 0: 223 | playbooks_to_run.add(p) 224 | 225 | for path in changed_files: 226 | if fnmatch.fnmatch(path, configuration['deploy_pattern']): 227 | playbooks_to_run.add("%s/%s" % (args.path, path)) 228 | 229 | if path_match_local_playbook(path): 230 | local_playbooks_to_run.add("%s/%s" % (args.path, path)) 231 | 232 | if 'hosts' in changed_files: 233 | old = extract_list_hosts_git(args.old, args.git) 234 | new = extract_list_hosts_git(args.new, args.git) 235 | 236 | def get_hostname(x): 237 | return x.get('name', '') 238 | diff = set(map(get_hostname, new)) - set(map(get_hostname, old)) 239 | if len(diff) > 0: 240 | for hostname in diff: 241 | # No need for a full fledged verification, just making 242 | # sure there is no funky chars for shell, and no space 243 | if re.search('^[\\w.-]+$', hostname): 244 | # avoid using the ssh stuff on salt bus host 245 | h = list(filter((lambda f: f['name'] == hostname), new))[0] 246 | if h['connection'] == 'ssh': 247 | commands_to_run.append("ssh %s -o " 248 | "PreferredAuthentications=publickey" 249 | " -o StrictHostKeyChecking=no %s id" 250 | % (h['ssh_args'], hostname)) 251 | 252 | for p in get_playbooks_to_run(args.path): 253 | if os.path.basename(p) in configuration['run_on_new_host']: 254 | playbooks_to_run.add(p) 255 | 256 | if update_requirements: 257 | commands_to_run.append('sudo /usr/local/bin/update_galaxy.sh') 258 | 259 | for p in playbooks_to_run: 260 | if os.path.exists(p): 261 | commands_to_run.append('ansible-playbook -D %s' % p) 262 | 263 | for p in local_playbooks_to_run: 264 | commands_to_run.append('sudo /usr/local/bin/ansible_local.py %s' % p) 265 | 266 | for c in commands_to_run: 267 | if args.dry_run: 268 | print(c) 269 | else: 270 | syslog.syslog("Running {}".format(c)) 271 | subprocess.call(shlex.split(c), cwd=args.path) 272 | -------------------------------------------------------------------------------- /files/help: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | if tty -s 4 | then 5 | echo "Run 'help' for help, or 'exit' to leave. Available commands:" 6 | else 7 | echo "Run 'help' for help. Available commands:" 8 | fi 9 | 10 | cd "$(dirname "$0")" 11 | 12 | for cmd in * 13 | do 14 | case "$cmd" in 15 | help) ;; 16 | *) [ -f "$cmd" ] && [ -x "$cmd" ] && echo "$cmd" ;; 17 | esac 18 | done 19 | -------------------------------------------------------------------------------- /files/list: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | print_if_bare_repo=' 4 | if "$(git --git-dir="$1" rev-parse --is-bare-repository)" = true 5 | then 6 | printf "%s\n" "${1#./}" 7 | fi 8 | ' 9 | 10 | find -type d -o -type l -name "*.git" -exec sh -c "$print_if_bare_repo" -- \{} \; -prune 2>/dev/null 11 | -------------------------------------------------------------------------------- /files/update_ansible_config.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | GIT_WORK_TREE=/etc/ansible/ git checkout -q -f 3 | -------------------------------------------------------------------------------- /files/update_external_roles: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | sudo -n -u root /usr/local/bin/update_galaxy.sh 4 | -------------------------------------------------------------------------------- /files/update_galaxy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | REQ_PATH=/etc/ansible/requirements.yml 3 | ROLES_PATH=/etc/ansible/roles 4 | INSTALL_ARGS="install -f -r $REQ_PATH" 5 | INSTALL_ROLES_ARGS="$INSTALL_ARGS -p $ROLES_PATH" 6 | 7 | [ -w $ROLES_PATH ] || (echo "Cannot write $ROLES_PATH, aborting"; exit 1) 8 | cd /etc/ansible 9 | NEW=0 10 | if grep -q 'roles:' $REQ_PATH; then 11 | /usr/bin/ansible-galaxy role $INSTALL_ROLES_ARGS 12 | NEW=1 13 | fi; 14 | 15 | if grep -q 'collections:' $REQ_PATH; then 16 | /usr/bin/ansible-galaxy collection $INSTALL_ARGS 17 | NEW=1 18 | fi; 19 | 20 | if [ $NEW -ne 1 ]; then 21 | /usr/bin/ansible-galaxy $INSTALL_ROLES_ARGS >/dev/null 22 | fi; 23 | -------------------------------------------------------------------------------- /meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | galaxy_info: 3 | author: Michael Scherer 4 | description: Ansible bastion host to control a set of servers, act like a puppet or salt master 5 | company: Red Hat 6 | license: MIT 7 | # use default_omit, need 1.8 8 | min_ansible_version: 1.8 9 | # Only tested with EL 7 and EL 6. No reason anything do 10 | # not work anywhere else, just no time to test. 11 | # I would welcome patches 12 | platforms: 13 | - name: EL 14 | versions: 15 | - 6 16 | - 7 17 | categories: 18 | - system 19 | dependencies: 20 | - role: tor 21 | when: enable_onion_support or use_tor_proxy 22 | -------------------------------------------------------------------------------- /tasks/compat_ansible_admin_group.yml: -------------------------------------------------------------------------------- 1 | - name: Display message about deprecated variable 2 | pause: 3 | seconds: 30 4 | prompt: "The variable ansible_admin_group is deprecated, please convert to ansible_committers_group" 5 | 6 | - set_fact: 7 | ansible_committers_group: "{{ ansible_admin_group }}" 8 | -------------------------------------------------------------------------------- /tasks/create_repo.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Create {{ repo }} git repos 3 | file: 4 | path: "{{ git_repositories_dir }}/{{ repo }}" 5 | state: directory 6 | group: "{{ ansible_committers_group | default(omit) }}" 7 | mode: "u=rwx,g=rwxs" 8 | 9 | - name: Initialize the {{ repo }} git repos 10 | shell: git init --bare -q --shared=group 11 | args: 12 | creates: "{{ git_repositories_dir }}/{{ repo }}/config" 13 | chdir: "{{ git_repositories_dir }}/{{ repo }}" 14 | 15 | - name: Set {{ repo }} git repos hooks 16 | template: 17 | dest: "{{ git_repositories_dir }}/{{ repo }}/hooks/{{ item }}" 18 | src: hooks/receive-hook.sh 19 | mode: 0755 20 | with_items: 21 | - post-receive 22 | - pre-receive 23 | 24 | - name: Set {{ repo }} git repos hooks directories 25 | file: 26 | path: "{{ git_repositories_dir }}/{{ repo }}/hooks/{{ item }}.d/" 27 | state: directory 28 | with_items: 29 | - post-receive 30 | - pre-receive 31 | -------------------------------------------------------------------------------- /tasks/create_repos.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Create directory holding git repos 3 | file: 4 | path: "{{ git_repositories_dir }}" 5 | state: directory 6 | 7 | - include_tasks: create_repo.yml 8 | with_items: 9 | - public 10 | - private 11 | loop_control: 12 | loop_var: repo 13 | -------------------------------------------------------------------------------- /tasks/git_shell.yml: -------------------------------------------------------------------------------- 1 | - set_fact: 2 | git_home: "/srv/git" 3 | 4 | - name: Create user {{ git_username }} 5 | user: 6 | name: "{{ git_username }}" 7 | shell: /usr/bin/git-shell 8 | home: "{{ git_home }}" 9 | group: "{{ ansible_committers_group | default(omit) }}" 10 | 11 | - name: Create git-shell-commands directory 12 | file: 13 | name: "{{ git_home }}/git-shell-commands" 14 | state: directory 15 | 16 | - name: Copy commands in the git-shell-commands directory 17 | copy: 18 | dest: "{{ git_home }}/git-shell-commands/{{ item }}" 19 | src: "{{ item }}" 20 | mode: "0755" 21 | with_items: 22 | - list 23 | - help 24 | - update_external_roles 25 | 26 | - name: Copy templated command in the git-shell-commands directory 27 | template: 28 | dest: "{{ git_home }}/git-shell-commands/{{ item }}" 29 | src: "{{ item }}" 30 | mode: "0755" 31 | with_items: 32 | - clean_ssh_public_keys 33 | 34 | - name: Link repositories in the home 35 | file: 36 | state: link 37 | dest: "{{ git_home }}/{{ item }}.git" 38 | src: "{{ git_repositories_dir }}/{{ item }}" 39 | with_items: 40 | - public 41 | - private 42 | -------------------------------------------------------------------------------- /tasks/install_ansible_git.yml: -------------------------------------------------------------------------------- 1 | - set_fact: 2 | checkout_dir: "/usr/local/src/ansible" 3 | 4 | - name: Install dependencies of ansible 5 | package: 6 | name: "{{ item }}" 7 | state: present 8 | with_items: 9 | - python-jinja2 10 | - python-paramiko 11 | - python-httplib2 12 | - PyYAML 13 | - python-keyczar 14 | - python-netaddr 15 | 16 | - name: Remove ansible package 17 | package: 18 | name: ansible 19 | state: absent 20 | 21 | - name: Checkout git in {{ checkout_dir }} 22 | git: 23 | repo: "https://github.com/ansible/ansible.git" 24 | dest: "{{ checkout_dir }}" 25 | version: "{{ ansible_git_version }}" 26 | 27 | - name: Create helper scripts 28 | template: 29 | dest: "/usr/local/bin/{{ item }}" 30 | src: "{{ item }}" 31 | mode: 0755 32 | with_items: 33 | - ansible_git 34 | - ansible_git_update 35 | 36 | - name: Create compatibility links 37 | file: 38 | dest: "/usr/bin/{{ item }}" 39 | src: /usr/local/bin/ansible_git 40 | state: link 41 | with_items: 42 | - ansible 43 | - ansible-doc 44 | - ansible-galaxy 45 | - ansible-playbook 46 | - ansible-pull 47 | - ansible-vault 48 | 49 | - name: Create link for library 50 | file: 51 | dest: /usr/local/lib/ansible 52 | src: /usr/local/src/ansible/lib/ansible 53 | state: link 54 | 55 | - name: Create /etc/ansible directory 56 | file: 57 | dest: /etc/ansible 58 | state: directory 59 | 60 | - name: Set cron to automatically update the checkout 61 | cron: 62 | name: "update ansible git" 63 | minute: "*/20" 64 | job: /usr/local/bin/ansible_git_update 65 | -------------------------------------------------------------------------------- /tasks/install_ansible_rpm.yml: -------------------------------------------------------------------------------- 1 | - name: Install ansible from package (classic) 2 | package: 3 | name: 4 | - ansible 5 | - python3-netaddr 6 | state: present 7 | when: ansible_distribution_major_version|int < 35 8 | 9 | - name: Install ansible from package 10 | package: 11 | name: 12 | - ansible-base 13 | - python3-netaddr 14 | state: present 15 | when: ansible_distribution_major_version|int >= 35 16 | 17 | - name: Install rpm collections 18 | package: 19 | name: "ansible-collection-{{ item | replace('.', '-') }}" 20 | state: present 21 | with_items: "{{ rpm_collections }}" 22 | -------------------------------------------------------------------------------- /tasks/install_callback_plugins.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # remove after september 2017 3 | - name: Cleanup old callbacks 4 | file: 5 | name: "/etc/ansible/callback_plugins/{{ item }}" 6 | state: absent 7 | with_items: 8 | - lock_run.py 9 | - log_sqlite.py 10 | 11 | - name: Create dir for callback plugins 12 | file: 13 | name: "~{{ ansible_username }}/.ansible/plugins/callback" 14 | state: directory 15 | 16 | - name: Install callback plugins 17 | copy: 18 | src: "callback/{{ item }}" 19 | dest: "~{{ ansible_username }}/.ansible/plugins/callback/{{ item }}" 20 | with_items: 21 | - lock_run.py 22 | - log_sqlite.py 23 | -------------------------------------------------------------------------------- /tasks/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - include_tasks: compat_ansible_admin_group.yml 3 | when: ansible_admin_group is defined 4 | 5 | - name: Verify that the groups are all defined 6 | fail: 7 | msg: "ansible_admins_group is defined while ansible_committers_group is not" 8 | when: ansible_admins_group is defined and ansible_committers_group is undefined 9 | 10 | - name: Fail is old option are used 11 | fail: 12 | msg: "allow_ansible_commands is deprecated, please see ansible_admins_group in README.md" 13 | when: allow_ansible_commands is defined 14 | 15 | - name: Install tools 16 | package: 17 | name: 18 | - git 19 | - sudo 20 | state: present 21 | 22 | - name: Install socat 23 | package: 24 | name: socat 25 | state: present 26 | when: enable_onion_support or use_tor_proxy 27 | 28 | - name: Create ansible user {{ ansible_username }} 29 | user: 30 | name: "{{ ansible_username }}" 31 | generate_ssh_key: yes 32 | ssh_key_type: "{{ ssh_key_type }}" 33 | 34 | # heck, since update_galaxy run as root, and I do not want to reintroduce 35 | # a config file 36 | - name: Add link for collections 37 | file: 38 | state: link 39 | dest: /root/.ansible/collections 40 | src: "~{{ ansible_username }}/.ansible/collections" 41 | 42 | - name: Create {{ ansible_admins_group }} group 43 | group: 44 | name: "{{ ansible_admins_group }}" 45 | when: ansible_admins_group is defined 46 | 47 | - name: Create {{ ansible_committers_group }} group 48 | group: 49 | name: "{{ ansible_committers_group }}" 50 | when: ansible_committers_group is defined 51 | 52 | - include_tasks: install_ansible_rpm.yml 53 | when: not use_ansible_git 54 | 55 | - include_tasks: install_ansible_git.yml 56 | when: use_ansible_git 57 | 58 | - name: Setup ssh config for {{ ansible_username }} 59 | template: 60 | dest: "~{{ ansible_username }}/.ssh/config" 61 | src: ssh_config 62 | 63 | - name: Setup the bastion configuration file 64 | template: 65 | dest: /etc/ansible_bastion.yml 66 | src: ansible_bastion.yml 67 | mode: 0644 68 | 69 | - name: Create cache folder {{ cache_folder }} 70 | file: 71 | path: "{{ cache_folder }}" 72 | state: directory 73 | owner: "{{ ansible_username }}" 74 | mode: 0755 75 | 76 | - name: Add sudo config for admins 77 | template: 78 | dest: "/etc/sudoers.d/{{ ansible_committers_group }}_cmd_config" 79 | src: cmd_config.sudoers 80 | validate: 'visudo -cf %s' 81 | when: ansible_committers_group is defined 82 | 83 | - include_tasks: create_repos.yml 84 | 85 | - include_tasks: install_callback_plugins.yml 86 | 87 | - name: Add post-receive hook to checkout playbooks 88 | template: 89 | dest: "{{ git_repositories_dir }}/{{ item }}/hooks/post-receive.d/00_update_checkout.sh" 90 | src: hooks/update_checkout.sh 91 | mode: 0755 92 | with_items: 93 | - public 94 | - private 95 | 96 | - name: Add post-receive hook to trigger a run 97 | template: 98 | dest: "{{ git_repositories_dir }}/public/hooks/post-receive.d/01_trigger_run.sh" 99 | src: hooks/trigger_run.sh 100 | mode: 0755 101 | 102 | - include_tasks: pre_receive_checks.yml 103 | 104 | - name: Deploy various helper scripts 105 | copy: 106 | src: "{{ item }}" 107 | dest: "/usr/local/bin/{{ item }}" 108 | mode: 0755 109 | with_items: 110 | - checkout_git_repos.sh 111 | - ansible_run_all.sh 112 | - generate_ansible_command.py 113 | - update_galaxy.sh 114 | - update_ansible_config.sh 115 | - ansible_local.py 116 | - clean_ssh_public_keys.py 117 | 118 | - name: Deploy the library of function for bastion 119 | copy: 120 | src: bastion_lib.py 121 | dest: /usr/local/lib/bastion_lib.py 122 | 123 | - name: Deploy various sudo configs 124 | template: 125 | src: "{{ item }}.sudoers" 126 | dest: "/etc/sudoers.d/{{ item }}" 127 | validate: 'visudo -cf %s' 128 | mode: 'ug+r' 129 | with_items: 130 | - update_galaxy 131 | - ansible_local 132 | 133 | - name: Deploy various scripts 134 | template: 135 | src: "{{ item }}" 136 | dest: "/usr/local/bin/{{ item }}" 137 | mode: 0755 138 | with_items: 139 | - file_changed_commit.sh 140 | - rerun_last_commit.sh 141 | 142 | - name: Add cron to merge change in git to /etc/ansible 143 | cron: 144 | name: "merge git {{ git_repositories_dir }}" 145 | job: "/usr/local/bin/checkout_git_repos.sh {{ git_repositories_dir }}" 146 | minute: "*/5" 147 | 148 | - name: Add cron to run ansible change every night 149 | cron: 150 | name: "ansible run" 151 | user: "{{ ansible_username }}" 152 | job: /usr/local/bin/ansible_run_all.sh 153 | minute: "{{ ansible_run_minute | default(omit) }}" 154 | hour: "{{ ansible_run_hour | default(omit) }}" 155 | 156 | - name: Add cron to update the modules every 6h 157 | cron: 158 | name: "ansible galaxy update" 159 | user: "{{ ansible_username }}" 160 | job: "sudo /usr/local/bin/update_galaxy.sh" 161 | hour: "*/6" 162 | minute: 0 163 | 164 | - include_tasks: git_shell.yml 165 | when: use_git_shell 166 | 167 | - include_tasks: push_remote.yml 168 | when: remotes|length > 0 169 | 170 | - include_tasks: use_tor_proxy.yml 171 | when: use_tor_proxy 172 | -------------------------------------------------------------------------------- /tasks/pre_receive_checks.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Deploy the pre-receive check 3 | template: 4 | dest: "{{ git_repositories_dir }}/public/hooks/pre-receive.d/{{ item.order }}_{{ item.script }}" 5 | src: hooks/{{ item.script }} 6 | mode: 0755 7 | with_items: 8 | - { order: '01', script: check_git_ignore.py } 9 | - { order: '02', script: check_branch_master.py } 10 | -------------------------------------------------------------------------------- /tasks/push_remote.yml: -------------------------------------------------------------------------------- 1 | - name: Create user {{ pusher_username }} to sync repos 2 | user: 3 | name: "{{ pusher_username }}" 4 | generate_ssh_key: yes 5 | 6 | - name: Add the repositories to safe directory list 7 | git_config: 8 | scope: global 9 | name: safe.directory 10 | value: "{{ git_repositories_dir }}/public" 11 | become: yes 12 | become_user: "{{ pusher_username }}" 13 | 14 | - name: Deploy the sync script 15 | template: 16 | src: push_remote_public.sh 17 | dest: /usr/local/bin/push_remote_public.sh 18 | mode: 0755 19 | 20 | - name: Add remote repositories to git 21 | ini_file: 22 | dest: "{{ git_repositories_dir }}/public/config" 23 | option: url 24 | value: "{{ item.url }}" 25 | section: "remote \"{{ item.name }}\"" 26 | with_items: "{{ remotes }}" 27 | 28 | - name: Set the push.default config to simple 29 | ini_file: 30 | dest: "{{ git_repositories_dir }}/public/config" 31 | option: "default" 32 | value: "simple" 33 | section: "push" 34 | 35 | - name: Deploy hook to push to remote repositories 36 | template: 37 | dest: "{{ git_repositories_dir }}/public/hooks/post-receive.d/02_push_remote.sh" 38 | src: hooks/push_remote.sh 39 | mode: 0755 40 | 41 | - name: Add sudo config for pushing to remote repositories 42 | template: 43 | dest: /etc/sudoers.d/push_remote_public 44 | src: push_remote_public.sudoers 45 | validate: 'visudo -cf %s' 46 | when: ansible_committers_group is defined 47 | -------------------------------------------------------------------------------- /tasks/use_tor_proxy.yml: -------------------------------------------------------------------------------- 1 | - name: Set proxy config to use tor 2 | git_config: 3 | scope: "global" 4 | name: "{{ item }}.proxy" 5 | value: "socks5h://127.0.0.1:9050" 6 | with_items: 7 | - http 8 | - https 9 | -------------------------------------------------------------------------------- /templates/ansible_bastion.yml: -------------------------------------------------------------------------------- 1 | {% if run_on_new_host|length > 0 %} 2 | run_on_new_host: 3 | {% for i in run_on_new_host %} 4 | - {{ i }} 5 | {% endfor %} 6 | {% endif %} 7 | -------------------------------------------------------------------------------- /templates/ansible_git: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2016 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | cd "{{ checkout_dir }}" 27 | source hacking/env-setup -q 28 | cd - >/dev/null 29 | $(basename $0) $* 30 | -------------------------------------------------------------------------------- /templates/ansible_git_update: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2016 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | cd "{{ checkout_dir }}" 27 | git pull --rebase -q 28 | git submodule update -q 29 | -------------------------------------------------------------------------------- /templates/ansible_local.sudoers: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | Defaults!/usr/local/bin/ansible_local.py !requiretty 4 | 5 | # in a ideal world, this would be restricted to the exact 3 number of file, 6 | # but this test is pushed in the wrapper script (who happen to run as root, 7 | # which is unfortunate from a design perspective, but unlikely to 8 | # cause trouble in practice 9 | {{ ansible_username }} ALL = NOPASSWD: /usr/local/bin/ansible_local.py 10 | -------------------------------------------------------------------------------- /templates/clean_ssh_public_keys: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2017 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | # this is just a wrapper to run a python script to remove ssh keys from .ssh/know_hosts 27 | 28 | TARGET=$1 29 | sudo -n -u {{ ansible_username }} /usr/local/bin/clean_ssh_public_keys.py "$TARGET" 30 | -------------------------------------------------------------------------------- /templates/cmd_config.sudoers: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | Cmnd_Alias UPDATE_EXTERNAL_ROLES = /usr/local/bin/update_galaxy.sh 4 | 5 | Cmnd_Alias UPDATE_ANSIBLE_CONFIG = /usr/local/bin/update_ansible_config.sh 6 | Defaults!UPDATE_ANSIBLE_CONFIG !requiretty 7 | 8 | Cmnd_Alias GENERATE_ANSIBLE_COMMAND = /usr/local/bin/generate_ansible_command.py 9 | Defaults!GENERATE_ANSIBLE_COMMAND !requiretty 10 | Defaults!GENERATE_ANSIBLE_COMMAND env_keep=ANSIBLE_FORCE_COLOR 11 | Defaults!GENERATE_ANSIBLE_COMMAND env_keep+=ANSIBLE_STDOUT_CALLBACK 12 | Defaults!GENERATE_ANSIBLE_COMMAND env_keep+=ANSIBLE_RETRY_FILES_ENABLED 13 | 14 | Cmnd_Alias CLEAN_SSH_PUBLIC_KEYS = /usr/local/bin/clean_ssh_public_keys.py 15 | Defaults!CLEAN_SSH_PUBLIC_KEYS !requiretty 16 | 17 | %{{ ansible_committers_group }} ALL=(root) NOPASSWD: UPDATE_ANSIBLE_CONFIG 18 | %{{ ansible_committers_group }} ALL=(root) NOPASSWD: UPDATE_EXTERNAL_ROLES 19 | %{{ ansible_committers_group }} ALL=({{ ansible_username }}) NOPASSWD: GENERATE_ANSIBLE_COMMAND 20 | %{{ ansible_committers_group }} ALL=({{ ansible_username }}) NOPASSWD: CLEAN_SSH_PUBLIC_KEYS 21 | 22 | {% if ansible_admins_group is defined %} 23 | Cmnd_Alias ANSIBLE_COMMAND = /usr/bin/ansible,/usr/bin/ansible-playbook 24 | %{{ ansible_admins_group }} ALL=({{ ansible_username }}) NOPASSWD: ANSIBLE_COMMAND 25 | {% endif %} 26 | -------------------------------------------------------------------------------- /templates/file_changed_commit.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2014 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | refname="$1" 27 | oldrev="$2" 28 | newrev="$3" 29 | 30 | SCRIPT=$(mktemp /tmp/ansible_commit-XXXXXX) 31 | 32 | echo "#!/bin/bash" >> $SCRIPT 33 | echo "cd /etc/ansible" >> $SCRIPT 34 | 35 | generate_ansible_command.py $( git diff --name-status --diff-filter=ACDMR ${oldrev}..${newrev} | awk '{print $2}') >> $SCRIPT 36 | echo "rm -f $SCRIPT" >> $SCRIPT 37 | chown {{ ansible_username }} $SCRIPT 38 | chmod +rx $SCRIPT 39 | 40 | su - {{ ansible_username }} -c "$SCRIPT" 41 | -------------------------------------------------------------------------------- /templates/hooks/check_branch_master.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # 6 | # Copyright (c) 2016 Michael Scherer 7 | # 8 | # Permission is hereby granted, free of charge, to any person obtaining a copy 9 | # of this software and associated documentation files (the "Software"), to deal 10 | # in the Software without restriction, including without limitation the rights 11 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 12 | # copies of the Software, and to permit persons to whom the Software is 13 | # furnished to do so, subject to the following conditions: 14 | # 15 | # The above copyright notice and this permission notice shall be included in all 16 | # copies or substantial portions of the Software. 17 | # 18 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 19 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 20 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 21 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 22 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 23 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 24 | # SOFTWARE. 25 | # 26 | # 27 | # This script prevent pushing on anything but master. 28 | # 29 | 30 | import sys 31 | 32 | for l in sys.stdin.readlines(): 33 | if l.split()[2] != 'refs/heads/master': 34 | print("Push on another branch than master is not authorized") 35 | sys.exit(1) 36 | -------------------------------------------------------------------------------- /templates/hooks/check_git_ignore.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python3 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2016,2020 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | # 27 | # This script verify that all modules in requirements.yml are also 28 | # in the .gitignore file 29 | # It doesn't handle branch, and will only verify the last commit of a whole 30 | # series of patch, mostly because that's a minor issue (and because that's easier to 31 | # code) 32 | # 33 | 34 | import yaml 35 | import sys 36 | import subprocess 37 | 38 | last_ref = '0' 39 | for line in sys.stdin.readlines(): 40 | last_ref = line.split(' ')[1] 41 | 42 | req = subprocess.check_output(['git', 'show', last_ref + ':requirements.yml']) 43 | doc = yaml.safe_load(req) 44 | if type(doc) == type({}): 45 | doc = doc['roles'] 46 | module_names = set([i['name'] for i in doc]) 47 | 48 | gitignore = subprocess.check_output(['git', 'show', last_ref + ':.gitignore']).decode('utf-8') 49 | ignored_dirs = set([l.replace('roles/','') for l in gitignore.split('\n') if l.startswith("roles/")]) 50 | 51 | diff = module_names.difference(ignored_dirs) 52 | if len(diff) > 0: 53 | print("error, there is modules in requirements.yml who are not in .gitignore") 54 | print("please copy this:") 55 | for d in diff: 56 | print("roles/" + d) 57 | sys.exit(1) 58 | -------------------------------------------------------------------------------- /templates/hooks/push_remote.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2014-2016 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | echo "* Pushing git commit to public repository" 27 | sudo -n -u {{ pusher_username }} /usr/local/bin/push_remote_public.sh 28 | -------------------------------------------------------------------------------- /templates/hooks/receive-hook.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2016 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | HOOK_DIR=$(dirname $0) 27 | while read OLDREV NEWREV REF 28 | do 29 | for script in $HOOK_DIR/{{ item }}.d/*{sh,py} ; do 30 | if [ -f $script -a -x $script ]; then 31 | echo $OLDREV $NEWREV $REF | $script 32 | if [ $? -ne 0 ]; then 33 | echo "error, blocking commit" 34 | exit 1 35 | fi 36 | fi 37 | done 38 | done 39 | -------------------------------------------------------------------------------- /templates/hooks/trigger_run.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2014-2016 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | echo "* Triggering a run of ansible" 27 | while read OLDREV NEWREV REF 28 | do 29 | # run ansible 30 | sudo -n -u {{ ansible_username }} ANSIBLE_FORCE_COLOR=1 ANSIBLE_RETRY_FILES_ENABLED=False /usr/local/bin/generate_ansible_command.py --old $OLDREV --new $NEWREV --git $(pwd) 31 | done 32 | -------------------------------------------------------------------------------- /templates/hooks/update_checkout.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2014 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | echo "* Extracting git repo in /etc/ansible" 26 | sudo -n /usr/local/bin/update_ansible_config.sh 27 | -------------------------------------------------------------------------------- /templates/push_remote_public.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # {{ ansible_managed }} 4 | # 5 | # Copyright (c) 2016 Michael Scherer 6 | # 7 | # Permission is hereby granted, free of charge, to any person obtaining a copy 8 | # of this software and associated documentation files (the "Software"), to deal 9 | # in the Software without restriction, including without limitation the rights 10 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 11 | # copies of the Software, and to permit persons to whom the Software is 12 | # furnished to do so, subject to the following conditions: 13 | # 14 | # The above copyright notice and this permission notice shall be included in all 15 | # copies or substantial portions of the Software. 16 | # 17 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 20 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 21 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 22 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 23 | # SOFTWARE. 24 | # 25 | 26 | 27 | cd {{ git_repositories_dir }}/public 28 | 29 | {% for remote in remotes %} 30 | git push --mirror {{ remote.name }} 31 | {% endfor %} 32 | -------------------------------------------------------------------------------- /templates/push_remote_public.sudoers: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | Cmnd_Alias PUSH_REMOTE_PUBLIC = /usr/local/bin/push_remote_public.sh 4 | 5 | Defaults!PUSH_REMOTE_PUBLIC !requiretty 6 | 7 | %{{ ansible_committers_group }} ALL=({{ pusher_username }}) NOPASSWD: PUSH_REMOTE_PUBLIC 8 | -------------------------------------------------------------------------------- /templates/rerun_last_commit.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | generate_ansible_command.py --old HEAD~1 --new HEAD --git "{{ git_repositories_dir }}/public/" 3 | -------------------------------------------------------------------------------- /templates/ssh_config: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | Host *.{{ ansible_domain }} 3 | GSSAPIDelegateCredentials yes 4 | 5 | {% if enable_onion_support or use_tor_proxy %} 6 | {% if use_tor_proxy %} 7 | Host * 8 | {% else %} 9 | Host *.onion 10 | {% endif %} 11 | ProxyCommand socat - SOCKS4A:localhost:%h:%p,socksport=9050 12 | {% endif %} 13 | -------------------------------------------------------------------------------- /templates/update_galaxy.sudoers: -------------------------------------------------------------------------------- 1 | # {{ ansible_managed }} 2 | 3 | Defaults!/usr/local/bin/update_galaxy.sh !requiretty 4 | 5 | {{ ansible_username }} ALL = NOPASSWD: /usr/local/bin/update_galaxy.sh 6 | --------------------------------------------------------------------------------