├── .gitignore ├── README.md ├── ansible.cfg ├── hosts.yml └── inventory ├── aliyun.example.ini └── aliyun.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.py[cod] 2 | 3 | # C extensions 4 | *.so 5 | 6 | # Packages 7 | *.egg 8 | *.egg-info 9 | dist 10 | build 11 | eggs 12 | parts 13 | bin 14 | var 15 | sdist 16 | develop-eggs 17 | .installed.cfg 18 | lib 19 | lib64 20 | __pycache__ 21 | 22 | # Installer logs 23 | pip-log.txt 24 | 25 | # Unit test / coverage reports 26 | .coverage 27 | .tox 28 | nosetests.xml 29 | 30 | # Translations 31 | *.mo 32 | 33 | # Mr Developer 34 | .mr.developer.cfg 35 | .project 36 | .pydevproject 37 | 38 | /inventory/aliyun.ini 39 | /hosts 40 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Aliyun Ansbile Inventory 2 | 3 | [Aliyun][]'s Ansible [Dynamic Inventory][] script 4 | 5 | [Ansible][] 是自动化配置工具，Dynamic Inventory 允许使用脚本获到需要配置的机器列表和信息。 6 | 7 | 可以参考 hosts.yml 的示例 ansible playbook 如何为所有 ecs 生成 hosts 文件来方便访问，如果换成调用 DNSPod API 就能实现自动更新 DNS 记录。 8 | 9 | 10 | # 开始 11 | 12 | ## 安装 13 | 14 | 可以直接 clone 本项目作为 ansible playbook 的根目录，或者把 inventory 目录复制到您 ansible playbook 的根目录下，并使用 inventory 目录作为 inventtory host file 15 | 16 | > 设置 host file 可以使用命令参数 `-i` 或者在 ansible.cfg 中配置，参考本项目中的 ansible.cfg 17 | 18 | 如果您还有其它的 inventory 条目，也放到 inventory 目录下，参考 ansible multiple inventory sources [相关文档](http://docs.ansible.com/intro_dynamic_inventory.html#using-multiple-inventory-sources) 19 | 20 | 脚本 aliyun.py 使用 python 2，在默认 python 版本为 3 的环境下使用自行修改 aliyun.py 的第一行。 21 | 22 | 该脚本依赖阿里去的命令行工具，可以使用 `pip` 安装 23 | 24 | sudo pip install aliyuncli 25 | 26 | ## 配置 27 | 28 | 首先需要配置好 `aliyuncli` 29 | 30 | aliyuncli configure 31 | 32 | 复制示例配置文件并进行修改。配置文件必须命名为 `aliyun.ini` 并且和 `aliyun.py` 在同一目录下。 33 | 34 | cp inventory/aliyun.example.ini inventory/aliyun.ini 35 | 36 | 然后配置 SSH 连接信息，ecs 对应云主机，配置中支持 Python 的 `%` 替换，比如 ecs 中的 `%(InstanceName)s` 会替换成云主机的名称。括号中可以是任何 `aliyuncli` 返回结果中的字段，另外为了方便使用 IP，还有以下额外字段可以使用 37 | 38 | - `PublicIp` BGP 机房出口 IP 39 | - `InnerIp` 内网 IP 40 | 41 | 配置文件还支持对某个 ecs 进行单独配置，只要新建新的小节，以资源类型和名称作为小节的名字，比如 ecs ops 会优先使用 `ecs.ops` 小节中的配置，见下面例子。命名规则见本文档后面的内容。 42 | 43 | [ecs.ops] 44 | host = ops.example.com 45 | port = 2222 46 | user = ops 47 | 48 | ## 测试 49 | 50 | 直接运行脚本 `aliyun.py`，没有错误应该会打印出符合 ansible dynamic inventory 要求的 JSON，然后可以运行 ansible 列表所有机器 51 | 52 | ansible all -i inventory --list-hosts 53 | 54 | 如果一切正常可以测试下 demo playbook hosts.yml 55 | 56 | ansible-playbook hosts.yml 57 | 58 | 该 playbook 会在当前目录生成 hosts，如果覆盖 /etc/hosts 可以使用里面配置的主机名比如 `ops.ecs` 来访问 ecs 主机了。而如果云主机已经配置能使用 ubuntu sudo 进行 ansible 操作，那么这些主机名也更新到所有的云主机上了。 59 | 60 | ## 缓存 61 | 62 | 示例配置中默认开启了缓存，如果改变了 aliyun 的设置想要立即更新主机信息，可以手动执行下面命令刷新缓存 63 | 64 | inventory/aliyun.py --refresh-cache 65 | 66 | 还可以在inventory/aliyun.ini中关闭缓存功能 67 | 68 | cache_disable = True 69 | 70 | # 命名规则 71 | 72 | ## 主机名 73 | 74 | 即相应资源在 ansible 中使用的 host 名称。 75 | 76 | ecs 使用 InstanceName 字段, 也就是管理后台中可以修改的名称。 77 | 78 | ## 主机组 79 | 80 | 首先所有的资源都按照类型进行了分组，ecs 主机都属于 ecs 这个组。 81 | 82 | 另外 ecs 还支持自定义分组，规则是使用『描述』(对应 API 返回结果中的 Description)。业务组名称使用英文逗号分隔之后并加上 `des_` 前缀即为该主机要加入的 ansible 主机组。比如主机 ops 的业务组名称是 `dev,public` 那么在 ansible 中会包含在组 `des_dev` 和 `des_public` 中。Aliyun 本身的 Tag 也会转成对应的分组，规则是 `KEY_VALUE`。 83 | 84 | ## 主机变量名 85 | 86 | 所有 API 返回结果以及上面提到的额外 IP 字段都会嵌套在主机变量 aliyun 下，比如在 jinja2 模板中引用出口 IP 87 | 88 | {{ aliyun.PublicIp }} 89 | 90 | [ansible]: http://www.ansible.com 91 | [dynamic inventory]: http://docs.ansible.com/intro_dynamic_inventory.html 92 | [aliyun]: http://www.aliyun.com 93 | -------------------------------------------------------------------------------- /ansible.cfg: -------------------------------------------------------------------------------- 1 | [defaults] 2 | hostfile = inventory 3 | roles_path = ./roles 4 | -------------------------------------------------------------------------------- /hosts.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - name: Generate hosts file in localhost for all ecs instances 3 | hosts: localhost 4 | tasks: 5 | - name: Copy hosts file 6 | command: cp /etc/hosts hosts 7 | - name: Update ecs IPs in hosts file 8 | lineinfile: 9 | dest: hosts 10 | regexp: "^{{ hostvars[item].aliyun.PublicIp | replace('.', '\\.') }}" 11 | line: "{{ hostvars[item].aliyun.PublicIp }} {{ item }}.aliyun" 12 | with_items: groups.ecs 13 | - debug: msg="Copy hosts to overwrite your /etc/hosts" 14 | - name: Generate hosts file in all ecs instances 15 | hosts: ecs 16 | sudo: true 17 | tasks: 18 | - name: Update ecs IPs in hosts file 19 | lineinfile: 20 | dest: /etc/hosts 21 | regexp: "^{{ hostvars[item].aliyun.PublicIp | replace('.', '\\.') }}" 22 | line: "{{ hostvars[item].aliyun.PublicIp }} {{ item }}.aliyun" 23 | with_items: groups.ecs 24 | - name: Update ecs private IPs in hosts file 25 | lineinfile: 26 | dest: /etc/hosts 27 | regexp: "^{{ hostvars[item].aliyun.InnerIp | replace('.', '\\.') }}" 28 | line: "{{ hostvars[item].aliyun.InnerIp }} {{ item }}.private" 29 | with_items: groups.ecs 30 | -------------------------------------------------------------------------------- /inventory/aliyun.example.ini: -------------------------------------------------------------------------------- 1 | [cache] 2 | path = /tmp/ansible-aliyun.cache 3 | max_age = 86400 4 | ;; Disable Cache(True|False) 5 | ;cache_disable = False 6 | 7 | ;; General ssh options for all ecs instances 8 | [ecs] 9 | ;; Use domain 10 | ; host = %(InstanceName)s.example.com 11 | ;; Use Public IP 12 | host = %(PublicIp)s 13 | ;; Use EIP 14 | ; host = %(EipAddress)s 15 | port = 22 16 | user = ubuntu 17 | 18 | ;; Override the options for a specific instance 19 | ; [ecs.ops] 20 | ; host = ops.example.com 21 | ; port = 2222 22 | ; user = ops 23 | -------------------------------------------------------------------------------- /inventory/aliyun.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import argparse 4 | import ConfigParser 5 | import os 6 | import subprocess 7 | import re 8 | import errno 9 | 10 | from time import time 11 | from collections import defaultdict 12 | 13 | try: 14 | import json 15 | except ImportError: 16 | import simplejson as json 17 | 18 | class AliyunClient: 19 | BATCH_SIZE = 100 20 | 21 | def describe(self, resource): 22 | page = 1 23 | received = 0 24 | total = 0 25 | 26 | while page == 1 or received < total: 27 | command = ['aliyuncli', resource, 'DescribeInstances', '--output', 'json', '--PageSize', repr(self.BATCH_SIZE), '--PageNumber', repr(page)] 28 | resp = json.loads(subprocess.check_output(command)) 29 | page += 1 30 | received += self.BATCH_SIZE 31 | total = resp['TotalCount'] 32 | 33 | for item in resp['Instances']['Instance']: 34 | yield item 35 | 36 | class AliyunInventory: 37 | def _empty_index(self): 38 | index = defaultdict(list, {'_meta': {'hostvars' : {}}}) 39 | return index 40 | 41 | def __init__(self): 42 | self.client = AliyunClient() 43 | self.read_settings() 44 | self.parse_cli_args() 45 | self.load_inventory() 46 | 47 | if self.args.host: 48 | host_vars = self.inventory['index']['_meta']['hostvars'][self.args.host] 49 | data_to_print = host_vars or {} 50 | else: 51 | data_to_print = self.inventory['index'] 52 | 53 | print self.json_format_dict(data_to_print, True) 54 | 55 | def read_settings(self): 56 | """ Reads the settings from the aliyun.ini file """ 57 | 58 | config = self.config = ConfigParser.RawConfigParser() 59 | script_file = os.path.realpath(__file__) 60 | config_dir = os.path.dirname(script_file) 61 | config_basename = os.path.basename(script_file).rsplit('.', 1)[0] + '.ini' 62 | config.read('/'.join([config_dir, config_basename])) 63 | 64 | # Cache related 65 | self.cache_path = config.get('cache', 'path') 66 | cache_dir = os.path.dirname(self.cache_path) 67 | try: 68 | os.makedirs(cache_dir) 69 | except OSError as exc: 70 | if exc.errno == errno.EEXIST and os.path.isdir(cache_dir): 71 | pass 72 | else: raise 73 | 74 | # Determine whether max_age parameters exist 75 | try: 76 | config.getint('cache', 'max_age') 77 | except: 78 | self.cache_max_age = 86400 79 | else: 80 | self.cache_max_age = config.getint('cache', 'max_age') 81 | 82 | # Determine whether cache_disable parameters exist 83 | try: 84 | config.getboolean('cache', 'cache_disable') 85 | except: 86 | self.cache_disable = False 87 | else: 88 | self.cache_disable = config.getboolean('cache', 'cache_disable') 89 | 90 | 91 | 92 | def is_cache_valid(self): 93 | """ Determines if the cache files have expired, or if it is still valid """ 94 | 95 | if os.path.isfile(self.cache_path): 96 | mod_time = os.path.getmtime(self.cache_path) 97 | current_time = time() 98 | return (mod_time + self.cache_max_age) > current_time 99 | else: 100 | return False 101 | 102 | 103 | def build_inventory(self): 104 | index = self._empty_index() 105 | self.add_ecs(index) 106 | 107 | return { 'index': index } 108 | 109 | 110 | def add_ecs(self, index): 111 | for ecs in self.client.describe('ecs'): 112 | safe_name = self.to_safe(ecs['InstanceName']) 113 | index['ecs'].append(safe_name) 114 | for tag in ecs['Description'].split(','): 115 | tag = tag.strip() 116 | if len(tag) > 0: 117 | index['des_' + self.to_safe(tag)].append(safe_name) 118 | if 'Tags' in ecs.keys(): 119 | for k in ecs['Tags']['Tag']: 120 | index[self.to_safe(k['TagKey']) + "_" + self.to_safe(k['TagValue'])].append(safe_name) 121 | ecs = self.extract_ips(ecs) 122 | ssh_options = self.ssh_options('ecs', safe_name, ecs) 123 | index['_meta']['hostvars'][safe_name] = dict(ssh_options, aliyun = ecs) 124 | 125 | def extract_ips(self, instance): 126 | ips = dict() 127 | for key, value in instance.iteritems(): 128 | if isinstance(value, dict) and 'IpAddress' in value and isinstance(value['IpAddress'], list) and len(value['IpAddress']) > 0 and key.endswith('IpAddress'): 129 | ips[key[:-len('Address')]] = value['IpAddress'][0] 130 | 131 | instance.update(ips) 132 | 133 | eips = dict() 134 | for key, value in instance.iteritems(): 135 | if isinstance(value, dict) and 'IpAddress' in value and len(value['IpAddress']) > 0 and key.endswith('EipAddress'): 136 | eips['EipAddress'] = value['IpAddress'] 137 | 138 | instance.update(eips) 139 | 140 | vips = dict() 141 | for key, value in instance.iteritems(): 142 | if isinstance(value, dict) and 'PrivateIpAddress' in value and len(value['PrivateIpAddress']['IpAddress']) > 0 and key.endswith('VpcAttributes'): 143 | vips['Vip'] = value['PrivateIpAddress']['IpAddress'][0] 144 | 145 | instance.update(vips) 146 | return instance 147 | 148 | def ssh_options(self, kind, name, instance): 149 | options = dict(self.config.items(kind)) 150 | specific_section = '.'.join([kind, name]) 151 | if self.config.has_section(specific_section): 152 | options.update(self.config.items(specific_section)) 153 | 154 | return { 155 | 'ansible_ssh_user': options['user'] % instance, 156 | 'ansible_ssh_host': options['host'] % instance, 157 | 'ansible_ssh_port': options['port'] % instance 158 | } 159 | 160 | 161 | def load_inventory(self): 162 | if self.args.refresh_cache or not self.is_cache_valid() or self.cache_disable: 163 | self.inventory = self.build_inventory() 164 | self.write_cache() 165 | else: 166 | self.read_cache() 167 | 168 | 169 | def write_cache(self): 170 | json_data = self.json_format_dict(self.inventory, True) 171 | cache = open(self.cache_path, 'w') 172 | cache.write(json_data) 173 | cache.close() 174 | 175 | 176 | def read_cache(self): 177 | cache = open(self.cache_path, 'r') 178 | json_data = cache.read() 179 | self.inventory = json.loads(json_data) 180 | 181 | 182 | def to_safe(self, word): 183 | ''' Converts 'bad' characters in a string to underscores so they can be 184 | used as Ansible groups ''' 185 | 186 | return re.sub("[^A-Za-z0-9\-]", "_", word) 187 | 188 | 189 | def parse_cli_args(self): 190 | ''' Command line argument processing ''' 191 | 192 | parser = argparse.ArgumentParser(description='Produce an Ansible Inventory file based on Aliyun') 193 | parser.add_argument('--list', action='store_true', default=True, 194 | help='List instances (default: True)') 195 | parser.add_argument('--host', action='store', 196 | help='Get all the variables about a specific instance') 197 | parser.add_argument('--refresh-cache', action='store_true', default=False, 198 | help='Force refresh of cache by making API requests to Aliyun (default: False - use cache files)') 199 | self.args = parser.parse_args() 200 | 201 | 202 | def json_format_dict(self, data, pretty=False): 203 | if pretty: 204 | return json.dumps(data, sort_keys=True, indent=2) 205 | else: 206 | return json.dumps(data) 207 | 208 | 209 | AliyunInventory() 210 | --------------------------------------------------------------------------------