├── LICENSE ├── README.md ├── ajax.js ├── app.py ├── cpis.html ├── flask-api.py ├── flask-celery-client.py ├── flask-celery-server.py ├── hosts.py ├── templates └── index.html ├── test.yaml └── test_playbook.py /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Miguel Grinberg 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![996.icu](https://img.shields.io/badge/link-996.icu-red.svg)](https://996.icu) 2 | 3 | ## 介绍 4 | 5 | ansible是一个基于SSH协议的自动化部署工具,但官网提供的例子大多都是在命令行下执行的, 6 | 对于python api只提供了很简单的信息,对于批量任务(playbook)的api,官方甚至没有提供 7 | 文档。 8 | 9 | 但是要用ansible实现一套自动化部署系统,是必须要有个界面提供给使用者的, 使用者可以请求 10 | 执行一个批量部署任务,并获取任务执行进度和最终状态,在这期间,不能长时间占用web worker 11 | 的时间。 12 | 13 | 所以查了一些资料,最后用flask,celery,ansible来写了个demo 14 | 15 | ## 执行 16 | 17 | 启动redis,作为celery的broker和result backend 18 | $ service redis start 19 | 启动celery来接受任务处理 20 | $ celery worker -A app.celery --loglevel=info 21 | 启动flask web app 22 | $ python app.py 23 | 浏览器里打开http://localhost/play,按界面执行就可以了 24 | 25 | 26 | ## 文件介绍 27 | 28 | ### hosts.py 29 | 30 | 使用ansible进行自动化部署,需要先定义Inventory,一般来说Inventory是一个静态文本文件, 31 | 里面定义了ansible可管理的主机列表,但是有时候每个用户可管理的机器列表是不一样的, 32 | 或者要管理的机器列表经常变化,比如要和公司的CMDB去集成,这就需要用到 Dynamic Inventory 33 | 34 | 这个脚本是根据环境变量来获取指定的机器分组, 具体实现可从数据里读取可管理的机器列表。 35 | 36 | 参考链接: 37 | 38 | - [Dynamic Inventory](http://docs.ansible.com/intro_dynamic_inventory.html) 39 | - [Developing Dynamic Inventory Sources](http://docs.ansible.com/developing_inventory.html) 40 | 41 | ### test.yaml 42 | 43 | 这里定义了一个ansible playbook,就是一个部署任务,安装nginx,并启动nginx。 44 | 可以用如下代码执行该playbook 45 | 46 | $ ansible-playbook -i hosts.py test.yaml 47 | 48 | 参考链接: 49 | 50 | - [Playbooks](http://docs.ansible.com/playbooks.html) 51 | 52 | ### test_playbook.py 53 | 54 | 该文件是利用了ansible的python api,去以编程的方式,而不是命令行的方式去执行一个playbook。 55 | 可以直接用如下命令去执行 56 | 57 | $ python test_playbook.py 58 | 59 | 注意这里的`PlaybookRunnerCallbacks`和`PlaybookCallbacks`两个类,是我自己定义过的,主要目的 60 | 是在每个task执行开始,执行成功或失败时,调用`update_state`去更新celery任务状态, 每个类的 61 | `celery_taski`成员是一个celery的task,在构造函数里传入的。 62 | 63 | 参考链接 64 | 65 | - [celery custom states](http://docs.celeryproject.org/en/latest/userguide/tasks.html#custom-states) 66 | - [ansible python api](http://docs.ansible.com/developing_api.html) 67 | - [Running ansible-playbook using Python API](http://stackoverflow.com/questions/27590039/running-ansible-playbook-using-python-api) 68 | 69 | ### app.py 70 | 71 | 这是一个flask web app,其中`/play/longtask`是发起一个异步任务,也就是执行playbook,该请求会返回 72 | 轮询检测任务执行状态的url,包含了该异步任务的ID, 如`/play/status/111` 。 73 | 74 | 参考链接 75 | 76 | - [Using Celery with Flask](http://blog.miguelgrinberg.com/post/using-celery-with-flask) 77 | 78 | ### templates/index.html 79 | 80 | 在界面上点击按钮发起执行playbook的请求,然后自动轮询获取任务执行状态并显示。在整个任务执行过程中 81 | web应用不会被hang住,真正的任务执行是celery执行的。 82 | 83 | ## 参考链接: 84 | 85 | - [自动化运维工具之ansible](http://guoting.blog.51cto.com/8886857/1553446) 86 | - [配置管理工具之Ansible视频教程(共10课时)_](http://edu.51cto.com/index.php?do=lession&id=38985) 87 | 88 | [![LICENSE](https://img.shields.io/badge/license-NPL%20(The%20996%20Prohibited%20License)-blue.svg)](https://github.com/996icu/996.ICU/blob/master/LICENSE) 89 | -------------------------------------------------------------------------------- /ajax.js: -------------------------------------------------------------------------------- 1 | 35 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | import os 2 | import random 3 | import time 4 | from flask import Flask, request, render_template, session, flash, redirect, \ 5 | url_for, jsonify 6 | from celery import Celery 7 | from test_playbook import get_pb 8 | 9 | app = Flask(__name__) 10 | app.config['SECRET_KEY'] = 'top-secret!' 11 | 12 | # Celery configuration 13 | app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0' 14 | app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost:6379/0' 15 | 16 | 17 | # Initialize Celery 18 | celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL']) 19 | celery.conf.update(app.config) 20 | 21 | 22 | @celery.task(bind=True) 23 | def long_task(self): 24 | self.logs = [] 25 | r = get_pb(self).run() 26 | self.logs.append("finish playbook") 27 | self.logs.append(str(r)) 28 | return self.logs 29 | 30 | 31 | @app.route('/play', methods=['GET', 'POST']) 32 | def index(): 33 | return render_template('index.html', email=session.get('email', '')) 34 | 35 | 36 | @app.route('/play/longtask', methods=['POST']) 37 | def longtask(): 38 | task = long_task.apply_async() 39 | return jsonify({}), 202, {'Location': url_for('taskstatus', task_id=task.id)} 40 | 41 | 42 | @app.route('/play/status/') 43 | def taskstatus(task_id): 44 | task = long_task.AsyncResult(task_id) 45 | if task.state == 'PENDING': 46 | response = { 47 | 'state': task.state, 48 | 'status': 'Pending...' 49 | } 50 | elif task.state != 'FAILURE': 51 | response = { 52 | 'state': task.state, 53 | 'status': task.info 54 | } 55 | if 'result' in task.info: 56 | response['result'] = task.info['result'] 57 | else: 58 | # something went wrong in the background job 59 | response = { 60 | 'state': task.state, 61 | 'status': task.info, # this is the exception raised 62 | } 63 | return jsonify(response) 64 | 65 | 66 | if __name__ == '__main__': 67 | app.run(debug=True) 68 | -------------------------------------------------------------------------------- /cpis.html: -------------------------------------------------------------------------------- 1 | ##########成功设备列表########### 2 | {% for x,y in cpis.items() %} 3 | ########主机{{ x }}########### 4 | {{ y }} 5 | {% endfor %} 6 | ##########失败设备列表######## 7 | {% for a,b in cpis.items() %} 8 | ############主机 {{a}} ######### 9 | {{b}} 10 | {%endfor%} 11 | -------------------------------------------------------------------------------- /flask-api.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | #coding:utf-8 3 | from ansible.inventory import Inventory 4 | from ansible.playbook import PlayBook 5 | from ansible import callbacks 6 | import ansible.runner 7 | from flask import Flask,request,jsonify,render_template,abort 8 | import commands,json 9 | app = Flask(__name__) 10 | 11 | hostfile='./hosts' 12 | ''' 13 | http://127.0.0.1:5000/API/Ansible/playbook?ip=2.2.2.2&palybook=test 14 | ''' 15 | @app.route('/API/Ansible/playbook') 16 | def Playbook(): 17 | vars={} 18 | inventory = Inventory(hostfile) 19 | stats = callbacks.AggregateStats() 20 | playbook_cb =callbacks.PlaybookCallbacks() 21 | runner_cb =callbacks.PlaybookRunnerCallbacks(stats) 22 | hosts=request.args.get('ip') 23 | task=request.args.get('playbook') 24 | vars['hosts'] = hosts 25 | play=task + '.yml' 26 | results = PlayBook(playbook=play,callbacks=playbook_cb,runner_callbacks=runner_cb,stats=stats,inventory=inventory,extra_vars=vars) 27 | res = results.run() 28 | return json.dumps(res,indent=4) 29 | 30 | ''' 31 | curl -H "Content-Type: application/json" -X POST -d '{"ip":"1.1.1.1","module":"shell","args":"ls -l"}' http://127.0.0.1:5000/API/Ansible/runner 32 | ''' 33 | @app.route('/API/Ansible/runner',methods=['POST']) 34 | def Runner(): 35 | print request.json 36 | if not request.json or not 'ip' in request.json or not 'module' in request.json or not 'args' in request.json: 37 | abort(400) 38 | hosts=request.json['ip'] 39 | module = request.json['module'] 40 | args=request.json['args'] 41 | runner = ansible.runner.Runner(module_name=module,module_args=args,pattern=hosts,forks=10,host_list=hostfile) 42 | tasks=runner.run() 43 | cpis={} 44 | cpis1={} 45 | for (hostname, result) in tasks['contacted'].items(): 46 | if not 'failed' in result: 47 | cpis[hostname] = result['stdout'] 48 | for (hostname, result) in tasks['dark'].items(): 49 | cpis1[hostname] = result['msg'] 50 | return render_template('cpis.html',cpis=cpis,cpis1=cpis1) 51 | 52 | if __name__ == "__main__": 53 | app.run(debug=True,host='0.0.0.0') 54 | -------------------------------------------------------------------------------- /flask-celery-client.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | #coding:utf-8 3 | import requests 4 | import json 5 | import argparse 6 | ppm={'server-1': '1.1.1.1','server-2': '2.2.2.2'} 7 | 8 | def tolist(fn): 9 | ips = [] 10 | with open(fn) as f: 11 | for ip in f: 12 | ips.append(ip.strip()) 13 | return ips 14 | 15 | def run(target,action,ips,users): 16 | p = {'ips': ips, 'users': users } 17 | r = requests.post('http://{0}:5000/{1}'.format(ppm[target],action), data = p) 18 | gto = r.json()['goto'] 19 | while 1: 20 | if requests.get("http://{0}:5000/{1}/result/{2}".format(ppm[target],action,gto)).json()['state'] == "PENDING": 21 | print "task running please wait........." 22 | time.sleep(1) 23 | continue 24 | else: 25 | print " " 26 | print "===============task running result==================" 27 | res=requests.get("http://{0}:5000/{1}/result/{2}".format(ppm[target],action,gto)).json()['status'] 28 | for i in res: 29 | print i,str(res[i]).replace("u","") 30 | break 31 | 32 | if __name__ == '__main__': 33 | parser = argparse.ArgumentParser() 34 | parser.add_argument('-i', '--ips', help='ips files') 35 | parser.add_argument('-u', '--users', help='uses files') 36 | parser.add_argument('-a', '--action', help='user manage action ex: add of del') 37 | parser.add_argument('-t', '--target', help='PPM IDC info ex: server-1 server-2 ....') 38 | args = vars(parser.parse_args()) 39 | if args['ips'] and args['users'] and args['action'] in ['add','del'] and args['target'] in ['server-1','server-2'] : 40 | ips=tolist(args['ips']) 41 | users=tolist(args['users']) 42 | run(args['target'],args['action'],ips,users) 43 | else: 44 | print parser.print_help() 45 | -------------------------------------------------------------------------------- /flask-celery-server.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/python 2 | #coding:utf-8 3 | from celery import Celery 4 | import json 5 | from flask import Flask, abort, jsonify, request, session 6 | from ansible.inventory import Inventory 7 | from ansible.playbook import PlayBook 8 | from ansible import callbacks 9 | import jinja2 10 | from tempfile import NamedTemporaryFile 11 | 12 | app = Flask(__name__) 13 | app.config['SECRET_KEY'] = 'top-secret!' 14 | app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0' 15 | app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost:6379/0' 16 | celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL']) 17 | celery.conf.update(app.config) 18 | 19 | @celery.task 20 | def adduser(ips, users): 21 | inventory =""" 22 | {% for i in hosts -%} 23 | {{ i }} 24 | {% endfor %} 25 | """ 26 | inventory_template = jinja2.Template(inventory) 27 | rendered_inventory = inventory_template.render({'hosts': ips}) 28 | hosts = NamedTemporaryFile(delete=False,suffix='tmp',dir='/tmp/ansible/') 29 | hosts.write(rendered_inventory) 30 | hosts.close() 31 | inventory = Inventory(hosts.name) 32 | stats = callbacks.AggregateStats() 33 | playbook_cb =callbacks.PlaybookCallbacks() 34 | runner_cb =callbacks.PlaybookRunnerCallbacks(stats) 35 | vars={} 36 | vars['users'] = users 37 | results = PlayBook(playbook='user.yaml',callbacks=playbook_cb,runner_callbacks=runner_cb,stats=stats,inventory=inventory,extra_vars=vars) 38 | res = results.run() 39 | logs = [] 40 | logs.append("finish playbook\n") 41 | logs.append(str(res)) 42 | return logs 43 | 44 | @app.route('/', methods=['GET', 'POST']) 45 | def index(): 46 | return render_template('index.html') 47 | 48 | @app.route("/add",methods=['POST']) 49 | def one(): 50 | ips = [ i.encode('ascii') for i in request.form.getlist('ips') ] 51 | users = [ i.encode('ascii') for i in request.form.getlist('users') ] 52 | res = adduser.apply_async((ips, users)) 53 | context = {"id": res.task_id, "ips": ips, "users": users} 54 | result = "add((ips){0}, (users){1})".format(context['ips'], context['users']) 55 | goto = "{0}".format(context['id']) 56 | return jsonify(result=result, goto=goto) 57 | 58 | @app.route("/add/result/") 59 | def show_add_result(task_id): 60 | task = adduser.AsyncResult(task_id) 61 | if task.state == 'PENDING': 62 | response = { 63 | 'state': task.state, 64 | 'status': 'Pending...' 65 | } 66 | elif task.state != 'FAILURE': 67 | response = { 68 | 'state': task.state, 69 | 'status': task.info 70 | } 71 | if 'result' in task.info: 72 | response['result'] = task.info['result'] 73 | else: 74 | response = { 75 | 'state': task.state, 76 | 'status': task.info, 77 | } 78 | return jsonify(response) 79 | 80 | 81 | if __name__ == "__main__": 82 | app.run(host='0.0.0.0', port=5000, debug=True) 83 | -------------------------------------------------------------------------------- /hosts.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | ''' 4 | ''' 5 | 6 | import json 7 | import os 8 | group = os.environ.get('ANSBILE_HOST_GROUP') 9 | 10 | hosts = { 11 | 'group1': ['localhost', '127.0.0.1'], 12 | 'group2': ['127.0.0.1'], 13 | } 14 | 15 | ret = hosts.get(group, []) 16 | ret = {"default": ret} 17 | 18 | print json.dumps(ret, indent=4) 19 | -------------------------------------------------------------------------------- /templates/index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | Flask + Celery + Ansible 4 | 5 | 6 |

Flask + Celery + Ansible

7 |

8 |

 9 |     
10 |     
11 |     
48 |   
49 | 
50 | 


--------------------------------------------------------------------------------
/test.yaml:
--------------------------------------------------------------------------------
 1 | - name: web service
 2 |   remote_user: phpa 
 3 |   hosts: localhost 
 4 |   vars:
 5 |     packages: nginx 
 6 |   tasks:
 7 |     - name: install nginx 
 8 |       yum: name={{ packages }} state=present
 9 |       tags: install
10 |     - name: service nginx start
11 |       service: name=nginx enabled=yes state=started
12 |       sudo: yes
13 |       tags: start
14 | 


--------------------------------------------------------------------------------
/test_playbook.py:
--------------------------------------------------------------------------------
 1 | from ansible.playbook import PlayBook
 2 | from ansible.inventory import Inventory
 3 | from ansible import callbacks
 4 | from ansible import utils
 5 | 
 6 | 
 7 | class PlaybookRunnerCallbacks(callbacks.PlaybookRunnerCallbacks):
 8 |     def __init__(self, task, stats, verbose=None):
 9 |         super(PlaybookRunnerCallbacks, self).__init__(stats, verbose)
10 |         self.celery_task = task
11 | 
12 |     def on_ok(self, host, host_result):
13 |         super(PlaybookRunnerCallbacks, self).on_ok(host, host_result)
14 |         self.celery_task.logs.append("ok:[%s]" % host)
15 |         self.celery_task.update_state(state='PROGRESS', meta={'msg': self.celery_task.logs})
16 | 
17 |     def on_unreachable(self, host, results):
18 |         super(PlaybookRunnerCallbacks, self).on_unreachable(host, results)
19 |         self.celery_task.logs.append("unreachable:[%s] %s" % (host, results))
20 |         self.celery_task.update_state(state='FAILURE', meta={'msg': self.celery_task.logs})
21 | 
22 |     def on_failed(self, host, results, ignore_errors=False):
23 |         super(PlaybookRunnerCallbacks, self).on_failed(host, results, ignore_errors)
24 |         self.celery_task.logs.append("failed:[%s] %s" % (host, results))
25 |         self.celery_task.update_state(state='FAILURE', meta={'msg': self.celery_task.logs})
26 | 
27 | 
28 | class PlaybookCallbacks(callbacks.PlaybookCallbacks):
29 |     def __init__(self, task, verbose=False):
30 |         super(PlaybookCallbacks, self).__init__(verbose);
31 |         self.celery_task = task
32 | 
33 |     def on_setup(self):
34 |         super(PlaybookCallbacks, self).on_setup()
35 |         self.celery_task.logs.append("GATHERING FACTS")
36 |         self.celery_task.update_state(state='PROGRESS', meta={'msg': self.celery_task.logs})
37 | 
38 |     def on_task_start(self, name, is_conditional):
39 |         super(PlaybookCallbacks, self).on_task_start(name, is_conditional)
40 |         self.celery_task.logs.append("TASK: [%s]" % name)
41 |         self.celery_task.update_state(state='PROGRESS', meta={'msg': self.celery_task.logs})
42 | 
43 | 
44 | hostfile = './hosts.py'
45 | inventory = Inventory(hostfile) 
46 | stats = callbacks.AggregateStats()
47 | vars = {"hosts":['127.0.0.1']}
48 | 
49 | 
50 | def get_pb(task):
51 |     if task:
52 |         runner_cb = PlaybookRunnerCallbacks(task, stats, verbose=utils.VERBOSITY)
53 |         playbook_cb = PlaybookCallbacks(task, verbose=utils.VERBOSITY)
54 |     else:
55 |         runner_cb = callbacks.PlaybookRunnerCallbacks(stats, verbose=utils.VERBOSITY)
56 |         playbook_cb = callbacks.PlaybookCallbacks(verbose=utils.VERBOSITY)
57 | 
58 |     pb = PlayBook(playbook='./test.yaml', 
59 |                   callbacks=playbook_cb,
60 |                   runner_callbacks=runner_cb,
61 |                   stats=stats,
62 |                   inventory=inventory,
63 |                   extra_vars=vars,
64 |                   )
65 |     return pb
66 | 
67 | if __name__ == '__main__':
68 |     get_pb(None).run()
69 | 


--------------------------------------------------------------------------------