├── LICENSE
├── README.md
├── ajax.js
├── app.py
├── cpis.html
├── flask-api.py
├── flask-celery-client.py
├── flask-celery-server.py
├── hosts.py
├── templates
└── index.html
├── test.yaml
└── test_playbook.py
/LICENSE:
--------------------------------------------------------------------------------
1 | The MIT License (MIT)
2 |
3 | Copyright (c) 2014 Miguel Grinberg
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
23 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | [](https://996.icu)
2 |
3 | ## 介绍
4 |
5 | ansible是一个基于SSH协议的自动化部署工具,但官网提供的例子大多都是在命令行下执行的,
6 | 对于python api只提供了很简单的信息,对于批量任务(playbook)的api,官方甚至没有提供
7 | 文档。
8 |
9 | 但是要用ansible实现一套自动化部署系统,是必须要有个界面提供给使用者的, 使用者可以请求
10 | 执行一个批量部署任务,并获取任务执行进度和最终状态,在这期间,不能长时间占用web worker
11 | 的时间。
12 |
13 | 所以查了一些资料,最后用flask,celery,ansible来写了个demo
14 |
15 | ## 执行
16 |
17 | 启动redis,作为celery的broker和result backend
18 | $ service redis start
19 | 启动celery来接受任务处理
20 | $ celery worker -A app.celery --loglevel=info
21 | 启动flask web app
22 | $ python app.py
23 | 浏览器里打开http://localhost/play,按界面执行就可以了
24 |
25 |
26 | ## 文件介绍
27 |
28 | ### hosts.py
29 |
30 | 使用ansible进行自动化部署,需要先定义Inventory,一般来说Inventory是一个静态文本文件,
31 | 里面定义了ansible可管理的主机列表,但是有时候每个用户可管理的机器列表是不一样的,
32 | 或者要管理的机器列表经常变化,比如要和公司的CMDB去集成,这就需要用到 Dynamic Inventory
33 |
34 | 这个脚本是根据环境变量来获取指定的机器分组, 具体实现可从数据里读取可管理的机器列表。
35 |
36 | 参考链接:
37 |
38 | - [Dynamic Inventory](http://docs.ansible.com/intro_dynamic_inventory.html)
39 | - [Developing Dynamic Inventory Sources](http://docs.ansible.com/developing_inventory.html)
40 |
41 | ### test.yaml
42 |
43 | 这里定义了一个ansible playbook,就是一个部署任务,安装nginx,并启动nginx。
44 | 可以用如下代码执行该playbook
45 |
46 | $ ansible-playbook -i hosts.py test.yaml
47 |
48 | 参考链接:
49 |
50 | - [Playbooks](http://docs.ansible.com/playbooks.html)
51 |
52 | ### test_playbook.py
53 |
54 | 该文件是利用了ansible的python api,去以编程的方式,而不是命令行的方式去执行一个playbook。
55 | 可以直接用如下命令去执行
56 |
57 | $ python test_playbook.py
58 |
59 | 注意这里的`PlaybookRunnerCallbacks`和`PlaybookCallbacks`两个类,是我自己定义过的,主要目的
60 | 是在每个task执行开始,执行成功或失败时,调用`update_state`去更新celery任务状态, 每个类的
61 | `celery_taski`成员是一个celery的task,在构造函数里传入的。
62 |
63 | 参考链接
64 |
65 | - [celery custom states](http://docs.celeryproject.org/en/latest/userguide/tasks.html#custom-states)
66 | - [ansible python api](http://docs.ansible.com/developing_api.html)
67 | - [Running ansible-playbook using Python API](http://stackoverflow.com/questions/27590039/running-ansible-playbook-using-python-api)
68 |
69 | ### app.py
70 |
71 | 这是一个flask web app,其中`/play/longtask`是发起一个异步任务,也就是执行playbook,该请求会返回
72 | 轮询检测任务执行状态的url,包含了该异步任务的ID, 如`/play/status/111` 。
73 |
74 | 参考链接
75 |
76 | - [Using Celery with Flask](http://blog.miguelgrinberg.com/post/using-celery-with-flask)
77 |
78 | ### templates/index.html
79 |
80 | 在界面上点击按钮发起执行playbook的请求,然后自动轮询获取任务执行状态并显示。在整个任务执行过程中
81 | web应用不会被hang住,真正的任务执行是celery执行的。
82 |
83 | ## 参考链接:
84 |
85 | - [自动化运维工具之ansible](http://guoting.blog.51cto.com/8886857/1553446)
86 | - [配置管理工具之Ansible视频教程(共10课时)_](http://edu.51cto.com/index.php?do=lession&id=38985)
87 |
88 | [-blue.svg)](https://github.com/996icu/996.ICU/blob/master/LICENSE)
89 |
--------------------------------------------------------------------------------
/ajax.js:
--------------------------------------------------------------------------------
1 |
35 |
--------------------------------------------------------------------------------
/app.py:
--------------------------------------------------------------------------------
1 | import os
2 | import random
3 | import time
4 | from flask import Flask, request, render_template, session, flash, redirect, \
5 | url_for, jsonify
6 | from celery import Celery
7 | from test_playbook import get_pb
8 |
9 | app = Flask(__name__)
10 | app.config['SECRET_KEY'] = 'top-secret!'
11 |
12 | # Celery configuration
13 | app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0'
14 | app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost:6379/0'
15 |
16 |
17 | # Initialize Celery
18 | celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL'])
19 | celery.conf.update(app.config)
20 |
21 |
22 | @celery.task(bind=True)
23 | def long_task(self):
24 | self.logs = []
25 | r = get_pb(self).run()
26 | self.logs.append("finish playbook")
27 | self.logs.append(str(r))
28 | return self.logs
29 |
30 |
31 | @app.route('/play', methods=['GET', 'POST'])
32 | def index():
33 | return render_template('index.html', email=session.get('email', ''))
34 |
35 |
36 | @app.route('/play/longtask', methods=['POST'])
37 | def longtask():
38 | task = long_task.apply_async()
39 | return jsonify({}), 202, {'Location': url_for('taskstatus', task_id=task.id)}
40 |
41 |
42 | @app.route('/play/status/')
43 | def taskstatus(task_id):
44 | task = long_task.AsyncResult(task_id)
45 | if task.state == 'PENDING':
46 | response = {
47 | 'state': task.state,
48 | 'status': 'Pending...'
49 | }
50 | elif task.state != 'FAILURE':
51 | response = {
52 | 'state': task.state,
53 | 'status': task.info
54 | }
55 | if 'result' in task.info:
56 | response['result'] = task.info['result']
57 | else:
58 | # something went wrong in the background job
59 | response = {
60 | 'state': task.state,
61 | 'status': task.info, # this is the exception raised
62 | }
63 | return jsonify(response)
64 |
65 |
66 | if __name__ == '__main__':
67 | app.run(debug=True)
68 |
--------------------------------------------------------------------------------
/cpis.html:
--------------------------------------------------------------------------------
1 | ##########成功设备列表###########
2 | {% for x,y in cpis.items() %}
3 | ########主机{{ x }}###########
4 | {{ y }}
5 | {% endfor %}
6 | ##########失败设备列表########
7 | {% for a,b in cpis.items() %}
8 | ############主机 {{a}} #########
9 | {{b}}
10 | {%endfor%}
11 |
--------------------------------------------------------------------------------
/flask-api.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python
2 | #coding:utf-8
3 | from ansible.inventory import Inventory
4 | from ansible.playbook import PlayBook
5 | from ansible import callbacks
6 | import ansible.runner
7 | from flask import Flask,request,jsonify,render_template,abort
8 | import commands,json
9 | app = Flask(__name__)
10 |
11 | hostfile='./hosts'
12 | '''
13 | http://127.0.0.1:5000/API/Ansible/playbook?ip=2.2.2.2&palybook=test
14 | '''
15 | @app.route('/API/Ansible/playbook')
16 | def Playbook():
17 | vars={}
18 | inventory = Inventory(hostfile)
19 | stats = callbacks.AggregateStats()
20 | playbook_cb =callbacks.PlaybookCallbacks()
21 | runner_cb =callbacks.PlaybookRunnerCallbacks(stats)
22 | hosts=request.args.get('ip')
23 | task=request.args.get('playbook')
24 | vars['hosts'] = hosts
25 | play=task + '.yml'
26 | results = PlayBook(playbook=play,callbacks=playbook_cb,runner_callbacks=runner_cb,stats=stats,inventory=inventory,extra_vars=vars)
27 | res = results.run()
28 | return json.dumps(res,indent=4)
29 |
30 | '''
31 | curl -H "Content-Type: application/json" -X POST -d '{"ip":"1.1.1.1","module":"shell","args":"ls -l"}' http://127.0.0.1:5000/API/Ansible/runner
32 | '''
33 | @app.route('/API/Ansible/runner',methods=['POST'])
34 | def Runner():
35 | print request.json
36 | if not request.json or not 'ip' in request.json or not 'module' in request.json or not 'args' in request.json:
37 | abort(400)
38 | hosts=request.json['ip']
39 | module = request.json['module']
40 | args=request.json['args']
41 | runner = ansible.runner.Runner(module_name=module,module_args=args,pattern=hosts,forks=10,host_list=hostfile)
42 | tasks=runner.run()
43 | cpis={}
44 | cpis1={}
45 | for (hostname, result) in tasks['contacted'].items():
46 | if not 'failed' in result:
47 | cpis[hostname] = result['stdout']
48 | for (hostname, result) in tasks['dark'].items():
49 | cpis1[hostname] = result['msg']
50 | return render_template('cpis.html',cpis=cpis,cpis1=cpis1)
51 |
52 | if __name__ == "__main__":
53 | app.run(debug=True,host='0.0.0.0')
54 |
--------------------------------------------------------------------------------
/flask-celery-client.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python
2 | #coding:utf-8
3 | import requests
4 | import json
5 | import argparse
6 | ppm={'server-1': '1.1.1.1','server-2': '2.2.2.2'}
7 |
8 | def tolist(fn):
9 | ips = []
10 | with open(fn) as f:
11 | for ip in f:
12 | ips.append(ip.strip())
13 | return ips
14 |
15 | def run(target,action,ips,users):
16 | p = {'ips': ips, 'users': users }
17 | r = requests.post('http://{0}:5000/{1}'.format(ppm[target],action), data = p)
18 | gto = r.json()['goto']
19 | while 1:
20 | if requests.get("http://{0}:5000/{1}/result/{2}".format(ppm[target],action,gto)).json()['state'] == "PENDING":
21 | print "task running please wait........."
22 | time.sleep(1)
23 | continue
24 | else:
25 | print " "
26 | print "===============task running result=================="
27 | res=requests.get("http://{0}:5000/{1}/result/{2}".format(ppm[target],action,gto)).json()['status']
28 | for i in res:
29 | print i,str(res[i]).replace("u","")
30 | break
31 |
32 | if __name__ == '__main__':
33 | parser = argparse.ArgumentParser()
34 | parser.add_argument('-i', '--ips', help='ips files')
35 | parser.add_argument('-u', '--users', help='uses files')
36 | parser.add_argument('-a', '--action', help='user manage action ex: add of del')
37 | parser.add_argument('-t', '--target', help='PPM IDC info ex: server-1 server-2 ....')
38 | args = vars(parser.parse_args())
39 | if args['ips'] and args['users'] and args['action'] in ['add','del'] and args['target'] in ['server-1','server-2'] :
40 | ips=tolist(args['ips'])
41 | users=tolist(args['users'])
42 | run(args['target'],args['action'],ips,users)
43 | else:
44 | print parser.print_help()
45 |
--------------------------------------------------------------------------------
/flask-celery-server.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python
2 | #coding:utf-8
3 | from celery import Celery
4 | import json
5 | from flask import Flask, abort, jsonify, request, session
6 | from ansible.inventory import Inventory
7 | from ansible.playbook import PlayBook
8 | from ansible import callbacks
9 | import jinja2
10 | from tempfile import NamedTemporaryFile
11 |
12 | app = Flask(__name__)
13 | app.config['SECRET_KEY'] = 'top-secret!'
14 | app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0'
15 | app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost:6379/0'
16 | celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL'])
17 | celery.conf.update(app.config)
18 |
19 | @celery.task
20 | def adduser(ips, users):
21 | inventory ="""
22 | {% for i in hosts -%}
23 | {{ i }}
24 | {% endfor %}
25 | """
26 | inventory_template = jinja2.Template(inventory)
27 | rendered_inventory = inventory_template.render({'hosts': ips})
28 | hosts = NamedTemporaryFile(delete=False,suffix='tmp',dir='/tmp/ansible/')
29 | hosts.write(rendered_inventory)
30 | hosts.close()
31 | inventory = Inventory(hosts.name)
32 | stats = callbacks.AggregateStats()
33 | playbook_cb =callbacks.PlaybookCallbacks()
34 | runner_cb =callbacks.PlaybookRunnerCallbacks(stats)
35 | vars={}
36 | vars['users'] = users
37 | results = PlayBook(playbook='user.yaml',callbacks=playbook_cb,runner_callbacks=runner_cb,stats=stats,inventory=inventory,extra_vars=vars)
38 | res = results.run()
39 | logs = []
40 | logs.append("finish playbook\n")
41 | logs.append(str(res))
42 | return logs
43 |
44 | @app.route('/', methods=['GET', 'POST'])
45 | def index():
46 | return render_template('index.html')
47 |
48 | @app.route("/add",methods=['POST'])
49 | def one():
50 | ips = [ i.encode('ascii') for i in request.form.getlist('ips') ]
51 | users = [ i.encode('ascii') for i in request.form.getlist('users') ]
52 | res = adduser.apply_async((ips, users))
53 | context = {"id": res.task_id, "ips": ips, "users": users}
54 | result = "add((ips){0}, (users){1})".format(context['ips'], context['users'])
55 | goto = "{0}".format(context['id'])
56 | return jsonify(result=result, goto=goto)
57 |
58 | @app.route("/add/result/")
59 | def show_add_result(task_id):
60 | task = adduser.AsyncResult(task_id)
61 | if task.state == 'PENDING':
62 | response = {
63 | 'state': task.state,
64 | 'status': 'Pending...'
65 | }
66 | elif task.state != 'FAILURE':
67 | response = {
68 | 'state': task.state,
69 | 'status': task.info
70 | }
71 | if 'result' in task.info:
72 | response['result'] = task.info['result']
73 | else:
74 | response = {
75 | 'state': task.state,
76 | 'status': task.info,
77 | }
78 | return jsonify(response)
79 |
80 |
81 | if __name__ == "__main__":
82 | app.run(host='0.0.0.0', port=5000, debug=True)
83 |
--------------------------------------------------------------------------------
/hosts.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python
2 |
3 | '''
4 | '''
5 |
6 | import json
7 | import os
8 | group = os.environ.get('ANSBILE_HOST_GROUP')
9 |
10 | hosts = {
11 | 'group1': ['localhost', '127.0.0.1'],
12 | 'group2': ['127.0.0.1'],
13 | }
14 |
15 | ret = hosts.get(group, [])
16 | ret = {"default": ret}
17 |
18 | print json.dumps(ret, indent=4)
19 |
--------------------------------------------------------------------------------
/templates/index.html:
--------------------------------------------------------------------------------
1 |
2 |
3 | Flask + Celery + Ansible
4 |
5 |
6 | Flask + Celery + Ansible
7 |
8 |
9 |
10 |
11 |
48 |
49 |
50 |
--------------------------------------------------------------------------------
/test.yaml:
--------------------------------------------------------------------------------
1 | - name: web service
2 | remote_user: phpa
3 | hosts: localhost
4 | vars:
5 | packages: nginx
6 | tasks:
7 | - name: install nginx
8 | yum: name={{ packages }} state=present
9 | tags: install
10 | - name: service nginx start
11 | service: name=nginx enabled=yes state=started
12 | sudo: yes
13 | tags: start
14 |
--------------------------------------------------------------------------------
/test_playbook.py:
--------------------------------------------------------------------------------
1 | from ansible.playbook import PlayBook
2 | from ansible.inventory import Inventory
3 | from ansible import callbacks
4 | from ansible import utils
5 |
6 |
7 | class PlaybookRunnerCallbacks(callbacks.PlaybookRunnerCallbacks):
8 | def __init__(self, task, stats, verbose=None):
9 | super(PlaybookRunnerCallbacks, self).__init__(stats, verbose)
10 | self.celery_task = task
11 |
12 | def on_ok(self, host, host_result):
13 | super(PlaybookRunnerCallbacks, self).on_ok(host, host_result)
14 | self.celery_task.logs.append("ok:[%s]" % host)
15 | self.celery_task.update_state(state='PROGRESS', meta={'msg': self.celery_task.logs})
16 |
17 | def on_unreachable(self, host, results):
18 | super(PlaybookRunnerCallbacks, self).on_unreachable(host, results)
19 | self.celery_task.logs.append("unreachable:[%s] %s" % (host, results))
20 | self.celery_task.update_state(state='FAILURE', meta={'msg': self.celery_task.logs})
21 |
22 | def on_failed(self, host, results, ignore_errors=False):
23 | super(PlaybookRunnerCallbacks, self).on_failed(host, results, ignore_errors)
24 | self.celery_task.logs.append("failed:[%s] %s" % (host, results))
25 | self.celery_task.update_state(state='FAILURE', meta={'msg': self.celery_task.logs})
26 |
27 |
28 | class PlaybookCallbacks(callbacks.PlaybookCallbacks):
29 | def __init__(self, task, verbose=False):
30 | super(PlaybookCallbacks, self).__init__(verbose);
31 | self.celery_task = task
32 |
33 | def on_setup(self):
34 | super(PlaybookCallbacks, self).on_setup()
35 | self.celery_task.logs.append("GATHERING FACTS")
36 | self.celery_task.update_state(state='PROGRESS', meta={'msg': self.celery_task.logs})
37 |
38 | def on_task_start(self, name, is_conditional):
39 | super(PlaybookCallbacks, self).on_task_start(name, is_conditional)
40 | self.celery_task.logs.append("TASK: [%s]" % name)
41 | self.celery_task.update_state(state='PROGRESS', meta={'msg': self.celery_task.logs})
42 |
43 |
44 | hostfile = './hosts.py'
45 | inventory = Inventory(hostfile)
46 | stats = callbacks.AggregateStats()
47 | vars = {"hosts":['127.0.0.1']}
48 |
49 |
50 | def get_pb(task):
51 | if task:
52 | runner_cb = PlaybookRunnerCallbacks(task, stats, verbose=utils.VERBOSITY)
53 | playbook_cb = PlaybookCallbacks(task, verbose=utils.VERBOSITY)
54 | else:
55 | runner_cb = callbacks.PlaybookRunnerCallbacks(stats, verbose=utils.VERBOSITY)
56 | playbook_cb = callbacks.PlaybookCallbacks(verbose=utils.VERBOSITY)
57 |
58 | pb = PlayBook(playbook='./test.yaml',
59 | callbacks=playbook_cb,
60 | runner_callbacks=runner_cb,
61 | stats=stats,
62 | inventory=inventory,
63 | extra_vars=vars,
64 | )
65 | return pb
66 |
67 | if __name__ == '__main__':
68 | get_pb(None).run()
69 |
--------------------------------------------------------------------------------