├── .gitignore ├── README.md ├── app ├── __init__.py ├── app.py ├── projects.json └── shells │ └── yshells.sh ├── requirements.txt ├── screenshot1.png ├── screenshot2.png └── screenshot3.png /.gitignore: -------------------------------------------------------------------------------- 1 | #### joe made this: http://goel.io/joe 2 | 3 | #####=== Python ===##### 4 | 5 | # Byte-compiled / optimized / DLL files 6 | __pycache__/ 7 | *.py[cod] 8 | *$py.class 9 | .idea/ 10 | # C extensions 11 | *.so 12 | 13 | # Distribution / packaging 14 | .Python 15 | env/ 16 | build/ 17 | develop-eggs/ 18 | dist/ 19 | downloads/ 20 | eggs/ 21 | .eggs/ 22 | lib/ 23 | lib64/ 24 | parts/ 25 | sdist/ 26 | var/ 27 | *.egg-info/ 28 | .installed.cfg 29 | *.egg 30 | 31 | # PyInstaller 32 | # Usually these files are written by a python script from a template 33 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 34 | *.manifest 35 | *.spec 36 | 37 | # Installer logs 38 | pip-log.txt 39 | pip-delete-this-directory.txt 40 | 41 | # Unit test / coverage reports 42 | htmlcov/ 43 | .tox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *,cover 50 | 51 | # Translations 52 | *.mo 53 | *.pot 54 | 55 | # Django stuff: 56 | *.log 57 | 58 | # Sphinx documentation 59 | docs/_build/ 60 | 61 | # PyBuilder 62 | target/ 63 | 64 | venv/ 65 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # smartwebhook 2 | 3 | A smart git webhook bases on bottle supports to github and coding or other git platform. 4 | 5 | # Usage 6 | 7 | 使用比较简单,大致意思就是在设置了webhook的git项目中,当ping或push代码后, git平台就会触发webhook 8 | 9 | 使用截图: 10 | 11 | ![](./screenshot3.png) 12 | 13 | ![](./screenshot1.png) 14 | 15 | ![](./screenshot2.png) 16 | 17 | 18 | 在我们的设置的webhook服务器上, 一旦监听到有ping或push请求就会处理。处理分以下几步: 19 | 20 | 1. 验证请求路径中对应的项目是否存在 (确定项目) 21 | 2. 验证请求头部是否合法并判断是哪个平台 (确定平台, coding.net, github或其他平台) 22 | 3. 验证Token是否正确 (确定合法请求) 23 | 4. 验证是否进行pull操作 (确定动作, 在配置文件中`pull_flag`中设置的pull操作的条件, 如果`git commit` 的提交日志中以`pull_flag`设置的值开头,则触发pull) 24 | 4. 执行命令(在配置文件中`command`选项下可设置shell脚本路径或直接写bash命令) 25 | 5. 发送邮件(确定结果, 在配置文件中可设置邮件信息, 处理结果会以邮件的形式反馈) 26 | 27 | ## Step 1 设置webhook: 28 | 29 | 格式如下: 30 | 31 | http://yourdomain/push/ 32 | 33 | 如我在coding平台下名为yshells的项目中设置的是: 34 | 35 | http://xxxx.com:7777/push/yshells 36 | 令牌是:abcdefg.. 37 | 38 | 根据这种方式可以在多个项目中设置不同的webhook请求 39 | 40 | ## Step 2 写配置文件 41 | 42 | 在 app/projects.json中包括多个webhook的项目配置文件, 说明如下: 43 | 44 | { 45 | "project_name_1": { // 你的项目名字,如yshells 46 | "path": "/path/to/your/yshells", // 项目的路径,在服务器中的绝对路径 47 | "command": "/path/to/your/yshells.sh", // shell脚本路径,在Shell脚本中可自由处理 48 | "pull_flag": "#online#", // pull动作执行的条件,根据git commit 日志匹配 49 | "token": "PL22m@3%!qaz", // 令牌环,这个需与webhook设置的一样 50 | "mail": { // 邮箱信息 51 | "mail_host": "smtp.163.com", // host,这里我使用163邮箱 52 | "mail_port": 25, // 端口,一般smtp服务用25端口 53 | "mail_user": "xinxinyu2011@163.com", // 邮箱用户 54 | "mail_pass": "****", // 邮箱密码 55 | "mail_sender": "xinxinyu2011@163.com", // 发送人 56 | "receivers": ["1373763906@qq.com"] // 接收人 57 | }, 58 | "git": { // git相关信息 59 | "coding" : { // 哪个平台,如github, coding, 等 60 | "X-Coding-Event": "push", // 请求Header的处理[必不可少],可参看相关平台上webhook帮助 61 | "User-Agent": "Coding.net Hook" // 请求Header的处理[必不可少] 62 | } 63 | } 64 | 65 | }, 66 | "project_name_2": { 67 | ... 68 | }, 69 | ... 70 | } 71 | 72 | ## Step 3 写shell脚本 73 | 确实写什么脚本都行,无法改下代码而已,最常见的如shell脚本(例子: yshells.sh): 74 | 75 | #!/usr/bin/env bash 76 | git pull origin master 77 | echo "pull successful!" 78 | 79 | 别忘了赋予可执行权限: 80 | 81 | chmod a+x yshells.sh 82 | 83 | ## Step 4 启动你的项目开始使用吧~ 84 | 85 | 如果你的`pull_flag` 写的是`#online#`, 那么: 86 | 87 | 如提交代码且远程更新, 在提交日志开头加上pull_flag值即可: 88 | 89 | .... 90 | git ci -m "#online# 你的提交日志..." 91 | .... 92 | 93 | 如提交代码但不更新: 94 | 95 | .... 96 | git ci -m "你的提交日志..." 97 | .... 98 | 99 | 注意: ping请求或 提交但不更新不会有email反馈。只有远程更新操作才会有email反馈。 100 | 101 | # Reference Documents 102 | 103 | - [coding](https://coding.net/help/doc/git/webhook.html) 104 | - [github](https://developer.github.com/webhooks/) 105 | 106 | 107 | # TODO: 108 | 109 | - [ ] github webhook处理 110 | - [ ] 自建git服务器中,webhook处理 111 | - [ ] 处理更多自定义事件 -------------------------------------------------------------------------------- /app/__init__.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | """ 3 | desc.. 4 | :copyright: (c) 2016 by fangpeng(@beginman.cn). 5 | :license: MIT, see LICENSE for more details. 6 | """ -------------------------------------------------------------------------------- /app/app.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | """ 3 | webhook 4 | :copyright: (c) 2016 by fangpeng(@beginman.cn). 5 | :license: MIT, see LICENSE for more details. 6 | """ 7 | import json 8 | import threading 9 | import os 10 | import subprocess 11 | from bottle import Bottle, run, request 12 | 13 | base = os.path.dirname(__file__) 14 | project_file = os.path.join(base, 'projects.json') 15 | app = Bottle() 16 | 17 | 18 | @app.post('/push/') 19 | def push(project_name): 20 | msg = 'prepare pulling for project: [%s]....\n' % project_name 21 | pull_status = '[Fail]' 22 | is_send_mail = True 23 | project_info = {} 24 | try: 25 | with open(project_file, 'r') as fb: 26 | projects = json.load(fb) 27 | if project_name not in projects: 28 | raise ValueError("project dose not exist!") 29 | 30 | data = dict(request.forms) 31 | headers = dict(request.headers) 32 | project_info = projects[project_name] 33 | 34 | # valid request headers 35 | git_platform = valid_request_headers(headers, project_info) 36 | if isinstance(git_platform, basestring): # ping request 37 | is_send_mail = False 38 | else: # push request 39 | platform = git_platform.keys()[0] 40 | if platform == 'coding': 41 | result = handel_coding(data, project_info) 42 | is_send_mail = result['pull'] 43 | stdout_msg = result['msg'] 44 | is_pull_succeeded = result['is_pull_succeeded'] 45 | if is_send_mail: 46 | if is_pull_succeeded: 47 | pull_status = '[OK]' 48 | 49 | msg += stdout_msg 50 | 51 | except Exception, e: 52 | # send mail 53 | msg += str(e) 54 | print e 55 | 56 | finally: 57 | if is_send_mail: 58 | mail = project_info.get('mail', None) 59 | if mail: 60 | t = HookThread(send_mail, (msg, project_name, pull_status, mail)) 61 | t.setDaemon(True) 62 | t.start() 63 | # send_mail(msg, project_name, pull_status, mail) 64 | 65 | return {} 66 | 67 | 68 | def valid_request_headers(headers, project_info): 69 | git = project_info['git'] 70 | for k, v in git.items(): 71 | if not headers['User-Agent'].startswith(v['User-Agent']): 72 | raise ValueError("Invalid User-Agent:%s" % headers['User-Agent']) 73 | 74 | if headers['X-Coding-Event'] != v['X-Coding-Event']: 75 | if headers['X-Coding-Event'] == 'ping': 76 | return 'ping' 77 | 78 | raise ValueError("Invalid User-Agent, excepted X-Coding-Event:%s but got %s" \ 79 | % (v['X-Coding-Event'], headers['X-Coding-Event'])) 80 | 81 | for item_k in v: 82 | if item_k not in headers: 83 | raise ValueError("request headers missed key:%s " % item_k) 84 | 85 | return {k: v} 86 | 87 | raise ValueError("Invalid User-Agent") 88 | 89 | 90 | def send_mail(text, project, status, mail_info): 91 | import smtplib 92 | from email.mime.text import MIMEText 93 | 94 | msg = MIMEText(text, 'plain', 'utf-8') 95 | msg['Subject'] = '%s pull %s' % (project, status) 96 | msg['From'] = "SmartWebHook" 97 | try: 98 | s = smtplib.SMTP() 99 | s.connect(mail_info['mail_host'], mail_info['mail_port']) 100 | s.login(mail_info['mail_user'], mail_info['mail_pass']) 101 | s.sendmail(mail_info['mail_sender'], mail_info['receivers'], msg.as_string()) 102 | except smtplib.SMTPException as ex: 103 | print "error: send mail failed, ", ex 104 | 105 | 106 | def handel_coding(data, project_info): 107 | try: 108 | data = data.keys()[0] 109 | 110 | if isinstance(data, (basestring,)): 111 | data = json.loads(data) 112 | 113 | if data.get('event', '') != 'push': 114 | raise ValueError("Only support push event") 115 | 116 | # valid token 117 | valid_token(data, project_info) 118 | 119 | # check short message whether is equal to value of the `pull_flag` flag. 120 | # running command when it's true or it's none. 121 | commits = data['commits'][0] 122 | pull_flag = project_info['pull_flag'] 123 | project_path = project_info['path'] 124 | command = 'cd %s;/bin/bash %s' % (project_path, project_info['command']) 125 | print command 126 | after_commit_id = data['after'] 127 | is_pull = False 128 | if pull_flag: 129 | if commits['short_message'].startswith(pull_flag): 130 | is_pull = True 131 | else: 132 | is_pull = True 133 | 134 | stdout = '' 135 | if is_pull: 136 | stdout = run_command(command) 137 | command = 'cd %s; %s' % (project_path, "git log -n 1 --pretty=oneline") 138 | print command 139 | is_pull_succeeded = valid_pull_status(command, after_commit_id) 140 | if is_pull_succeeded: 141 | stdout = "[OK] Pull Succeeded!\n" + stdout 142 | else: 143 | stdout = "[Fail] Pull Failed!\n" + stdout 144 | return {'pull': True, 'msg': stdout, 'is_pull_succeeded': is_pull_succeeded} 145 | 146 | return {'pull': False, 'msg': stdout} 147 | 148 | except Exception as ex: 149 | raise Exception(ex) 150 | 151 | 152 | def valid_token(data, project_info): 153 | if 'token' not in data: 154 | raise ValueError("Missing Token") 155 | 156 | if project_info['token'] != data['token']: 157 | raise ValueError("Invalid Token") 158 | 159 | 160 | def run_command(command): 161 | m = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) 162 | stdout = '' 163 | for line in m.stdout.readlines(): 164 | stdout += (line +'\n') 165 | return stdout 166 | 167 | 168 | def valid_pull_status(command, after_commit_id): 169 | m = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) 170 | msg = m.stdout.readlines()[0] 171 | last_commit_id = msg.split(' ')[0] 172 | return after_commit_id == last_commit_id 173 | 174 | 175 | class HookThread(threading.Thread): 176 | def __init__(self, func, args): 177 | threading.Thread.__init__(self) 178 | self.func = func 179 | self.args = args 180 | 181 | def run(self): 182 | apply(self.func, self.args) 183 | 184 | run(app, host="0.0.0.0", port=7777, debug=True) -------------------------------------------------------------------------------- /app/projects.json: -------------------------------------------------------------------------------- 1 | { 2 | "yshells": { 3 | "branch": "master", 4 | "path": "/home/team/team/yshells", 5 | "command": "/home/team/team/smartwebhook/app/shells/yshells.sh", 6 | "pull_flag": "#online#", 7 | "token": "PL,Yjm@3%!qaz", 8 | "mail": { 9 | "mail_host": "smtp.163.com", 10 | "mail_port": 25, 11 | "mail_user": "xinxinyu2011@163.com", 12 | "mail_pass": "****", 13 | "mail_sender": "xinxinyu2011@163.com", 14 | "receivers": ["1373763906@qq.com"] 15 | }, 16 | "git": { 17 | "coding" : { 18 | "X-Coding-Event": "push", 19 | "User-Agent": "Coding.net Hook" 20 | } 21 | } 22 | 23 | }, 24 | "main-website": { 25 | "branch": "master", 26 | "path": "/path/to/project", 27 | "command": "/path/to/shell", 28 | "pull_flag": "#online#" 29 | } 30 | } -------------------------------------------------------------------------------- /app/shells/yshells.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | cd /home/team/team/yshells 3 | git pull origin master 4 | echo "pull successful!" 5 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | bottle==0.12.9 2 | -------------------------------------------------------------------------------- /screenshot1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/limboinf/smartwebhook/2d7290e191ea9af3b24b6902b4705cee42c4748e/screenshot1.png -------------------------------------------------------------------------------- /screenshot2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/limboinf/smartwebhook/2d7290e191ea9af3b24b6902b4705cee42c4748e/screenshot2.png -------------------------------------------------------------------------------- /screenshot3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/limboinf/smartwebhook/2d7290e191ea9af3b24b6902b4705cee42c4748e/screenshot3.png --------------------------------------------------------------------------------