├── .gitignore ├── LICENSE ├── README.md ├── README.rst ├── README_zh.md ├── gitagent ├── __init__.py ├── __main__.py ├── agent.py ├── auth.py └── client.py └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.swp 2 | *.swo 3 | *.pyc 4 | .DS_Store 5 | gitagent/config.json 6 | dist/* 7 | GitAgent.egg-info/* 8 | __pycache__ -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU LESSER GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | 9 | This version of the GNU Lesser General Public License incorporates 10 | the terms and conditions of version 3 of the GNU General Public 11 | License, supplemented by the additional permissions listed below. 12 | 13 | 0. Additional Definitions. 14 | 15 | As used herein, "this License" refers to version 3 of the GNU Lesser 16 | General Public License, and the "GNU GPL" refers to version 3 of the GNU 17 | General Public License. 18 | 19 | "The Library" refers to a covered work governed by this License, 20 | other than an Application or a Combined Work as defined below. 21 | 22 | An "Application" is any work that makes use of an interface provided 23 | by the Library, but which is not otherwise based on the Library. 24 | Defining a subclass of a class defined by the Library is deemed a mode 25 | of using an interface provided by the Library. 26 | 27 | A "Combined Work" is a work produced by combining or linking an 28 | Application with the Library. The particular version of the Library 29 | with which the Combined Work was made is also called the "Linked 30 | Version". 31 | 32 | The "Minimal Corresponding Source" for a Combined Work means the 33 | Corresponding Source for the Combined Work, excluding any source code 34 | for portions of the Combined Work that, considered in isolation, are 35 | based on the Application, and not on the Linked Version. 36 | 37 | The "Corresponding Application Code" for a Combined Work means the 38 | object code and/or source code for the Application, including any data 39 | and utility programs needed for reproducing the Combined Work from the 40 | Application, but excluding the System Libraries of the Combined Work. 41 | 42 | 1. Exception to Section 3 of the GNU GPL. 43 | 44 | You may convey a covered work under sections 3 and 4 of this License 45 | without being bound by section 3 of the GNU GPL. 46 | 47 | 2. Conveying Modified Versions. 48 | 49 | If you modify a copy of the Library, and, in your modifications, a 50 | facility refers to a function or data to be supplied by an Application 51 | that uses the facility (other than as an argument passed when the 52 | facility is invoked), then you may convey a copy of the modified 53 | version: 54 | 55 | a) under this License, provided that you make a good faith effort to 56 | ensure that, in the event an Application does not supply the 57 | function or data, the facility still operates, and performs 58 | whatever part of its purpose remains meaningful, or 59 | 60 | b) under the GNU GPL, with none of the additional permissions of 61 | this License applicable to that copy. 62 | 63 | 3. Object Code Incorporating Material from Library Header Files. 64 | 65 | The object code form of an Application may incorporate material from 66 | a header file that is part of the Library. You may convey such object 67 | code under terms of your choice, provided that, if the incorporated 68 | material is not limited to numerical parameters, data structure 69 | layouts and accessors, or small macros, inline functions and templates 70 | (ten or fewer lines in length), you do both of the following: 71 | 72 | a) Give prominent notice with each copy of the object code that the 73 | Library is used in it and that the Library and its use are 74 | covered by this License. 75 | 76 | b) Accompany the object code with a copy of the GNU GPL and this license 77 | document. 78 | 79 | 4. Combined Works. 80 | 81 | You may convey a Combined Work under terms of your choice that, 82 | taken together, effectively do not restrict modification of the 83 | portions of the Library contained in the Combined Work and reverse 84 | engineering for debugging such modifications, if you also do each of 85 | the following: 86 | 87 | a) Give prominent notice with each copy of the Combined Work that 88 | the Library is used in it and that the Library and its use are 89 | covered by this License. 90 | 91 | b) Accompany the Combined Work with a copy of the GNU GPL and this license 92 | document. 93 | 94 | c) For a Combined Work that displays copyright notices during 95 | execution, include the copyright notice for the Library among 96 | these notices, as well as a reference directing the user to the 97 | copies of the GNU GPL and this license document. 98 | 99 | d) Do one of the following: 100 | 101 | 0) Convey the Minimal Corresponding Source under the terms of this 102 | License, and the Corresponding Application Code in a form 103 | suitable for, and under terms that permit, the user to 104 | recombine or relink the Application with a modified version of 105 | the Linked Version to produce a modified Combined Work, in the 106 | manner specified by section 6 of the GNU GPL for conveying 107 | Corresponding Source. 108 | 109 | 1) Use a suitable shared library mechanism for linking with the 110 | Library. A suitable mechanism is one that (a) uses at run time 111 | a copy of the Library already present on the user's computer 112 | system, and (b) will operate properly with a modified version 113 | of the Library that is interface-compatible with the Linked 114 | Version. 115 | 116 | e) Provide Installation Information, but only if you would otherwise 117 | be required to provide such information under section 6 of the 118 | GNU GPL, and only to the extent that such information is 119 | necessary to install and execute a modified version of the 120 | Combined Work produced by recombining or relinking the 121 | Application with a modified version of the Linked Version. (If 122 | you use option 4d0, the Installation Information must accompany 123 | the Minimal Corresponding Source and Corresponding Application 124 | Code. If you use option 4d1, you must provide the Installation 125 | Information in the manner specified by section 6 of the GNU GPL 126 | for conveying Corresponding Source.) 127 | 128 | 5. Combined Libraries. 129 | 130 | You may place library facilities that are a work based on the 131 | Library side by side in a single library together with other library 132 | facilities that are not Applications and are not covered by this 133 | License, and convey such a combined library under terms of your 134 | choice, if you do both of the following: 135 | 136 | a) Accompany the combined library with a copy of the same work based 137 | on the Library, uncombined with any other library facilities, 138 | conveyed under the terms of this License. 139 | 140 | b) Give prominent notice with the combined library that part of it 141 | is a work based on the Library, and explaining where to find the 142 | accompanying uncombined form of the same work. 143 | 144 | 6. Revised Versions of the GNU Lesser General Public License. 145 | 146 | The Free Software Foundation may publish revised and/or new versions 147 | of the GNU Lesser General Public License from time to time. Such new 148 | versions will be similar in spirit to the present version, but may 149 | differ in detail to address new problems or concerns. 150 | 151 | Each version is given a distinguishing version number. If the 152 | Library as you received it specifies that a certain numbered version 153 | of the GNU Lesser General Public License "or any later version" 154 | applies to it, you have the option of following the terms and 155 | conditions either of that published version or of any later version 156 | published by the Free Software Foundation. If the Library as you 157 | received it does not specify a version number of the GNU Lesser 158 | General Public License, you may choose any version of the GNU Lesser 159 | General Public License ever published by the Free Software Foundation. 160 | 161 | If the Library as you received it specifies that a proxy can decide 162 | whether future versions of the GNU Lesser General Public License shall 163 | apply, that proxy's public statement of acceptance of any version is 164 | permanent authorization for you to choose that version for the 165 | Library. 166 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # GitAgent 2 | A web server receive HTTP request to pull local repository 3 | 4 | [中文文档](https://github.com/alexazhou/GitAgent/blob/master/README_zh.md) 5 | 6 | ## Desc 7 | 8 | GitAgent run as a webserver. It receive command from http requests and do operation to local git repositorys. 9 | 10 | So GitAgent let you can do git operation over http request. 11 | 12 | With GitAgent, you can a git repository on other machine to: 13 | 14 | * get current status 15 | * pull latest code 16 | * checkout branch 17 | ... 18 | 19 | GitAgent also support execute some commant after pull success, and use a password to protect http request. 20 | 21 | ## install 22 | 23 | python3 -m pip install gitagent 24 | 25 | ## require 26 | 27 | GitAgent based on python3, and those libs was required. 28 | 29 | * Tornado 30 | * GitPython 31 | * ws4py 32 | 33 | if you use pip install GitAgent, the requirements will be install automatic. 34 | 35 | ## config 36 | 37 | ##### Basic config format 38 | The basic format of config.json is like this. 39 | 40 | ``` 41 | example_config = { 42 | "bind_ip":"0.0.0.0", 43 | "port":10000, 44 | "repo":{ 45 | "self":{ 46 | "repo_path":"./", 47 | } 48 | }, 49 | } 50 | ``` 51 | If need, you can put more than one repon into it. 52 | 53 | 54 | ##### full config format 55 | 56 | if you need use password, or execute command after pull, you can add some args to config file like that. 57 | 58 | ``` 59 | example_config_full = { 60 | "bind_ip":"0.0.0.0", 61 | "port":10000, 62 | "repo":{ 63 | "self":{ 64 | "repo_path":"./", 65 | "command":{ 66 | "cmd1":"the command 1", 67 | "cmd2":"the command 2", 68 | } 69 | } 70 | }, 71 | "password":"123456" 72 | } 73 | 74 | ``` 75 | 76 | 77 | ## Usage 78 | 79 | #### step 1: Write default config file 80 | ```python3 -m gitagent [-c config.json] write``` 81 | 82 | The default config file will by written to config.json, then you can known the config format. 83 | 84 | if the -c arg don't gived, gitagent will write the config.json to current directory 85 | 86 | #### step 2: Edit the config.json 87 | 88 | Just edit the config file as you need 89 | 90 | #### step 3: Run gitagent 91 | ```python3 -m gitagent [-c config.json] run``` 92 | 93 | If you havn't see any error message, the gitagent is running. 94 | 95 | ## API 96 | 97 | #### list all repos 98 | 99 | ```curl -v 'http://localhost:10000/repo'``` 100 | 101 | Return: 102 | 103 | ``` 104 | [ 105 | "demo1", 106 | "demo2", 107 | "demo3" 108 | ] 109 | ``` 110 | 111 | 112 | #### repo status 113 | 114 | ```curl -v 'http://localhost:10000/repo/demo1'``` 115 | 116 | Return: 117 | 118 | ``` 119 | { 120 | { 121 | "author": "AlexaZhou", 122 | "busy": false, 123 | "changed_files": { 124 | "A": [], 125 | "D": [], 126 | "M": [ 127 | "agent.py" 128 | ], 129 | "R": [] 130 | }, 131 | "dirty": true, 132 | "hash": "c8c082d898c2dc18adb8e79f8992c074fb2294ce", 133 | "message": "some message text", 134 | "untracked_files": [ 135 | "config.json" 136 | ] 137 | } 138 | ``` 139 | 140 | busy means the repo is processing a pull request or other action 141 | 142 | #### repo pull / switch branch / switch hash 143 | 144 | ```curl -v -d 'git_branch=master&git_hash=abcdefg&command=cmd1&block=1' 'http://localhost:10000/repo/demo1/pull'``` 145 | 146 | Return: 147 | 148 | ``` 149 | { 150 | "ret": "success" 151 | } 152 | ``` 153 | 154 | args: 155 | 156 | * ****git_branch****: the branch you want to checkout. 157 | * ****git_hash****: is a optional arg. if git_hash not given, gitagent will checkout lastest commit on the target branch 158 | * ****command****: is a optional arg. if command was gived, it will be execute after pull. 159 | * ****block****: can be 0/1, if block = 1, the request will block until the git work finish 160 | 161 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | GitAgent 2 | ======== 3 | 4 | A web server receive HTTP request to pull local repository 5 | 6 | install 7 | ------- 8 | 9 | python3 -m pip install gitagent 10 | 11 | require 12 | ------- 13 | 14 | GitAgent based on python3, and those libs was required. 15 | 16 | - Tornado 17 | - GitPython 18 | 19 | if you use pip install GitAgent, the requirements will be install 20 | automatic. 21 | 22 | Desc 23 | ---- 24 | 25 | GitAgent run as a webserver. It receive command from http requests and 26 | do operation to local git repositorys. 27 | 28 | So GitAgent let you can do git operation over http request. 29 | 30 | With GitAgent, you can a git repository on other machine to: 31 | 32 | - get current status 33 | - pull latest code 34 | - checkout branch … 35 | 36 | GitAgent also support execute some commant after pull success, and use a 37 | password to protect http request. 38 | 39 | You can see detail documents over https://github.com/alexazhou/GitAgent -------------------------------------------------------------------------------- /README_zh.md: -------------------------------------------------------------------------------- 1 | # GitAgent 2 | 一个允许你通过 Http 请求来操作其他机器上 Git 仓库的服务 3 | 4 | ## 介绍 5 | 6 | GitAgent 作为一个web服务来运行. 接收来自 Http 请求的命令来对本地的 Git 仓库进行操作 7 | 8 | 有了 GitAgent，你可以对其他机器上的 Git 仓库做下面这些事情 9 | 10 | * 获取当前仓库的状态 11 | * pull 最新的代码 12 | * checkout 分支／版本 13 | ... 14 | 15 | GitAgent 还支持: 16 | 17 | * 在 pull 成功之后执行指定的命令（主要是为了方便完成部署的附加工作） 18 | * 也允许设置密码来保护接口的安全性 19 | * 通过 websocket 实时回传git pull 和命令执行过程中的日志输出 😎 20 | 21 | ## 安装 22 | 23 | GitAgent 已经封装成库，通过以下命令即可安装 24 | 25 | ``` 26 | python3 -m pip install gitagent 27 | ``` 28 | 29 | 30 | ## 依赖 31 | 32 | GitAgent 基于 Python3，下面这些库是需要的。 33 | 34 | * Tornado 35 | * GitPython 36 | * ws4py 37 | 38 | 如果你是通过 pip 安装的 GitAgent, 那么这些依赖会被自动装好. 39 | 40 | ## 配置 41 | 42 | ##### 最简配置割格式 43 | 44 | 最简化的配置文件格式如下 45 | ``` 46 | example_config = { 47 | "bind_ip":"0.0.0.0", 48 | "port":10000, 49 | "repo":{ 50 | "self":{ 51 | "repo_path":"./", 52 | } 53 | }, 54 | } 55 | ``` 56 | 如果需要可以在里面定义多个仓库 57 | 58 | 59 | ##### 完整配置格式 60 | 61 | 如果需要使用密码，或者在 pull 之后执行命令，那么像下面这样多定义一些字段。 62 | 63 | ``` 64 | example_config_full = { 65 | "bind_ip":"0.0.0.0", 66 | "port":10000, 67 | "repo":{ 68 | "self":{ 69 | "repo_path":"./", 70 | "command":{ 71 | "cmd1":"the command 1", 72 | "cmd2":"the command 2", 73 | } 74 | } 75 | }, 76 | "password":"123456" 77 | } 78 | 79 | ``` 80 | 81 | command 中可以定义多个命令，通过 http 请求来控制 pull 的时候，可以指定在 pull 完成之后，来执行这里面的命令 82 | 83 | 84 | ## Usage 85 | 86 | #### step 1: 生成配置文件 87 | ```python3 -m gitagent [-c config.json] write``` 88 | 89 | 执行这条命令之后，默认配置模版会被写入到指定的文件 90 | 91 | 如果没有给出 -c 参数, gitagent 会写入配置模板到当前目录的 config.json 92 | 93 | #### step 2: 编辑配置文件 94 | 95 | 按照自己的情况编辑配置模板 96 | 97 | #### step 3: 运行 gitagent 98 | ```python3 -m gitagent [-c config.json] run``` 99 | 100 | 如果没有报错，那么gitagent就已经在运行了（目前 gitagent 在前台运行，如果需要的话可以使用 supervisor 使其在后台运行 ） 101 | 102 | ## API 103 | 104 | #### 列出当前仓库 105 | 106 | ```curl -v 'http://localhost:10000/repo'``` 107 | 108 | Return: 109 | 110 | ``` 111 | [ 112 | "demo1", 113 | "demo2", 114 | "demo3" 115 | ] 116 | ``` 117 | 118 | 119 | ####仓库状态 120 | 121 | ```curl -v 'http://localhost:10000/repo/demo1'``` 122 | 123 | Return: 124 | 125 | ``` 126 | { 127 | { 128 | "author": "AlexaZhou", 129 | "busy": false, 130 | "changed_files": { 131 | "A": [], 132 | "D": [], 133 | "M": [ 134 | "agent.py" 135 | ], 136 | "R": [] 137 | }, 138 | "dirty": true, 139 | "hash": "c8c082d898c2dc18adb8e79f8992c074fb2294ce", 140 | "message": "some message text", 141 | "untracked_files": [ 142 | "config.json" 143 | ] 144 | } 145 | ``` 146 | 147 | * author: 当前所在 commit 的作者 148 | * hash: 当前所在 commit 的版本 149 | * message: 当前所在 commit 的 log 150 | * busy: 仓库当前是否正在执行其他的操作 151 | * dirty: 仓库的文件是否有变更 152 | * changed_files: 修改过的文件 153 | * untracked_files: 未跟踪的文件 154 | 155 | #### 对仓库进行 pull / 切换分支 / 切换版本 156 | 157 | ```curl -v -d 'git_branch=master&git_hash=abcdefg&command=cmd1&block=1' 'http://localhost:10000/repo/demo1/pull'``` 158 | 159 | Return: 160 | 161 | ``` 162 | { 163 | "ret": "success" 164 | } 165 | ``` 166 | 167 | 参数: 168 | 169 | * ****git_branch****: 需要 pull／checkout 的分支 170 | * ****git_hash****: 可选参数. 如果没有指定 git_hash, gitagent 将自动 checkout 目标分支上面的最新一个提交 171 | * ****command****: 可选参数. 如果指定了 command，那么这个 command 将会被在 pull 成功之后来执行 172 | * ****block****: 可以是 0/1, 如果 block = 1, 那么这个请求会阻塞到操作全部完成之后才返回 173 | 174 | 175 | #### 身份验证 176 | 177 | 通过在 config.json 中添加 password 项，可以对接口进行保护。 178 | 添加 password 项之后，调用 API 时，需要带上身份验证信息才可以正常使用，否则会被阻止。身份验证信息有以下两种方式： 179 | 180 | * 参数中添加一项 password，值和设置的 password 一致 （安全性弱） 181 | * 参数中添加两项 time 和 sign，其中 time 为当前的时间戳，sign 为使用 password 对请求进行签名的结果 182 | 183 | 第二种方式中签名计算方式如下: 184 | 185 | ``` 186 | sign = md5( method + uri + '?' + 参数字符串按字母排序后拼接 + password ) 187 | 188 | 当 password 为 123456 时，以这个请求为例： 189 | curl -v -d 'git_branch=master&git_hash=abcdefg&command=cmd1&block=1' 'http://localhost:10000/repo/demo1/pull 190 | 191 | 签名计算方式为： 192 | sign = md5( 'POST' + '/repo/demo1/pull' + '?' + 'block=1&command=cmd1&time=1469938982&git_branch=master&git_hash=abcdefg' + '123456' ) 193 | 194 | 计算得到 sign 为 '217ead22e7d680a3fe5a31b0e557b1c7' 195 | 196 | 那最后添加验证信息后的请求应该是 197 | curl -v -d 'git_branch=master&git_hash=abcdefg&command=cmd1&block=1&time=1469938982&sign=217ead22e7d680a3fe5a31b0e557b1c7' 'http://localhost:10000/repo/demo1/pull 198 | 199 | ``` 200 | 201 | 202 | 203 | 204 | ## Client 205 | 206 | GitAgent 还包含了一个 client 😈，基于 requests 库，封装了通过 http 请求操作 GitAgent 的相关代码。如果使用 python 的话，只需要通过 207 | 208 | ```from gitagent import client``` 209 | 210 | import 之后，就可以直接使用啦 211 | 212 | #### 创建 client 对象 213 | 214 | ``` 215 | agent_client = client.AgentClient( SERVER_ADDR, SERVER_PORT, password=None ) 216 | ``` 217 | 218 | #### 获取仓库列表 219 | 220 | ``` 221 | agent_client.repo_list() 222 | 223 | >> ['repo1','repo2','repo3'] 224 | ``` 225 | 226 | #### 获取仓库状态 227 | 228 | ``` 229 | agent_client.repo_status('repo1') 230 | {'untracked_files': ['a.txt', 'config.json', 'xxx.json'], 'busy': False, 'hash': '827b39799a543fee30a174d44cd0c5451776e413', 'dirty': True, 'changed_files': {'R': [], 'A': [], 'D': [], 'M': []}, 'author': 'AlexaZhou', 'branch': 'master', 'message': '\u66f4\u65b0\u6587\u6863\n'} 231 | ``` 232 | 233 | #### 对仓库进行操作 234 | ``` 235 | agent_client.pull('repo1', branch='master', hash='abcdefg', command='cmd1', block=1) 236 | >>{'ret': 'success', 'err_msg': None} 237 | ``` 238 | 注： 参数含义参考前面 API 部分的介绍 239 | 240 | 241 | 242 | -------------------------------------------------------------------------------- /gitagent/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Date : 2016-07-23 10:20 4 | # @Author : Alexa (AlexaZhou@163.com) 5 | # @Link : 6 | # @Disc : 7 | 8 | __version__ = '0.0.16' 9 | 10 | -------------------------------------------------------------------------------- /gitagent/__main__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Date : 2016-07-23 4 | # @Author : Alexa (AlexaZhou@163.com) 5 | # @Link : 6 | # @Disc : 7 | 8 | import sys 9 | import os 10 | import json 11 | import getopt 12 | import gitagent.agent as agent 13 | 14 | help_doc = '''usage: [options] cmd 15 | [-c filename] [--type=simple/full] write: write default config format into a file 16 | [-c filename] run: run with config 17 | ''' 18 | 19 | example_config = { 20 | "bind_ip":"0.0.0.0", 21 | "port":10000, 22 | "repo":{ 23 | "self":{ 24 | "repo_path":"./", 25 | } 26 | }, 27 | } 28 | 29 | example_config_full = { 30 | "bind_ip":"0.0.0.0", 31 | "port":10000, 32 | "repo":{ 33 | "self":{ 34 | "repo_path":"./", 35 | "command":{ 36 | "cmd1":"the command 1", 37 | "cmd2":"the command 2", 38 | } 39 | } 40 | }, 41 | "password":"123456" 42 | } 43 | 44 | def write_example_config( config_name, config_type='simple' ): 45 | print(' write_example_config ') 46 | 47 | if os.path.exists( config_name ): 48 | print('the file already exists') 49 | sys.exit(1) 50 | 51 | if config_type == 'simple': 52 | config_dict = example_config 53 | elif config_type == 'full': 54 | config_dict = example_config_full 55 | else: 56 | raise Exception("Unknown config type") 57 | 58 | with open( config_name,'w' ) as f: 59 | f.write( json.dumps( config_dict, sort_keys=True, indent=4, ensure_ascii=False ) ) 60 | print('write config to %s successed'%config_name) 61 | 62 | 63 | def load_config( config_name ): 64 | print(' load_config:',config_name) 65 | config = None 66 | with open( config_name, 'r' ) as f: 67 | config = json.load(f) 68 | 69 | return config 70 | 71 | def exit_with_message( message ): 72 | print( message ) 73 | print(help_doc) 74 | sys.exit(1) 75 | 76 | 77 | if __name__ == "__main__": 78 | opts, args = getopt.getopt(sys.argv[1:], 'c:', ['type=']) 79 | 80 | if len(args) != 1: 81 | exit_with_message('args error') 82 | 83 | cmd = args[0] 84 | if cmd not in ['write','run']: 85 | exit_with_message('args error') 86 | 87 | config_name = './config.json' 88 | config_type = 'simple' 89 | for option, value in opts: 90 | #print("option:%s --> value:%s"%(option, value)) 91 | if option == '-c': 92 | config_name = value 93 | elif option == '--type': 94 | config_type = value 95 | 96 | if cmd == 'write': 97 | write_example_config( config_name, config_type ) 98 | else: 99 | config = load_config( config_name ) 100 | agent.set_config( config ) 101 | agent.start_server( ) 102 | 103 | sys.exit(0) 104 | 105 | -------------------------------------------------------------------------------- /gitagent/agent.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Date : 2016-07-14 14:35 4 | # @Author : Alexa (AlexaZhou@163.com) 5 | # @Link : 6 | # @Disc : 7 | 8 | import os 9 | import fcntl 10 | import subprocess 11 | import tornado.ioloop 12 | import tornado.web 13 | import tornado.websocket 14 | import time 15 | import json 16 | import git 17 | import threading 18 | import logging 19 | from gitagent import auth 20 | 21 | 22 | settings = { 23 | 'debug' : True, 24 | "static_path": os.path.join(os.path.dirname(__file__), "static") 25 | } 26 | 27 | repo_lock = {} 28 | client_sockets = {} 29 | 30 | pretty_json_dump = lambda x:json.dumps( x,sort_keys=True,indent=4,ensure_ascii=False ) 31 | 32 | config = None 33 | 34 | def get_config( ): 35 | return config 36 | 37 | def set_config( value ): 38 | global config 39 | config = value 40 | 41 | class git_work_progress( git.RemoteProgress ): 42 | def __init__(self,delegate): 43 | git.RemoteProgress.__init__(self) 44 | self.delegate = delegate 45 | 46 | def update(self,op_code,cur_count,max_count=None,message=""): 47 | print( '-->',op_code,cur_count,max_count,message ) 48 | self.delegate.console_output( '\r' + "working with op_code[%s] progress[%s/%s] %s"%(op_code,cur_count,max_count,message) ) 49 | 50 | class GitWorker(): 51 | def __init__(self, repo_path, git_branch, git_hash, command=None, console_id=None, GIT_SSH_COMMAND=None): 52 | self.repo_path = repo_path 53 | self.git_branch = git_branch 54 | self.git_hash = git_hash 55 | self.console_id = console_id 56 | self.GIT_SSH_COMMAND = GIT_SSH_COMMAND 57 | self.finish_ret = None 58 | self.output = '' 59 | self.err_msg = None 60 | self.command = command 61 | 62 | def console_output(self,s): 63 | print('console [%s]>>'%self.console_id,s ) 64 | if self.console_id != None: 65 | 66 | try: 67 | ws_cocket = client_sockets[ self.console_id ] 68 | msg = {} 69 | msg['type'] = 'output' 70 | msg['content'] = s 71 | ws_cocket.write_message( msg ) 72 | 73 | except Exception as e: 74 | print('write to websocket failed:',e) 75 | 76 | def non_block_read(self, output): 77 | fd = output.fileno() 78 | fl = fcntl.fcntl(fd, fcntl.F_GETFL) 79 | fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK) 80 | try: 81 | ret = output.read().decode('utf-8') 82 | if ret == None: 83 | ret = "" 84 | return ret 85 | except: 86 | return "" 87 | 88 | def worker(self): 89 | print( "-"*20 + "git checkout " + "-"*20 ) 90 | print( "branch:" + self.git_branch ) 91 | print( "hash:" + str(self.git_hash)) 92 | 93 | progress_delegate = git_work_progress( self ) 94 | 95 | try: 96 | repo=git.Repo( self.repo_path ) 97 | if self.GIT_SSH_COMMAND != None: 98 | repo.git.update_environment( GIT_SSH_COMMAND=self.GIT_SSH_COMMAND ) 99 | 100 | branch_now = None 101 | if repo.head.is_detached == False: 102 | branch_now = repo.active_branch.name 103 | 104 | print( 'Now repo is on branch:',branch_now ) 105 | if self.git_branch in repo.branches: 106 | #make sure on right branch 107 | 108 | if branch_now != self.git_branch: 109 | self.console_output( 'checkout branch %s...'%self.git_branch ) 110 | repo.branches[self.git_branch].checkout() 111 | #pull 112 | self.console_output( 'pull...' ) 113 | repo.remotes['origin'].pull( progress= progress_delegate ) 114 | else: 115 | #if the target branch is not existed in local, checkout out it at first 116 | self.console_output( 'branch %s not existed local. update remote branches...'%self.git_branch ) 117 | origin = repo.remotes['origin'] 118 | origin.update( ) 119 | self.console_output( 'checkout branch %s...'%self.git_branch ) 120 | origin.refs[self.git_branch].checkout( b=self.git_branch ) 121 | 122 | if self.git_hash != None: 123 | #TODO:判断本地是否存在 hash 对应的commit，如果存在则跳过 pull 的动作 124 | self.console_output( 'git checkout %s...'%self.git_hash ) 125 | git_exec = repo.git 126 | git_exec.checkout( self.git_hash ) 127 | 128 | if self.command != None: 129 | self.console_output('Exec command: %s'%self.command) 130 | p_command = subprocess.Popen(self.command, shell=True, bufsize=1024000, cwd=self.repo_path, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) 131 | p_returncode = None 132 | 133 | while True: 134 | p_output = self.non_block_read(p_command.stdout) 135 | if len(p_output) != 0: 136 | self.console_output( p_output ) 137 | 138 | p_returncode = p_command.poll() 139 | if p_returncode != None: 140 | break 141 | 142 | time.sleep(0.01) 143 | 144 | if p_returncode != 0: 145 | raise Exception("exce command [%s] return code !=0"%self.command) 146 | 147 | self.finish_ret = 'success' 148 | except Exception as e: 149 | print('Exception:',e) 150 | self.err_msg = str(e) 151 | self.finish_ret = 'failed' 152 | 153 | print( "-"*20 + "git checkout finish:" + self.finish_ret + "-"*20 ) 154 | 155 | def start(self): 156 | t = threading.Thread( target = self.worker ) 157 | t.start() 158 | 159 | 160 | def return_json(fn): 161 | def wrapper( self, *args, **kwargs ): 162 | self.set_header("Content-Type", "application/json; charset=UTF-8") 163 | return fn( self, *args, **kwargs ) 164 | 165 | return wrapper 166 | 167 | def verify_request( request ): 168 | 169 | config = get_config() 170 | if 'password' in config: 171 | password = config['password'] 172 | 173 | method = request.method 174 | uri = request.path 175 | request_args = {} 176 | for name in request.arguments: 177 | request_args[name] = request.arguments[name][0].decode('utf-8') 178 | 179 | #use password directly auth 180 | if request_args.get('password') == password: 181 | return True 182 | 183 | #use password sign auth 184 | request_time = int(request_args['time']) 185 | if abs(request_time - time.time()) > 60: 186 | print( 'time diff too much, auth failed' ) 187 | return False 188 | 189 | sign = request_args['sign'] 190 | del request_args['sign'] 191 | 192 | sign_right = auth.sign( method, uri, request_args, password, time_stamp = request_args['time'] )['sign'] 193 | print( 'sign_right:',sign_right ) 194 | 195 | if sign_right != sign: 196 | print( 'verify request failed due to sign not match' ) 197 | return False 198 | 199 | return True 200 | 201 | def auth_verify(fn): 202 | def wrapper( self, *args, **kwargs ): 203 | if verify_request(self.request) == False: 204 | raise tornado.web.HTTPError(501) 205 | return fn( self, *args, **kwargs ) 206 | 207 | return wrapper 208 | 209 | class MainHandler(tornado.web.RequestHandler): 210 | def get(self): 211 | self.write("Hello, GitAgent~") 212 | 213 | class RepoHandler(tornado.web.RequestHandler): 214 | @return_json 215 | @auth_verify 216 | def get(self): 217 | config = get_config() 218 | ret = list(config['repo'].keys()) 219 | self.write( pretty_json_dump(ret) ) 220 | 221 | class StatusHandler(tornado.web.RequestHandler): 222 | @return_json 223 | @auth_verify 224 | def get(self,repo): 225 | config = get_config() 226 | if repo not in config['repo']: 227 | raise tornado.web.HTTPError(404) 228 | 229 | repo_path = config['repo'][ repo ]['repo_path'] 230 | repo = git.Repo( repo_path ) 231 | commit = repo.commit("HEAD") 232 | 233 | info = {} 234 | 235 | if repo.head.is_detached == True: 236 | info['branch'] = None 237 | else: 238 | info['branch'] = repo.active_branch.name 239 | 240 | info['hash'] = commit.hexsha 241 | info['author'] = str(commit.author) 242 | info['message'] = commit.message 243 | info['busy'] = repo in repo_lock 244 | info['dirty'] = repo.is_dirty() 245 | info['untracked_files'] = repo.untracked_files 246 | info['changed_files'] = {} 247 | 248 | change = info['changed_files'] 249 | diff = repo.index.diff(None) 250 | 251 | name_getter = lambda diff:diff.a_path 252 | for change_type in "ADRM": 253 | print( 'change_type:',change_type ) 254 | change[change_type] = list(map( name_getter, diff.iter_change_type( change_type ))) 255 | 256 | self.write( pretty_json_dump(info) ) 257 | 258 | class PullHandle(tornado.web.RequestHandler): 259 | @tornado.web.asynchronous 260 | @tornado.gen.coroutine 261 | def post(self,repo): 262 | self.set_header("Content-Type", "application/json; charset=UTF-8") 263 | config = get_config() 264 | 265 | if verify_request(self.request) == False: 266 | raise tornado.web.HTTPError(501) 267 | 268 | if repo not in config['repo']: 269 | raise tornado.web.HTTPError(404) 270 | 271 | block = self.get_argument( 'block', '0') 272 | git_branch = self.get_argument( 'git_branch', 'master') 273 | git_hash = self.get_argument( 'git_hash', None) 274 | console_id = self.get_argument( 'console_id', None) 275 | cmd = self.get_argument( 'command', None) 276 | GIT_SSH_COMMAND = None 277 | 278 | ret = {} 279 | ret['ret'] = 'success' 280 | ret['err_msg'] = None 281 | command = None 282 | 283 | if cmd != None: 284 | if cmd not in config['repo'][repo]['command']: 285 | raise tornado.web.HTTPError(501) 286 | else: 287 | command = config['repo'][repo]['command'][cmd] 288 | 289 | if repo in repo_lock: 290 | ret['ret'] = 'failure' 291 | ret['err_msg'] = 'repo is busying' 292 | self.write( pretty_json_dump(ret)) 293 | self.finish() 294 | return 295 | else: 296 | repo_lock[repo] = True 297 | 298 | repo_path = config['repo'][ repo ]['repo_path'] 299 | if 'GIT_SSH_COMMAND' in config['repo'][repo]: 300 | GIT_SSH_COMMAND = config['repo'][repo]['GIT_SSH_COMMAND'] 301 | print('use GIT_SSH_COMMAND:',GIT_SSH_COMMAND) 302 | 303 | git_worker = GitWorker( repo_path, git_branch, git_hash, command, console_id, GIT_SSH_COMMAND) 304 | git_worker.start() 305 | 306 | if block == '0':#no block 307 | ret['ret'] = 'success' 308 | else:#block until git worker finish 309 | while git_worker.finish_ret == None: 310 | yield tornado.gen.sleep(0.01) 311 | 312 | ret['ret'] = git_worker.finish_ret 313 | ret['err_msg'] = git_worker.err_msg 314 | 315 | self.write( pretty_json_dump(ret)) 316 | self.finish() 317 | del repo_lock[repo] 318 | 319 | class CommandHandle(tornado.web.RequestHandler): 320 | def post(self,repo): 321 | pass 322 | 323 | class ConsoleHandler(tornado.websocket.WebSocketHandler): 324 | """docstring for ConsoleHandler""" 325 | 326 | def check_origin(self, origin): 327 | return True 328 | 329 | def open(self): 330 | print("websocket open") 331 | self.write_message(json.dumps({ 332 | 'type': 'sys', 333 | 'message': 'Welcome to WebSocket', 334 | 'id': str(id(self)), 335 | })) 336 | client_sockets[ str(id(self)) ] = self 337 | 338 | def on_close(self): 339 | print("websocket close") 340 | del client_sockets[ str(id(self)) ] 341 | 342 | 343 | application = tornado.web.Application([ 344 | ("/repo/([^/]+)/exec", CommandHandle), 345 | ("/repo/([^/]+)/pull", PullHandle), 346 | ("/repo/([^/]+)", StatusHandler), 347 | ("/repo", RepoHandler ), 348 | ("/console", ConsoleHandler ), 349 | ("/", MainHandler), 350 | ],**settings) 351 | 352 | def start_server(): 353 | 354 | logging.basicConfig() 355 | logging.root.setLevel(logging.INFO) 356 | 357 | config = get_config() 358 | application.listen( config['port'], address=config['bind_ip'] ) 359 | tornado.ioloop.IOLoop.instance().start() 360 | -------------------------------------------------------------------------------- /gitagent/auth.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Date : 2016-07-24 4 | # @Author : Alexa (AlexaZhou@163.com) 5 | # @Link : 6 | # @Disc : 7 | 8 | import time 9 | import hashlib 10 | 11 | def sign( method, uri, args, sign_key, time_stamp = None ): 12 | if time_stamp == None: 13 | time_stamp = str(int(time.time())) 14 | 15 | args['time'] = time_stamp 16 | args_keys = list(args.keys()) 17 | args_keys.sort() 18 | 19 | args_str = '' 20 | for key in args_keys: 21 | if len(args_str) != 0: 22 | args_str += '&' 23 | 24 | args_str += key + '=' + str(args[key]) 25 | 26 | str_to_sign = method + uri + '?' + args_str + sign_key 27 | #print('str_to_sign:',str_to_sign) 28 | 29 | m = hashlib.md5() 30 | m.update(str_to_sign.encode('utf-8')) 31 | 32 | #print('md5:%s'%m.hexdigest()) 33 | args['sign'] = m.hexdigest() 34 | return args 35 | -------------------------------------------------------------------------------- /gitagent/client.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Date : 2016-07-24 4 | # @Author : Alexa (AlexaZhou@163.com) 5 | # @Link : 6 | # @Disc : 7 | 8 | import requests 9 | import threading 10 | import time 11 | import json 12 | from gitagent import auth 13 | from ws4py.client.threadedclient import WebSocketClient 14 | 15 | 16 | class WebSocketConsole(WebSocketClient): 17 | def __init__(self, ip, port, console_receiver): 18 | self.websocket_id = None 19 | self.console_receiver = console_receiver 20 | url = "ws://%s:%s/console"%(ip, port) 21 | WebSocketClient.__init__(self, url, protocols=['http-only', 'chat']) 22 | 23 | def opened(self): 24 | self.console_output('websocket console connected') 25 | 26 | def closed(self, code, reason=None): 27 | self.console_output("websocket console disconnected") 28 | 29 | def received_message(self, m): 30 | #print('console ->',m) 31 | data = json.loads( m.data.decode('utf-8') ) 32 | if data['type'] == 'sys': 33 | self.websocket_id = data['id'] 34 | elif data['type'] == 'output': 35 | self.console_output( data['content'] ) 36 | 37 | def console_output(self,s): 38 | self.console_receiver( s ) 39 | 40 | 41 | class AgentClient(): 42 | def __init__(self, ip, port, password=None, console_receiver=print): 43 | self.web_console = None 44 | self.ip = ip 45 | self.port = str(port) 46 | self.base_url = ip + ':' + self.port 47 | self.password = password 48 | self.console_receiver = console_receiver 49 | 50 | def request_sign(self, method, uri, args): 51 | 52 | if self.password != None: 53 | args = auth.sign( method, uri, args, self.password ) 54 | 55 | return args 56 | 57 | def repo_list(self): 58 | uri = '/repo' 59 | args = {} 60 | args = self.request_sign( 'GET', uri, args ) 61 | 62 | r = requests.get( 'http://' + self.base_url + uri, data=args, timeout=10 ) 63 | if r.status_code != 200: 64 | raise Exception('Request failed with status_code:%s response:%s'%(r.status_code, r.content)) 65 | return r.json() 66 | 67 | def repo_status(self, repo): 68 | uri = '/repo/' + repo 69 | args = {} 70 | args = self.request_sign( 'GET', uri, args ) 71 | r = requests.get( 'http://' + self.base_url + uri , data=args, timeout=10 ) 72 | if r.status_code != 200: 73 | raise Exception('Request failed with status_code:%s response:%s'%(r.status_code, r.content)) 74 | return r.json() 75 | 76 | def pull(self, repo, git_branch='master', git_hash=None, command=None, block = 1): 77 | args ={ 'git_branch':git_branch, 'block':block } 78 | if git_hash != None: 79 | args['git_hash'] = git_hash 80 | 81 | if self.web_console != None: 82 | args['console_id'] = self.web_console.websocket_id 83 | 84 | if command != None: 85 | args['command'] = command 86 | 87 | uri = '/repo/' + repo + '/pull' 88 | args = self.request_sign( 'POST', uri, args ) 89 | 90 | #print( 'connect GitAgent to deploy...' ) 91 | r = requests.post( 'http://' + self.base_url + uri , data=args, timeout=600 ) 92 | if r.status_code != 200: 93 | raise Exception('Request failed with status_code:%s response:%s'%(r.status_code, r.content)) 94 | r_json = r.json() 95 | return r_json 96 | 97 | def connect_websocket(self): 98 | self.web_console = WebSocketConsole( self.ip, self.port, self.console_receiver ) 99 | def console_worker(): 100 | self.web_console.connect() 101 | self.web_console.run_forever() 102 | 103 | t = threading.Thread( target=console_worker ) 104 | t.setDaemon( True ) 105 | t.start() 106 | 107 | time_begin = time.time() 108 | while time.time() - time_begin < 5: 109 | if self.web_console.websocket_id != None: 110 | break 111 | time.sleep(0.1) 112 | else: 113 | raise Exception('WebSocket Connect Error') 114 | 115 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Date : 2016-07-23 10:20 4 | # @Author : Alexa (AlexaZhou@163.com) 5 | # @Link : 6 | # @Disc : 7 | 8 | """A setuptools based setup module. 9 | See: 10 | https://packaging.python.org/en/latest/distributing.html 11 | https://github.com/pypa/sampleproject 12 | """ 13 | 14 | # Always prefer setuptools over distutils 15 | from setuptools import setup, find_packages 16 | # To use a consistent encoding 17 | from codecs import open 18 | from os import path 19 | 20 | import gitagent 21 | 22 | here = path.abspath(path.dirname(__file__)) 23 | 24 | # Get the long description from the README file 25 | with open(path.join(here, 'README.rst'), encoding='utf-8') as f: 26 | long_description = f.read() 27 | 28 | setup( 29 | name='GitAgent', 30 | 31 | # Versions should comply with PEP440. For a discussion on single-sourcing 32 | # the version across setup.py and the project code, see 33 | # https://packaging.python.org/en/latest/single_source_version.html 34 | version=gitagent.__version__, 35 | 36 | description='A web server receive HTTP request to pull local repository', 37 | long_description=long_description, 38 | 39 | # The project's main homepage. 40 | url='https://github.com/alexazhou/GitAgent', 41 | 42 | # Author details 43 | author='AlexaZhou', 44 | author_email='AlexaZhou@163.com', 45 | 46 | # Choose your license 47 | license='LGPL', 48 | 49 | # See https://pypi.python.org/pypi?%3Aaction=list_classifiers 50 | classifiers=[ 51 | # How mature is this project? Common values are 52 | # 3 - Alpha 53 | # 4 - Beta 54 | # 5 - Production/Stable 55 | 'Development Status :: 3 - Alpha', 56 | 57 | # Indicate who your project is intended for 58 | 'Intended Audience :: Developers', 59 | 'Topic :: Software Development :: Build Tools', 60 | 61 | # Pick your license as you wish (should match "license" above) 62 | 'License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)', 63 | 64 | # Specify the Python versions you support here. In particular, ensure 65 | # that you indicate whether you support Python 2, Python 3 or both. 66 | 'Programming Language :: Python :: 3', 67 | 'Programming Language :: Python :: 3.3', 68 | 'Programming Language :: Python :: 3.4', 69 | 'Programming Language :: Python :: 3.5', 70 | ], 71 | 72 | # What does your project relate to? 73 | keywords='git pull hook', 74 | 75 | # You can just specify the packages manually here if your project is 76 | # simple. Or you can use find_packages(). 77 | packages=find_packages(exclude=['contrib', 'docs', 'tests']), 78 | 79 | # Alternatively, if you want to distribute just a my_module.py, uncomment 80 | # this: 81 | # py_modules=["my_module"], 82 | 83 | # List run-time dependencies here. These will be installed by pip when 84 | # your project is installed. For an analysis of "install_requires" vs pip's 85 | # requirements files see: 86 | # https://packaging.python.org/en/latest/requirements.html 87 | install_requires=['tornado','GitPython','requests','ws4py'], 88 | 89 | # List additional groups of dependencies here (e.g. development 90 | # dependencies). You can install these using the following syntax, 91 | # for example: 92 | # $ pip install -e .[dev,test] 93 | extras_require={ 94 | 'dev': [], 95 | 'test': [], 96 | }, 97 | 98 | # If there are data files included in your packages that need to be 99 | # installed, specify them here. If using Python 2.6 or less, then these 100 | # have to be included in MANIFEST.in as well. 101 | package_data={ 102 | 'sample': [], 103 | }, 104 | 105 | # Although 'package_data' is the preferred approach, in some case you may 106 | # need to place data files outside of your packages. See: 107 | # http://docs.python.org/3.4/distutils/setupscript.html#installing-additional-files # noqa 108 | # In this case, 'data_file' will be installed into '/my_data' 109 | data_files=[ ], 110 | 111 | # To provide executable scripts, use entry points in preference to the 112 | # "scripts" keyword. Entry points provide cross-platform support and allow 113 | # pip to create the appropriate form of executable for the target platform. 114 | entry_points={ 115 | }, 116 | ) 117 | --------------------------------------------------------------------------------