├── .gitignore ├── README.md ├── __init__.py ├── httpserver.py ├── log.py ├── manager.py ├── socketserver.py ├── statichandler.py ├── template.py └── web.py /.gitignore: -------------------------------------------------------------------------------- 1 | .idea/* 2 | *.pyc 3 | test/* 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # python2实现的web框架 2 | 3 | ## 简介 4 | 5 | httppy是作者在经历过一段时间Django开发后， 6 | 有感于Django的强大与实用以及一些令人诟病之处后出于学习目的开发的一套web框架。 7 | Django框架在实际的web开发中是非常快捷、实用的。 8 | 对于大中小型的项目Django都能吃得开。功能完善，开发快捷。 9 | 但是Django框架内部的耦合性非常强，这导致了在使用中对于具体项目的深度定制非常困难。 10 | 比如Django的模板系统以及ORM在不少地方引人诟病，可如果想要享受Django带来的便利就必须忍受这些瑕疵。 11 | 在此情况下作者参考了Django的许多设计概念，并在一些引人诟病处实现了自己的想法，从而开发了httppy框架。 12 | 13 | 14 | **httppy更多的只是一个用于学习的作品，而不是一个可以用作生产环境的工具。通过简单的代码去实现对HTTP协议的解析、封装，在这个过程中了解一个服务器的工作流程究竟是怎样的。** 15 | 16 | 17 | 框架命名为httppy原因有二 18 | 19 | 1. 最初是想简单的命名为web.py但是由于web.py这个标志性的名字已经被Aaron Swartz大神占用了只能放弃 20 | 2. httppy这套框架不仅是应用层的框架，而是从socket层开始一步步向上在自己实现的http服务器基础上开发而来 21 | 22 | httppy框架的特色 23 | 24 | 1. 吸收了大量django的设计模式，在开发中可以参考django的开发范式快速开发。 25 | 2. 整套框架代码量非常少，结构清晰，实现的功能较齐全。易于阅读、学习、定制。 26 | 3. 使用多进程、线程池技术，性能强劲，足以支撑绝大多数小型个人站点的点击量。 27 | 28 | httppy的不足 29 | 30 | 目前为止httppy完全由作者一人编写、维护。httppy没有经过什么项目的实战。因此httppy框架显得过于单薄。 31 | 相对于成熟的框架存在非常多没有考虑到的场景，有待于日后的优化、升级。 32 | 欢迎各位有兴趣的朋友来共同学习、维护httppy框架。 33 | 34 | 35 | ## 快速开始 36 | 37 | 让我们从一个hello world工程来快速领略httppy的使用吧 38 | 39 | 创建hello.py文件 40 | 41 | import httppy 42 | from httppy import web 43 | 44 | 45 | class Index(web.RequestHandler): 46 | def handle(self): 47 | self.response.set_body('Hello httppy!') 48 | 49 | url = [ 50 | (r'^/index/$', Index) 51 | ] 52 | 53 | server_address = [ 54 | ('', 7777), 55 | ] 56 | 57 | manager = httppy.Manager(server_address, url) 58 | manager.server_start() 59 | 60 | 61 | 执行使用python2执行hello.py 62 | 63 | $python hello.py 64 | 2015-09-21 20:19:04 socketserver.py[line:58] INFO Server start 65 | 2015-09-21 20:19:04 socketserver.py[line:61] INFO bind:('0.0.0.0', 7777) 66 | 67 | 接下来在浏览器中打开 http://127.0.0.1:7777/index/ 就可以看到 Hello httppy! 的字样了 68 | 69 | 在demo目录中有一个示例工程展示了框架的使用方式 70 | 71 | 72 | ## 框架工作模式 73 | 74 | 75 | |----------------------------------------------------------------------------------------------- 76 | | +-----------+ 77 | | | Server() | 78 | | +-----------+ 79 | | | 80 | | v 81 | | +-----------+ 82 | | | bind() | 83 | | +-----------+ 84 | | | 85 | | v 86 | | +-----------+ 87 | | | listen() | 88 | | +-----------+ 89 | | | 90 | | v 91 | | +------------+ connect +-----------+ 92 | | v---------------------------| accept() |<----------------| Client() | 93 | | | +------------+ +-----------+ 94 | | |从线程池中找到一个 　继续等待下一个请求 95 | | |空闲Handler处理connect | 96 | | |connect | 97 | | |客户端套接字连入 | 98 | | v---------------------------------- 99 | | +-----------------+ 100 | | | SocketHandler() | 101 | | +-----------------+ 102 | | |recv -> data 103 | | v在TCP套接字层面接受数据 104 | | +-----------------+ 105 | | | HttpHandler() | 106 | | +-----------------+ 107 | | |parse_http -> Request 108 | | v在http协议层解析生成Request对象 109 | |+------------------------+ 110 | || Web.RequestHandler() |----v将request对象交给UrlRoute对象处理 111 | |+------------------------+ | 112 | | | | +--------------+ 113 | | | >--->| UrlRoute() |-v根据request url调用用户指定的RequestHandler 114 | | | +--------------+ | 115 | | | | +--------------------+ 116 | | | >--| RequestHandler() | 117 | | | +--------------------+ 118 | | | |handler -> Response 119 | | | |用户对请求进行处理 120 | | v-----------------------------------------------------<并生成Response对象作为响应 121 | | | 122 | | | 123 | | | 124 | | v 125 | | +----------------+ 126 | | | HttpHandler() | 127 | | +----------------+ 128 | | |get_response -> result 129 | | v将Response对象转化为套接字数据 130 | | +-----------------+ 131 | | | SocketHandler() | 132 | | +-----------------+ 133 | | |sendall 134 | | |将数据发送给客户端 135 | | v 136 | |----------------------------------------------------------------------------------------------- 137 | 138 | 139 | 1. Server层 140 | > 服务器绑定监听一个端口等待客户端连入 141 | > 当客户端连入后将连接转交给线程池中的空闲Handler处理 142 | 2. SocketHandler层 143 | > 在套接字层recv客户端发来的数据 144 | > `self.recv() -> self.data` 145 | 3. HttpHandler层 146 | > 根据http协议解析套接字接收到的数据 147 | > head body method url get post file ... 148 | > `self.parse_http() -> self.request` 149 | 4. Web.RequestHandler层 150 | > 将self.request对象交给UrlRoute处理 151 | > `self.url_route.route(self.http_request)` 152 | 5. UrlRoute 153 | > 根据传入的request url在用户预先写好的url匹配表中匹配 154 | > 实例化相应的RequestHandler对象 将request传给他 155 | 6. RequestHandler 156 | > 用户继承对象并重载handle方法进行处理 157 | > 产生相应的Response对象 158 | 7. HttpHandler层 159 | > 将Response解析为完整可发送的对象 160 | > `self.get_http_response(http_response)` 161 | 8. SocketHandler层 162 | > sendall发送数据到客户端 163 | > 关闭连接 164 | 165 | 166 | 167 | ## URL配置 168 | 169 | url配置方法完全基于python自带的re正则模块 170 | 171 | url = [ 172 | (r'^/url1/$', handler1), 173 | (r'^/url2/$', handler2), 174 | ... 175 | ] 176 | 177 | 每条匹配规则写在一个元组中 178 | 所有的url匹配规则写在一个list中 179 | 180 | 支持re模块的分组功能 181 | 182 | (r'^/(?P[^/]+)/$', handler3) 183 | 184 | 该正则可以将 /xxx/ 形式的url中的xxx解析提取出 185 | 并可以在RequestHandler中通过`self.request.url_param['param1']`访问 186 | 187 | httppy中不论传入的url是否使用 / 结尾 都会强制规范以 / 结尾 188 | 如: 189 | > / 190 | /url1/ 191 | /url2/url3/ 192 | 193 | 绝对不会出现 194 | > /url4 195 | 196 | 这种不以 / 结尾的url 197 | 因此在编写url时要特别注意`正则`的表达 198 | 199 | 200 | ## 静态文件 201 | 202 | httppy自己实现了静态文件的处理 203 | 可以通过配置url达到对静态文件的响应 204 | 205 | from httppy.statichandler import static_handler 206 | url = [ 207 | ..., 208 | (r'/static/(?P.*)$', static_handler(os.path.join(os.getcwd(), 'static'))), 209 | ..., 210 | ] 211 | 212 | 以该模式配置url即可通过 /static/xxx 访问static目录下的文件 213 | 214 | os.path.join(os.getcwd(), 'static')) 215 | 216 | 此处是为了获取指定目录的绝对路径 217 | 在实际使用中请获取绝对路径填入 218 | 避免无谓的混淆 219 | 如果有多个目录方式静态文件 220 | 也可以编写多条对应到不同路径的静态文件url 221 | 222 | httppy实现的静态文件处理功能比较粗糙，仅适合`调试或者流量较小站点使用` 223 | 更好的方案是使用`nginx`处理静态文件 224 | httppy仅处理动态的请求 225 | 226 | 227 | ## 响应的生成 228 | 229 | 用户重载web.RequestHandler来对一个request进行处理、响应 230 | 重载handle方法实际进行操作 231 | 该对象具备两个重要的属性 232 | 233 | self.request 234 | self.response 235 | 236 | self.request可以提取解析后的请求 请查看源代码了解该对象所具备的属性 237 | self.response是返回给客户端的响应 238 | Response对象具备以下常用方法 239 | 240 | set_header 241 | set_cookie 242 | set_status 243 | set_body 244 | redirect 245 | 246 | 有一些简单的http协议的了解对这些方法应该很容易理解 247 | 如set_body设置了html页面内容 248 | 使用范例 249 | 250 | self.response.set_body('Hello httppy!') 251 | self.response.set_header('Content-Type', 'text/html') 252 | self.response.set_cookie('key', 'value') 253 | self.response.set_status(404) 254 | self.response.redirect('/index/') 255 | 256 | 特别注意set_header的使用 257 | 请一定注意使用驼峰风格来编写 Content-Type 而不是 content-type 258 | 目前在httppy中是大小写敏感的的 259 | 通常无需设置Content-Type默认为text/html 260 | 261 | 262 | ## 模板系统 263 | 264 | 模板系统采用jinja2 265 | 如果需要使用模板系统请自行安装jinja2 266 | httppy.template.py中对jinja2进行了一些简单的包装 267 | 268 | 首先需要设置jinja2的工作目录 269 | 270 | from httppy import template 271 | template.render = template.get_template_render('template') 272 | 273 | 该语句设置工程下的template目录为默认工作路径 274 | 建议你在任何时刻填写绝对路径以避免混淆 275 | 当然，只要你足够了解python的规则，填写相对路径会更加简洁 276 | 277 | 现在来使用模板系统 278 | 279 | from httppy.template import render 280 | class Template(web.RequestHandler): 281 | def handle(self): 282 | # template中的render对象用于进行模板渲染 283 | # 模板系统使用jinja2 284 | # template中仅仅是对jinja2的包装 285 | import datetime 286 | self.response.set_body(render.render('template.html', {'hello': 'world'})) 287 | 288 | 更多的内容请移步[jinja2文档](http://www.pythonfan.org/docs/python/jinja2/zh/templates.html) 289 | 290 | 291 | ## ORM 292 | 293 | httppy框架没有对ORM进行整合 294 | 如有必要请自行整合ORM 295 | 296 | 297 | ## 服务器配置 298 | 299 | 300 | import httppy 301 | from httppy import web 302 | 303 | 304 | class Index(web.RequestHandler): 305 | def handle(self): 306 | self.response.set_body('Hello httppy!') 307 | 308 | url = [ 309 | (r'^/index/$', Index) 310 | ] 311 | 312 | server_address = [ 313 | ('', 7777), 314 | ] 315 | 316 | conf = { 317 | # 设定套接字超时时间 318 | 'connect_timeout': 5, 319 | # 设定套接字监听队列长度 320 | 'request_queue_size': 5, 321 | # 设定每条进程的线程池大小 322 | 'thread_pool_size': 10, 323 | } 324 | 325 | manager = httppy.Manager(server_address, url, **conf) 326 | manager.server_start() 327 | 328 | 将服务器要绑定的(主机名、端口)以及配置好的url列表交给Manager类 329 | Manager负责完成框架的启动 330 | httppy框架的每个服务器在一个进程中运行，并维护一个进程池 331 | 由于python GIL的限制，只能使用一个cpu核心 332 | 因此在生产环境中应开启与cpu核心数一样多的进程 333 | 这些工作Manager类都一并负责完成 334 | 335 | server_address = [ 336 | ('', 7777), 337 | ('', 7778), 338 | ] 339 | 340 | 只需要在这里添加新的端口信息就可以开启相应的进程 341 | 342 | 在生产环境中应开启多条进程占用多个端口 343 | 并使用Nginx的负载均衡来进行反向代理 344 | 345 | 346 | ## 推荐开发范式 347 | 348 | 349 | 目录结构 350 | Project 351 | > manage.py 主文件 程序入口 352 | > urls.py url配置文件 353 | > settings.py 框架配置文件 354 | > httppy/ httppy库 355 | > demoapp/ 一个典型的应用 356 | >> \__init__.py 357 | >> handler.py 358 | >> urls.py 359 | 360 | > static/ 静态文件目录 361 | > template/ 模板文件路径 362 | 363 | 推荐在linux服务器上运行 364 | 使用Nginx作为前端反向代理反向代理服务器 365 | Nginx完成负载均衡、处理静态文件请求 366 | 367 | 368 | ---------------------- 369 | -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | import log 3 | import web 4 | from manager import Manager 5 | import sys 6 | reload(sys) 7 | sys.setdefaultencoding('utf-8') 8 | 9 | 10 | __author__ = 'titorx' 11 | __version__ = 'httppy/v0.1' 12 | -------------------------------------------------------------------------------- /httpserver.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | from socketserver import BaseSocketHandler, TreadPoolTCPServer 3 | import urllib 4 | import StringIO 5 | import datetime 6 | from __init__ import __version__ 7 | 8 | 9 | class BaseHttpServer(TreadPoolTCPServer): 10 | pass 11 | 12 | 13 | class HttpRequest: 14 | 15 | """ 16 | http请求对象 每个浏览器发送到服务器的请求对应生成一个HTTPRequest对象 17 | 18 | header 原始报文首部 19 | body 原始报文主体 20 | request_line 原始报文首部中的请求行 21 | ip port 发起请求者的ip port 22 | http_version 请求使用的http版本 23 | method 请求方法 24 | 25 | url 请求的url 26 | get_string 请求的get部分字符串 27 | url?get_string 原始的完整请求url 28 | 29 | META 字典(dict)形式的解析后的报文首部 30 | 首部字段的key全部转化为大写(.upper()) 31 | 如: 32 | self.META = { 33 | 'CONNECTION': 'keep-alive' 34 | } 35 | 36 | GET 字典(dict)形式的解析后的get参数 37 | COOKIE 字典(dict)形式的解析后的cookie参数 38 | POST 字典(dict)形式的解析后的post参数 39 | 40 | FILE 字典(dict)形式的解析后的上传文件 41 | self.FILE = { 42 | 'file1': { 43 | 'CONTENT-TYPE': 'image/jpeg', 44 | 'filename': 'test.jpg', 45 | 'content': StringIO.StringIO(文件二进制) 46 | } 47 | } 48 | 49 | """ 50 | 51 | def __init__(self): 52 | self.header = '' 53 | self.body = '' 54 | self.ip = '' 55 | self.port = None 56 | self.http_version = '' 57 | self.request_line = '' 58 | self.method = '' 59 | self.url = '' 60 | self.get_string = '' 61 | self.META = {} 62 | self.GET = {} 63 | self.COOKIE = {} 64 | self.POST = {} 65 | self.FILE = {} 66 | 67 | 68 | class HttpResponse: 69 | 70 | """ 71 | 对http请求的响应对象 72 | """ 73 | 74 | STATUS = { 75 | 200: '200 OK', 76 | 204: '204 No Content', 77 | 206: '206 Partial Content', 78 | 301: '301 Moved Permanently', 79 | 302: '302 Found', 80 | 303: '303 See Other', 81 | 304: '304 Not Modified', 82 | 307: '307 Temporary Redirect', 83 | 400: '400 Bad Request', 84 | 401: '401 Unauthorized', 85 | 403: '403 Forbidden', 86 | 404: '404 Not Found', 87 | 500: '500 Internal Server Error', 88 | 503: '503 Service Unavailable', 89 | } 90 | 91 | class _Cookie: 92 | 93 | """ 94 | 用于cookie处理 95 | """ 96 | 97 | def __init__(self): 98 | self.COOKIE = {} 99 | 100 | def set_cookie(self, key, value, expires, path, domain): 101 | """ 102 | 设置cookie 103 | """ 104 | 105 | value = urllib.quote(value) 106 | self.COOKIE[key] = '='.join([key, value]) + ';' 107 | 108 | if expires: 109 | expires = (datetime.datetime.utcnow() + datetime.timedelta(seconds=expires)) 110 | self.COOKIE[key] += ' expires=%s;' % expires.strftime('%a, %d %b %Y %H:%M:%S UTC') 111 | if path: 112 | self.COOKIE[key] += ' path=%s;' % path 113 | if domain: 114 | self.COOKIE[key] += ' domain=%s;' % domain 115 | 116 | def get_set_cookie_meta(self): 117 | if self.COOKIE: 118 | set_cookie = ['Set-Cookie: ' + i for i in self.COOKIE.values()] 119 | set_cookie_string = '\r\n'.join(set_cookie) 120 | return set_cookie_string 121 | else: 122 | return None 123 | 124 | def __init__(self): 125 | self.http_version = 'HTTP/1.1' 126 | self.status = self.STATUS[200] 127 | self.header = '' 128 | self.body = '' 129 | self.META = { 130 | 'SERVER': __version__, 131 | 'CONNECTION': 'close', # 默认关闭长连接 132 | } 133 | self.cookie = self._Cookie() 134 | 135 | def set_header(self, key, value): 136 | """ 137 | :type key: str 138 | :type value: str 139 | """ 140 | self.META[key.upper()] = value 141 | 142 | def set_cookie(self, key, value, expires=None, path=None, domain=None): 143 | self.cookie.set_cookie(key, value, expires, path, domain) 144 | 145 | def set_status(self, status_code): 146 | """ 通过状态码设定响应状态 """ 147 | self.status = self.STATUS[status_code] 148 | 149 | def set_body(self, content): 150 | """ 设置响应正文 """ 151 | self.body = content 152 | 153 | def make_header(self): 154 | """ 生成响应首部 """ 155 | headers = [key + ': ' + value for key, value in self.META.iteritems()] 156 | set_cookie = self.cookie.get_set_cookie_meta() 157 | if set_cookie: 158 | headers.append(set_cookie) 159 | self.header = '\r\n'.join(headers) 160 | 161 | def get_response(self): 162 | """ 该方法返回一个字符串形式的http响应 """ 163 | self.make_header() 164 | response_line = ' '.join([self.http_version, self.status]) 165 | response = response_line + '\r\n' + self.header + '\r\n\r\n' + self.body 166 | return response 167 | 168 | 169 | class BaseHttpHandler(BaseSocketHandler): 170 | 171 | """ 172 | 进行HTTP协议的解析 173 | self.http_request 请求对象 174 | self.http_response 响应对象 175 | """ 176 | 177 | def __init__(self, socket_request, client_address, server): 178 | self.http_request = HttpRequest() 179 | BaseSocketHandler.__init__(self, socket_request, client_address, server) 180 | 181 | def handle_socket_request(self): 182 | self.parse_http() 183 | self.server.logger.info('[%s %s]' % (self.http_request.ip, self.http_request.request_line)) 184 | http_response = self.handle_http_request() 185 | self.get_http_response(http_response) 186 | 187 | def handle_http_request(self): 188 | """ 189 | 由用户进行重载 对http request进行处理 190 | 该方法必须返回一个HttpResponse对象 191 | """ 192 | return HttpResponse() 193 | 194 | def parse_http(self): 195 | """ 解析http """ 196 | # 获取客户端的ip port 197 | self.http_request.ip, self.http_request.port = self.client_address 198 | 199 | # 区分报文头部与主体 200 | header_len = self.data.find('\r\n\r\n') 201 | self.http_request.header = self.data[:header_len] 202 | self.http_request.body = self.data[header_len + 4:] 203 | 204 | self.parse_header() 205 | self.parse_get() 206 | self.parse_cookie() 207 | if self.http_request.method == 'POST': 208 | self.parse_post() 209 | 210 | def parse_header(self): 211 | """ 解析请求行以及请求头部 """ 212 | header_lines = self.http_request.header.splitlines() 213 | # 解析请求行 214 | request_line = header_lines[0] 215 | self.http_request.request_line = request_line 216 | request_line = request_line.split() 217 | self.http_request.method = request_line[0] 218 | 219 | request = request_line[1].split('?') 220 | # 当url中没有?时 如:/index/index 添加一个空字符串('')使get_string为空字符串('') 221 | request.append('') 222 | self.http_request.url, self.http_request.get_string = request[0], urllib.unquote(request[1]) 223 | 224 | self.http_request.http_version = request_line[2] 225 | 226 | # 解析报文首部 227 | for one_line in header_lines[1:]: 228 | meta = one_line.split(': ') 229 | self.http_request.META[meta[0].upper()] = meta[1] 230 | 231 | def parse_get(self): 232 | """ 解析get参数 """ 233 | if self.http_request.get_string: 234 | for get_string in self.http_request.get_string.split('&'): 235 | get = get_string.split('=') 236 | # 当出现get参数没写完整的情况时 如:?a&b=1 添加一个''使不完整的参数赋值为'' 237 | get.append('') 238 | self.http_request.GET[get[0]] = get[1] 239 | 240 | def parse_cookie(self): 241 | """ 解析cookie参数 """ 242 | cookie_string = self.http_request.META.get('COOKIE', '') 243 | if cookie_string: 244 | for cookies in cookie_string.split('; '): 245 | cookie = cookies.split('=') 246 | self.http_request.COOKIE[cookie[0]] = urllib.unquote(cookie[1]) 247 | 248 | def parse_post(self): 249 | """ 解析post得到的参数以及文件 """ 250 | content_type = self.http_request.META.get('CONTENT-TYPE', '') 251 | content_length = int(self.http_request.META.get('CONTENT-LENGTH', -1)) + 1 252 | 253 | if 'multipart/form-data' in content_type: 254 | boundary = content_type.split('; ')[1].split('=')[1] 255 | chunks = self.http_request.body.split('--' + boundary)[1:-1] 256 | for chunk in chunks: 257 | chunk_header_len = chunk.find('\r\n\r\n') 258 | chunk_header = chunk[:chunk_header_len] 259 | # [:-2] 截取掉末尾的\r\n 260 | chunk_body = chunk[chunk_header_len + 4:-2] 261 | chunk_meta = {} 262 | 263 | # 解析chunk header 264 | for header_line in chunk_header.splitlines(): 265 | if header_line: 266 | header_line = header_line.split('; ') 267 | 268 | meta = header_line[0].split(': ') 269 | chunk_meta[meta[0].upper()] = meta[1] 270 | 271 | if len(header_line) > 1: 272 | for header in header_line[1:]: 273 | meta = header.split('=') 274 | chunk_meta[meta[0]] = meta[1].strip('\"') 275 | 276 | # 解析 chunk body 277 | if not chunk_meta.get('CONTENT-TYPE', ''): 278 | self.http_request.POST[chunk_meta['name']] = chunk_body 279 | else: 280 | self.http_request.FILE[chunk_meta['name']] = { 281 | 'CONTENT-TYPE': chunk_meta['CONTENT-TYPE'], 282 | 'filename': chunk_meta['filename'], 283 | 'content': StringIO.StringIO(chunk_body) 284 | } 285 | 286 | else: 287 | body = self.http_request.body[:content_length] 288 | for posts in body.split('&'): 289 | post = posts.split('=') 290 | self.http_request.POST[post[0]] = urllib.unquote(post[1].replace('+', ' ')) 291 | 292 | def get_http_response(self, http_response): 293 | self.result = http_response.get_response() 294 | -------------------------------------------------------------------------------- /log.py: -------------------------------------------------------------------------------- 1 | import logging 2 | 3 | logging.basicConfig( 4 | level=logging.DEBUG, 5 | format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s', 6 | datefmt='%F %T', 7 | ) 8 | -------------------------------------------------------------------------------- /manager.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | import web 3 | from multiprocessing import Process 4 | 5 | 6 | class Manager: 7 | 8 | """ 9 | 一个对httppy库使用流程进行包装简化的管理框架类 10 | """ 11 | 12 | def __init__(self, addresses, urls, **kwargs): 13 | """ 14 | :type addresses: list 15 | :type urls: list 16 | """ 17 | self.addresses = addresses 18 | self.urls = urls 19 | self.kwargs = kwargs 20 | self.servers = [] 21 | self.url_route = None 22 | 23 | self.create_url_route() 24 | self.create_servers() 25 | self.setup() 26 | 27 | def create_url_route(self): 28 | self.url_route = web.UrlRoute(self.urls) 29 | 30 | def create_servers(self): 31 | for address in self.addresses: 32 | app = web.WebServer(address, self.url_route) 33 | self.servers.append(Process(target=app.server_start)) 34 | 35 | def setup(self): 36 | """ 37 | 对框架进行初始化 38 | 可配置项 39 | connect_timeout 套接字连接超时时间 40 | request_queue_size 套接字listen监听队列 41 | thread_pool_size 线程池线程数 42 | 404page 404页面 43 | 500page 500页面 44 | template_path 模板目录 45 | """ 46 | 47 | # 服务器设置 48 | def set_server(setup_server, key, value): 49 | """ 对服务器进行设置 """ 50 | if value: 51 | setattr(setup_server, key, value) 52 | 53 | for server in self.servers: 54 | set_server(server, 'connect_timeout', self.kwargs.get('connect_timeout')) 55 | set_server(server, 'request_queue_size', self.kwargs.get('request_queue_size')) 56 | set_server(server, 'thread_pool_size', self.kwargs.get('thread_pool_size')) 57 | #################################################################################### 58 | # 404状态页 59 | page_404 = self.kwargs.get('404page') 60 | if page_404: 61 | web.UrlRoute.response404 = page_404 62 | #################################################################################### 63 | # 500状态页 64 | page_500 = self.kwargs.get('500page') 65 | if page_500: 66 | web.UrlRoute.response404 = page_500 67 | #################################################################################### 68 | # 模板目录 69 | template_path = self.kwargs.get('template_path') 70 | if template_path: 71 | import template 72 | template.Render = template.get_template_render(template_path) 73 | 74 | def server_start(self): 75 | for server in self.servers: 76 | server.start() 77 | 78 | for server in self.servers: 79 | server.join() 80 | -------------------------------------------------------------------------------- /socketserver.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | import socket 3 | import threading 4 | import logging 5 | import os 6 | import traceback 7 | 8 | 9 | class BaseTCPServer(object): 10 | 11 | """ 12 | 基于TCP套接字的单线程服务器类 13 | 用于完成套接字层面的网络操作 14 | 15 | 工作流程: 16 | bind() 17 | ↓ 18 | listen() 19 | | 20 | | loop ←------- ← 21 | ↓ ↑ 22 | get_request() → handle() 23 | 24 | """ 25 | 26 | def __init__(self, server_address, request_handler_class): 27 | """ 28 | :type server_address: (str, int) 29 | """ 30 | self.logger = logging.getLogger(str(os.getpid())) 31 | self.server_address = server_address 32 | self.request_handler_class = request_handler_class 33 | self.request_queue_size = 5 34 | self.connect_timeout = 5 35 | 36 | self.socket = socket.socket() 37 | self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) 38 | 39 | def server_bind(self): 40 | """ 套接字对象绑定到服务器端口 """ 41 | self.socket.bind(self.server_address) 42 | self.server_address = self.socket.getsockname() 43 | 44 | def server_listen(self): 45 | """ 开始监听 """ 46 | self.socket.listen(self.request_queue_size) 47 | 48 | def server_start(self): 49 | """ 开启服务 """ 50 | self.server_bind() 51 | self.logger.info('Server start. pid:%s' % str(os.getpid())) 52 | 53 | self.server_listen() 54 | self.logger.info('bind:' + str(self.server_address)) 55 | 56 | while 1: 57 | socket_request, client_address = self.get_request() 58 | socket_request.settimeout(self.connect_timeout) 59 | self.handle_socket_request(socket_request, client_address) 60 | 61 | def get_request(self): 62 | """ 获取连接 """ 63 | return self.socket.accept() 64 | 65 | def handle_socket_request(self, socket_request, client_address): 66 | """ 67 | 处理套接字请求 68 | :type socket_request: socket._socketobject 69 | :type client_address: (str, int) 70 | """ 71 | self.request_handler_class(socket_request, client_address, self) 72 | 73 | def server_close(self): 74 | """ 关闭服务 """ 75 | self.socket.close() 76 | 77 | 78 | class TreadPoolTCPServer(BaseTCPServer): 79 | 80 | """ 81 | 基于线程池的TCP服务器 82 | """ 83 | 84 | class _Handler(threading.Thread): 85 | def __init__(self, server): 86 | threading.Thread.__init__(self) 87 | self.handler = None 88 | self.work_signal = threading.Event() 89 | self.server = server 90 | self.socket_request = None 91 | self.client_address = None 92 | self.setDaemon(True) 93 | self.start() 94 | 95 | def set_socket_request(self, socket_request, client_address): 96 | self.socket_request = socket_request 97 | self.client_address = client_address 98 | 99 | def run(self): 100 | while True: 101 | # 等待工作开始的信号 102 | self.work_signal.wait() 103 | try: 104 | self.handle_request() 105 | self.socket_request = None 106 | self.client_address = None 107 | except Exception as e: 108 | self.socket_request.close() 109 | self.server.logger.warn('\n'.join([str(e), traceback.format_exc()])) 110 | self.work_signal.clear() 111 | # 工作完成后将自身添加回线程池中 112 | self.server.thread_pool.append(self) 113 | 114 | def set_request_handler(self, request_handler_class): 115 | """ 设置处理请求的类 """ 116 | self.handler = request_handler_class 117 | 118 | def handle_request(self): 119 | self.handler(self.socket_request, self.client_address, self.server) 120 | 121 | def __init__(self, server_address, request_handler_class): 122 | BaseTCPServer.__init__(self, server_address, request_handler_class) 123 | # 线程池的最大线程数 124 | self.thread_pool_size = 10 125 | self.thread_pool = [] 126 | 127 | def build_thread_pool(self): 128 | """ 创建线程池 """ 129 | for i in range(self.thread_pool_size): 130 | handler = self.create_thread() 131 | handler.set_request_handler(self.request_handler_class) 132 | self.thread_pool.append(handler) 133 | 134 | def create_thread(self): 135 | """ 创建一条工作线程 """ 136 | return self._Handler(self) 137 | 138 | def server_start(self): 139 | self.build_thread_pool() 140 | BaseTCPServer.server_start(self) 141 | 142 | def handle_socket_request(self, socket_request, client_address): 143 | """ 处理套接字请求 """ 144 | if self.thread_pool: 145 | # 从线程池中取一条空闲线程 146 | handler = self.thread_pool.pop() 147 | handler.set_socket_request(socket_request, client_address) 148 | # 通知线程开始工作 149 | handler.work_signal.set() 150 | else: 151 | # 线程池中没有空闲线程 直接关闭套接字连接 152 | socket_request.close() 153 | 154 | 155 | class BaseSocketHandler: 156 | 157 | def __init__(self, socket_request, client_address, server): 158 | """ 159 | :type socket_request: socket._socketobject 160 | :type client_address: (str, int) 161 | """ 162 | self.recv_size = 1024 163 | self.data = '' 164 | self.result = '' 165 | self.socket_request = socket_request 166 | self.client_address = client_address 167 | self.server = server 168 | self.setup() 169 | try: 170 | self.recv() 171 | if self.data != '': 172 | self.handle_socket_request() 173 | finally: 174 | self.finish() 175 | 176 | def setup(self): 177 | pass 178 | 179 | def recv(self): 180 | while True: 181 | recv = self.socket_request.recv(self.recv_size) 182 | self.data += recv 183 | if len(recv) < self.recv_size: 184 | break 185 | 186 | def handle_socket_request(self): 187 | pass 188 | 189 | def finish(self): 190 | self.socket_request.sendall(self.result) 191 | self.socket_request.close() 192 | -------------------------------------------------------------------------------- /statichandler.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | import web 3 | import os 4 | 5 | 6 | MIME = { 7 | "rtf": "application/rtf", 8 | "jpeg": "image/jpeg", 9 | "tcl": "application/x-tcl", 10 | "jng": "image/x-jng", 11 | "mp3": "audio/mpeg", 12 | "mng": "video/x-mng", 13 | "xml": "text/xml", 14 | "img": "application/octet-stream", 15 | "jar": "application/java-archive", 16 | "ts": "video/mp2t", 17 | "tk": "application/x-tcl", 18 | "wml": "text/vnd.wap.wml", 19 | "jad": "text/vnd.sun.j2me.app-descriptor", 20 | "m4a": "audio/x-m4a", 21 | "bin": "application/octet-stream", 22 | "mpg": "video/mpeg", 23 | "docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document", 24 | "woff": "application/font-woff", 25 | "mov": "video/quicktime", 26 | "tif": "image/tiff", 27 | "hqx": "application/mac-binhex40", 28 | "jardiff": "application/x-java-archive-diff", 29 | "crt": "application/x-x509-ca-cert", 30 | "mp4": "video/mp4", 31 | "xls": "application/vnd.ms-excel", 32 | "htc": "text/x-component", 33 | "htm": "text/html", 34 | "webm": "video/webm", 35 | "webp": "image/webp", 36 | "sea": "application/x-sea", 37 | "pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation", 38 | "mpeg": "video/mpeg", 39 | "ico": "image/x-icon", 40 | "rpm": "application/x-redhat-package-manager", 41 | "kml": "application/vnd.google-earth.kml+xml", 42 | "dll": "application/octet-stream", 43 | "pem": "application/x-x509-ca-cert", 44 | "ra": "audio/x-realaudio", 45 | "kmz": "application/vnd.google-earth.kmz", 46 | "run": "application/x-makeself", 47 | "m3u8": "application/vnd.apple.mpegurl", 48 | "asx": "video/x-ms-asf", 49 | "js": "application/javascript", 50 | "msp": "application/octet-stream", 51 | "asf": "video/x-ms-asf", 52 | "mml": "text/mathml", 53 | "iso": "application/octet-stream", 54 | "pdf": "application/pdf", 55 | "pdb": "application/x-pilot", 56 | "xspf": "application/xspf+xml", 57 | "tiff": "image/tiff", 58 | "wmv": "video/x-ms-wmv", 59 | "ppt": "application/vnd.ms-powerpoint", 60 | "txt": "text/plain", 61 | "wbmp": "image/vnd.wap.wbmp", 62 | "jnlp": "application/x-java-jnlp-file", 63 | "ps": "application/postscript", 64 | "xpi": "application/x-xpinstall", 65 | "zip": "application/zip", 66 | "json": "application/json", 67 | "swf": "application/x-shockwave-flash", 68 | "gif": "image/gif", 69 | "msm": "application/octet-stream", 70 | "msi": "application/octet-stream", 71 | "war": "application/java-archive", 72 | "pl": "application/x-perl", 73 | "pm": "application/x-perl", 74 | "xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", 75 | "bmp": "image/x-ms-bmp", 76 | "atom": "application/atom+xml", 77 | "ogg": "audio/ogg", 78 | "ear": "application/java-archive", 79 | "png": "image/png", 80 | "rss": "application/rss+xml", 81 | "dmg": "application/octet-stream", 82 | "der": "application/x-x509-ca-cert", 83 | "midi": "audio/midi", 84 | "flv": "video/x-flv", 85 | "doc": "application/msword", 86 | "deb": "application/octet-stream", 87 | "svgz": "image/svg+xml", 88 | "prc": "application/x-pilot", 89 | "ai": "application/postscript", 90 | "jpg": "image/jpeg", 91 | "rar": "application/x-rar-compressed", 92 | "avi": "video/x-msvideo", 93 | "7z": "application/x-7z-compressed", 94 | "shtml": "text/html", 95 | "sit": "application/x-stuffit", 96 | "3gpp": "video/3gpp", 97 | "mid": "audio/midi", 98 | "html": "text/html", 99 | "xhtml": "application/xhtml+xml", 100 | "css": "text/css", 101 | "3gp": "video/3gpp", 102 | "kar": "audio/midi", 103 | "m4v": "video/x-m4v", 104 | "exe": "application/octet-stream", 105 | "svg": "image/svg+xml", 106 | "eps": "application/postscript", 107 | "cco": "application/x-cocoa", 108 | "eot": "application/vnd.ms-fontobject", 109 | "wmlc": "application/vnd.wap.wmlc" 110 | } 111 | 112 | 113 | def static_handler(path): 114 | 115 | class StaticFileHandler(web.RequestHandler): 116 | 117 | """ 118 | 用以处理静态文件请求 119 | """ 120 | 121 | static_dir_path = '' 122 | 123 | def handle(self): 124 | file_path = os.path.abspath(os.path.join(self.static_dir_path, self.request.url_param['path'].rstrip('/'))) 125 | # 判断目标文件是否位于指定的目录下 以及 目标文件是否存在 126 | if (self.static_dir_path in file_path) and os.path.isfile(file_path): 127 | # 发送文件 128 | with open(file_path) as f: 129 | self.response.set_body(f.read()) 130 | self.response.set_header('Content-Length', str(len(self.response.body))) 131 | # 获取文件的类型 132 | mime = MIME.get(os.path.basename(file_path).split('.')[-1], None) 133 | if mime: 134 | self.response.set_header('Content-Type', mime) 135 | else: 136 | self.response = web.Response404() 137 | 138 | setattr(StaticFileHandler, 'static_dir_path', os.path.abspath(path)) 139 | return StaticFileHandler 140 | -------------------------------------------------------------------------------- /template.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | import jinja2 3 | import os 4 | 5 | 6 | def get_template_render(path): 7 | 8 | class _Render: 9 | 10 | path = '' 11 | 12 | def __init__(self): 13 | self.env = jinja2.Environment(loader=jinja2.FileSystemLoader(self.path)) 14 | 15 | def render(self, template, *args, **kwargs): 16 | content = self.env.get_template(template).render(*args, **kwargs) 17 | if type(content) is unicode: 18 | content = content.encode('utf-8') 19 | return content 20 | 21 | setattr(_Render, 'path', os.path.abspath(path)) 22 | return _Render() 23 | 24 | Render = get_template_render('template') 25 | 26 | 27 | def render(template, *args, **kwargs): 28 | return Render.render(template, *args, **kwargs) 29 | -------------------------------------------------------------------------------- /web.py: -------------------------------------------------------------------------------- 1 | # coding=utf-8 2 | import httpserver 3 | import re 4 | import traceback 5 | 6 | 7 | class Request(httpserver.HttpRequest): 8 | 9 | """ 10 | httppy web 框架的基本请求类 11 | """ 12 | 13 | def __init__(self): 14 | httpserver.HttpRequest.__init__(self) 15 | # 存储从url中解析得到的参数 16 | self.url_param = {} 17 | 18 | 19 | class Response(httpserver.HttpResponse): 20 | 21 | """ 22 | httppy web 框架的基本响应类 23 | """ 24 | 25 | def __init__(self): 26 | httpserver.HttpResponse.__init__(self) 27 | 28 | def redirect(self, redirect_to): 29 | """ 30 | 将请求重定向到新的url 31 | :type redirect_to: str 32 | """ 33 | self.set_status(302) 34 | self.set_header('Location', redirect_to) 35 | 36 | 37 | class Response404(Response): 38 | 39 | """ 40 | 404响应类 41 | """ 42 | 43 | def __init__(self): 44 | Response.__init__(self) 45 | self.set_status(404) 46 | self.handler() 47 | 48 | def handler(self): 49 | pass 50 | 51 | 52 | class Response500(Response): 53 | 54 | """ 55 | 500响应类 56 | """ 57 | 58 | def __init__(self): 59 | Response.__init__(self) 60 | self.set_status(500) 61 | self.handler() 62 | 63 | def handler(self): 64 | pass 65 | 66 | 67 | class WebHandler(httpserver.BaseHttpHandler): 68 | 69 | """ 70 | httppy web 框架的handler 71 | """ 72 | 73 | def __init__(self, socket_request, client_address, server, url_route): 74 | """ 75 | :type socket_request: socket._socketobject 76 | :type client_address: (str, int) 77 | :type server: WebServer 78 | :type url_route: UrlRoute 79 | """ 80 | self.url_route = url_route 81 | httpserver.BaseHttpHandler.__init__(self, socket_request, client_address, server) 82 | 83 | def handle_http_request(self): 84 | if not self.http_request.url.endswith('/'): 85 | self.http_request.url += '/' 86 | return self.url_route.route(self.http_request, self.server) 87 | 88 | 89 | class WebServer(httpserver.BaseHttpServer): 90 | class _Handler(httpserver.BaseHttpServer._Handler): 91 | def __init__(self, server, url_route): 92 | self.url_route = url_route 93 | httpserver.BaseHttpServer._Handler.__init__(self, server) 94 | 95 | def handle_request(self): 96 | self.handler(self.socket_request, self.client_address, self.server, self.url_route) 97 | 98 | def __init__(self, server_address, url_route): 99 | """ 100 | :type server_address: (str, int) 101 | :type url_route: UrlRoute 102 | """ 103 | self.url_route = url_route 104 | httpserver.BaseHttpServer.__init__(self, server_address, WebHandler) 105 | 106 | def create_thread(self): 107 | return self._Handler(self, self.url_route) 108 | 109 | 110 | class RequestHandler: 111 | 112 | """ 113 | 处理UrlRoute分发的请求 114 | """ 115 | 116 | def __init__(self, request, server): 117 | """ 118 | :type request: Request 119 | :type server: WebServer 120 | """ 121 | self.request = request 122 | self.server = server 123 | self.logger = self.server.logger 124 | self.response = Response() 125 | # 默认返回内容的类型为html页面 126 | self.response.set_header('Content-Type', "text/html") 127 | 128 | self.setup() 129 | self.handle() 130 | self.finish() 131 | 132 | def setup(self): 133 | pass 134 | 135 | def handle(self): 136 | pass 137 | 138 | def finish(self): 139 | pass 140 | 141 | def get_response(self): 142 | return self.response 143 | 144 | 145 | class UrlRoute: 146 | 147 | """ 148 | url路由 149 | 负责将对应url请求分发给对应handler 150 | """ 151 | 152 | response404 = Response404 153 | response500 = Response500 154 | 155 | def __init__(self, route_table): 156 | """ 157 | :type route_table: list 158 | """ 159 | self.route_table = route_table 160 | self.convert_route_table() 161 | 162 | def convert_route_table(self): 163 | route_table = [] 164 | for url, handler in self.route_table: 165 | route_table.append((re.compile(url), handler)) 166 | self.route_table = route_table 167 | 168 | def route(self, request, server): 169 | """ 170 | :type request: Request 171 | :type server: WebServer 172 | """ 173 | response = None 174 | for url, handler in self.route_table: 175 | result = url.match(request.url) 176 | if result: 177 | request.url_param = result.groupdict() 178 | try: 179 | response = handler(request, server).get_response() 180 | except Exception as e: 181 | # handler出错返回500错误 182 | server.logger.warn('\n'.join([str(e), traceback.format_exc()])) 183 | response = self.response500() 184 | break 185 | 186 | if not response: 187 | response = self.response404() 188 | return response 189 | --------------------------------------------------------------------------------