├── README.md ├── app.py ├── client.py ├── config.py └── requirement.txt /README.md: -------------------------------------------------------------------------------- 1 | async-flask-sqlalchemy-postgresql 2 | ================================= 3 | 4 | 异步的数据库连接(postgresql)示例 5 | 6 | * 安装所需的插件: 7 | 8 | pip install -r requirements.txt 9 | 10 | * 需要在postgresql中创建数据库 test_asyn 11 | 12 | * 运行 python app.py -c 用来创建测试的表； 13 | 14 | * 运行 Python app.py 来启动服务器，运行 python client 来测试 15 | 16 | 结果如下:: 17 | 18 | localhost:async-flask-sqlalchemy-postgresql sixu05202004$ python client.py 19 | Sending 5 requests for http://localhost:8080/test/postgres/... 20 | @ 5.05s got response [200] 21 | @ 5.05s got response [200] 22 | @ 5.05s got response [200] 23 | @ 5.05s got response [200] 24 | @ 5.05s got response [200] 25 | = 5.06s TOTAL 26 | SUM TOTAL = 5.06s 27 | 28 | 29 | 注意： 30 | 31 | 1.config.py 中需要修改测试数据库的用户名和密码，并且可以修改pool_size的数量； 32 | 33 | 2.python client.py num,比如：python client.py 100可以用来模拟更多的连接； 34 | 35 | 3.如果python client.py num中得num数大于config.py中的SQLALCHEMY_POOL_SIZE = 100时候，我们会发现有些数据库连接又存在阻塞的情况。 36 | 比如，我们把SQLALCHEMY_POOL_SIZE改成10，使用python client.py 30来测试，结果如下:: 37 | 38 | localhost:async-flask-sqlalchemy-postgresql sixu05202004$ python client.py 30 39 | Sending 30 requests for http://localhost:8080/test/postgres/... 40 | @ 5.07s got response [200] 41 | @ 5.07s got response [200] 42 | @ 5.08s got response [200] 43 | @ 5.09s got response [200] 44 | @ 5.09s got response [200] 45 | @ 5.13s got response [200] 46 | @ 5.12s got response [200] 47 | @ 5.12s got response [200] 48 | @ 5.13s got response [200] 49 | @ 5.19s got response [200] 50 | @ 5.19s got response [200] 51 | @ 5.19s got response [200] 52 | @ 5.19s got response [200] 53 | @ 5.19s got response [200] 54 | @ 5.19s got response [200] 55 | @ 5.19s got response [200] 56 | @ 5.19s got response [200] 57 | @ 5.20s got response [200] 58 | @ 5.20s got response [200] 59 | @ 5.20s got response [200] 60 | @ 10.14s got response [200] 61 | @ 10.15s got response [200] 62 | @ 10.15s got response [200] 63 | @ 10.24s got response [200] 64 | @ 10.24s got response [200] 65 | @ 10.24s got response [200] 66 | @ 10.24s got response [200] 67 | @ 10.25s got response [200] 68 | @ 10.25s got response [200] 69 | @ 10.26s got response [200] 70 | = 10.28s TOTAL 71 | SUM TOTAL = 10.28s 72 | 73 | 74 | 这是因为 连接数 》SQLALCHEMY_POOL_SIZE + SQLALCHEMY_MAX_OVERFLOW ，我们可以通过一些设置来避免这种情况（NullPool ），但是实际上 postgresql 规定了最大连接数，这个是无法避免的，因此上述的设置最好不要使用 75 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import sys 5 | from flask import Flask,jsonify 6 | from werkzeug import cached_property 7 | from flask.ext.sqlalchemy import SQLAlchemy, BaseQuery 8 | 9 | from gevent.monkey import patch_all 10 | patch_all() 11 | from psycogreen.gevent import patch_psycopg 12 | patch_psycopg() 13 | 14 | 15 | app = Flask(__name__) 16 | app.config.from_pyfile('config.py') 17 | 18 | 19 | db = SQLAlchemy(app) 20 | db.engine.pool._use_threadlocal = True 21 | 22 | 23 | class AsyndbQuery(BaseQuery): 24 | 25 | '''Provide all kinds of query functions.''' 26 | 27 | def jsonify(self): 28 | '''Converted datas into JSON.''' 29 | for item in self.all(): 30 | yield item.as_dict 31 | 32 | 33 | class Ayndb(db.Model): 34 | query_class = AsyndbQuery 35 | 36 | id = db.Column(db.Integer, primary_key=True) 37 | title = db.Column(db.String(60)) 38 | done = db.Column(db.Boolean) 39 | priority = db.Column(db.Integer) 40 | 41 | @cached_property 42 | def as_dict(self): 43 | return { 44 | 'id': self.id, 45 | 'title': self.title, 46 | 'done': self.done, 47 | 'priority': self.priority 48 | } 49 | 50 | 51 | @app.route('/test/postgres/') 52 | def sleep_postgres(): 53 | db.session.execute('SELECT pg_sleep(5)') 54 | return jsonify(data = list(Ayndb.query.jsonify())) 55 | 56 | 57 | def create_data(): 58 | """ A helper function to create our tables and some Todo objects. 59 | """ 60 | db.create_all() 61 | alldata = [] 62 | for i in range(50): 63 | item = Ayndb( 64 | title="test for postgres,this is {0}".format(i), 65 | done=(i % 2 == 0), 66 | priority=(i % 5) 67 | ) 68 | alldata.append(item) 69 | db.session.add_all(alldata) 70 | db.session.commit() 71 | db.session.close() 72 | 73 | 74 | if __name__ == '__main__': 75 | 76 | if '-c' in sys.argv: 77 | create_data() 78 | else: 79 | #app.run() 80 | 81 | from gevent.pywsgi import WSGIServer 82 | http_server = WSGIServer(('', 8080), app) 83 | http_server.serve_forever() 84 | 85 | -------------------------------------------------------------------------------- /client.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import sys 5 | import gevent 6 | import time 7 | from gevent import monkey 8 | monkey.patch_all() 9 | import urllib2 10 | 11 | 12 | def fetch_url(url): 13 | t0 = time.time() 14 | try: 15 | resp = urllib2.urlopen(url) 16 | resp_code = resp.code 17 | except urllib2.HTTPError, e: 18 | resp_code = e.code 19 | t1 = time.time() 20 | print("\t@ %5.2fs got response [%d]" % (t1 - t0, resp_code)) 21 | 22 | 23 | def time_fetch_urls(url, num_jobs): 24 | print("Sending %d requests for %s..." % (num_jobs, url)) 25 | t0 = time.time() 26 | gevent.joinall([gevent.spawn(fetch_url, url) for i in range(num_jobs)]) 27 | t1 = time.time() 28 | print("SUM TOTAL = %.2fs" % (t1 - t0)) 29 | 30 | 31 | if __name__ == '__main__': 32 | 33 | try: 34 | num_requests = int(sys.argv[1]) 35 | except IndexError: 36 | num_requests = 10 37 | 38 | time_fetch_urls("http://localhost:8080/test/postgres/", num_requests) 39 | -------------------------------------------------------------------------------- /config.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://postgres:postgres@localhost/test_asyn' 5 | SQLALCHEMY_ECHO = False 6 | SQLALCHEMY_POOL_SIZE = 10 7 | SECRET_KEY = 'A0Zr98j/3yX R~XHH!jmN]LWX/,?RT' 8 | DEBUG = True 9 | -------------------------------------------------------------------------------- /requirement.txt: -------------------------------------------------------------------------------- 1 | Flask 2 | Flask-SQLAlchemy 3 | psycopg2 4 | psycogreen 5 | gevent --------------------------------------------------------------------------------