├── .gitignore ├── setup.py ├── flask_replicated.py └── README.rst /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | build 3 | dist 4 | MANIFEST 5 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | with open('README.rst') as fp: 4 | readme = fp.read() 5 | 6 | setup( 7 | name='flask_replicated', 8 | description=( 9 | 'Flask SqlAlchemy router for stateful master-slave replication' 10 | ), 11 | long_description=readme, 12 | author='Peter Demin', 13 | author_email='peterdemin@gmail.com', 14 | url='https://github.com/peterdemin/python-flask-replicated', 15 | license="BSD", 16 | zip_safe=False, 17 | keywords='flask sqlalchemy replication master slave', 18 | classifiers=[ 19 | 'Development Status :: 5 - Production/Stable', 20 | 'Intended Audience :: Developers', 21 | 'License :: OSI Approved :: BSD License', 22 | 'Natural Language :: English', 23 | 'Programming Language :: Python :: 2', 24 | 'Programming Language :: Python :: 2.6', 25 | 'Programming Language :: Python :: 2.7', 26 | 'Programming Language :: Python :: 3', 27 | 'Programming Language :: Python :: 3.5', 28 | 'Programming Language :: Python :: 3.6', 29 | 'Programming Language :: Python :: 3.7', 30 | ], 31 | version='2.1', 32 | py_modules=['flask_replicated'], 33 | ) 34 | -------------------------------------------------------------------------------- /flask_replicated.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | 3 | from flask import current_app, g, request 4 | 5 | 6 | class FlaskReplicated(object): 7 | READONLY_METHODS = set(['GET', 'HEAD']) 8 | 9 | def __init__(self, app=None): 10 | if app is not None: 11 | self.init_app(app) 12 | self.AUTO_SLAVE = app.config.get('AUTO_READ_ON_SLAVE', True) 13 | 14 | def init_app(self, app): 15 | assert hasattr(app, 'extensions') 16 | assert 'sqlalchemy' in app.extensions 17 | if 'replicated' not in app.extensions: 18 | app.extensions['replicated'] = self 19 | binds = app.config.get('SQLALCHEMY_BINDS') or {} 20 | if 'slave' in binds: 21 | app.before_request(self._pick_database_replica) 22 | db = app.extensions['sqlalchemy'].db 23 | get_engine_vanilla = db.get_engine 24 | 25 | def get_replicated_engine(app=app, bind=None): 26 | if bind is None and g: 27 | use_slave = getattr(g, 'use_slave', False) 28 | use_master = getattr(g, 'use_master', False) 29 | if use_slave and not use_master: 30 | bind = 'slave' 31 | return get_engine_vanilla(app, bind) 32 | db.get_engine = get_replicated_engine 33 | 34 | def _pick_database_replica(self): 35 | func = current_app.view_functions.get(request.endpoint) 36 | if getattr(func, 'use_master_database', False): 37 | g.use_master = True 38 | g.use_slave = ( request.method in self.READONLY_METHODS and self.AUTO_SLAVE ) or getattr(func, 'use_slave_database', False) 39 | 40 | def use_master_database(func): 41 | func.use_master_database = True 42 | return func 43 | 44 | 45 | def use_slave_database(func): 46 | func.use_slave_database = True 47 | return func -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | .. image:: https://img.shields.io/pypi/v/flask_replicated.svg 2 | :target: https://pypi.python.org/pypi/flask_replicated 3 | 4 | SUMMARY 5 | ------- 6 | 7 | Flask replicated is a Flask extension, designed to work with 8 | SqlAlchemy. It's purpose it to provide more or less automatic 9 | master-slave replication. On each request, extension determines database 10 | usage intention (to read or to write into a database). Then, it picks 11 | right database url inside overriden ``db.get_engine()`` whenever request 12 | handler tries to access database depending on REST verb used. 13 | 14 | Flask replicated comes with a security kill-switch to enable developer mode 15 | to easily keep control on the feature ``AUTO_READ_ON_SLAVE`` true by default, 16 | once deactivated only master database will be used unless explicit use of the decorators. 17 | 18 | INSTALLATION 19 | ------------ 20 | 21 | 1. Install flask replicated distribution using ``pip install flask_replicated``. 22 | Or add ``flask-replicated==1.4`` in requirements.txt and requirements 23 | ``pip3 install -r requirements.txt``. 24 | 25 | 2. Import library ``from flask_replicated import FlaskReplicated`` or 26 | ``import flask_replicated`` depending on how you want to call the functions or decorators 27 | 28 | 3. In flask ``app.config`` configure your database bindings a standard way:: 29 | 30 | AUTO_READ_ON_SLAVE = True 31 | SQLALCHEMY_DATABASE_URI = '%(schema)s://%(user)s:%(password)s@%(master_host)s/%(database)s' 32 | SQLALCHEMY_BINDS = { 33 | 'master': SQLALCHEMY_DATABASE_URI, 34 | 'slave': '%(schema)s://%(user)s:%(password)s@%(slave_host)s/%(database)s' 35 | } 36 | 37 | 4. Register app extension:: 38 | 39 | app = Flask(...) 40 | ... 41 | FlaskReplicated(app) 42 | 43 | USAGE 44 | ----- 45 | 46 | Flask replicated routes SQL queries into different databases based on 47 | request method. If method is one of ``READONLY_METHODS`` which are defined 48 | as set(['GET', 'HEAD']) and config ``AUTO_READ_ON_SLAVE`` has not been set 49 | 50 | While this is usually enough there are cases when DB access is not 51 | controlled explicitly by your business logic. Good examples are implicit 52 | creation of sessions on first access, writing some bookkeeping info, 53 | implicit registration of a user account somewhere inside the system. 54 | These things can happen at arbitrary moments of time, including during 55 | GET requests. 56 | 57 | To handle these situations wrap appropriate view function with 58 | ``@flask_replicated.use_master_database`` decorator. It will mark function to 59 | always use master database url. 60 | 61 | Conversely, wrap the view function with the ``@flask_replicated.use_slave_database`` 62 | decorator if you want to ensure that it always uses the slave replica. 63 | 64 | 65 | 66 | GET after POST 67 | ~~~~~~~~~~~~~~ 68 | 69 | There is a special case that needs addressing when working with 70 | asynchronous replication scheme. Replicas can lag behind a master 71 | database on receiving updates. In practice this mean that after 72 | submitting a POST form that redirects to a page with updated data this 73 | page may be requested from a slave replica that wasn't updated yet. And 74 | the user will have an impression that the submit didn't work. 75 | --------------------------------------------------------------------------------