├── .gitignore ├── CHANGES.txt ├── LICENSE ├── README.md ├── README.rst ├── setup.py ├── tests ├── __init__.py └── test.py └── tomorrow ├── __init__.py └── tomorrow.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | 5 | # C extensions 6 | *.so 7 | 8 | # Distribution / packaging 9 | .Python 10 | env/ 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | *.egg-info/ 23 | .installed.cfg 24 | *.egg 25 | 26 | # PyInstaller 27 | # Usually these files are written by a python script from a template 28 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 29 | *.manifest 30 | *.spec 31 | 32 | # Installer logs 33 | pip-log.txt 34 | pip-delete-this-directory.txt 35 | 36 | # Unit test / coverage reports 37 | htmlcov/ 38 | .tox/ 39 | .coverage 40 | .coverage.* 41 | .cache 42 | nosetests.xml 43 | coverage.xml 44 | *,cover 45 | 46 | # Translations 47 | *.mo 48 | *.pot 49 | 50 | # Django stuff: 51 | *.log 52 | 53 | # Sphinx documentation 54 | docs/_build/ 55 | 56 | # PyBuilder 57 | target/ 58 | -------------------------------------------------------------------------------- /CHANGES.txt: -------------------------------------------------------------------------------- 1 | 0.1.0 2 | - Initial commit 3 | - Basic support for thread-based concurrency 4 | 5 | 0.2.0 6 | - Documentation update 7 | - Code cleanliness 8 | 9 | 0.2.1 10 | - Package structure edits 11 | - Install instruction edits 12 | 13 | 0.2.2 14 | - Simplify Tomorrow.__init__ 15 | - Tests for timeout argument 16 | - Documentation for timeout argument 17 | 18 | 0.2.3 19 | - Sync README.rst with README.md 20 | - Change to relative import 21 | - Test to ensure functions are valid return types 22 | 23 | 0.2.4 24 | - Remove `tests` folder from installed packages -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Madison May 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [ ![Codeship Status for madisonmay/Tomorrow](https://codeship.com/projects/9a3b4c60-1b5b-0133-5ec7-7e346f2e432c/status?branch=master)](https://codeship.com/projects/94472) 2 | 3 | # Tomorrow 4 | **Magic decorator syntax for asynchronous code in Python 2.7.** 5 | 6 | 7 | **Please don't actually use this in production. It's more of a thought experiment than anything else, and relies heavily on behavior specific to Python's old style classes. Pull requests, issues, comments and suggestions welcome.** 8 | 9 | Installation 10 | ------------ 11 | 12 | Tomorrow is conveniently available via pip: 13 | ``` 14 | pip install tomorrow 15 | ``` 16 | 17 | or installable via `git clone` and `setup.py` 18 | ``` 19 | git clone git@github.com:madisonmay/Tomorrow.git 20 | sudo python setup.py install 21 | ``` 22 | 23 | To ensure Tomorrow is properly installed, you can run the unittest suite from the project root: 24 | ``` 25 | nosetests -v 26 | ``` 27 | 28 | Usage 29 | ----- 30 | The tomorrow library enables you to utilize the benefits of multi-threading with minimal concern about the implementation details. 31 | 32 | Behind the scenes, the library is a thin wrapper around the `Future` object in `concurrent.futures` that resolves the `Future` whenever you try to access any of its attributes. 33 | 34 | Enough of the implementation details, let's take a look at how simple it is to speed up an inefficient chunk of blocking code with minimal effort. 35 | 36 | 37 | Naive Web Scraper 38 | ----------------- 39 | You've collected a list of urls and are looking to download the HTML of the lot. The following is a perfectly reasonable first stab at solving the task. 40 | 41 | For the following examples, we'll be using the top sites from the Alexa rankings. 42 | 43 | ```python 44 | urls = [ 45 | 'http://google.com', 46 | 'http://facebook.com', 47 | 'http://youtube.com', 48 | 'http://baidu.com', 49 | 'http://yahoo.com', 50 | ] 51 | ``` 52 | 53 | Right then, let's get on to the code. 54 | 55 | ```python 56 | import time 57 | import requests 58 | 59 | def download(url): 60 | return requests.get(url) 61 | 62 | if __name__ == "__main__": 63 | 64 | start = time.time() 65 | responses = [download(url) for url in urls] 66 | html = [response.text for response in responses] 67 | end = time.time() 68 | print "Time: %f seconds" % (end - start) 69 | ``` 70 | 71 | More Efficient Web Scraper 72 | -------------------------- 73 | 74 | Using tomorrow's decorator syntax, we can define a function that executes in multiple threads. Individual calls to `download` are non-blocking, but we can largely ignore this fact and write code identically to how we would in a synchronous paradigm. 75 | 76 | ```python 77 | import time 78 | import requests 79 | 80 | from tomorrow import threads 81 | 82 | @threads(5) 83 | def download(url): 84 | return requests.get(url) 85 | 86 | if __name__ == "__main__": 87 | start = time.time() 88 | responses = [download(url) for url in urls] 89 | html = [response.text for response in responses] 90 | end = time.time() 91 | print "Time: %f seconds" % (end - start) 92 | 93 | ``` 94 | 95 | Awesome! With a single line of additional code (and no explicit threading logic) we can now download websites ~10x as efficiently. 96 | 97 | You can also optionally pass in a timeout argument, to prevent hanging on a task that is not guaranteed to return. 98 | 99 | ```python 100 | import time 101 | 102 | from tomorrow import threads 103 | 104 | @threads(1, timeout=0.1) 105 | def raises_timeout_error(): 106 | time.sleep(1) 107 | 108 | if __name__ == "__main__": 109 | print raises_timeout_error() 110 | ``` 111 | 112 | How Does it Work? 113 | ----------------- 114 | 115 | Feel free to read the source for a peek behind the scenes -- it's less than 50 lines of code. 116 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | Tomorrow 2 | ======== 3 | 4 | Magic decorator syntax for asynchronous code in Python 5 | 6 | Installation 7 | ------------ 8 | 9 | Tomorrow is conveniently available via pip: 10 | 11 | :: 12 | 13 | pip install tomorrow 14 | 15 | or installable via ``git clone`` and ``setup.py`` 16 | 17 | :: 18 | 19 | git clone git@github.com:madisonmay/Tomorrow.git 20 | sudo python setup.py install 21 | 22 | Usage 23 | ----- 24 | 25 | The tomorrow library enables you to utilize the benefits of 26 | multi-threading with minimal concern about the implementation details. 27 | 28 | Behind the scenes, the library is a thin wrapper around the ``Future`` 29 | object in ``concurrent.futures`` that resolves the ``Future`` whenever 30 | you try to access any of its attributes. 31 | 32 | Enough of the implementation details, let's take a look at how simple it 33 | is to speed up an inefficient chunk of blocking code with minimal 34 | effort. 35 | 36 | Naive Web Scraper 37 | ----------------- 38 | 39 | You've collected a list of urls and are looking to download the HTML of 40 | the lot. The following is a perfectly reasonable first stab at solving 41 | the task. 42 | 43 | For the following examples, we'll be using the top sites from the Alexa 44 | rankings. 45 | 46 | .. code:: python 47 | 48 | urls = [ 49 | 'http://google.com', 50 | 'http://facebook.com', 51 | 'http://youtube.com', 52 | 'http://baidu.com', 53 | 'http://yahoo.com', 54 | ] 55 | 56 | Right then, let's get on to the code. 57 | 58 | .. code:: python 59 | 60 | import time 61 | import requests 62 | 63 | def download(url): 64 | return requests.get(url) 65 | 66 | if __name__ == "__main__": 67 | 68 | start = time.time() 69 | responses = [download(url) for url in urls] 70 | html = [response.text for response in responses] 71 | end = time.time() 72 | print "Time: %f seconds" % (end - start) 73 | 74 | More Efficient Web Scraper 75 | -------------------------- 76 | 77 | Using tomorrow's decorator syntax, we can define a function that 78 | executes in multiple threads. Individual calls to ``download`` are 79 | non-blocking, but we can largely ignore this fact and write code 80 | identically to how we would in a synchronous paradigm. 81 | 82 | .. code:: python 83 | 84 | import time 85 | import requests 86 | 87 | from tomorrow import threads 88 | 89 | @threads(5) 90 | def download(url): 91 | return requests.get(url) 92 | 93 | if __name__ == "__main__": 94 | import time 95 | 96 | start = time.time() 97 | responses = [download(url) for url in urls] 98 | html = [response.text for response in responses] 99 | end = time.time() 100 | print "Time: %f seconds" % (end - start) 101 | 102 | Awesome! With a single line of additional code (and no explicit 103 | threading logic) we can now download websites ~10x as efficiently. 104 | 105 | You can also optionally pass in a timeout argument, to prevent hanging 106 | on a task that is not guaranteed to return. 107 | 108 | .. code:: python 109 | 110 | import time 111 | 112 | from tomorrow import threads 113 | 114 | @threads(1, timeout=0.1) 115 | def raises_timeout_error(): 116 | time.sleep(1) 117 | 118 | if __name__ == "__main__": 119 | print raises_timeout_error() 120 | 121 | How Does it Work? 122 | ----------------- 123 | 124 | Feel free to read the source for a peek behind the scenes -- it's less 125 | that 50 lines of code. 126 | 127 | .. |Codeship Status for madisonmay/Tomorrow| image:: https://codeship.com/projects/9a3b4c60-1b5b-0133-5ec7-7e346f2e432c/status?branch=master 128 | :target: https://codeship.com/projects/94472 129 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | 3 | setup( 4 | name="tomorrow", 5 | version="0.2.4", 6 | author="Madison May", 7 | author_email="madison@indico.io", 8 | packages=find_packages( 9 | exclude=[ 10 | 'tests' 11 | ] 12 | ), 13 | install_requires=[ 14 | "futures >= 2.2.0" 15 | ], 16 | description=""" 17 | Magic decorator syntax for asynchronous code. 18 | """, 19 | license="MIT License (See LICENSE)", 20 | long_description=open("README.rst").read(), 21 | url="https://github.com/madisonmay/tomorrow" 22 | ) 23 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/madisonmay/Tomorrow/34e0bf422e5f81fd60872367c4fdf2b89d26ec14/tests/__init__.py -------------------------------------------------------------------------------- /tests/test.py: -------------------------------------------------------------------------------- 1 | import time 2 | import unittest 3 | 4 | from concurrent.futures import ThreadPoolExecutor, TimeoutError 5 | 6 | from tomorrow import threads 7 | 8 | DELAY = 0.5 9 | TIMEOUT = 0.1 10 | N = 2 11 | 12 | 13 | class TomorrowTestCase(unittest.TestCase): 14 | 15 | def test_threads_decorator(self): 16 | 17 | def slow_add(x, y): 18 | time.sleep(DELAY) 19 | return x + y 20 | 21 | @threads(N) 22 | def async_add(x, y): 23 | time.sleep(DELAY) 24 | return x + y 25 | 26 | x, y = 2, 2 27 | 28 | start = time.time() 29 | 30 | results = [] 31 | for i in range(N): 32 | results.append(async_add(x, y)) 33 | 34 | checkpoint = time.time() 35 | 36 | for result in results: 37 | print result 38 | 39 | end = time.time() 40 | assert (checkpoint - start) < DELAY 41 | assert DELAY < (end - start) < (DELAY * N) 42 | 43 | 44 | def test_shared_executor(self): 45 | 46 | executor = ThreadPoolExecutor(N) 47 | 48 | @threads(executor) 49 | def f(x): 50 | time.sleep(DELAY) 51 | return x 52 | 53 | @threads(executor) 54 | def g(x): 55 | time.sleep(DELAY) 56 | return x 57 | 58 | start = time.time() 59 | 60 | results = [] 61 | for i in range(N): 62 | results.append(g(f(i))) 63 | 64 | for result in results: 65 | print result 66 | 67 | end = time.time() 68 | assert (N * DELAY) < (end - start) < (2 * N * DELAY) 69 | 70 | 71 | def test_timeout(self): 72 | 73 | @threads(N, timeout=TIMEOUT) 74 | def raises_timeout_error(): 75 | time.sleep(DELAY) 76 | 77 | with self.assertRaises(TimeoutError): 78 | print raises_timeout_error() 79 | 80 | @threads(N, timeout=2*DELAY) 81 | def no_timeout_error(): 82 | time.sleep(DELAY) 83 | 84 | print no_timeout_error() 85 | 86 | def test_future_function(self): 87 | 88 | @threads(N) 89 | def returns_function(): 90 | def f(): 91 | return True 92 | return f 93 | 94 | true = returns_function() 95 | assert true() 96 | 97 | def test_wait(self): 98 | 99 | mutable = [] 100 | 101 | @threads(N) 102 | def side_effects(): 103 | mutable.append(True) 104 | 105 | result = side_effects() 106 | result._wait() 107 | assert mutable[0] 108 | 109 | @threads(N, timeout=0.1) 110 | def side_effects_timeout(): 111 | time.sleep(1) 112 | 113 | result = side_effects_timeout() 114 | with self.assertRaises(TimeoutError): 115 | result._wait() 116 | 117 | 118 | if __name__ == "__main__": 119 | unittest.main() 120 | -------------------------------------------------------------------------------- /tomorrow/__init__.py: -------------------------------------------------------------------------------- 1 | from .tomorrow import threads 2 | -------------------------------------------------------------------------------- /tomorrow/tomorrow.py: -------------------------------------------------------------------------------- 1 | from functools import wraps 2 | 3 | from concurrent.futures import ThreadPoolExecutor 4 | 5 | 6 | class Tomorrow(): 7 | 8 | def __init__(self, future, timeout): 9 | self._future = future 10 | self._timeout = timeout 11 | 12 | def __getattr__(self, name): 13 | result = self._wait() 14 | return result.__getattribute__(name) 15 | 16 | @property 17 | def result(self): 18 | return self._wait() 19 | 20 | def __iter__(self): 21 | result = self._wait() 22 | return result.__iter__() 23 | 24 | def _wait(self): 25 | return self._future.result(self._timeout) 26 | 27 | 28 | def async_(n, base_type, timeout=None): 29 | def decorator(f): 30 | if isinstance(n, int): 31 | pool = base_type(n) 32 | elif isinstance(n, base_type): 33 | pool = n 34 | else: 35 | raise TypeError( 36 | "Invalid type: %s" 37 | % type(base_type) 38 | ) 39 | @wraps(f) 40 | def wrapped(*args, **kwargs): 41 | return Tomorrow( 42 | pool.submit(f, *args, **kwargs), 43 | timeout=timeout 44 | ) 45 | return wrapped 46 | return decorator 47 | 48 | 49 | def threads(n, timeout=None): 50 | return async_(n, ThreadPoolExecutor, timeout) 51 | --------------------------------------------------------------------------------