├── setup.py ├── .gitignore ├── README.md ├── test.py ├── sharearray.py └── LICENSE /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | 4 | from codecs import open 5 | from os import path 6 | 7 | here = path.abspath(path.dirname(__file__)) 8 | with open(path.join(here, 'README.md'), encoding='utf-8') as f: 9 | long_description = f.read() 10 | 11 | setup( 12 | name='sharearray', 13 | version='0.1', 14 | py_modules=['sharearray'], 15 | 16 | url='https://github.com/bshillingford/python-sharearray', 17 | description=("Share numpy arrays across processes efficiently " 18 | "(ideal for large, read-only datasets)"), 19 | long_description=long_description, 20 | 21 | author='Brendan Shillingford', 22 | license='Apache Software License 2.0', 23 | install_requires=['numpy'], 24 | 25 | classifiers=[ 26 | 'Development Status :: 4 - Beta', 27 | 'Intended Audience :: Developers', 28 | 'License :: OSI Approved :: Apache Software License', 29 | 30 | 'Programming Language :: Python :: 2', 31 | 'Programming Language :: Python :: 2.7', 32 | 'Programming Language :: Python :: 3', 33 | 'Programming Language :: Python :: 3.3', 34 | 'Programming Language :: Python :: 3.4', 35 | 'Programming Language :: Python :: 3.5', 36 | ], 37 | ) 38 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # OSX useful to ignore 7 | *.DS_Store 8 | .AppleDouble 9 | .LSOverride 10 | 11 | # Thumbnails 12 | ._* 13 | 14 | # Files that might appear in the root of a volume 15 | .DocumentRevisions-V100 16 | .fseventsd 17 | .Spotlight-V100 18 | .TemporaryItems 19 | .Trashes 20 | .VolumeIcon.icns 21 | .com.apple.timemachine.donotpresent 22 | 23 | # Directories potentially created on remote AFP share 24 | .AppleDB 25 | .AppleDesktop 26 | Network Trash Folder 27 | Temporary Items 28 | .apdisk 29 | 30 | # C extensions 31 | *.so 32 | 33 | # Distribution / packaging 34 | .Python 35 | env/ 36 | build/ 37 | develop-eggs/ 38 | dist/ 39 | downloads/ 40 | eggs/ 41 | .eggs/ 42 | lib/ 43 | lib64/ 44 | parts/ 45 | sdist/ 46 | var/ 47 | *.egg-info/ 48 | .installed.cfg 49 | *.egg 50 | 51 | # PyInstaller 52 | # Usually these files are written by a python script from a template 53 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 54 | *.manifest 55 | *.spec 56 | 57 | # Installer logs 58 | pip-log.txt 59 | pip-delete-this-directory.txt 60 | 61 | # Unit test / coverage reports 62 | htmlcov/ 63 | .tox/ 64 | .coverage 65 | .coverage.* 66 | .cache 67 | nosetests.xml 68 | coverage.xml 69 | *,cover 70 | .hypothesis/ 71 | 72 | # Translations 73 | *.mo 74 | *.pot 75 | 76 | # Django stuff: 77 | *.log 78 | 79 | # Sphinx documentation 80 | docs/_build/ 81 | 82 | # IntelliJ Idea family of suites 83 | .idea 84 | *.iml 85 | ## File-based project format: 86 | *.ipr 87 | *.iws 88 | ## mpeltonen/sbt-idea plugin 89 | .idea_modules/ 90 | 91 | # PyBuilder 92 | target/ 93 | 94 | # Cookiecutter 95 | output/ 96 | python_boilerplate/ 97 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # sharearray 2 | Have you worried about creating large identical numpy arrays across processes due to RAM wastage, e.g. datasets that are big enough to fit in RAM but large enough to cause concern when running multiple jobs using the same data? 3 | `sharearray` efficiently caches numpy arrays in RAM (using shared memory in `/dev/shm`, no root needed) locally on a machine. 4 | 5 | Usage is simple, using the `cache` function or `decorator` decorator. 6 | A first call saves the result of the call into the built-in RAM disk, and 7 | returns a read-only memory-mapped view into it. 8 | Since it's in RAM, there's no performance penalty. 9 | Any subsequent calls with the same ID will return an identical read-only memory mapped view, 10 | even across processes. The IDs are **global**. 11 | 12 | Installation: 13 | ``` 14 | pip install git+https://github.com/bshillingford/python-sharearray 15 | ``` 16 | or 17 | ``` 18 | git clone https://github.com/bshillingford/python-sharearray 19 | python setup.py install 20 | ``` 21 | 22 | ## Usage 23 | ### Using `decorator`: 24 | ```python 25 | @sharearray.decorator('some_unique_id', verbose=False) 26 | def get_training_data(): 27 | # create largeish / expensive-to-generate data 28 | return my_array # some instance of np.ndarray 29 | 30 | # first call, across all processes, creates the array 31 | arr_view = get_training_data() 32 | 33 | # all further calls are cached/memoized: we return a view into memory 34 | arr_view_2 = get_training_data() 35 | ``` 36 | 37 | ### Using the `cache` function: 38 | ```python 39 | import sharearray 40 | import numpy as np 41 | arr = sharearray.cache('my_global_id', lambda: create_large_array()) 42 | # or: 43 | arr = sharearray.cache('my_global_id', lambda: create_large_array()) 44 | ``` 45 | where, for instance, `create_large_array` returns a large training set, potentially performing expensive feature transformations or data augmentations first. 46 | 47 | By default, the file is at `/dev/shm/sharearray_my_global_id.npy`, and to avoid concurrency 48 | issues when first generating the array, and to avoid duplicated computation, 49 | 50 | For futher details, read the docstrings. You may be interested in the `timeout`, `verbose`, and `log_func` arguments (to either `cache` or `decorator`). 51 | 52 | ### PyTorch 53 | Since PyTorch does not yet support memmapped files (at time of writing), we can instead just create torch Tensors that point to the memory mapped by numpy: 54 | ```python 55 | data_numpy = get_training_data() # numpy.ndarray 56 | data_torch = torch.from_numpy(data_numpy) # torch.Tensor 57 | ``` 58 | 59 | ## Notes 60 | TODO: support returning multiple arrays (e.g. as a tuple or dict) from the callback / decorated function 61 | 62 | There exist similar libraries in Python already, but this just makes it easier to do as a memoization-style API. Also, this module is a single file, and does not write anything in C. 63 | -------------------------------------------------------------------------------- /test.py: -------------------------------------------------------------------------------- 1 | # Copyright 2017 Brendan Shillingford 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | import unittest 16 | import os.path 17 | 18 | import numpy as np 19 | 20 | import sharearray 21 | 22 | 23 | class TestCache(unittest.TestCase): 24 | def test_simple(self): 25 | identifier = 'testcache_simple' 26 | arr = np.random.normal(size=(5, 6)) 27 | 28 | # create from scratch: 29 | arr1 = sharearray.cache(identifier, 30 | array_or_callback=arr, 31 | timeout=-1, 32 | verbose=True) 33 | self.assertTrue((arr1 == arr).all()) 34 | # opens view 35 | arr2 = sharearray.cache(identifier, 36 | array_or_callback=arr, 37 | timeout=-1, 38 | verbose=True) 39 | self.assertTrue((arr2 == arr).all()) 40 | 41 | # close views: 42 | del arr1 43 | del arr2 44 | 45 | # free memory: 46 | sharearray.free(identifier) 47 | 48 | def test_callback(self): 49 | identifier = 'testcache_callback' 50 | 51 | # ensure already freed: 52 | sharearray.free(identifier) 53 | fn, fn_lock = sharearray._build_path(identifier, 54 | shm_path='/dev/shm', 55 | prefix='sharearray_') 56 | self.assertTrue(not os.path.exists(fn) and not os.path.exists(fn_lock)) 57 | 58 | # create from scratch or open view 59 | arr = sharearray.cache(identifier, 60 | array_or_callback=lambda: np.ones((5, 6)), 61 | timeout=-1, 62 | verbose=True) 63 | self.assertTrue(os.path.exists(fn) and not os.path.exists(fn_lock)) 64 | del arr 65 | # should still exist after removing memmapped view 66 | self.assertTrue(os.path.exists(fn) and not os.path.exists(fn_lock)) 67 | 68 | # free memory: 69 | sharearray.free(identifier) 70 | self.assertTrue(not os.path.exists(fn) and not os.path.exists(fn_lock)) 71 | 72 | def test_decorator(self): 73 | identifier = 'testcache_decorator' 74 | 75 | @sharearray.decorator(identifier, verbose=False) 76 | def data(): 77 | return np.ones((5, 6)) 78 | 79 | # ensure memory is freed; neither lock nor file should exist 80 | sharearray.free(identifier) 81 | fn, fn_lock = sharearray._build_path(identifier, 82 | shm_path='/dev/shm', 83 | prefix='sharearray_') 84 | self.assertTrue(not os.path.exists(fn) and not os.path.exists(fn_lock)) 85 | # 1st call should create item, and lock should be gone 86 | data1 = data() 87 | self.assertTrue((data1 == 1).all()) 88 | self.assertTrue(os.path.exists(fn) and not os.path.exists(fn_lock)) 89 | 90 | # 2nd call should just view 91 | data2 = data() 92 | self.assertTrue((data2 == 1).all()) 93 | self.assertTrue(os.path.exists(fn) and not os.path.exists(fn_lock)) 94 | 95 | # free memory: 96 | sharearray.free(identifier) 97 | self.assertTrue(not os.path.exists(fn) and not os.path.exists(fn_lock)) 98 | 99 | if __name__ == '__main__': 100 | unittest.main() 101 | 102 | # TODO: test exceptions? 103 | -------------------------------------------------------------------------------- /sharearray.py: -------------------------------------------------------------------------------- 1 | # Copyright 2017 Brendan Shillingford 2 | # 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | import functools 16 | import os 17 | import os.path 18 | import re 19 | import time 20 | import sys 21 | 22 | import numpy as np 23 | 24 | __all__ = ['cache', 'decorator', 'valid_id', 'TimeoutException'] 25 | 26 | 27 | if sys.version_info[0] == 2: 28 | FileExistsError = OSError 29 | 30 | 31 | class TimeoutException(Exception): 32 | pass 33 | 34 | 35 | _ID_REGEX = re.compile(r"^[A-Za-z0-9=+._-]+$") 36 | 37 | 38 | def valid_id(id): 39 | if _ID_REGEX.match(id): 40 | return True 41 | return False 42 | 43 | 44 | def _memmapped_view(filename): 45 | return np.lib.format.open_memmap(filename, mode='r') 46 | 47 | 48 | def _build_path(id, prefix, shm_path): 49 | fn = os.path.join(shm_path, prefix + id + '.npy') 50 | fn_lock = fn + '.lock' 51 | return fn, fn_lock 52 | 53 | 54 | def free(id, shm_path='/dev/shm', prefix='sharearray_'): 55 | fn, fn_lock = _build_path(id, prefix=prefix, shm_path=shm_path) 56 | fn_exists = os.path.exists(fn) 57 | fn_lock_exists = os.path.exists(fn_lock) 58 | 59 | if fn_lock_exists: 60 | import warnings 61 | warnings.warn("lock still exists") 62 | os.unlink(fn_lock) 63 | 64 | if fn_exists: 65 | os.unlink(fn) 66 | 67 | 68 | def cache(id, array_or_callback, 69 | shm_path='/dev/shm', 70 | prefix='sharearray_', 71 | timeout=-1, 72 | verbose=True, 73 | log_func=None): 74 | """ 75 | Stores a `numpy` `ndarray` into shared memory, caching subsequent requests 76 | (globally, across all processes) to the function so they point to the same 77 | memory. 78 | 79 | By default, does this be creating a file at `/dev/shm/shareddataset_`. 80 | If: 81 | 82 | 1. The file is not created yet, saves `array_or_callback` to the path 83 | listed above (see NOTE 1). Then, returns a read-only memmapped view to 84 | this numpy array. 85 | 86 | 2. The file is already created. We return a read-only memmapped view of it. 87 | 88 | Args: 89 | id (str): identifier for shared array, global across system. 90 | Must match `[A-Za-z0-9=+._-]+`. You may want to include your 91 | program's name to prevent name collisions. 92 | array_or_callback: either a `numpy.ndarray` containing value types, or 93 | a callback function taking no arguments that returns one. 94 | shm_path (str, optional): path to the Linux shared memory 95 | tmpfs mountpoint. In almost all kernel builds, one lives at 96 | `/dev/shm` with size defaulting to half the RAM; sometimes it's a 97 | symlink to `/run/shm`. 98 | timeout (int, optional): number of seconds to wait before timing out 99 | waiting for lock to be released, when file is already being created. 100 | If -1, waits indefinitely. 101 | prefix (str, optional): prefix added to files in `shm_path`. 102 | verbose (bool): if True, prints useful information (2-3 lines). 103 | log_func (callable): if verbose is True, this is used if specified. 104 | Else just uses `print`. 105 | 106 | Returns: 107 | A `numpy.ndarray` read-only view into the shared memory, whether it 108 | was newly created or previously created. 109 | 110 | Raises: 111 | ValueError: `id` is not a valid identifier (must match 112 | `[A-Za-z0-9=+._-]+`), or `array_or_callback` is not a callback 113 | or returns 114 | TimeoutException: if `timeout` is positive, the lock file exists, and 115 | we have waited at least `timeout` seconds yet the lock still exists. 116 | 117 | Notes: 118 | 119 | NOTE 1: For concurrency safety, this function creates a lock file at 120 | `/dev/shm/shareddataset_.lock` when initially writing the file 121 | (lock file is empty, doesn't contain PID). File creating is hence 122 | checked using the lock, rather than the file's existence itself. We 123 | don't use the standard create-rename method of ensuring atomicity, 124 | since `array_or_callback` may be expensive to call or large to write. 125 | 126 | NOTE 2: `id`s are currently global to the system. Include your program's 127 | name to prevent name collisions. 128 | 129 | NOTE 3: memmapped views are created using 130 | `numpy.lib.format.open_memmap`. 131 | 132 | NOTE 4: If the array is very large, to save memory, you may want to 133 | immediately remove all references to the original array, then do a full 134 | garbage collection (`import gc; gc.collect()`). 135 | 136 | Examples: 137 | 138 | An expensive operation (e.g. data preprocessing) that is the same 139 | across all running instances of this program: 140 | 141 | x_times_y = cache("myprog_x_times_y", lambda: np.dot(x, y)) 142 | 143 | A large (large enough to warrant concern, but small enough to fit in 144 | RAM once) training set that we only want one instance of across 145 | many training jobs: 146 | 147 | def load_training_set(): 148 | # load and/or preprocess training_set once here 149 | return training_set 150 | training_set = cache("myprog_training_set", load_training_set) 151 | 152 | Only passing a callback to array_or_callback makes sense here, 153 | of course. 154 | """ 155 | if not valid_id(id): 156 | raise ValueError('invalid id: ' + id) 157 | 158 | if not (hasattr(array_or_callback, '__call__') 159 | or isinstance(array_or_callback, np.ndarray)): 160 | raise ValueError( 161 | 'array_or_callback should be ndarray or zero-argument callable') 162 | 163 | if verbose and log_func: 164 | print_ = log_func 165 | elif verbose: 166 | def print_(s): 167 | print(s) 168 | else: 169 | def print_(s): 170 | pass 171 | 172 | fn, fn_lock = _build_path(id, prefix=prefix, shm_path=shm_path) 173 | fd_lock = -1 174 | try: 175 | fd_lock = os.open(fn_lock, os.O_CREAT | os.O_EXCL) 176 | if fd_lock < 0: 177 | raise OSError("Lock open failure (bug?)", fn_lock, fd_lock) 178 | 179 | except FileExistsError: 180 | if timeout < 0: 181 | print_(("'{}' is being created by another process. " 182 | "Waiting indefinitely... (timeout < 0)").format(id)) 183 | 184 | while os.path.exists(fn_lock): 185 | time.sleep(1) 186 | 187 | else: 188 | print_(("'{}' is being created by another process. " 189 | "Waiting up to {} seconds...").format(id, timeout)) 190 | 191 | for _ in range(timeout): 192 | time.sleep(1) 193 | if not os.path.exists(fn_lock): 194 | break 195 | else: 196 | raise TimeoutException( 197 | "timed out waiting for %s to unlock (be created)" % id) 198 | else: 199 | if not os.path.exists(fn): 200 | print_("'%s' doesn't exist yet. Locking and creating..." % id) 201 | 202 | if isinstance(array_or_callback, np.ndarray): 203 | array = array_or_callback 204 | else: 205 | array = array_or_callback() 206 | if not isinstance(array, np.ndarray): 207 | raise ValueError( 208 | 'callback did not return a numpy.ndarray, returned:', 209 | type(array)) 210 | 211 | np.save(fn, array, allow_pickle=False) 212 | print_("'%s': written." % id) 213 | 214 | finally: 215 | if fd_lock > 0: 216 | os.close(fd_lock) 217 | os.unlink(fn_lock) 218 | 219 | print_("'%s': returning memmapped view." % id) 220 | return _memmapped_view(fn) 221 | 222 | 223 | def decorator(id, **kwargs): 224 | """ 225 | Decorator version of `cache`, analogous to a memoization decorator. 226 | 227 | Besides `array_or_callback` which isn't needed, arguments are identical to 228 | those of `cache`, see there for docs. They must be passed as keyword args 229 | except for `id`. 230 | 231 | Note that `id` can't depend on the arguments to the decorated function. 232 | For that, use `cache` directly. 233 | 234 | Example: 235 | Alternative to callback syntax above. 236 | 237 | @decorator("my_large_array") 238 | def foo(): 239 | # ...do some expensive computation to generate arr... 240 | return arr 241 | 242 | arr = foo() # first call, in shared memory arr global to system 243 | arr2 = foo() # here or another script, returns read-only view 244 | """ 245 | if not valid_id(id): 246 | raise ValueError('invalid id: ' + id) 247 | 248 | def decorate(f): 249 | @functools.wraps(f) 250 | def wrapped(): 251 | return cache(id, f, **kwargs) 252 | 253 | return wrapped 254 | 255 | return decorate 256 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "[]" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright [yyyy] [name of copyright owner] 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. 203 | --------------------------------------------------------------------------------