├── .build.yml ├── .dockerignore ├── .gitignore ├── AUTHORS.rst ├── CONTRIBUTING.rst ├── Dockerfile ├── HISTORY.rst ├── LICENSE ├── MANIFEST.in ├── README.rst ├── docker-compose.yml ├── docs ├── Makefile ├── authors.rst ├── conf.py ├── contributing.rst ├── history.rst ├── index.rst ├── installation.rst ├── make.bat ├── readme.rst └── usage.rst ├── etcd3 ├── __init__.py ├── client.py ├── etcdrpc │ ├── __init__.py │ ├── auth_pb2.py │ ├── kv_pb2.py │ ├── rpc_pb2.py │ └── rpc_pb2_grpc.py ├── events.py ├── exceptions.py ├── leases.py ├── locks.py ├── members.py ├── proto │ ├── auth.proto │ ├── kv.proto │ └── rpc.proto ├── transactions.py ├── utils.py └── watch.py ├── requirements ├── base.in ├── base.txt ├── test.in └── test.txt ├── setup.cfg ├── setup.py ├── tests ├── __init__.py ├── ca.crt ├── client.crt ├── client.key └── test_etcd3.py └── tox.ini /.build.yml: -------------------------------------------------------------------------------- 1 | image: archlinux 2 | packages: 3 | - python 4 | - python-pip 5 | - python39 6 | sources: 7 | - https://github.com/kragniz/python-etcd3 8 | environment: 9 | TEST_ETCD_VERSION: v3.3.10 10 | tasks: 11 | - test: | 12 | cd python-etcd3 13 | pip install --progress-bar off -U tox 14 | curl -L https://github.com/coreos/etcd/releases/download/$TEST_ETCD_VERSION/etcd-$TEST_ETCD_VERSION-linux-amd64.tar.gz -o etcd-$TEST_ETCD_VERSION-linux-amd64.tar.gz 15 | tar xzvf etcd-$TEST_ETCD_VERSION-linux-amd64.tar.gz 16 | export PATH=$PATH:~/.local/bin:etcd-$TEST_ETCD_VERSION-linux-amd64 17 | 18 | tox 19 | -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | .direnv/ 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *,cover 47 | .hypothesis/ 48 | 49 | # Translations 50 | *.mo 51 | *.pot 52 | 53 | # Django stuff: 54 | *.log 55 | 56 | # Sphinx documentation 57 | docs/_build/ 58 | 59 | # PyBuilder 60 | target/ 61 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | env/ 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | 27 | # PyInstaller 28 | # Usually these files are written by a python script from a template 29 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 30 | *.manifest 31 | *.spec 32 | 33 | # Installer logs 34 | pip-log.txt 35 | pip-delete-this-directory.txt 36 | 37 | # Unit test / coverage reports 38 | htmlcov/ 39 | .tox/ 40 | .coverage 41 | .coverage.* 42 | .cache 43 | nosetests.xml 44 | coverage.xml 45 | *,cover 46 | .hypothesis/ 47 | 48 | # Translations 49 | *.mo 50 | *.pot 51 | 52 | # Django stuff: 53 | *.log 54 | 55 | # Sphinx documentation 56 | docs/_build/ 57 | 58 | # PyBuilder 59 | target/ 60 | 61 | # Editor files 62 | .sw? 63 | .idea/ 64 | -------------------------------------------------------------------------------- /AUTHORS.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | Credits 3 | ======= 4 | 5 | Development Lead 6 | ---------------- 7 | 8 | * Louis Taylor 9 | 10 | Contributors 11 | ------------ 12 | 13 | * Giuseppe Lavagetto 14 | * InvalidInterrupt 15 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | .. highlight:: shell 2 | 3 | ============ 4 | Contributing 5 | ============ 6 | 7 | Contributions are welcome, and they are greatly appreciated! Every 8 | little bit helps, and credit will always be given. 9 | 10 | You can contribute in many ways: 11 | 12 | Types of Contributions 13 | ---------------------- 14 | 15 | Report Bugs 16 | ~~~~~~~~~~~ 17 | 18 | Report bugs at https://github.com/kragniz/python-etcd3/issues. 19 | 20 | If you are reporting a bug, please include: 21 | 22 | * Your operating system name and version. 23 | * Any details about your local setup that might be helpful in troubleshooting. 24 | * Detailed steps to reproduce the bug. 25 | 26 | Fix Bugs 27 | ~~~~~~~~ 28 | 29 | Look through the GitHub issues for bugs. Anything tagged with "bug" 30 | and "help wanted" is open to whoever wants to implement it. 31 | 32 | Implement Features 33 | ~~~~~~~~~~~~~~~~~~ 34 | 35 | Look through the GitHub issues for features. Anything tagged with "enhancement" 36 | and "help wanted" is open to whoever wants to implement it. 37 | 38 | Write Documentation 39 | ~~~~~~~~~~~~~~~~~~~ 40 | 41 | python-etcd3 could always use more documentation, whether as part of the 42 | official python-etcd3 docs, in docstrings, or even on the web in blog posts, 43 | articles, and such. 44 | 45 | Submit Feedback 46 | ~~~~~~~~~~~~~~~ 47 | 48 | The best way to send feedback is to file an issue at https://github.com/kragniz/python-etcd3/issues. 49 | 50 | If you are proposing a feature: 51 | 52 | * Explain in detail how it would work. 53 | * Keep the scope as narrow as possible, to make it easier to implement. 54 | * Remember that this is a volunteer-driven project, and that contributions 55 | are welcome :) 56 | 57 | Get Started! 58 | ------------ 59 | 60 | Ready to contribute? Here's how to set up `python-etcd3` for local development. 61 | 62 | 1. Fork the `python-etcd3` repo on GitHub. 63 | 2. Clone your fork locally:: 64 | 65 | $ git clone git@github.com:your_name_here/python-etcd3.git 66 | 67 | 3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:: 68 | 69 | $ mkvirtualenv etcd3 70 | $ cd etcd3/ 71 | $ python setup.py develop 72 | 73 | 4. Create a branch for local development:: 74 | 75 | $ git checkout -b name-of-your-bugfix-or-feature 76 | 77 | Now you can make your changes locally. 78 | 79 | 5. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:: 80 | 81 | $ flake8 etcd3 tests 82 | $ python setup.py test or py.test 83 | $ tox 84 | 85 | To get flake8 and tox, just pip install them into your virtualenv. 86 | 87 | 6. Commit your changes and push your branch to GitHub:: 88 | 89 | $ git add . 90 | $ git commit -m "Your detailed description of your changes." 91 | $ git push origin name-of-your-bugfix-or-feature 92 | 93 | 7. Submit a pull request through the GitHub website. 94 | 95 | Pull Request Guidelines 96 | ----------------------- 97 | 98 | Before you submit a pull request, check that it meets these guidelines: 99 | 100 | 1. The pull request should include tests. 101 | 2. If the pull request adds functionality, the docs should be updated. Put 102 | your new functionality into a function with a docstring, and add the 103 | feature to the list in README.rst. 104 | 3. The pull request should work for Python 2.6, 2.7, 3.3, 3.4 and 3.5, and for PyPy. Check 105 | https://travis-ci.org/kragniz/python-etcd3/pull_requests 106 | and make sure that the tests pass for all supported Python versions. 107 | 108 | Generating protobuf stubs 109 | ------------------------- 110 | 111 | If the upstream protobuf files changes, copy the stubs:: 112 | 113 | $ cp etcd/etcdserver/etcdserverpb/*.proto python-etcd3/etcd3/proto/ 114 | 115 | Then:: 116 | 117 | $ cd python-etcd3 118 | $ tox -e genproto 119 | 120 | 121 | Cutting new releases 122 | -------------------- 123 | 124 | The release process to PyPi is automated using travis deploys and bumpversion. 125 | 126 | 1. Check changes since the last release: 127 | 128 | .. code-block:: bash 129 | 130 | $ git log $(git describe --tags --abbrev=0)..HEAD --oneline 131 | 132 | 2. Bump the version (respecting semver, one of ``major``, ``minor`` or 133 | ``patch``): 134 | 135 | .. code-block:: bash 136 | 137 | $ bumpversion patch 138 | 139 | 3. Push to github: 140 | 141 | .. code-block:: bash 142 | 143 | $ git push 144 | $ git push --tags 145 | 146 | 4. Wait for travis tests to run and deploy to PyPI 147 | 148 | 149 | Dependency management 150 | --------------------- 151 | 152 | This project uses ``pip-compile-multi`` (https://pypi.org/project/pip-compile-multi/) for hard-pinning dependencies versions. 153 | Please see its documentation for usage instructions. 154 | In short, ``requirements/base.in`` contains the list of direct requirements with occasional version constraints (like ``Django<2``) 155 | and `requirements/base.txt` is automatically generated from it by adding recursive tree of dependencies with fixed versions. 156 | The same goes for ``test`` and ``dev``. 157 | 158 | To upgrade dependency versions, run ``pip-compile-multi``. 159 | 160 | To add a new dependency without upgrade, add it to `requirements/base.in` and run `pip-compile-multi --no-upgrade`. 161 | 162 | For installation always use ``.txt`` files. For example, command ``pip install -Ue . -r requirements/dev.txt`` will install 163 | this project in development mode, testing requirements and development tools. 164 | Another useful command is ``pip-sync requirements/dev.txt``, it uninstalls packages from your virtualenv that aren't listed in the file. 165 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM python:3.9 2 | 3 | ARG HTTP_PROXY 4 | ARG http_proxy 5 | ARG HTTPS_PROXY 6 | ARG https_proxy 7 | ENV TEST_ETCD_VERSION v3.3.10 8 | 9 | RUN curl -L https://github.com/etcd-io/etcd/releases/download/${TEST_ETCD_VERSION}/etcd-${TEST_ETCD_VERSION}-linux-amd64.tar.gz | tar xzvf - 10 | ENV PATH $PATH:/etcd-${TEST_ETCD_VERSION}-linux-amd64 11 | 12 | RUN pip install -U tox 13 | 14 | RUN mkdir python-etcd3 15 | WORKDIR python-etcd3 16 | # Rebuild this layer .tox when tox.ini or requirements changes 17 | COPY tox.ini ./ 18 | COPY requirements/base.txt requirements/test.txt ./requirements/ 19 | 20 | RUN tox -epy39 --notest 21 | 22 | COPY . ./ 23 | 24 | ENV ETCDCTL_API 3 25 | CMD ["tox", "-epy39"] 26 | -------------------------------------------------------------------------------- /HISTORY.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | History 3 | ======= 4 | 5 | 0.1.0 (2016-09-30) 6 | ------------------ 7 | 8 | * First release on PyPI. 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | 2 | include AUTHORS.rst 3 | 4 | include CONTRIBUTING.rst 5 | include HISTORY.rst 6 | include LICENSE 7 | include README.rst 8 | include requirements/base.txt 9 | include requirements/test.txt 10 | 11 | recursive-include tests * 12 | recursive-exclude * __pycache__ 13 | recursive-exclude * *.py[co] 14 | 15 | recursive-include docs *.rst conf.py Makefile make.bat *.jpg *.png *.gif 16 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | python-etcd3 3 | ============ 4 | 5 | 6 | .. image:: https://img.shields.io/pypi/v/etcd3.svg 7 | :target: https://pypi.python.org/pypi/etcd3 8 | 9 | .. image:: https://img.shields.io/travis/kragniz/python-etcd3.svg 10 | :target: https://travis-ci.org/kragniz/python-etcd3 11 | 12 | .. image:: https://readthedocs.org/projects/python-etcd3/badge/?version=latest 13 | :target: https://python-etcd3.readthedocs.io/en/latest/?badge=latest 14 | :alt: Documentation Status 15 | 16 | .. image:: https://pyup.io/repos/github/kragniz/python-etcd3/shield.svg 17 | :target: https://pyup.io/repos/github/kragniz/python-etcd3/ 18 | :alt: Updates 19 | 20 | .. image:: https://codecov.io/github/kragniz/python-etcd3/coverage.svg?branch=master 21 | :target: https://codecov.io/github/kragniz/python-etcd3?branch=master 22 | 23 | 24 | Python client for the etcd API v3, supported under python 2.7, 3.4 and 3.5. 25 | 26 | **Warning: the API is mostly stable, but may change in the future** 27 | 28 | If you're interested in using this library, please get involved. 29 | 30 | * Free software: Apache Software License 2.0 31 | * Documentation: https://python-etcd3.readthedocs.io. 32 | 33 | Basic usage: 34 | 35 | .. code-block:: python 36 | 37 | import etcd3 38 | 39 | etcd = etcd3.client() 40 | 41 | etcd.get('foo') 42 | etcd.put('bar', 'doot') 43 | etcd.delete('bar') 44 | 45 | # locks 46 | lock = etcd.lock('thing') 47 | lock.acquire() 48 | # do something 49 | lock.release() 50 | 51 | with etcd.lock('doot-machine') as lock: 52 | # do something 53 | 54 | # transactions 55 | etcd.transaction( 56 | compare=[ 57 | etcd.transactions.value('/doot/testing') == 'doot', 58 | etcd.transactions.version('/doot/testing') > 0, 59 | ], 60 | success=[ 61 | etcd.transactions.put('/doot/testing', 'success'), 62 | ], 63 | failure=[ 64 | etcd.transactions.put('/doot/testing', 'failure'), 65 | ] 66 | ) 67 | 68 | # watch key 69 | watch_count = 0 70 | events_iterator, cancel = etcd.watch("/doot/watch") 71 | for event in events_iterator: 72 | print(event) 73 | watch_count += 1 74 | if watch_count > 10: 75 | cancel() 76 | 77 | # watch prefix 78 | watch_count = 0 79 | events_iterator, cancel = etcd.watch_prefix("/doot/watch/prefix/") 80 | for event in events_iterator: 81 | print(event) 82 | watch_count += 1 83 | if watch_count > 10: 84 | cancel() 85 | 86 | # recieve watch events via callback function 87 | def watch_callback(event): 88 | print(event) 89 | 90 | watch_id = etcd.add_watch_callback("/anotherkey", watch_callback) 91 | 92 | # cancel watch 93 | etcd.cancel_watch(watch_id) 94 | 95 | # recieve watch events for a prefix via callback function 96 | def watch_callback(event): 97 | print(event) 98 | 99 | watch_id = etcd.add_watch_prefix_callback("/doot/watch/prefix/", watch_callback) 100 | 101 | # cancel watch 102 | etcd.cancel_watch(watch_id) 103 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '2' 2 | services: 3 | python: 4 | build: 5 | context: . 6 | args: 7 | - HTTP_PROXY 8 | - http_proxy 9 | - HTTPS_PROXY 10 | - https_proxy 11 | links: 12 | - etcd 13 | environment: 14 | - PYTHON_ETCD_HTTP_URL=http://etcd:2379 15 | etcd: 16 | ports: 17 | - "2379:2379" 18 | image: quay.io/coreos/etcd 19 | command: etcd --initial-cluster-state new --listen-client-urls http://0.0.0.0:2379 --advertise-client-urls http://127.0.0.1:2379 20 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # User-friendly check for sphinx-build 11 | ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) 12 | $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) 13 | endif 14 | 15 | # Internal variables. 16 | PAPEROPT_a4 = -D latex_paper_size=a4 17 | PAPEROPT_letter = -D latex_paper_size=letter 18 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 19 | # the i18n builder cannot share the environment and doctrees with the others 20 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 21 | 22 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 23 | 24 | help: 25 | @echo "Please use \`make ' where is one of" 26 | @echo " html to make standalone HTML files" 27 | @echo " dirhtml to make HTML files named index.html in directories" 28 | @echo " singlehtml to make a single large HTML file" 29 | @echo " pickle to make pickle files" 30 | @echo " json to make JSON files" 31 | @echo " htmlhelp to make HTML files and a HTML help project" 32 | @echo " qthelp to make HTML files and a qthelp project" 33 | @echo " devhelp to make HTML files and a Devhelp project" 34 | @echo " epub to make an epub" 35 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 36 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 37 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 38 | @echo " text to make text files" 39 | @echo " man to make manual pages" 40 | @echo " texinfo to make Texinfo files" 41 | @echo " info to make Texinfo files and run them through makeinfo" 42 | @echo " gettext to make PO message catalogs" 43 | @echo " changes to make an overview of all changed/added/deprecated items" 44 | @echo " xml to make Docutils-native XML files" 45 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 46 | @echo " linkcheck to check all external links for integrity" 47 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 48 | 49 | clean: 50 | rm -rf $(BUILDDIR)/* 51 | 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | dirhtml: 58 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 59 | @echo 60 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 61 | 62 | singlehtml: 63 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 64 | @echo 65 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 66 | 67 | pickle: 68 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 69 | @echo 70 | @echo "Build finished; now you can process the pickle files." 71 | 72 | json: 73 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 74 | @echo 75 | @echo "Build finished; now you can process the JSON files." 76 | 77 | htmlhelp: 78 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 79 | @echo 80 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 81 | ".hhp project file in $(BUILDDIR)/htmlhelp." 82 | 83 | qthelp: 84 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 85 | @echo 86 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 87 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 88 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/etcd3.qhcp" 89 | @echo "To view the help file:" 90 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/etcd3.qhc" 91 | 92 | devhelp: 93 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 94 | @echo 95 | @echo "Build finished." 96 | @echo "To view the help file:" 97 | @echo "# mkdir -p $$HOME/.local/share/devhelp/etcd3" 98 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/etcd3" 99 | @echo "# devhelp" 100 | 101 | epub: 102 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 103 | @echo 104 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 105 | 106 | latex: 107 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 108 | @echo 109 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 110 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 111 | "(use \`make latexpdf' here to do that automatically)." 112 | 113 | latexpdf: 114 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 115 | @echo "Running LaTeX files through pdflatex..." 116 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 117 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 118 | 119 | latexpdfja: 120 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 121 | @echo "Running LaTeX files through platex and dvipdfmx..." 122 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 123 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 124 | 125 | text: 126 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 127 | @echo 128 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 129 | 130 | man: 131 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 132 | @echo 133 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 134 | 135 | texinfo: 136 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 137 | @echo 138 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 139 | @echo "Run \`make' in that directory to run these through makeinfo" \ 140 | "(use \`make info' here to do that automatically)." 141 | 142 | info: 143 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 144 | @echo "Running Texinfo files through makeinfo..." 145 | make -C $(BUILDDIR)/texinfo info 146 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 147 | 148 | gettext: 149 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 150 | @echo 151 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 152 | 153 | changes: 154 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 155 | @echo 156 | @echo "The overview file is in $(BUILDDIR)/changes." 157 | 158 | linkcheck: 159 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 160 | @echo 161 | @echo "Link check complete; look for any errors in the above output " \ 162 | "or in $(BUILDDIR)/linkcheck/output.txt." 163 | 164 | doctest: 165 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 166 | @echo "Testing of doctests in the sources finished, look at the " \ 167 | "results in $(BUILDDIR)/doctest/output.txt." 168 | 169 | xml: 170 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 171 | @echo 172 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 173 | 174 | pseudoxml: 175 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 176 | @echo 177 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 178 | -------------------------------------------------------------------------------- /docs/authors.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../AUTHORS.rst 2 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # 4 | # etcd3 documentation build configuration file, created by 5 | # sphinx-quickstart on Tue Jul 9 22:26:36 2013. 6 | # 7 | # This file is execfile()d with the current directory set to its 8 | # containing dir. 9 | # 10 | # Note that not all possible configuration values are present in this 11 | # autogenerated file. 12 | # 13 | # All configuration values have a default; values that are commented out 14 | # serve to show the default. 15 | 16 | import functools 17 | import sys 18 | import os 19 | 20 | 21 | def no_op_wraps(func): 22 | """Replaces functools.wraps in order to undo wrapping. 23 | 24 | Can be used to preserve the decorated function's signature 25 | in the documentation generated by Sphinx. 26 | """ 27 | def wrapper(decorator): 28 | return func 29 | return wrapper 30 | 31 | 32 | functools.wraps = no_op_wraps 33 | 34 | # If extensions (or modules to document with autodoc) are in another 35 | # directory, add these directories to sys.path here. If the directory is 36 | # relative to the documentation root, use os.path.abspath to make it 37 | # absolute, like shown here. 38 | #sys.path.insert(0, os.path.abspath('.')) 39 | 40 | # Get the project root dir, which is the parent dir of this 41 | cwd = os.getcwd() 42 | project_root = os.path.dirname(cwd) 43 | 44 | # Insert the project root dir as the first element in the PYTHONPATH. 45 | # This lets us ensure that the source package is imported, and that its 46 | # version is used. 47 | sys.path.insert(0, project_root) 48 | 49 | import etcd3 50 | 51 | # -- General configuration --------------------------------------------- 52 | 53 | # If your documentation needs a minimal Sphinx version, state it here. 54 | #needs_sphinx = '1.0' 55 | 56 | # Add any Sphinx extension module names here, as strings. They can be 57 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 58 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] 59 | 60 | # Add any paths that contain templates here, relative to this directory. 61 | templates_path = ['_templates'] 62 | 63 | # The suffix of source filenames. 64 | source_suffix = '.rst' 65 | 66 | # The encoding of source files. 67 | #source_encoding = 'utf-8-sig' 68 | 69 | # The master toctree document. 70 | master_doc = 'index' 71 | 72 | # General information about the project. 73 | project = u'python-etcd3' 74 | copyright = u"2016, Louis Taylor" 75 | 76 | # The version info for the project you're documenting, acts as replacement 77 | # for |version| and |release|, also used in various other places throughout 78 | # the built documents. 79 | # 80 | # The short X.Y version. 81 | version = etcd3.__version__ 82 | # The full version, including alpha/beta/rc tags. 83 | release = etcd3.__version__ 84 | 85 | # The language for content autogenerated by Sphinx. Refer to documentation 86 | # for a list of supported languages. 87 | #language = None 88 | 89 | # There are two options for replacing |today|: either, you set today to 90 | # some non-false value, then it is used: 91 | #today = '' 92 | # Else, today_fmt is used as the format for a strftime call. 93 | #today_fmt = '%B %d, %Y' 94 | 95 | # List of patterns, relative to source directory, that match files and 96 | # directories to ignore when looking for source files. 97 | exclude_patterns = ['_build'] 98 | 99 | # The reST default role (used for this markup: `text`) to use for all 100 | # documents. 101 | #default_role = None 102 | 103 | # If true, '()' will be appended to :func: etc. cross-reference text. 104 | #add_function_parentheses = True 105 | 106 | # If true, the current module name will be prepended to all description 107 | # unit titles (such as .. function::). 108 | #add_module_names = True 109 | 110 | # If true, sectionauthor and moduleauthor directives will be shown in the 111 | # output. They are ignored by default. 112 | #show_authors = False 113 | 114 | # The name of the Pygments (syntax highlighting) style to use. 115 | pygments_style = 'sphinx' 116 | 117 | # A list of ignored prefixes for module index sorting. 118 | #modindex_common_prefix = [] 119 | 120 | # If true, keep warnings as "system message" paragraphs in the built 121 | # documents. 122 | #keep_warnings = False 123 | 124 | autodoc_member_order = 'bysource' 125 | 126 | 127 | # -- Options for HTML output ------------------------------------------- 128 | 129 | # The theme to use for HTML and HTML Help pages. See the documentation for 130 | # a list of builtin themes. 131 | html_theme = 'alabaster' 132 | 133 | # Theme options are theme-specific and customize the look and feel of a 134 | # theme further. For a list of options available for each theme, see the 135 | # documentation. 136 | #html_theme_options = {} 137 | 138 | # Add any paths that contain custom themes here, relative to this directory. 139 | #html_theme_path = [] 140 | 141 | # The name for this set of Sphinx documents. If None, it defaults to 142 | # " v documentation". 143 | #html_title = None 144 | 145 | # A shorter title for the navigation bar. Default is the same as 146 | # html_title. 147 | #html_short_title = None 148 | 149 | # The name of an image file (relative to this directory) to place at the 150 | # top of the sidebar. 151 | #html_logo = None 152 | 153 | # The name of an image file (within the static path) to use as favicon 154 | # of the docs. This file should be a Windows icon file (.ico) being 155 | # 16x16 or 32x32 pixels large. 156 | #html_favicon = None 157 | 158 | # Add any paths that contain custom static files (such as style sheets) 159 | # here, relative to this directory. They are copied after the builtin 160 | # static files, so a file named "default.css" will overwrite the builtin 161 | # "default.css". 162 | html_static_path = ['_static'] 163 | 164 | # If not '', a 'Last updated on:' timestamp is inserted at every page 165 | # bottom, using the given strftime format. 166 | #html_last_updated_fmt = '%b %d, %Y' 167 | 168 | # If true, SmartyPants will be used to convert quotes and dashes to 169 | # typographically correct entities. 170 | #html_use_smartypants = True 171 | 172 | # Custom sidebar templates, maps document names to template names. 173 | #html_sidebars = {} 174 | 175 | # Additional templates that should be rendered to pages, maps page names 176 | # to template names. 177 | #html_additional_pages = {} 178 | 179 | # If false, no module index is generated. 180 | #html_domain_indices = True 181 | 182 | # If false, no index is generated. 183 | #html_use_index = True 184 | 185 | # If true, the index is split into individual pages for each letter. 186 | #html_split_index = False 187 | 188 | # If true, links to the reST sources are added to the pages. 189 | #html_show_sourcelink = True 190 | 191 | # If true, "Created using Sphinx" is shown in the HTML footer. 192 | # Default is True. 193 | #html_show_sphinx = True 194 | 195 | # If true, "(C) Copyright ..." is shown in the HTML footer. 196 | # Default is True. 197 | #html_show_copyright = True 198 | 199 | # If true, an OpenSearch description file will be output, and all pages 200 | # will contain a tag referring to it. The value of this option 201 | # must be the base URL from which the finished HTML is served. 202 | #html_use_opensearch = '' 203 | 204 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 205 | #html_file_suffix = None 206 | 207 | # Output file base name for HTML help builder. 208 | htmlhelp_basename = 'etcd3doc' 209 | 210 | 211 | # -- Options for LaTeX output ------------------------------------------ 212 | 213 | latex_elements = { 214 | # The paper size ('letterpaper' or 'a4paper'). 215 | #'papersize': 'letterpaper', 216 | 217 | # The font size ('10pt', '11pt' or '12pt'). 218 | #'pointsize': '10pt', 219 | 220 | # Additional stuff for the LaTeX preamble. 221 | #'preamble': '', 222 | } 223 | 224 | # Grouping the document tree into LaTeX files. List of tuples 225 | # (source start file, target name, title, author, documentclass 226 | # [howto/manual]). 227 | latex_documents = [ 228 | ('index', 'etcd3.tex', 229 | u'python-etcd3 Documentation', 230 | u'Louis Taylor', 'manual'), 231 | ] 232 | 233 | # The name of an image file (relative to this directory) to place at 234 | # the top of the title page. 235 | #latex_logo = None 236 | 237 | # For "manual" documents, if this is true, then toplevel headings 238 | # are parts, not chapters. 239 | #latex_use_parts = False 240 | 241 | # If true, show page references after internal links. 242 | #latex_show_pagerefs = False 243 | 244 | # If true, show URL addresses after external links. 245 | #latex_show_urls = False 246 | 247 | # Documents to append as an appendix to all manuals. 248 | #latex_appendices = [] 249 | 250 | # If false, no module index is generated. 251 | #latex_domain_indices = True 252 | 253 | 254 | # -- Options for manual page output ------------------------------------ 255 | 256 | # One entry per manual page. List of tuples 257 | # (source start file, name, description, authors, manual section). 258 | man_pages = [ 259 | ('index', 'etcd3', 260 | u'python-etcd3 Documentation', 261 | [u'Louis Taylor'], 1) 262 | ] 263 | 264 | # If true, show URL addresses after external links. 265 | #man_show_urls = False 266 | 267 | 268 | # -- Options for Texinfo output ---------------------------------------- 269 | 270 | # Grouping the document tree into Texinfo files. List of tuples 271 | # (source start file, target name, title, author, 272 | # dir menu entry, description, category) 273 | texinfo_documents = [ 274 | ('index', 'etcd3', 275 | u'python-etcd3 Documentation', 276 | u'Louis Taylor', 277 | 'etcd3', 278 | 'One line description of project.', 279 | 'Miscellaneous'), 280 | ] 281 | 282 | # Documents to append as an appendix to all manuals. 283 | #texinfo_appendices = [] 284 | 285 | # If false, no module index is generated. 286 | #texinfo_domain_indices = True 287 | 288 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 289 | #texinfo_show_urls = 'footnote' 290 | 291 | # If true, do not generate a @detailmenu in the "Top" node's menu. 292 | #texinfo_no_detailmenu = False 293 | -------------------------------------------------------------------------------- /docs/contributing.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CONTRIBUTING.rst 2 | -------------------------------------------------------------------------------- /docs/history.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../HISTORY.rst 2 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. etcd3 documentation master file, created by 2 | sphinx-quickstart on Tue Jul 9 22:26:36 2013. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | Welcome to python-etcd3's documentation! 7 | ======================================== 8 | 9 | Contents: 10 | 11 | .. toctree:: 12 | :maxdepth: 2 13 | 14 | readme 15 | installation 16 | usage 17 | contributing 18 | authors 19 | history 20 | 21 | Indices and tables 22 | ================== 23 | 24 | * :ref:`genindex` 25 | * :ref:`modindex` 26 | * :ref:`search` 27 | -------------------------------------------------------------------------------- /docs/installation.rst: -------------------------------------------------------------------------------- 1 | .. highlight:: shell 2 | 3 | ============ 4 | Installation 5 | ============ 6 | 7 | 8 | Stable release 9 | -------------- 10 | 11 | To install python-etcd3, run this command in your terminal: 12 | 13 | .. code-block:: console 14 | 15 | $ pip install etcd3 16 | 17 | This is the preferred method to install python-etcd3, as it will always install the most recent stable release. 18 | 19 | If you don't have `pip`_ installed, this `Python installation guide`_ can guide 20 | you through the process. 21 | 22 | .. _pip: https://pip.pypa.io 23 | .. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/ 24 | 25 | 26 | From sources 27 | ------------ 28 | 29 | The sources for python-etcd3 can be downloaded from the `Github repo`_. 30 | 31 | You can either clone the public repository: 32 | 33 | .. code-block:: console 34 | 35 | $ git clone git://github.com/kragniz/python-etcd3 36 | 37 | Or download the `tarball`_: 38 | 39 | .. code-block:: console 40 | 41 | $ curl -OL https://github.com/kragniz/python-etcd3/tarball/master 42 | 43 | Once you have a copy of the source, you can install it with: 44 | 45 | .. code-block:: console 46 | 47 | $ python setup.py install 48 | 49 | 50 | .. _Github repo: https://github.com/kragniz/python-etcd3 51 | .. _tarball: https://github.com/kragniz/python-etcd3/tarball/master 52 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | REM Command file for Sphinx documentation 4 | 5 | if "%SPHINXBUILD%" == "" ( 6 | set SPHINXBUILD=sphinx-build 7 | ) 8 | set BUILDDIR=_build 9 | set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . 10 | set I18NSPHINXOPTS=%SPHINXOPTS% . 11 | if NOT "%PAPER%" == "" ( 12 | set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% 13 | set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% 14 | ) 15 | 16 | if "%1" == "" goto help 17 | 18 | if "%1" == "help" ( 19 | :help 20 | echo.Please use `make ^` where ^ is one of 21 | echo. html to make standalone HTML files 22 | echo. dirhtml to make HTML files named index.html in directories 23 | echo. singlehtml to make a single large HTML file 24 | echo. pickle to make pickle files 25 | echo. json to make JSON files 26 | echo. htmlhelp to make HTML files and a HTML help project 27 | echo. qthelp to make HTML files and a qthelp project 28 | echo. devhelp to make HTML files and a Devhelp project 29 | echo. epub to make an epub 30 | echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter 31 | echo. text to make text files 32 | echo. man to make manual pages 33 | echo. texinfo to make Texinfo files 34 | echo. gettext to make PO message catalogs 35 | echo. changes to make an overview over all changed/added/deprecated items 36 | echo. xml to make Docutils-native XML files 37 | echo. pseudoxml to make pseudoxml-XML files for display purposes 38 | echo. linkcheck to check all external links for integrity 39 | echo. doctest to run all doctests embedded in the documentation if enabled 40 | goto end 41 | ) 42 | 43 | if "%1" == "clean" ( 44 | for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i 45 | del /q /s %BUILDDIR%\* 46 | goto end 47 | ) 48 | 49 | 50 | %SPHINXBUILD% 2> nul 51 | if errorlevel 9009 ( 52 | echo. 53 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 54 | echo.installed, then set the SPHINXBUILD environment variable to point 55 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 56 | echo.may add the Sphinx directory to PATH. 57 | echo. 58 | echo.If you don't have Sphinx installed, grab it from 59 | echo.http://sphinx-doc.org/ 60 | exit /b 1 61 | ) 62 | 63 | if "%1" == "html" ( 64 | %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html 65 | if errorlevel 1 exit /b 1 66 | echo. 67 | echo.Build finished. The HTML pages are in %BUILDDIR%/html. 68 | goto end 69 | ) 70 | 71 | if "%1" == "dirhtml" ( 72 | %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml 73 | if errorlevel 1 exit /b 1 74 | echo. 75 | echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. 76 | goto end 77 | ) 78 | 79 | if "%1" == "singlehtml" ( 80 | %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml 81 | if errorlevel 1 exit /b 1 82 | echo. 83 | echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. 84 | goto end 85 | ) 86 | 87 | if "%1" == "pickle" ( 88 | %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle 89 | if errorlevel 1 exit /b 1 90 | echo. 91 | echo.Build finished; now you can process the pickle files. 92 | goto end 93 | ) 94 | 95 | if "%1" == "json" ( 96 | %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json 97 | if errorlevel 1 exit /b 1 98 | echo. 99 | echo.Build finished; now you can process the JSON files. 100 | goto end 101 | ) 102 | 103 | if "%1" == "htmlhelp" ( 104 | %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp 105 | if errorlevel 1 exit /b 1 106 | echo. 107 | echo.Build finished; now you can run HTML Help Workshop with the ^ 108 | .hhp project file in %BUILDDIR%/htmlhelp. 109 | goto end 110 | ) 111 | 112 | if "%1" == "qthelp" ( 113 | %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp 114 | if errorlevel 1 exit /b 1 115 | echo. 116 | echo.Build finished; now you can run "qcollectiongenerator" with the ^ 117 | .qhcp project file in %BUILDDIR%/qthelp, like this: 118 | echo.^> qcollectiongenerator %BUILDDIR%\qthelp\etcd3.qhcp 119 | echo.To view the help file: 120 | echo.^> assistant -collectionFile %BUILDDIR%\qthelp\etcd3.ghc 121 | goto end 122 | ) 123 | 124 | if "%1" == "devhelp" ( 125 | %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp 126 | if errorlevel 1 exit /b 1 127 | echo. 128 | echo.Build finished. 129 | goto end 130 | ) 131 | 132 | if "%1" == "epub" ( 133 | %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub 134 | if errorlevel 1 exit /b 1 135 | echo. 136 | echo.Build finished. The epub file is in %BUILDDIR%/epub. 137 | goto end 138 | ) 139 | 140 | if "%1" == "latex" ( 141 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 142 | if errorlevel 1 exit /b 1 143 | echo. 144 | echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. 145 | goto end 146 | ) 147 | 148 | if "%1" == "latexpdf" ( 149 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 150 | cd %BUILDDIR%/latex 151 | make all-pdf 152 | cd %BUILDDIR%/.. 153 | echo. 154 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 155 | goto end 156 | ) 157 | 158 | if "%1" == "latexpdfja" ( 159 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 160 | cd %BUILDDIR%/latex 161 | make all-pdf-ja 162 | cd %BUILDDIR%/.. 163 | echo. 164 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 165 | goto end 166 | ) 167 | 168 | if "%1" == "text" ( 169 | %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text 170 | if errorlevel 1 exit /b 1 171 | echo. 172 | echo.Build finished. The text files are in %BUILDDIR%/text. 173 | goto end 174 | ) 175 | 176 | if "%1" == "man" ( 177 | %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man 178 | if errorlevel 1 exit /b 1 179 | echo. 180 | echo.Build finished. The manual pages are in %BUILDDIR%/man. 181 | goto end 182 | ) 183 | 184 | if "%1" == "texinfo" ( 185 | %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo 186 | if errorlevel 1 exit /b 1 187 | echo. 188 | echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. 189 | goto end 190 | ) 191 | 192 | if "%1" == "gettext" ( 193 | %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale 194 | if errorlevel 1 exit /b 1 195 | echo. 196 | echo.Build finished. The message catalogs are in %BUILDDIR%/locale. 197 | goto end 198 | ) 199 | 200 | if "%1" == "changes" ( 201 | %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes 202 | if errorlevel 1 exit /b 1 203 | echo. 204 | echo.The overview file is in %BUILDDIR%/changes. 205 | goto end 206 | ) 207 | 208 | if "%1" == "linkcheck" ( 209 | %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck 210 | if errorlevel 1 exit /b 1 211 | echo. 212 | echo.Link check complete; look for any errors in the above output ^ 213 | or in %BUILDDIR%/linkcheck/output.txt. 214 | goto end 215 | ) 216 | 217 | if "%1" == "doctest" ( 218 | %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest 219 | if errorlevel 1 exit /b 1 220 | echo. 221 | echo.Testing of doctests in the sources finished, look at the ^ 222 | results in %BUILDDIR%/doctest/output.txt. 223 | goto end 224 | ) 225 | 226 | if "%1" == "xml" ( 227 | %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml 228 | if errorlevel 1 exit /b 1 229 | echo. 230 | echo.Build finished. The XML files are in %BUILDDIR%/xml. 231 | goto end 232 | ) 233 | 234 | if "%1" == "pseudoxml" ( 235 | %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml 236 | if errorlevel 1 exit /b 1 237 | echo. 238 | echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. 239 | goto end 240 | ) 241 | 242 | :end 243 | -------------------------------------------------------------------------------- /docs/readme.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../README.rst 2 | -------------------------------------------------------------------------------- /docs/usage.rst: -------------------------------------------------------------------------------- 1 | ========= 2 | API Usage 3 | ========= 4 | 5 | To use python-etcd3 in a project: 6 | 7 | .. code-block:: python 8 | 9 | import etcd3 10 | 11 | and create a client: 12 | 13 | .. code-block:: python 14 | 15 | etcd = etcd3.client() 16 | 17 | This defaults to localhost, but you can specify the host and port: 18 | 19 | .. code-block:: python 20 | 21 | etcd = etcd3.client(host='etcd-host-01', port=2379) 22 | 23 | If you would like to specify options for the underlying GRPC connection, you can also pass it as a parameter: 24 | 25 | .. code-block:: python 26 | 27 | etcd = etcd3.client(grpc_options={ 28 | 'grpc.http2.true_binary': 1, 29 | 'grpc.http2.max_pings_without_data': 0, 30 | }.items()) 31 | 32 | Putting values into etcd 33 | ------------------------ 34 | 35 | Values can be stored with the ``put`` method: 36 | 37 | .. code-block:: python 38 | 39 | etcd.put('/key', 'dooot') 40 | 41 | You can check this has been stored correctly by testing with etcdctl: 42 | 43 | .. code-block:: bash 44 | 45 | $ ETCDCTL_API=3 etcdctl get /key 46 | /key 47 | dooot 48 | 49 | API 50 | === 51 | 52 | .. autoclass:: etcd3.MultiEndpointEtcd3Client 53 | :members: 54 | 55 | .. autoclass:: etcd3.Etcd3Client 56 | :members: 57 | 58 | .. autoclass:: etcd3.Endpoint 59 | :members: 60 | 61 | .. autoclass:: etcd3.Member 62 | :members: 63 | 64 | .. autoclass:: etcd3.Lease 65 | :members: 66 | 67 | .. autoclass:: etcd3.Lock 68 | :members: 69 | -------------------------------------------------------------------------------- /etcd3/__init__.py: -------------------------------------------------------------------------------- 1 | from __future__ import absolute_import 2 | 3 | import etcd3.etcdrpc as etcdrpc 4 | from etcd3.client import Endpoint 5 | from etcd3.client import Etcd3Client 6 | from etcd3.client import MultiEndpointEtcd3Client 7 | from etcd3.client import Transactions 8 | from etcd3.client import client 9 | from etcd3.exceptions import Etcd3Exception 10 | from etcd3.leases import Lease 11 | from etcd3.locks import Lock 12 | from etcd3.members import Member 13 | 14 | __author__ = 'Louis Taylor' 15 | __email__ = 'louis@kragniz.eu' 16 | __version__ = '0.12.0' 17 | 18 | __all__ = ( 19 | 'etcdrpc', 20 | 'Endpoint', 21 | 'Etcd3Client', 22 | 'Etcd3Exception', 23 | 'Transactions', 24 | 'client', 25 | 'Lease', 26 | 'Lock', 27 | 'Member', 28 | 'MultiEndpointEtcd3Client' 29 | ) 30 | -------------------------------------------------------------------------------- /etcd3/client.py: -------------------------------------------------------------------------------- 1 | import functools 2 | import random 3 | import threading 4 | import time 5 | 6 | import grpc 7 | import grpc._channel 8 | 9 | from six.moves import queue 10 | 11 | import etcd3.etcdrpc as etcdrpc 12 | import etcd3.exceptions as exceptions 13 | import etcd3.leases as leases 14 | import etcd3.locks as locks 15 | import etcd3.members 16 | import etcd3.transactions as transactions 17 | import etcd3.utils as utils 18 | import etcd3.watch as watch 19 | 20 | _EXCEPTIONS_BY_CODE = { 21 | grpc.StatusCode.INTERNAL: exceptions.InternalServerError, 22 | grpc.StatusCode.UNAVAILABLE: exceptions.ConnectionFailedError, 23 | grpc.StatusCode.DEADLINE_EXCEEDED: exceptions.ConnectionTimeoutError, 24 | grpc.StatusCode.FAILED_PRECONDITION: exceptions.PreconditionFailedError, 25 | } 26 | 27 | _FAILED_EP_CODES = [ 28 | grpc.StatusCode.UNAVAILABLE, 29 | grpc.StatusCode.DEADLINE_EXCEEDED, 30 | grpc.StatusCode.INTERNAL 31 | ] 32 | 33 | 34 | class Transactions(object): 35 | def __init__(self): 36 | self.value = transactions.Value 37 | self.version = transactions.Version 38 | self.create = transactions.Create 39 | self.mod = transactions.Mod 40 | 41 | self.put = transactions.Put 42 | self.get = transactions.Get 43 | self.delete = transactions.Delete 44 | self.txn = transactions.Txn 45 | 46 | 47 | class KVMetadata(object): 48 | def __init__(self, keyvalue, header): 49 | self.key = keyvalue.key 50 | self.create_revision = keyvalue.create_revision 51 | self.mod_revision = keyvalue.mod_revision 52 | self.version = keyvalue.version 53 | self.lease_id = keyvalue.lease 54 | self.response_header = header 55 | 56 | 57 | class Status(object): 58 | def __init__(self, version, db_size, leader, raft_index, raft_term): 59 | self.version = version 60 | self.db_size = db_size 61 | self.leader = leader 62 | self.raft_index = raft_index 63 | self.raft_term = raft_term 64 | 65 | 66 | class Alarm(object): 67 | def __init__(self, alarm_type, member_id): 68 | self.alarm_type = alarm_type 69 | self.member_id = member_id 70 | 71 | 72 | class Endpoint(object): 73 | """Represents an etcd cluster endpoint. 74 | 75 | :param str host: Endpoint host 76 | :param int port: Endpoint port 77 | :param bool secure: Use secure channel, default True 78 | :param creds: Credentials to use for secure channel, required if 79 | secure=True 80 | :type creds: grpc.ChannelCredentials, optional 81 | :param time_retry: Seconds to wait before retrying this endpoint after 82 | failure, default 300.0 83 | :type time_retry: int or float 84 | :param opts: Additional gRPC options 85 | :type opts: dict, optional 86 | """ 87 | 88 | def __init__(self, host, port, secure=True, creds=None, time_retry=300.0, 89 | opts=None): 90 | self.host = host 91 | self.netloc = "{host}:{port}".format(host=host, port=port) 92 | self.secure = secure 93 | self.protocol = 'https' if secure else 'http' 94 | if self.secure and creds is None: 95 | raise ValueError( 96 | 'Please set TLS credentials for secure connections') 97 | self.credentials = creds 98 | self.time_retry = time_retry 99 | self.in_use = False 100 | self.last_failed = 0 101 | self.channel = self._mkchannel(opts) 102 | 103 | def close(self): 104 | self.channel.close() 105 | 106 | def fail(self): 107 | """Transition the endpoint to a failed state.""" 108 | self.in_use = False 109 | self.last_failed = time.time() 110 | 111 | def use(self): 112 | """Transition the endpoint to an active state.""" 113 | if self.is_failed(): 114 | raise ValueError('Trying to use a failed node') 115 | self.in_use = True 116 | self.last_failed = 0 117 | return self.channel 118 | 119 | def __str__(self): 120 | return "Endpoint({}://{})".format(self.protocol, self.netloc) 121 | 122 | def is_failed(self): 123 | """Check if the current endpoint is failed.""" 124 | return ((time.time() - self.last_failed) < self.time_retry) 125 | 126 | def _mkchannel(self, opts): 127 | if self.secure: 128 | return grpc.secure_channel(self.netloc, self.credentials, 129 | options=opts) 130 | else: 131 | return grpc.insecure_channel(self.netloc, options=opts) 132 | 133 | 134 | class EtcdTokenCallCredentials(grpc.AuthMetadataPlugin): 135 | """Metadata wrapper for raw access token credentials.""" 136 | 137 | def __init__(self, access_token): 138 | self._access_token = access_token 139 | 140 | def __call__(self, context, callback): 141 | metadata = (('token', self._access_token),) 142 | callback(metadata, None) 143 | 144 | 145 | class MultiEndpointEtcd3Client(object): 146 | """ 147 | etcd v3 API client with multiple endpoints. 148 | 149 | When failover is enabled, requests still will not be auto-retried. 150 | Instead, the application may retry the request, and the ``Etcd3Client`` 151 | will then attempt to send it to a different endpoint that has not recently 152 | failed. If all configured endpoints have failed and are not ready to be 153 | retried, an ``exceptions.NoServerAvailableError`` will be raised. 154 | 155 | :param endpoints: Endpoints to use in lieu of host and port 156 | :type endpoints: Iterable(Endpoint), optional 157 | :param timeout: Timeout for all RPC in seconds 158 | :type timeout: int or float, optional 159 | :param user: Username for authentication 160 | :type user: str, optional 161 | :param password: Password for authentication 162 | :type password: str, optional 163 | :param bool failover: Failover between endpoints, default False 164 | """ 165 | 166 | def __init__(self, endpoints=None, timeout=None, user=None, password=None, 167 | failover=False): 168 | 169 | self.metadata = None 170 | self.failover = failover 171 | 172 | # Cache GRPC stubs here 173 | self._stubs = {} 174 | 175 | # Step 1: setup endpoints 176 | self.endpoints = {ep.netloc: ep for ep in endpoints} 177 | self._current_endpoint_label = random.choice( 178 | list(self.endpoints.keys()) 179 | ) 180 | 181 | # Step 2: if auth is enabled, call the auth endpoint 182 | self.timeout = timeout 183 | self.call_credentials = None 184 | cred_params = [c is not None for c in (user, password)] 185 | 186 | if all(cred_params): 187 | auth_request = etcdrpc.AuthenticateRequest( 188 | name=user, 189 | password=password 190 | ) 191 | 192 | resp = self.authstub.Authenticate(auth_request, self.timeout) 193 | self.metadata = (('token', resp.token),) 194 | self.call_credentials = grpc.metadata_call_credentials( 195 | EtcdTokenCallCredentials(resp.token)) 196 | 197 | elif any(cred_params): 198 | raise Exception( 199 | 'if using authentication credentials both user and password ' 200 | 'must be specified.' 201 | ) 202 | 203 | self.transactions = Transactions() 204 | 205 | def _create_stub_property(name, stub_class): 206 | def get_stub(self): 207 | stub = self._stubs.get(name) 208 | if stub is None: 209 | stub = self._stubs[name] = stub_class(self.channel) 210 | return stub 211 | return property(get_stub) 212 | 213 | authstub = _create_stub_property("authstub", etcdrpc.AuthStub) 214 | kvstub = _create_stub_property("kvstub", etcdrpc.KVStub) 215 | clusterstub = _create_stub_property("clusterstub", etcdrpc.ClusterStub) 216 | leasestub = _create_stub_property("leasestub", etcdrpc.LeaseStub) 217 | maintenancestub = _create_stub_property( 218 | "maintenancestub", etcdrpc.MaintenanceStub 219 | ) 220 | 221 | def get_watcher(self): 222 | watchstub = etcdrpc.WatchStub(self.channel) 223 | return watch.Watcher( 224 | watchstub, 225 | timeout=self.timeout, 226 | call_credentials=self.call_credentials, 227 | metadata=self.metadata 228 | ) 229 | 230 | @property 231 | def watcher(self): 232 | watcher = self._stubs.get("watcher") 233 | if watcher is None: 234 | watcher = self._stubs["watcher"] = self.get_watcher() 235 | return watcher 236 | 237 | @watcher.setter 238 | def watcher(self, value): 239 | self._stubs["watcher"] = value 240 | 241 | def _clear_old_stubs(self): 242 | old_watcher = self._stubs.get("watcher") 243 | self._stubs.clear() 244 | if old_watcher: 245 | old_watcher.close() 246 | 247 | @property 248 | def _current_endpoint_label(self): 249 | return self._current_ep_label 250 | 251 | @_current_endpoint_label.setter 252 | def _current_endpoint_label(self, value): 253 | if getattr(self, "_current_ep_label", None) is not value: 254 | self._clear_old_stubs() 255 | self._current_ep_label = value 256 | 257 | @property 258 | def endpoint_in_use(self): 259 | """Get the current endpoint in use.""" 260 | if self._current_endpoint_label is None: 261 | return None 262 | return self.endpoints[self._current_endpoint_label] 263 | 264 | @property 265 | def channel(self): 266 | """ 267 | Get an available channel on the first node that's not failed. 268 | 269 | Raises an exception if no node is available 270 | """ 271 | try: 272 | return self.endpoint_in_use.use() 273 | except ValueError: 274 | if not self.failover: 275 | raise 276 | # We're failing over. We get the first non-failed channel 277 | # we encounter, and use it by calling this function again, 278 | # recursively 279 | for label, endpoint in self.endpoints.items(): 280 | if endpoint.is_failed(): 281 | continue 282 | self._current_endpoint_label = label 283 | return self.channel 284 | raise exceptions.NoServerAvailableError( 285 | "No endpoint available and not failed") 286 | 287 | def close(self): 288 | """Call the GRPC channel close semantics.""" 289 | possible_watcher = self._stubs.get("watcher") 290 | if possible_watcher: 291 | possible_watcher.close() 292 | for endpoint in self.endpoints.values(): 293 | endpoint.close() 294 | 295 | def __enter__(self): 296 | return self 297 | 298 | def __exit__(self, *args): 299 | self.close() 300 | 301 | @staticmethod 302 | def get_secure_creds(ca_cert, cert_key=None, cert_cert=None): 303 | cert_key_file = None 304 | cert_cert_file = None 305 | 306 | with open(ca_cert, 'rb') as f: 307 | ca_cert_file = f.read() 308 | 309 | if cert_key is not None: 310 | with open(cert_key, 'rb') as f: 311 | cert_key_file = f.read() 312 | 313 | if cert_cert is not None: 314 | with open(cert_cert, 'rb') as f: 315 | cert_cert_file = f.read() 316 | 317 | return grpc.ssl_channel_credentials( 318 | ca_cert_file, 319 | cert_key_file, 320 | cert_cert_file 321 | ) 322 | 323 | def _manage_grpc_errors(self, exc): 324 | code = exc.code() 325 | if code in _FAILED_EP_CODES: 326 | # This sets the current node to failed. 327 | # If others are available, they will be used on 328 | # subsequent requests. 329 | self.endpoint_in_use.fail() 330 | self._clear_old_stubs() 331 | exception = _EXCEPTIONS_BY_CODE.get(code) 332 | if exception is None: 333 | raise 334 | raise exception() 335 | 336 | def _handle_errors(payload): 337 | @functools.wraps(payload) 338 | def handler(self, *args, **kwargs): 339 | try: 340 | return payload(self, *args, **kwargs) 341 | except grpc.RpcError as exc: 342 | self._manage_grpc_errors(exc) 343 | return handler 344 | 345 | def _handle_generator_errors(payload): 346 | @functools.wraps(payload) 347 | def handler(self, *args, **kwargs): 348 | try: 349 | for item in payload(self, *args, **kwargs): 350 | yield item 351 | except grpc.RpcError as exc: 352 | self._manage_grpc_errors(exc) 353 | return handler 354 | 355 | def _build_get_range_request(self, key, 356 | range_end=None, 357 | limit=None, 358 | revision=None, 359 | sort_order=None, 360 | sort_target='key', 361 | serializable=False, 362 | keys_only=False, 363 | count_only=False, 364 | min_mod_revision=None, 365 | max_mod_revision=None, 366 | min_create_revision=None, 367 | max_create_revision=None): 368 | range_request = etcdrpc.RangeRequest() 369 | range_request.key = utils.to_bytes(key) 370 | range_request.keys_only = keys_only 371 | range_request.count_only = count_only 372 | range_request.serializable = serializable 373 | 374 | if range_end is not None: 375 | range_request.range_end = utils.to_bytes(range_end) 376 | if limit is not None: 377 | range_request.limit = limit 378 | if revision is not None: 379 | range_request.revision = revision 380 | if min_mod_revision is not None: 381 | range_request.min_mod_revision = min_mod_revision 382 | if max_mod_revision is not None: 383 | range_request.max_mod_revision = max_mod_revision 384 | if min_create_revision is not None: 385 | range_request.min_mod_revision = min_create_revision 386 | if max_create_revision is not None: 387 | range_request.min_mod_revision = max_create_revision 388 | 389 | sort_orders = { 390 | None: etcdrpc.RangeRequest.NONE, 391 | 'ascend': etcdrpc.RangeRequest.ASCEND, 392 | 'descend': etcdrpc.RangeRequest.DESCEND, 393 | } 394 | request_sort_order = sort_orders.get(sort_order) 395 | if request_sort_order is None: 396 | raise ValueError('unknown sort order: "{}"'.format(sort_order)) 397 | range_request.sort_order = request_sort_order 398 | 399 | sort_targets = { 400 | None: etcdrpc.RangeRequest.KEY, 401 | 'key': etcdrpc.RangeRequest.KEY, 402 | 'version': etcdrpc.RangeRequest.VERSION, 403 | 'create': etcdrpc.RangeRequest.CREATE, 404 | 'mod': etcdrpc.RangeRequest.MOD, 405 | 'value': etcdrpc.RangeRequest.VALUE, 406 | } 407 | request_sort_target = sort_targets.get(sort_target) 408 | if request_sort_target is None: 409 | raise ValueError('sort_target must be one of "key", ' 410 | '"version", "create", "mod" or "value"') 411 | range_request.sort_target = request_sort_target 412 | 413 | return range_request 414 | 415 | @_handle_errors 416 | def get_response(self, key, **kwargs): 417 | """Get the value of a key from etcd.""" 418 | range_request = self._build_get_range_request( 419 | key, 420 | **kwargs 421 | ) 422 | 423 | return self.kvstub.Range( 424 | range_request, 425 | self.timeout, 426 | credentials=self.call_credentials, 427 | metadata=self.metadata 428 | ) 429 | 430 | def get(self, key, **kwargs): 431 | """ 432 | Get the value of a key from etcd. 433 | 434 | example usage: 435 | 436 | .. code-block:: python 437 | 438 | >>> import etcd3 439 | >>> etcd = etcd3.client() 440 | >>> etcd.get('/thing/key') 441 | 'hello world' 442 | 443 | :param key: key in etcd to get 444 | :returns: value of key and metadata 445 | :rtype: bytes, ``KVMetadata`` 446 | """ 447 | range_response = self.get_response(key, **kwargs) 448 | if range_response.count < 1: 449 | return None, None 450 | else: 451 | kv = range_response.kvs.pop() 452 | return kv.value, KVMetadata(kv, range_response.header) 453 | 454 | @_handle_errors 455 | def get_prefix_response(self, key_prefix, **kwargs): 456 | """Get a range of keys with a prefix.""" 457 | if any(kwarg in kwargs for kwarg in ("key", "range_end")): 458 | raise TypeError("Don't use key or range_end with prefix") 459 | 460 | range_request = self._build_get_range_request( 461 | key=key_prefix, 462 | range_end=utils.prefix_range_end(utils.to_bytes(key_prefix)), 463 | **kwargs 464 | ) 465 | 466 | return self.kvstub.Range( 467 | range_request, 468 | self.timeout, 469 | credentials=self.call_credentials, 470 | metadata=self.metadata 471 | ) 472 | 473 | def get_prefix(self, key_prefix, **kwargs): 474 | """ 475 | Get a range of keys with a prefix. 476 | 477 | :param key_prefix: first key in range 478 | :param keys_only: if True, retrieve only the keys, not the values 479 | 480 | :returns: sequence of (value, metadata) tuples 481 | """ 482 | range_response = self.get_prefix_response(key_prefix, **kwargs) 483 | return ( 484 | (kv.value, KVMetadata(kv, range_response.header)) 485 | for kv in range_response.kvs 486 | ) 487 | 488 | @_handle_errors 489 | def get_range_response(self, range_start, range_end, sort_order=None, 490 | sort_target='key', **kwargs): 491 | """Get a range of keys.""" 492 | range_request = self._build_get_range_request( 493 | key=range_start, 494 | range_end=range_end, 495 | sort_order=sort_order, 496 | sort_target=sort_target, 497 | **kwargs 498 | ) 499 | 500 | return self.kvstub.Range( 501 | range_request, 502 | self.timeout, 503 | credentials=self.call_credentials, 504 | metadata=self.metadata 505 | ) 506 | 507 | def get_range(self, range_start, range_end, **kwargs): 508 | """ 509 | Get a range of keys. 510 | 511 | :param range_start: first key in range 512 | :param range_end: last key in range 513 | :returns: sequence of (value, metadata) tuples 514 | """ 515 | range_response = self.get_range_response(range_start, range_end, 516 | **kwargs) 517 | for kv in range_response.kvs: 518 | yield (kv.value, KVMetadata(kv, range_response.header)) 519 | 520 | @_handle_errors 521 | def get_all_response(self, sort_order=None, sort_target='key', 522 | keys_only=False): 523 | """Get all keys currently stored in etcd.""" 524 | range_request = self._build_get_range_request( 525 | key=b'\0', 526 | range_end=b'\0', 527 | sort_order=sort_order, 528 | sort_target=sort_target, 529 | keys_only=keys_only, 530 | ) 531 | 532 | return self.kvstub.Range( 533 | range_request, 534 | self.timeout, 535 | credentials=self.call_credentials, 536 | metadata=self.metadata 537 | ) 538 | 539 | def get_all(self, **kwargs): 540 | """ 541 | Get all keys currently stored in etcd. 542 | 543 | :param keys_only: if True, retrieve only the keys, not the values 544 | :returns: sequence of (value, metadata) tuples 545 | """ 546 | range_response = self.get_all_response(**kwargs) 547 | for kv in range_response.kvs: 548 | yield (kv.value, KVMetadata(kv, range_response.header)) 549 | 550 | def _build_put_request(self, key, value, lease=None, prev_kv=False): 551 | put_request = etcdrpc.PutRequest() 552 | put_request.key = utils.to_bytes(key) 553 | put_request.value = utils.to_bytes(value) 554 | put_request.lease = utils.lease_to_id(lease) 555 | put_request.prev_kv = prev_kv 556 | 557 | return put_request 558 | 559 | @_handle_errors 560 | def put(self, key, value, lease=None, prev_kv=False): 561 | """ 562 | Save a value to etcd. 563 | 564 | Example usage: 565 | 566 | .. code-block:: python 567 | 568 | >>> import etcd3 569 | >>> etcd = etcd3.client() 570 | >>> etcd.put('/thing/key', 'hello world') 571 | 572 | :param key: key in etcd to set 573 | :param value: value to set key to 574 | :type value: bytes 575 | :param lease: Lease to associate with this key. 576 | :type lease: either :class:`.Lease`, or int (ID of lease) 577 | :param prev_kv: return the previous key-value pair 578 | :type prev_kv: bool 579 | :returns: a response containing a header and the prev_kv 580 | :rtype: :class:`.rpc_pb2.PutResponse` 581 | """ 582 | put_request = self._build_put_request(key, value, lease=lease, 583 | prev_kv=prev_kv) 584 | return self.kvstub.Put( 585 | put_request, 586 | self.timeout, 587 | credentials=self.call_credentials, 588 | metadata=self.metadata 589 | ) 590 | 591 | @_handle_errors 592 | def put_if_not_exists(self, key, value, lease=None): 593 | """ 594 | Atomically puts a value only if the key previously had no value. 595 | 596 | This is the etcdv3 equivalent to setting a key with the etcdv2 597 | parameter prevExist=false. 598 | 599 | :param key: key in etcd to put 600 | :param value: value to be written to key 601 | :type value: bytes 602 | :param lease: Lease to associate with this key. 603 | :type lease: either :class:`.Lease`, or int (ID of lease) 604 | :returns: state of transaction, ``True`` if the put was successful, 605 | ``False`` otherwise 606 | :rtype: bool 607 | """ 608 | status, _ = self.transaction( 609 | compare=[self.transactions.create(key) == '0'], 610 | success=[self.transactions.put(key, value, lease=lease)], 611 | failure=[], 612 | ) 613 | 614 | return status 615 | 616 | @_handle_errors 617 | def replace(self, key, initial_value, new_value): 618 | """ 619 | Atomically replace the value of a key with a new value. 620 | 621 | This compares the current value of a key, then replaces it with a new 622 | value if it is equal to a specified value. This operation takes place 623 | in a transaction. 624 | 625 | :param key: key in etcd to replace 626 | :param initial_value: old value to replace 627 | :type initial_value: bytes 628 | :param new_value: new value of the key 629 | :type new_value: bytes 630 | :returns: status of transaction, ``True`` if the replace was 631 | successful, ``False`` otherwise 632 | :rtype: bool 633 | """ 634 | status, _ = self.transaction( 635 | compare=[self.transactions.value(key) == initial_value], 636 | success=[self.transactions.put(key, new_value)], 637 | failure=[], 638 | ) 639 | 640 | return status 641 | 642 | def _build_delete_request(self, key, 643 | range_end=None, 644 | prev_kv=False): 645 | delete_request = etcdrpc.DeleteRangeRequest() 646 | delete_request.key = utils.to_bytes(key) 647 | delete_request.prev_kv = prev_kv 648 | 649 | if range_end is not None: 650 | delete_request.range_end = utils.to_bytes(range_end) 651 | 652 | return delete_request 653 | 654 | @_handle_errors 655 | def delete(self, key, prev_kv=False, return_response=False): 656 | """ 657 | Delete a single key in etcd. 658 | 659 | :param key: key in etcd to delete 660 | :param prev_kv: return the deleted key-value pair 661 | :type prev_kv: bool 662 | :param return_response: return the full response 663 | :type return_response: bool 664 | :returns: True if the key has been deleted when 665 | ``return_response`` is False and a response containing 666 | a header, the number of deleted keys and prev_kvs when 667 | ``return_response`` is True 668 | """ 669 | delete_request = self._build_delete_request(key, prev_kv=prev_kv) 670 | delete_response = self.kvstub.DeleteRange( 671 | delete_request, 672 | self.timeout, 673 | credentials=self.call_credentials, 674 | metadata=self.metadata 675 | ) 676 | if return_response: 677 | return delete_response 678 | return delete_response.deleted >= 1 679 | 680 | @_handle_errors 681 | def delete_prefix(self, prefix): 682 | """Delete a range of keys with a prefix in etcd.""" 683 | delete_request = self._build_delete_request( 684 | prefix, 685 | range_end=utils.prefix_range_end(utils.to_bytes(prefix)) 686 | ) 687 | return self.kvstub.DeleteRange( 688 | delete_request, 689 | self.timeout, 690 | credentials=self.call_credentials, 691 | metadata=self.metadata 692 | ) 693 | 694 | @_handle_errors 695 | def status(self): 696 | """Get the status of the responding member.""" 697 | status_request = etcdrpc.StatusRequest() 698 | status_response = self.maintenancestub.Status( 699 | status_request, 700 | self.timeout, 701 | credentials=self.call_credentials, 702 | metadata=self.metadata 703 | ) 704 | 705 | for m in self.members: 706 | if m.id == status_response.leader: 707 | leader = m 708 | break 709 | else: 710 | # raise exception? 711 | leader = None 712 | 713 | return Status(status_response.version, 714 | status_response.dbSize, 715 | leader, 716 | status_response.raftIndex, 717 | status_response.raftTerm) 718 | 719 | @_handle_errors 720 | def add_watch_callback(self, *args, **kwargs): 721 | """ 722 | Watch a key or range of keys and call a callback on every response. 723 | 724 | If timeout was declared during the client initialization and 725 | the watch cannot be created during that time the method raises 726 | a ``WatchTimedOut`` exception. 727 | 728 | :param key: key to watch 729 | :param callback: callback function 730 | 731 | :returns: watch_id. Later it could be used for cancelling watch. 732 | """ 733 | try: 734 | return self.watcher.add_callback(*args, **kwargs) 735 | except queue.Empty: 736 | raise exceptions.WatchTimedOut() 737 | 738 | @_handle_errors 739 | def add_watch_prefix_callback(self, key_prefix, callback, **kwargs): 740 | """ 741 | Watch a prefix and call a callback on every response. 742 | 743 | If timeout was declared during the client initialization and 744 | the watch cannot be created during that time the method raises 745 | a ``WatchTimedOut`` exception. 746 | 747 | :param key_prefix: prefix to watch 748 | :param callback: callback function 749 | 750 | :returns: watch_id. Later it could be used for cancelling watch. 751 | """ 752 | kwargs['range_end'] = \ 753 | utils.prefix_range_end(utils.to_bytes(key_prefix)) 754 | 755 | return self.add_watch_callback(key_prefix, callback, **kwargs) 756 | 757 | @_handle_errors 758 | def watch_response(self, key, **kwargs): 759 | """ 760 | Watch a key. 761 | 762 | Example usage: 763 | 764 | .. code-block:: python 765 | responses_iterator, cancel = etcd.watch_response('/doot/key') 766 | for response in responses_iterator: 767 | print(response) 768 | 769 | :param key: key to watch 770 | 771 | :returns: tuple of ``responses_iterator`` and ``cancel``. 772 | Use ``responses_iterator`` to get the watch responses, 773 | each of which contains a header and a list of events. 774 | Use ``cancel`` to cancel the watch request. 775 | """ 776 | response_queue = queue.Queue() 777 | 778 | def callback(response): 779 | response_queue.put(response) 780 | 781 | watch_id = self.add_watch_callback(key, callback, **kwargs) 782 | canceled = threading.Event() 783 | 784 | def cancel(): 785 | canceled.set() 786 | response_queue.put(None) 787 | self.cancel_watch(watch_id) 788 | 789 | def iterator(): 790 | try: 791 | while not canceled.is_set(): 792 | response = response_queue.get() 793 | if response is None: 794 | canceled.set() 795 | if isinstance(response, Exception): 796 | canceled.set() 797 | raise response 798 | if not canceled.is_set(): 799 | yield response 800 | except grpc.RpcError as exc: 801 | self._manage_grpc_errors(exc) 802 | 803 | return iterator(), cancel 804 | 805 | def watch(self, key, **kwargs): 806 | """ 807 | Watch a key. 808 | 809 | Example usage: 810 | 811 | .. code-block:: python 812 | events_iterator, cancel = etcd.watch('/doot/key') 813 | for event in events_iterator: 814 | print(event) 815 | 816 | :param key: key to watch 817 | 818 | :returns: tuple of ``events_iterator`` and ``cancel``. 819 | Use ``events_iterator`` to get the events of key changes 820 | and ``cancel`` to cancel the watch request. 821 | """ 822 | response_iterator, cancel = self.watch_response(key, **kwargs) 823 | return utils.response_to_event_iterator(response_iterator), cancel 824 | 825 | def watch_prefix_response(self, key_prefix, **kwargs): 826 | """ 827 | Watch a range of keys with a prefix. 828 | 829 | :param key_prefix: prefix to watch 830 | 831 | :returns: tuple of ``responses_iterator`` and ``cancel``. 832 | """ 833 | kwargs['range_end'] = \ 834 | utils.prefix_range_end(utils.to_bytes(key_prefix)) 835 | return self.watch_response(key_prefix, **kwargs) 836 | 837 | def watch_prefix(self, key_prefix, **kwargs): 838 | """ 839 | Watch a range of keys with a prefix. 840 | 841 | :param key_prefix: prefix to watch 842 | 843 | :returns: tuple of ``events_iterator`` and ``cancel``. 844 | """ 845 | kwargs['range_end'] = \ 846 | utils.prefix_range_end(utils.to_bytes(key_prefix)) 847 | return self.watch(key_prefix, **kwargs) 848 | 849 | @_handle_errors 850 | def watch_once_response(self, key, timeout=None, **kwargs): 851 | """ 852 | Watch a key and stop after the first response. 853 | 854 | If the timeout was specified and response didn't arrive method 855 | will raise ``WatchTimedOut`` exception. 856 | 857 | :param key: key to watch 858 | :param timeout: (optional) timeout in seconds. 859 | 860 | :returns: ``WatchResponse`` 861 | """ 862 | response_queue = queue.Queue() 863 | 864 | def callback(response): 865 | response_queue.put(response) 866 | 867 | watch_id = self.add_watch_callback(key, callback, **kwargs) 868 | 869 | try: 870 | return response_queue.get(timeout=timeout) 871 | except queue.Empty: 872 | raise exceptions.WatchTimedOut() 873 | finally: 874 | self.cancel_watch(watch_id) 875 | 876 | def watch_once(self, key, timeout=None, **kwargs): 877 | """ 878 | Watch a key and stop after the first event. 879 | 880 | If the timeout was specified and event didn't arrive method 881 | will raise ``WatchTimedOut`` exception. 882 | 883 | :param key: key to watch 884 | :param timeout: (optional) timeout in seconds. 885 | 886 | :returns: ``Event`` 887 | """ 888 | response = self.watch_once_response(key, timeout=timeout, **kwargs) 889 | return response.events[0] 890 | 891 | def watch_prefix_once_response(self, key_prefix, timeout=None, **kwargs): 892 | """ 893 | Watch a range of keys with a prefix and stop after the first response. 894 | 895 | If the timeout was specified and response didn't arrive method 896 | will raise ``WatchTimedOut`` exception. 897 | """ 898 | kwargs['range_end'] = \ 899 | utils.prefix_range_end(utils.to_bytes(key_prefix)) 900 | return self.watch_once_response(key_prefix, timeout=timeout, **kwargs) 901 | 902 | def watch_prefix_once(self, key_prefix, timeout=None, **kwargs): 903 | """ 904 | Watch a range of keys with a prefix and stop after the first event. 905 | 906 | If the timeout was specified and event didn't arrive method 907 | will raise ``WatchTimedOut`` exception. 908 | """ 909 | kwargs['range_end'] = \ 910 | utils.prefix_range_end(utils.to_bytes(key_prefix)) 911 | return self.watch_once(key_prefix, timeout=timeout, **kwargs) 912 | 913 | @_handle_errors 914 | def cancel_watch(self, watch_id): 915 | """ 916 | Stop watching a key or range of keys. 917 | 918 | :param watch_id: watch_id returned by ``add_watch_callback`` method 919 | """ 920 | self.watcher.cancel(watch_id) 921 | 922 | def _ops_to_requests(self, ops): 923 | """ 924 | Return a list of grpc requests. 925 | 926 | Returns list from an input list of etcd3.transactions.{Put, Get, 927 | Delete, Txn} objects. 928 | """ 929 | request_ops = [] 930 | for op in ops: 931 | if isinstance(op, transactions.Put): 932 | request = self._build_put_request(op.key, op.value, 933 | op.lease, op.prev_kv) 934 | request_op = etcdrpc.RequestOp(request_put=request) 935 | request_ops.append(request_op) 936 | 937 | elif isinstance(op, transactions.Get): 938 | request = self._build_get_range_request(op.key, op.range_end) 939 | request_op = etcdrpc.RequestOp(request_range=request) 940 | request_ops.append(request_op) 941 | 942 | elif isinstance(op, transactions.Delete): 943 | request = self._build_delete_request(op.key, op.range_end, 944 | op.prev_kv) 945 | request_op = etcdrpc.RequestOp(request_delete_range=request) 946 | request_ops.append(request_op) 947 | 948 | elif isinstance(op, transactions.Txn): 949 | compare = [c.build_message() for c in op.compare] 950 | success_ops = self._ops_to_requests(op.success) 951 | failure_ops = self._ops_to_requests(op.failure) 952 | request = etcdrpc.TxnRequest(compare=compare, 953 | success=success_ops, 954 | failure=failure_ops) 955 | request_op = etcdrpc.RequestOp(request_txn=request) 956 | request_ops.append(request_op) 957 | 958 | else: 959 | raise Exception( 960 | 'Unknown request class {}'.format(op.__class__)) 961 | return request_ops 962 | 963 | @_handle_errors 964 | def transaction(self, compare, success=None, failure=None): 965 | """ 966 | Perform a transaction. 967 | 968 | Example usage: 969 | 970 | .. code-block:: python 971 | 972 | etcd.transaction( 973 | compare=[ 974 | etcd.transactions.value('/doot/testing') == 'doot', 975 | etcd.transactions.version('/doot/testing') > 0, 976 | ], 977 | success=[ 978 | etcd.transactions.put('/doot/testing', 'success'), 979 | ], 980 | failure=[ 981 | etcd.transactions.put('/doot/testing', 'failure'), 982 | ] 983 | ) 984 | 985 | :param compare: A list of comparisons to make 986 | :param success: A list of operations to perform if all the comparisons 987 | are true 988 | :param failure: A list of operations to perform if any of the 989 | comparisons are false 990 | :return: A tuple of (operation status, responses) 991 | """ 992 | compare = [c.build_message() for c in compare] 993 | 994 | success_ops = self._ops_to_requests(success) 995 | failure_ops = self._ops_to_requests(failure) 996 | 997 | transaction_request = etcdrpc.TxnRequest(compare=compare, 998 | success=success_ops, 999 | failure=failure_ops) 1000 | txn_response = self.kvstub.Txn( 1001 | transaction_request, 1002 | self.timeout, 1003 | credentials=self.call_credentials, 1004 | metadata=self.metadata 1005 | ) 1006 | 1007 | responses = [] 1008 | for response in txn_response.responses: 1009 | response_type = response.WhichOneof('response') 1010 | if response_type in ['response_put', 'response_delete_range', 1011 | 'response_txn']: 1012 | responses.append(response) 1013 | 1014 | elif response_type == 'response_range': 1015 | range_kvs = [] 1016 | for kv in response.response_range.kvs: 1017 | range_kvs.append((kv.value, 1018 | KVMetadata(kv, txn_response.header))) 1019 | 1020 | responses.append(range_kvs) 1021 | 1022 | return txn_response.succeeded, responses 1023 | 1024 | @_handle_errors 1025 | def lease(self, ttl, lease_id=None): 1026 | """ 1027 | Create a new lease. 1028 | 1029 | All keys attached to this lease will be expired and deleted if the 1030 | lease expires. A lease can be sent keep alive messages to refresh the 1031 | ttl. 1032 | 1033 | :param ttl: Requested time to live 1034 | :param lease_id: Requested ID for the lease 1035 | 1036 | :returns: new lease 1037 | :rtype: :class:`.Lease` 1038 | """ 1039 | lease_grant_request = etcdrpc.LeaseGrantRequest(TTL=ttl, ID=lease_id) 1040 | lease_grant_response = self.leasestub.LeaseGrant( 1041 | lease_grant_request, 1042 | self.timeout, 1043 | credentials=self.call_credentials, 1044 | metadata=self.metadata 1045 | ) 1046 | return leases.Lease(lease_id=lease_grant_response.ID, 1047 | ttl=lease_grant_response.TTL, 1048 | etcd_client=self) 1049 | 1050 | @_handle_errors 1051 | def revoke_lease(self, lease_id): 1052 | """ 1053 | Revoke a lease. 1054 | 1055 | :param lease_id: ID of the lease to revoke. 1056 | """ 1057 | lease_revoke_request = etcdrpc.LeaseRevokeRequest(ID=lease_id) 1058 | self.leasestub.LeaseRevoke( 1059 | lease_revoke_request, 1060 | self.timeout, 1061 | credentials=self.call_credentials, 1062 | metadata=self.metadata 1063 | ) 1064 | 1065 | @_handle_generator_errors 1066 | def refresh_lease(self, lease_id): 1067 | keep_alive_request = etcdrpc.LeaseKeepAliveRequest(ID=lease_id) 1068 | request_stream = [keep_alive_request] 1069 | for response in self.leasestub.LeaseKeepAlive( 1070 | iter(request_stream), 1071 | self.timeout, 1072 | credentials=self.call_credentials, 1073 | metadata=self.metadata): 1074 | yield response 1075 | 1076 | @_handle_errors 1077 | def get_lease_info(self, lease_id): 1078 | # only available in etcd v3.1.0 and later 1079 | ttl_request = etcdrpc.LeaseTimeToLiveRequest(ID=lease_id, 1080 | keys=True) 1081 | return self.leasestub.LeaseTimeToLive( 1082 | ttl_request, 1083 | self.timeout, 1084 | credentials=self.call_credentials, 1085 | metadata=self.metadata 1086 | ) 1087 | 1088 | @_handle_errors 1089 | def lock(self, name, ttl=60): 1090 | """ 1091 | Create a new lock. 1092 | 1093 | :param name: name of the lock 1094 | :type name: string or bytes 1095 | :param ttl: length of time for the lock to live for in seconds. The 1096 | lock will be released after this time elapses, unless 1097 | refreshed 1098 | :type ttl: int 1099 | :returns: new lock 1100 | :rtype: :class:`.Lock` 1101 | """ 1102 | return locks.Lock(name, ttl=ttl, etcd_client=self) 1103 | 1104 | @_handle_errors 1105 | def add_member(self, urls): 1106 | """ 1107 | Add a member into the cluster. 1108 | 1109 | :returns: new member 1110 | :rtype: :class:`.Member` 1111 | """ 1112 | member_add_request = etcdrpc.MemberAddRequest(peerURLs=urls) 1113 | 1114 | member_add_response = self.clusterstub.MemberAdd( 1115 | member_add_request, 1116 | self.timeout, 1117 | credentials=self.call_credentials, 1118 | metadata=self.metadata 1119 | ) 1120 | 1121 | member = member_add_response.member 1122 | return etcd3.members.Member(member.ID, 1123 | member.name, 1124 | member.peerURLs, 1125 | member.clientURLs, 1126 | etcd_client=self) 1127 | 1128 | @_handle_errors 1129 | def remove_member(self, member_id): 1130 | """ 1131 | Remove an existing member from the cluster. 1132 | 1133 | :param member_id: ID of the member to remove 1134 | """ 1135 | member_rm_request = etcdrpc.MemberRemoveRequest(ID=member_id) 1136 | self.clusterstub.MemberRemove( 1137 | member_rm_request, 1138 | self.timeout, 1139 | credentials=self.call_credentials, 1140 | metadata=self.metadata 1141 | ) 1142 | 1143 | @_handle_errors 1144 | def update_member(self, member_id, peer_urls): 1145 | """ 1146 | Update the configuration of an existing member in the cluster. 1147 | 1148 | :param member_id: ID of the member to update 1149 | :param peer_urls: new list of peer urls the member will use to 1150 | communicate with the cluster 1151 | """ 1152 | member_update_request = etcdrpc.MemberUpdateRequest(ID=member_id, 1153 | peerURLs=peer_urls) 1154 | self.clusterstub.MemberUpdate( 1155 | member_update_request, 1156 | self.timeout, 1157 | credentials=self.call_credentials, 1158 | metadata=self.metadata 1159 | ) 1160 | 1161 | @property 1162 | def members(self): 1163 | """ 1164 | List of all members associated with the cluster. 1165 | 1166 | :type: sequence of :class:`.Member` 1167 | 1168 | """ 1169 | member_list_request = etcdrpc.MemberListRequest() 1170 | member_list_response = self.clusterstub.MemberList( 1171 | member_list_request, 1172 | self.timeout, 1173 | credentials=self.call_credentials, 1174 | metadata=self.metadata 1175 | ) 1176 | 1177 | for member in member_list_response.members: 1178 | yield etcd3.members.Member(member.ID, 1179 | member.name, 1180 | member.peerURLs, 1181 | member.clientURLs, 1182 | etcd_client=self) 1183 | 1184 | @_handle_errors 1185 | def compact(self, revision, physical=False): 1186 | """ 1187 | Compact the event history in etcd up to a given revision. 1188 | 1189 | All superseded keys with a revision less than the compaction revision 1190 | will be removed. 1191 | 1192 | :param revision: revision for the compaction operation 1193 | :param physical: if set to True, the request will wait until the 1194 | compaction is physically applied to the local database 1195 | such that compacted entries are totally removed from 1196 | the backend database 1197 | """ 1198 | compact_request = etcdrpc.CompactionRequest(revision=revision, 1199 | physical=physical) 1200 | self.kvstub.Compact( 1201 | compact_request, 1202 | self.timeout, 1203 | credentials=self.call_credentials, 1204 | metadata=self.metadata 1205 | ) 1206 | 1207 | @_handle_errors 1208 | def defragment(self): 1209 | """Defragment a member's backend database to recover storage space.""" 1210 | defrag_request = etcdrpc.DefragmentRequest() 1211 | self.maintenancestub.Defragment( 1212 | defrag_request, 1213 | self.timeout, 1214 | credentials=self.call_credentials, 1215 | metadata=self.metadata 1216 | ) 1217 | 1218 | @_handle_errors 1219 | def hash(self): 1220 | """ 1221 | Return the hash of the local KV state. 1222 | 1223 | :returns: kv state hash 1224 | :rtype: int 1225 | """ 1226 | hash_request = etcdrpc.HashRequest() 1227 | return self.maintenancestub.Hash(hash_request).hash 1228 | 1229 | def _build_alarm_request(self, alarm_action, member_id, alarm_type): 1230 | alarm_request = etcdrpc.AlarmRequest() 1231 | 1232 | if alarm_action == 'get': 1233 | alarm_request.action = etcdrpc.AlarmRequest.GET 1234 | elif alarm_action == 'activate': 1235 | alarm_request.action = etcdrpc.AlarmRequest.ACTIVATE 1236 | elif alarm_action == 'deactivate': 1237 | alarm_request.action = etcdrpc.AlarmRequest.DEACTIVATE 1238 | else: 1239 | raise ValueError('Unknown alarm action: {}'.format(alarm_action)) 1240 | 1241 | alarm_request.memberID = member_id 1242 | 1243 | if alarm_type == 'none': 1244 | alarm_request.alarm = etcdrpc.NONE 1245 | elif alarm_type == 'no space': 1246 | alarm_request.alarm = etcdrpc.NOSPACE 1247 | else: 1248 | raise ValueError('Unknown alarm type: {}'.format(alarm_type)) 1249 | 1250 | return alarm_request 1251 | 1252 | @_handle_errors 1253 | def create_alarm(self, member_id=0): 1254 | """Create an alarm. 1255 | 1256 | If no member id is given, the alarm is activated for all the 1257 | members of the cluster. Only the `no space` alarm can be raised. 1258 | 1259 | :param member_id: The cluster member id to create an alarm to. 1260 | If 0, the alarm is created for all the members 1261 | of the cluster. 1262 | :returns: list of :class:`.Alarm` 1263 | """ 1264 | alarm_request = self._build_alarm_request('activate', 1265 | member_id, 1266 | 'no space') 1267 | alarm_response = self.maintenancestub.Alarm( 1268 | alarm_request, 1269 | self.timeout, 1270 | credentials=self.call_credentials, 1271 | metadata=self.metadata 1272 | ) 1273 | 1274 | return [Alarm(alarm.alarm, alarm.memberID) 1275 | for alarm in alarm_response.alarms] 1276 | 1277 | @_handle_errors 1278 | def list_alarms(self, member_id=0, alarm_type='none'): 1279 | """List the activated alarms. 1280 | 1281 | :param member_id: 1282 | :param alarm_type: The cluster member id to create an alarm to. 1283 | If 0, the alarm is created for all the members 1284 | of the cluster. 1285 | :returns: sequence of :class:`.Alarm` 1286 | """ 1287 | alarm_request = self._build_alarm_request('get', 1288 | member_id, 1289 | alarm_type) 1290 | alarm_response = self.maintenancestub.Alarm( 1291 | alarm_request, 1292 | self.timeout, 1293 | credentials=self.call_credentials, 1294 | metadata=self.metadata 1295 | ) 1296 | 1297 | for alarm in alarm_response.alarms: 1298 | yield Alarm(alarm.alarm, alarm.memberID) 1299 | 1300 | @_handle_errors 1301 | def disarm_alarm(self, member_id=0): 1302 | """Cancel an alarm. 1303 | 1304 | :param member_id: The cluster member id to cancel an alarm. 1305 | If 0, the alarm is canceled for all the members 1306 | of the cluster. 1307 | :returns: List of :class:`.Alarm` 1308 | """ 1309 | alarm_request = self._build_alarm_request('deactivate', 1310 | member_id, 1311 | 'no space') 1312 | alarm_response = self.maintenancestub.Alarm( 1313 | alarm_request, 1314 | self.timeout, 1315 | credentials=self.call_credentials, 1316 | metadata=self.metadata 1317 | ) 1318 | 1319 | return [Alarm(alarm.alarm, alarm.memberID) 1320 | for alarm in alarm_response.alarms] 1321 | 1322 | @_handle_errors 1323 | def snapshot(self, file_obj): 1324 | """Take a snapshot of the database. 1325 | 1326 | :param file_obj: A file-like object to write the database contents in. 1327 | """ 1328 | snapshot_request = etcdrpc.SnapshotRequest() 1329 | snapshot_response = self.maintenancestub.Snapshot( 1330 | snapshot_request, 1331 | self.timeout, 1332 | credentials=self.call_credentials, 1333 | metadata=self.metadata 1334 | ) 1335 | 1336 | for response in snapshot_response: 1337 | file_obj.write(response.blob) 1338 | 1339 | 1340 | class Etcd3Client(MultiEndpointEtcd3Client): 1341 | """ 1342 | etcd v3 API client. 1343 | 1344 | :param host: Host to connect to, 'localhost' if not specified 1345 | :type host: str, optional 1346 | :param port: Port to connect to on host, 2379 if not specified 1347 | :type port: int, optional 1348 | :param ca_cert: Filesystem path of etcd CA certificate 1349 | :type ca_cert: str or os.PathLike, optional 1350 | :param cert_key: Filesystem path of client key 1351 | :type cert_key: str or os.PathLike, optional 1352 | :param cert_cert: Filesystem path of client certificate 1353 | :type cert_cert: str or os.PathLike, optional 1354 | :param timeout: Timeout for all RPC in seconds 1355 | :type timeout: int or float, optional 1356 | :param user: Username for authentication 1357 | :type user: str, optional 1358 | :param password: Password for authentication 1359 | :type password: str, optional 1360 | :param dict grpc_options: Additional gRPC options 1361 | :type grpc_options: dict, optional 1362 | """ 1363 | 1364 | def __init__(self, host='localhost', port=2379, ca_cert=None, 1365 | cert_key=None, cert_cert=None, timeout=None, user=None, 1366 | password=None, grpc_options=None): 1367 | 1368 | # Step 1: verify credentials 1369 | cert_params = [c is not None for c in (cert_cert, cert_key)] 1370 | if ca_cert is not None: 1371 | if all(cert_params): 1372 | credentials = self.get_secure_creds( 1373 | ca_cert, 1374 | cert_key, 1375 | cert_cert 1376 | ) 1377 | self.uses_secure_channel = True 1378 | elif any(cert_params): 1379 | # some of the cert parameters are set 1380 | raise ValueError( 1381 | 'to use a secure channel ca_cert is required by itself, ' 1382 | 'or cert_cert and cert_key must both be specified.') 1383 | else: 1384 | credentials = self.get_secure_creds(ca_cert, None, None) 1385 | self.uses_secure_channel = True 1386 | else: 1387 | self.uses_secure_channel = False 1388 | credentials = None 1389 | 1390 | # Step 2: create Endpoint 1391 | ep = Endpoint(host, port, secure=self.uses_secure_channel, 1392 | creds=credentials, opts=grpc_options) 1393 | 1394 | super(Etcd3Client, self).__init__(endpoints=[ep], timeout=timeout, 1395 | user=user, password=password) 1396 | 1397 | 1398 | def client(host='localhost', port=2379, 1399 | ca_cert=None, cert_key=None, cert_cert=None, timeout=None, 1400 | user=None, password=None, grpc_options=None): 1401 | """Return an instance of an Etcd3Client.""" 1402 | return Etcd3Client(host=host, 1403 | port=port, 1404 | ca_cert=ca_cert, 1405 | cert_key=cert_key, 1406 | cert_cert=cert_cert, 1407 | timeout=timeout, 1408 | user=user, 1409 | password=password, 1410 | grpc_options=grpc_options) 1411 | -------------------------------------------------------------------------------- /etcd3/etcdrpc/__init__.py: -------------------------------------------------------------------------------- 1 | from .rpc_pb2 import * 2 | from .rpc_pb2_grpc import * 3 | -------------------------------------------------------------------------------- /etcd3/etcdrpc/auth_pb2.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Generated by the protocol buffer compiler. DO NOT EDIT! 3 | # source: auth.proto 4 | """Generated protocol buffer code.""" 5 | from google.protobuf import descriptor as _descriptor 6 | from google.protobuf import descriptor_pool as _descriptor_pool 7 | from google.protobuf import message as _message 8 | from google.protobuf import reflection as _reflection 9 | from google.protobuf import symbol_database as _symbol_database 10 | # @@protoc_insertion_point(imports) 11 | 12 | _sym_db = _symbol_database.Default() 13 | 14 | 15 | 16 | 17 | DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\nauth.proto\x12\x06\x61uthpb\"5\n\x04User\x12\x0c\n\x04name\x18\x01 \x01(\x0c\x12\x10\n\x08password\x18\x02 \x01(\x0c\x12\r\n\x05roles\x18\x03 \x03(\t\"\x83\x01\n\nPermission\x12)\n\x08permType\x18\x01 \x01(\x0e\x32\x17.authpb.Permission.Type\x12\x0b\n\x03key\x18\x02 \x01(\x0c\x12\x11\n\trange_end\x18\x03 \x01(\x0c\"*\n\x04Type\x12\x08\n\x04READ\x10\x00\x12\t\n\x05WRITE\x10\x01\x12\r\n\tREADWRITE\x10\x02\"?\n\x04Role\x12\x0c\n\x04name\x18\x01 \x01(\x0c\x12)\n\rkeyPermission\x18\x02 \x03(\x0b\x32\x12.authpb.PermissionB\x15\n\x11io.etcd.jetcd.apiP\x01\x62\x06proto3') 18 | 19 | 20 | 21 | _USER = DESCRIPTOR.message_types_by_name['User'] 22 | _PERMISSION = DESCRIPTOR.message_types_by_name['Permission'] 23 | _ROLE = DESCRIPTOR.message_types_by_name['Role'] 24 | _PERMISSION_TYPE = _PERMISSION.enum_types_by_name['Type'] 25 | User = _reflection.GeneratedProtocolMessageType('User', (_message.Message,), { 26 | 'DESCRIPTOR' : _USER, 27 | '__module__' : 'auth_pb2' 28 | # @@protoc_insertion_point(class_scope:authpb.User) 29 | }) 30 | _sym_db.RegisterMessage(User) 31 | 32 | Permission = _reflection.GeneratedProtocolMessageType('Permission', (_message.Message,), { 33 | 'DESCRIPTOR' : _PERMISSION, 34 | '__module__' : 'auth_pb2' 35 | # @@protoc_insertion_point(class_scope:authpb.Permission) 36 | }) 37 | _sym_db.RegisterMessage(Permission) 38 | 39 | Role = _reflection.GeneratedProtocolMessageType('Role', (_message.Message,), { 40 | 'DESCRIPTOR' : _ROLE, 41 | '__module__' : 'auth_pb2' 42 | # @@protoc_insertion_point(class_scope:authpb.Role) 43 | }) 44 | _sym_db.RegisterMessage(Role) 45 | 46 | if _descriptor._USE_C_DESCRIPTORS == False: 47 | 48 | DESCRIPTOR._options = None 49 | DESCRIPTOR._serialized_options = b'\n\021io.etcd.jetcd.apiP\001' 50 | _USER._serialized_start=22 51 | _USER._serialized_end=75 52 | _PERMISSION._serialized_start=78 53 | _PERMISSION._serialized_end=209 54 | _PERMISSION_TYPE._serialized_start=167 55 | _PERMISSION_TYPE._serialized_end=209 56 | _ROLE._serialized_start=211 57 | _ROLE._serialized_end=274 58 | # @@protoc_insertion_point(module_scope) 59 | -------------------------------------------------------------------------------- /etcd3/etcdrpc/kv_pb2.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # Generated by the protocol buffer compiler. DO NOT EDIT! 3 | # source: kv.proto 4 | """Generated protocol buffer code.""" 5 | from google.protobuf import descriptor as _descriptor 6 | from google.protobuf import descriptor_pool as _descriptor_pool 7 | from google.protobuf import message as _message 8 | from google.protobuf import reflection as _reflection 9 | from google.protobuf import symbol_database as _symbol_database 10 | # @@protoc_insertion_point(imports) 11 | 12 | _sym_db = _symbol_database.Default() 13 | 14 | 15 | 16 | 17 | DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x08kv.proto\x12\x06mvccpb\"u\n\x08KeyValue\x12\x0b\n\x03key\x18\x01 \x01(\x0c\x12\x17\n\x0f\x63reate_revision\x18\x02 \x01(\x03\x12\x14\n\x0cmod_revision\x18\x03 \x01(\x03\x12\x0f\n\x07version\x18\x04 \x01(\x03\x12\r\n\x05value\x18\x05 \x01(\x0c\x12\r\n\x05lease\x18\x06 \x01(\x03\"\x91\x01\n\x05\x45vent\x12%\n\x04type\x18\x01 \x01(\x0e\x32\x17.mvccpb.Event.EventType\x12\x1c\n\x02kv\x18\x02 \x01(\x0b\x32\x10.mvccpb.KeyValue\x12!\n\x07prev_kv\x18\x03 \x01(\x0b\x32\x10.mvccpb.KeyValue\" \n\tEventType\x12\x07\n\x03PUT\x10\x00\x12\n\n\x06\x44\x45LETE\x10\x01\x42\x15\n\x11io.etcd.jetcd.apiP\x01\x62\x06proto3') 18 | 19 | 20 | 21 | _KEYVALUE = DESCRIPTOR.message_types_by_name['KeyValue'] 22 | _EVENT = DESCRIPTOR.message_types_by_name['Event'] 23 | _EVENT_EVENTTYPE = _EVENT.enum_types_by_name['EventType'] 24 | KeyValue = _reflection.GeneratedProtocolMessageType('KeyValue', (_message.Message,), { 25 | 'DESCRIPTOR' : _KEYVALUE, 26 | '__module__' : 'kv_pb2' 27 | # @@protoc_insertion_point(class_scope:mvccpb.KeyValue) 28 | }) 29 | _sym_db.RegisterMessage(KeyValue) 30 | 31 | Event = _reflection.GeneratedProtocolMessageType('Event', (_message.Message,), { 32 | 'DESCRIPTOR' : _EVENT, 33 | '__module__' : 'kv_pb2' 34 | # @@protoc_insertion_point(class_scope:mvccpb.Event) 35 | }) 36 | _sym_db.RegisterMessage(Event) 37 | 38 | if _descriptor._USE_C_DESCRIPTORS == False: 39 | 40 | DESCRIPTOR._options = None 41 | DESCRIPTOR._serialized_options = b'\n\021io.etcd.jetcd.apiP\001' 42 | _KEYVALUE._serialized_start=20 43 | _KEYVALUE._serialized_end=137 44 | _EVENT._serialized_start=140 45 | _EVENT._serialized_end=285 46 | _EVENT_EVENTTYPE._serialized_start=253 47 | _EVENT_EVENTTYPE._serialized_end=285 48 | # @@protoc_insertion_point(module_scope) 49 | -------------------------------------------------------------------------------- /etcd3/events.py: -------------------------------------------------------------------------------- 1 | class Event(object): 2 | 3 | def __init__(self, event): 4 | self.key = event.kv.key 5 | self._event = event 6 | 7 | def __getattr__(self, name): 8 | if name.startswith('prev_'): 9 | return getattr(self._event.prev_kv, name[5:]) 10 | return getattr(self._event.kv, name) 11 | 12 | def __str__(self): 13 | return '{type} key={key} value={value}'.format(type=self.__class__, 14 | key=self.key, 15 | value=self.value) 16 | 17 | 18 | class PutEvent(Event): 19 | pass 20 | 21 | 22 | class DeleteEvent(Event): 23 | pass 24 | 25 | 26 | def new_event(event): 27 | """ 28 | Wrap a raw gRPC event in a friendlier containing class. 29 | 30 | This picks the appropriate class from one of PutEvent or DeleteEvent and 31 | returns a new instance. 32 | """ 33 | op_name = event.EventType.DESCRIPTOR.values_by_number[event.type].name 34 | if op_name == 'PUT': 35 | cls = PutEvent 36 | elif op_name == 'DELETE': 37 | cls = DeleteEvent 38 | else: 39 | raise Exception('Invalid op_name') 40 | 41 | return cls(event) 42 | -------------------------------------------------------------------------------- /etcd3/exceptions.py: -------------------------------------------------------------------------------- 1 | class Etcd3Exception(Exception): 2 | pass 3 | 4 | 5 | class WatchTimedOut(Etcd3Exception): 6 | pass 7 | 8 | 9 | class InternalServerError(Etcd3Exception): 10 | pass 11 | 12 | 13 | class ConnectionFailedError(Etcd3Exception): 14 | def __str__(self): 15 | return "etcd connection failed" 16 | 17 | 18 | class ConnectionTimeoutError(Etcd3Exception): 19 | def __str__(self): 20 | return "etcd connection timeout" 21 | 22 | 23 | class PreconditionFailedError(Etcd3Exception): 24 | pass 25 | 26 | 27 | class RevisionCompactedError(Etcd3Exception): 28 | def __init__(self, compacted_revision): 29 | self.compacted_revision = compacted_revision 30 | super(RevisionCompactedError, self).__init__() 31 | 32 | 33 | class NoServerAvailableError(Etcd3Exception): 34 | pass 35 | -------------------------------------------------------------------------------- /etcd3/leases.py: -------------------------------------------------------------------------------- 1 | class Lease(object): 2 | """ 3 | A lease. 4 | 5 | :ivar id: ID of the lease 6 | :ivar ttl: time to live for this lease 7 | """ 8 | 9 | def __init__(self, lease_id, ttl, etcd_client=None): 10 | self.id = lease_id 11 | self.ttl = ttl 12 | 13 | self.etcd_client = etcd_client 14 | 15 | def _get_lease_info(self): 16 | return self.etcd_client.get_lease_info(self.id) 17 | 18 | def revoke(self): 19 | """Revoke this lease.""" 20 | self.etcd_client.revoke_lease(self.id) 21 | 22 | def refresh(self): 23 | """Refresh the time to live for this lease.""" 24 | return list(self.etcd_client.refresh_lease(self.id)) 25 | 26 | @property 27 | def remaining_ttl(self): 28 | return self._get_lease_info().TTL 29 | 30 | @property 31 | def granted_ttl(self): 32 | return self._get_lease_info().grantedTTL 33 | 34 | @property 35 | def keys(self): 36 | return self._get_lease_info().keys 37 | -------------------------------------------------------------------------------- /etcd3/locks.py: -------------------------------------------------------------------------------- 1 | import threading 2 | import time 3 | import uuid 4 | 5 | from etcd3 import events, exceptions 6 | 7 | 8 | class Lock(object): 9 | """ 10 | A distributed lock. 11 | 12 | This can be used as a context manager, with the lock being acquired and 13 | released as you would expect: 14 | 15 | .. code-block:: python 16 | 17 | etcd = etcd3.client() 18 | 19 | # create a lock that expires after 20 seconds 20 | with etcd.lock('toot', ttl=20) as lock: 21 | # do something that requires the lock 22 | print(lock.is_acquired()) 23 | 24 | # refresh the timeout on the lease 25 | lock.refresh() 26 | 27 | :param name: name of the lock 28 | :type name: string or bytes 29 | :param ttl: length of time for the lock to live for in seconds. The lock 30 | will be released after this time elapses, unless refreshed 31 | :type ttl: int 32 | """ 33 | 34 | lock_prefix = '/locks/' 35 | 36 | def __init__(self, name, ttl=60, 37 | etcd_client=None): 38 | self.name = name 39 | self.ttl = ttl 40 | if etcd_client is not None: 41 | self.etcd_client = etcd_client 42 | 43 | self.key = self.lock_prefix + self.name 44 | self.lease = None 45 | # store uuid as bytes, since it avoids having to decode each time we 46 | # need to compare 47 | self.uuid = uuid.uuid1().bytes 48 | 49 | def acquire(self, timeout=10): 50 | """Acquire the lock. 51 | 52 | :params timeout: Maximum time to wait before returning. `None` means 53 | forever, any other value equal or greater than 0 is 54 | the number of seconds. 55 | :returns: True if the lock has been acquired, False otherwise. 56 | 57 | """ 58 | if timeout is not None: 59 | deadline = time.time() + timeout 60 | 61 | while True: 62 | if self._try_acquire(): 63 | return True 64 | 65 | if timeout is not None: 66 | remaining_timeout = max(deadline - time.time(), 0) 67 | if remaining_timeout == 0: 68 | return False 69 | else: 70 | remaining_timeout = None 71 | 72 | self._wait_delete_event(remaining_timeout) 73 | 74 | def _try_acquire(self): 75 | self.lease = self.etcd_client.lease(self.ttl) 76 | 77 | success, metadata = self.etcd_client.transaction( 78 | compare=[ 79 | self.etcd_client.transactions.create(self.key) == 0 80 | ], 81 | success=[ 82 | self.etcd_client.transactions.put(self.key, self.uuid, 83 | lease=self.lease) 84 | ], 85 | failure=[ 86 | self.etcd_client.transactions.get(self.key) 87 | ] 88 | ) 89 | if success is True: 90 | self.revision = metadata[0].response_put.header.revision 91 | return True 92 | self.revision = metadata[0][0][1].mod_revision 93 | self.lease = None 94 | return False 95 | 96 | def _wait_delete_event(self, timeout): 97 | try: 98 | event_iter, cancel = self.etcd_client.watch( 99 | self.key, start_revision=self.revision + 1) 100 | except exceptions.WatchTimedOut: 101 | return 102 | 103 | if timeout is not None: 104 | timer = threading.Timer(timeout, cancel) 105 | timer.start() 106 | else: 107 | timer = None 108 | 109 | for event in event_iter: 110 | if isinstance(event, events.DeleteEvent): 111 | if timer is not None: 112 | timer.cancel() 113 | 114 | cancel() 115 | break 116 | 117 | def release(self): 118 | """Release the lock.""" 119 | success, _ = self.etcd_client.transaction( 120 | compare=[ 121 | self.etcd_client.transactions.value(self.key) == self.uuid 122 | ], 123 | success=[self.etcd_client.transactions.delete(self.key)], 124 | failure=[] 125 | ) 126 | return success 127 | 128 | def refresh(self): 129 | """Refresh the time to live on this lock.""" 130 | if self.lease is not None: 131 | return self.lease.refresh() 132 | else: 133 | raise ValueError('No lease associated with this lock - have you ' 134 | 'acquired the lock yet?') 135 | 136 | def is_acquired(self): 137 | """Check if this lock is currently acquired.""" 138 | uuid, _ = self.etcd_client.get(self.key) 139 | 140 | if uuid is None: 141 | return False 142 | 143 | return uuid == self.uuid 144 | 145 | def __enter__(self): 146 | self.acquire() 147 | return self 148 | 149 | def __exit__(self, exception_type, exception_value, traceback): 150 | self.release() 151 | -------------------------------------------------------------------------------- /etcd3/members.py: -------------------------------------------------------------------------------- 1 | class Member(object): 2 | """ 3 | A member of the etcd cluster. 4 | 5 | :ivar id: ID of the member 6 | :ivar name: human-readable name of the member 7 | :ivar peer_urls: list of URLs the member exposes to the cluster for 8 | communication 9 | :ivar client_urls: list of URLs the member exposes to clients for 10 | communication 11 | """ 12 | 13 | def __init__(self, id, name, peer_urls, client_urls, etcd_client=None): 14 | self.id = id 15 | self.name = name 16 | self.peer_urls = peer_urls 17 | self.client_urls = client_urls 18 | self._etcd_client = etcd_client 19 | 20 | def __str__(self): 21 | return ('Member {id}: peer urls: {peer_urls}, client ' 22 | 'urls: {client_urls}'.format(id=self.id, 23 | peer_urls=self.peer_urls, 24 | client_urls=self.client_urls)) 25 | 26 | def remove(self): 27 | """Remove this member from the cluster.""" 28 | self._etcd_client.remove_member(self.id) 29 | 30 | def update(self, peer_urls): 31 | """ 32 | Update the configuration of this member. 33 | 34 | :param peer_urls: new list of peer urls the member will use to 35 | communicate with the cluster 36 | """ 37 | self._etcd_client.update_member(self.id, peer_urls) 38 | 39 | @property 40 | def active_alarms(self): 41 | """Get active alarms of the member. 42 | 43 | :returns: Alarms 44 | """ 45 | return self._etcd_client.list_alarms(member_id=self.id) 46 | -------------------------------------------------------------------------------- /etcd3/proto/auth.proto: -------------------------------------------------------------------------------- 1 | // 2 | // Copyright 2016-2021 The jetcd authors 3 | // 4 | // Licensed under the Apache License, Version 2.0 (the "License"); 5 | // you may not use this file except in compliance with the License. 6 | // You may obtain a copy of the License at 7 | // 8 | // http://www.apache.org/licenses/LICENSE-2.0 9 | // 10 | // Unless required by applicable law or agreed to in writing, software 11 | // distributed under the License is distributed on an "AS IS" BASIS, 12 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | // See the License for the specific language governing permissions and 14 | // limitations under the License. 15 | // 16 | 17 | syntax = "proto3"; 18 | 19 | package authpb; 20 | 21 | option java_multiple_files = true; 22 | option java_package = "io.etcd.jetcd.api"; 23 | 24 | // User is a single entry in the bucket authUsers 25 | message User { 26 | bytes name = 1; 27 | bytes password = 2; 28 | repeated string roles = 3; 29 | } 30 | 31 | // Permission is a single entity 32 | message Permission { 33 | enum Type { 34 | READ = 0; 35 | WRITE = 1; 36 | READWRITE = 2; 37 | } 38 | Type permType = 1; 39 | 40 | bytes key = 2; 41 | bytes range_end = 3; 42 | } 43 | 44 | // Role is a single entry in the bucket authRoles 45 | message Role { 46 | bytes name = 1; 47 | 48 | repeated Permission keyPermission = 2; 49 | } -------------------------------------------------------------------------------- /etcd3/proto/kv.proto: -------------------------------------------------------------------------------- 1 | // 2 | // Copyright 2016-2021 The jetcd authors 3 | // 4 | // Licensed under the Apache License, Version 2.0 (the "License"); 5 | // you may not use this file except in compliance with the License. 6 | // You may obtain a copy of the License at 7 | // 8 | // http://www.apache.org/licenses/LICENSE-2.0 9 | // 10 | // Unless required by applicable law or agreed to in writing, software 11 | // distributed under the License is distributed on an "AS IS" BASIS, 12 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | // See the License for the specific language governing permissions and 14 | // limitations under the License. 15 | // 16 | 17 | syntax = "proto3"; 18 | 19 | package mvccpb; 20 | 21 | option java_multiple_files = true; 22 | option java_package = "io.etcd.jetcd.api"; 23 | 24 | message KeyValue { 25 | // key is the key in bytes. An empty key is not allowed. 26 | bytes key = 1; 27 | // create_revision is the revision of last creation on this key. 28 | int64 create_revision = 2; 29 | // mod_revision is the revision of last modification on this key. 30 | int64 mod_revision = 3; 31 | // version is the version of the key. A deletion resets 32 | // the version to zero and any modification of the key 33 | // increases its version. 34 | int64 version = 4; 35 | // value is the value held by the key, in bytes. 36 | bytes value = 5; 37 | // lease is the ID of the lease that attached to key. 38 | // When the attached lease expires, the key will be deleted. 39 | // If lease is 0, then no lease is attached to the key. 40 | int64 lease = 6; 41 | } 42 | 43 | message Event { 44 | enum EventType { 45 | PUT = 0; 46 | DELETE = 1; 47 | } 48 | // type is the kind of event. If type is a PUT, it indicates 49 | // new data has been stored to the key. If type is a DELETE, 50 | // it indicates the key was deleted. 51 | EventType type = 1; 52 | // kv holds the KeyValue for the event. 53 | // A PUT event contains current kv pair. 54 | // A PUT event with kv.Version=1 indicates the creation of a key. 55 | // A DELETE/EXPIRE event contains the deleted key with 56 | // its modification revision set to the revision of deletion. 57 | KeyValue kv = 2; 58 | 59 | // prev_kv holds the key-value pair before the event happens. 60 | KeyValue prev_kv = 3; 61 | } -------------------------------------------------------------------------------- /etcd3/proto/rpc.proto: -------------------------------------------------------------------------------- 1 | // 2 | // Copyright 2016-2021 The jetcd authors 3 | // 4 | // Licensed under the Apache License, Version 2.0 (the "License"); 5 | // you may not use this file except in compliance with the License. 6 | // You may obtain a copy of the License at 7 | // 8 | // http://www.apache.org/licenses/LICENSE-2.0 9 | // 10 | // Unless required by applicable law or agreed to in writing, software 11 | // distributed under the License is distributed on an "AS IS" BASIS, 12 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 | // See the License for the specific language governing permissions and 14 | // limitations under the License. 15 | // 16 | 17 | syntax = "proto3"; 18 | 19 | import "kv.proto"; 20 | import "auth.proto"; 21 | 22 | package etcdserverpb; 23 | 24 | option java_multiple_files = true; 25 | option java_package = "io.etcd.jetcd.api"; 26 | option java_outer_classname = "JetcdProto"; 27 | option objc_class_prefix = "Jetcd"; 28 | 29 | service KV { 30 | // Range gets the keys in the range from the key-value store. 31 | rpc Range(RangeRequest) returns (RangeResponse) {} 32 | 33 | // Put puts the given key into the key-value store. 34 | // A put request increments the revision of the key-value store 35 | // and generates one event in the event history. 36 | rpc Put(PutRequest) returns (PutResponse) {} 37 | 38 | // DeleteRange deletes the given range from the key-value store. 39 | // A delete request increments the revision of the key-value store 40 | // and generates a delete event in the event history for every deleted key. 41 | rpc DeleteRange(DeleteRangeRequest) returns (DeleteRangeResponse) {} 42 | 43 | // Txn processes multiple requests in a single transaction. 44 | // A txn request increments the revision of the key-value store 45 | // and generates events with the same revision for every completed request. 46 | // It is not allowed to modify the same key several times within one txn. 47 | rpc Txn(TxnRequest) returns (TxnResponse) {} 48 | 49 | // Compact compacts the event history in the etcd key-value store. The key-value 50 | // store should be periodically compacted or the event history will continue to grow 51 | // indefinitely. 52 | rpc Compact(CompactionRequest) returns (CompactionResponse) {} 53 | } 54 | 55 | service Watch { 56 | // Progress requests that a watch stream progress status 57 | // be sent in the watch response stream as soon as possible. 58 | // For watch progress responses, the header.revision indicates progress. All future events 59 | // received in this stream are guaranteed to have a higher revision number than the 60 | // header.revision number. 61 | rpc Progress(WatchProgressRequest) returns (WatchResponse) {} 62 | 63 | // Watch watches for events happening or that have happened. Both input and output 64 | // are streams; the input stream is for creating and canceling watchers and the output 65 | // stream sends events. One watch RPC can watch on multiple key ranges, streaming events 66 | // for several watches at once. The entire event history can be watched starting from the 67 | // last compaction revision. 68 | rpc Watch(stream WatchRequest) returns (stream WatchResponse) {} 69 | } 70 | 71 | service Lease { 72 | // LeaseGrant creates a lease which expires if the server does not receive a keepAlive 73 | // within a given time to live period. All keys attached to the lease will be expired and 74 | // deleted if the lease expires. Each expired key generates a delete event in the event history. 75 | rpc LeaseGrant(LeaseGrantRequest) returns (LeaseGrantResponse) {} 76 | 77 | // LeaseRevoke revokes a lease. All keys attached to the lease will expire and be deleted. 78 | rpc LeaseRevoke(LeaseRevokeRequest) returns (LeaseRevokeResponse) {} 79 | 80 | // LeaseKeepAlive keeps the lease alive by streaming keep alive requests from the client 81 | // to the server and streaming keep alive responses from the server to the client. 82 | rpc LeaseKeepAlive(stream LeaseKeepAliveRequest) returns (stream LeaseKeepAliveResponse) {} 83 | 84 | // LeaseTimeToLive retrieves lease information. 85 | rpc LeaseTimeToLive(LeaseTimeToLiveRequest) returns (LeaseTimeToLiveResponse) {} 86 | 87 | // TODO(xiangli) List all existing Leases? 88 | } 89 | 90 | service Cluster { 91 | // MemberAdd adds a member into the cluster. 92 | rpc MemberAdd(MemberAddRequest) returns (MemberAddResponse) {} 93 | 94 | // MemberRemove removes an existing member from the cluster. 95 | rpc MemberRemove(MemberRemoveRequest) returns (MemberRemoveResponse) {} 96 | 97 | // MemberUpdate updates the member configuration. 98 | rpc MemberUpdate(MemberUpdateRequest) returns (MemberUpdateResponse) {} 99 | 100 | // MemberList lists all the members in the cluster. 101 | rpc MemberList(MemberListRequest) returns (MemberListResponse) {} 102 | } 103 | 104 | service Maintenance { 105 | // Alarm activates, deactivates, and queries alarms regarding cluster health. 106 | rpc Alarm(AlarmRequest) returns (AlarmResponse) {} 107 | 108 | // Status gets the status of the member. 109 | rpc Status(StatusRequest) returns (StatusResponse) {} 110 | 111 | // Defragment defragments a member's backend database to recover storage space. 112 | rpc Defragment(DefragmentRequest) returns (DefragmentResponse) {} 113 | 114 | // Hash returns the hash of the local KV state for consistency checking purpose. 115 | // This is designed for testing; do not use this in production when there 116 | // are ongoing transactions. 117 | rpc Hash(HashRequest) returns (HashResponse) {} 118 | 119 | // HashKV computes the hash of all MVCC keys up to a given revision. 120 | rpc HashKV(HashKVRequest) returns (HashKVResponse) {} 121 | 122 | // Snapshot sends a snapshot of the entire backend from a member over a stream to a client. 123 | rpc Snapshot(SnapshotRequest) returns (stream SnapshotResponse) {} 124 | 125 | // MoveLeader requests current leader node to transfer its leadership to transferee. 126 | rpc MoveLeader(MoveLeaderRequest) returns (MoveLeaderResponse) {} 127 | } 128 | 129 | service Auth { 130 | // AuthEnable enables authentication. 131 | rpc AuthEnable(AuthEnableRequest) returns (AuthEnableResponse) {} 132 | 133 | // AuthDisable disables authentication. 134 | rpc AuthDisable(AuthDisableRequest) returns (AuthDisableResponse) {} 135 | 136 | // Authenticate processes an authenticate request. 137 | rpc Authenticate(AuthenticateRequest) returns (AuthenticateResponse) {} 138 | 139 | // UserAdd adds a new user. 140 | rpc UserAdd(AuthUserAddRequest) returns (AuthUserAddResponse) {} 141 | 142 | // UserGet gets detailed user information. 143 | rpc UserGet(AuthUserGetRequest) returns (AuthUserGetResponse) {} 144 | 145 | // UserList gets a list of all users. 146 | rpc UserList(AuthUserListRequest) returns (AuthUserListResponse) {} 147 | 148 | // UserDelete deletes a specified user. 149 | rpc UserDelete(AuthUserDeleteRequest) returns (AuthUserDeleteResponse) {} 150 | 151 | // UserChangePassword changes the password of a specified user. 152 | rpc UserChangePassword(AuthUserChangePasswordRequest) returns (AuthUserChangePasswordResponse) {} 153 | 154 | // UserGrant grants a role to a specified user. 155 | rpc UserGrantRole(AuthUserGrantRoleRequest) returns (AuthUserGrantRoleResponse) {} 156 | 157 | // UserRevokeRole revokes a role of specified user. 158 | rpc UserRevokeRole(AuthUserRevokeRoleRequest) returns (AuthUserRevokeRoleResponse) {} 159 | 160 | // RoleAdd adds a new role. 161 | rpc RoleAdd(AuthRoleAddRequest) returns (AuthRoleAddResponse) {} 162 | 163 | // RoleGet gets detailed role information. 164 | rpc RoleGet(AuthRoleGetRequest) returns (AuthRoleGetResponse) {} 165 | 166 | // RoleList gets lists of all roles. 167 | rpc RoleList(AuthRoleListRequest) returns (AuthRoleListResponse) {} 168 | 169 | // RoleDelete deletes a specified role. 170 | rpc RoleDelete(AuthRoleDeleteRequest) returns (AuthRoleDeleteResponse) {} 171 | 172 | // RoleGrantPermission grants a permission of a specified key or range to a specified role. 173 | rpc RoleGrantPermission(AuthRoleGrantPermissionRequest) returns (AuthRoleGrantPermissionResponse) {} 174 | 175 | // RoleRevokePermission revokes a key or range permission of a specified role. 176 | rpc RoleRevokePermission(AuthRoleRevokePermissionRequest) returns (AuthRoleRevokePermissionResponse) {} 177 | } 178 | 179 | message ResponseHeader { 180 | // cluster_id is the ID of the cluster which sent the response. 181 | uint64 cluster_id = 1; 182 | // member_id is the ID of the member which sent the response. 183 | uint64 member_id = 2; 184 | // revision is the key-value store revision when the request was applied. 185 | // For watch progress responses, the header.revision indicates progress. All future events 186 | // recieved in this stream are guaranteed to have a higher revision number than the 187 | // header.revision number. 188 | int64 revision = 3; 189 | // raft_term is the raft term when the request was applied. 190 | uint64 raft_term = 4; 191 | } 192 | 193 | message RangeRequest { 194 | enum SortOrder { 195 | NONE = 0; // default, no sorting 196 | ASCEND = 1; // lowest target value first 197 | DESCEND = 2; // highest target value first 198 | } 199 | enum SortTarget { 200 | KEY = 0; 201 | VERSION = 1; 202 | CREATE = 2; 203 | MOD = 3; 204 | VALUE = 4; 205 | } 206 | 207 | // key is the first key for the range. If range_end is not given, the request only looks up key. 208 | bytes key = 1; 209 | // range_end is the upper bound on the requested range [key, range_end). 210 | // If range_end is '\0', the range is all keys >= key. 211 | // If range_end is key plus one (e.g., "aa"+1 == "ab", "a\xff"+1 == "b"), 212 | // then the range request gets all keys prefixed with key. 213 | // If both key and range_end are '\0', then the range request returns all keys. 214 | bytes range_end = 2; 215 | // limit is a limit on the number of keys returned for the request. When limit is set to 0, 216 | // it is treated as no limit. 217 | int64 limit = 3; 218 | // revision is the point-in-time of the key-value store to use for the range. 219 | // If revision is less or equal to zero, the range is over the newest key-value store. 220 | // If the revision has been compacted, ErrCompacted is returned as a response. 221 | int64 revision = 4; 222 | 223 | // sort_order is the order for returned sorted results. 224 | SortOrder sort_order = 5; 225 | 226 | // sort_target is the key-value field to use for sorting. 227 | SortTarget sort_target = 6; 228 | 229 | // serializable sets the range request to use serializable member-local reads. 230 | // Range requests are linearizable by default; linearizable requests have higher 231 | // latency and lower throughput than serializable requests but reflect the current 232 | // consensus of the cluster. For better performance, in exchange for possible stale reads, 233 | // a serializable range request is served locally without needing to reach consensus 234 | // with other nodes in the cluster. 235 | bool serializable = 7; 236 | 237 | // keys_only when set returns only the keys and not the values. 238 | bool keys_only = 8; 239 | 240 | // count_only when set returns only the count of the keys in the range. 241 | bool count_only = 9; 242 | 243 | // min_mod_revision is the lower bound for returned key mod revisions; all keys with 244 | // lesser mod revisions will be filtered away. 245 | int64 min_mod_revision = 10; 246 | 247 | // max_mod_revision is the upper bound for returned key mod revisions; all keys with 248 | // greater mod revisions will be filtered away. 249 | int64 max_mod_revision = 11; 250 | 251 | // min_create_revision is the lower bound for returned key create revisions; all keys with 252 | // lesser create trevisions will be filtered away. 253 | int64 min_create_revision = 12; 254 | 255 | // max_create_revision is the upper bound for returned key create revisions; all keys with 256 | // greater create revisions will be filtered away. 257 | int64 max_create_revision = 13; 258 | } 259 | 260 | message RangeResponse { 261 | ResponseHeader header = 1; 262 | // kvs is the list of key-value pairs matched by the range request. 263 | // kvs is empty when count is requested. 264 | repeated mvccpb.KeyValue kvs = 2; 265 | // more indicates if there are more keys to return in the requested range. 266 | bool more = 3; 267 | // count is set to the number of keys within the range when requested. 268 | int64 count = 4; 269 | } 270 | 271 | message PutRequest { 272 | // key is the key, in bytes, to put into the key-value store. 273 | bytes key = 1; 274 | // value is the value, in bytes, to associate with the key in the key-value store. 275 | bytes value = 2; 276 | // lease is the lease ID to associate with the key in the key-value store. A lease 277 | // value of 0 indicates no lease. 278 | int64 lease = 3; 279 | 280 | // If prev_kv is set, etcd gets the previous key-value pair before changing it. 281 | // The previous key-value pair will be returned in the put response. 282 | bool prev_kv = 4; 283 | 284 | // If ignore_value is set, etcd updates the key using its current value. 285 | // Returns an error if the key does not exist. 286 | bool ignore_value = 5; 287 | 288 | // If ignore_lease is set, etcd updates the key using its current lease. 289 | // Returns an error if the key does not exist. 290 | bool ignore_lease = 6; 291 | } 292 | 293 | message PutResponse { 294 | ResponseHeader header = 1; 295 | // if prev_kv is set in the request, the previous key-value pair will be returned. 296 | mvccpb.KeyValue prev_kv = 2; 297 | } 298 | 299 | message DeleteRangeRequest { 300 | // key is the first key to delete in the range. 301 | bytes key = 1; 302 | // range_end is the key following the last key to delete for the range [key, range_end). 303 | // If range_end is not given, the range is defined to contain only the key argument. 304 | // If range_end is one bit larger than the given key, then the range is all the keys 305 | // with the prefix (the given key). 306 | // If range_end is '\0', the range is all keys greater than or equal to the key argument. 307 | bytes range_end = 2; 308 | 309 | // If prev_kv is set, etcd gets the previous key-value pairs before deleting it. 310 | // The previous key-value pairs will be returned in the delete response. 311 | bool prev_kv = 3; 312 | } 313 | 314 | message DeleteRangeResponse { 315 | ResponseHeader header = 1; 316 | // deleted is the number of keys deleted by the delete range request. 317 | int64 deleted = 2; 318 | // if prev_kv is set in the request, the previous key-value pairs will be returned. 319 | repeated mvccpb.KeyValue prev_kvs = 3; 320 | } 321 | 322 | message RequestOp { 323 | // request is a union of request types accepted by a transaction. 324 | oneof request { 325 | RangeRequest request_range = 1; 326 | PutRequest request_put = 2; 327 | DeleteRangeRequest request_delete_range = 3; 328 | TxnRequest request_txn = 4; 329 | } 330 | } 331 | 332 | message ResponseOp { 333 | // response is a union of response types returned by a transaction. 334 | oneof response { 335 | RangeResponse response_range = 1; 336 | PutResponse response_put = 2; 337 | DeleteRangeResponse response_delete_range = 3; 338 | TxnResponse response_txn = 4; 339 | } 340 | } 341 | 342 | message Compare { 343 | enum CompareResult { 344 | EQUAL = 0; 345 | GREATER = 1; 346 | LESS = 2; 347 | NOT_EQUAL = 3; 348 | } 349 | enum CompareTarget { 350 | VERSION = 0; 351 | CREATE = 1; 352 | MOD = 2; 353 | VALUE = 3; 354 | LEASE = 4; 355 | } 356 | // result is logical comparison operation for this comparison. 357 | CompareResult result = 1; 358 | // target is the key-value field to inspect for the comparison. 359 | CompareTarget target = 2; 360 | // key is the subject key for the comparison operation. 361 | bytes key = 3; 362 | oneof target_union { 363 | // version is the version of the given key 364 | int64 version = 4; 365 | // create_revision is the creation revision of the given key 366 | int64 create_revision = 5; 367 | // mod_revision is the last modified revision of the given key. 368 | int64 mod_revision = 6; 369 | // value is the value of the given key, in bytes. 370 | bytes value = 7; 371 | // lease is the lease id of the given key. 372 | int64 lease = 8; 373 | // leave room for more target_union field tags, jump to 64 374 | } 375 | 376 | // range_end compares the given target to all keys in the range [key, range_end). 377 | // See RangeRequest for more details on key ranges. 378 | bytes range_end = 64; 379 | // TODO: fill out with most of the rest of RangeRequest fields when needed. 380 | } 381 | 382 | // From google paxosdb paper: 383 | // Our implementation hinges around a powerful primitive which we call MultiOp. All other database 384 | // operations except for iteration are implemented as a single call to MultiOp. A MultiOp is applied atomically 385 | // and consists of three components: 386 | // 1. A list of tests called guard. Each test in guard checks a single entry in the database. It may check 387 | // for the absence or presence of a value, or compare with a given value. Two different tests in the guard 388 | // may apply to the same or different entries in the database. All tests in the guard are applied and 389 | // MultiOp returns the results. If all tests are true, MultiOp executes t op (see item 2 below), otherwise 390 | // it executes f op (see item 3 below). 391 | // 2. A list of database operations called t op. Each operation in the list is either an insert, delete, or 392 | // lookup operation, and applies to a single database entry. Two different operations in the list may apply 393 | // to the same or different entries in the database. These operations are executed 394 | // if guard evaluates to 395 | // true. 396 | // 3. A list of database operations called f op. Like t op, but executed if guard evaluates to false. 397 | message TxnRequest { 398 | // compare is a list of predicates representing a conjunction of terms. 399 | // If the comparisons succeed, then the success requests will be processed in order, 400 | // and the response will contain their respective responses in order. 401 | // If the comparisons fail, then the failure requests will be processed in order, 402 | // and the response will contain their respective responses in order. 403 | repeated Compare compare = 1; 404 | // success is a list of requests which will be applied when compare evaluates to true. 405 | repeated RequestOp success = 2; 406 | // failure is a list of requests which will be applied when compare evaluates to false. 407 | repeated RequestOp failure = 3; 408 | } 409 | 410 | message TxnResponse { 411 | ResponseHeader header = 1; 412 | // succeeded is set to true if the compare evaluated to true or false otherwise. 413 | bool succeeded = 2; 414 | // responses is a list of responses corresponding to the results from applying 415 | // success if succeeded is true or failure if succeeded is false. 416 | repeated ResponseOp responses = 3; 417 | } 418 | 419 | // CompactionRequest compacts the key-value store up to a given revision. All superseded keys 420 | // with a revision less than the compaction revision will be removed. 421 | message CompactionRequest { 422 | // revision is the key-value store revision for the compaction operation. 423 | int64 revision = 1; 424 | // physical is set so the RPC will wait until the compaction is physically 425 | // applied to the local database such that compacted entries are totally 426 | // removed from the backend database. 427 | bool physical = 2; 428 | } 429 | 430 | message CompactionResponse { 431 | ResponseHeader header = 1; 432 | } 433 | 434 | message HashRequest { 435 | } 436 | 437 | message HashResponse { 438 | ResponseHeader header = 1; 439 | // hash is the hash value computed from the responding member's key-value store. 440 | uint32 hash = 2; 441 | } 442 | 443 | message HashKVRequest { 444 | // revision is the key-value store revision for the hash operation. 445 | int64 revision = 1; 446 | } 447 | 448 | message HashKVResponse { 449 | ResponseHeader header = 1; 450 | // hash is the hash value computed from the responding member's MVCC keys up to a given revision. 451 | uint32 hash = 2; 452 | // compact_revision is the compacted revision of key-value store when hash begins. 453 | int64 compact_revision = 3; 454 | } 455 | 456 | message SnapshotRequest { 457 | } 458 | 459 | message SnapshotResponse { 460 | // header has the current key-value store information. The first header in the snapshot 461 | // stream indicates the point in time of the snapshot. 462 | ResponseHeader header = 1; 463 | 464 | // remaining_bytes is the number of blob bytes to be sent after this message 465 | uint64 remaining_bytes = 2; 466 | 467 | // blob contains the next chunk of the snapshot in the snapshot stream. 468 | bytes blob = 3; 469 | } 470 | 471 | message WatchRequest { 472 | // request_union is a request to either create a new watcher or cancel an existing watcher. 473 | oneof request_union { 474 | WatchCreateRequest create_request = 1; 475 | WatchCancelRequest cancel_request = 2; 476 | WatchProgressRequest progress_request = 3; 477 | } 478 | } 479 | 480 | message WatchCreateRequest { 481 | // key is the key to register for watching. 482 | bytes key = 1; 483 | // range_end is the end of the range [key, range_end) to watch. If range_end is not given, 484 | // only the key argument is watched. If range_end is equal to '\0', all keys greater than 485 | // or equal to the key argument are watched. 486 | // If the range_end is one bit larger than the given key, 487 | // then all keys with the prefix (the given key) will be watched. 488 | bytes range_end = 2; 489 | // start_revision is an optional revision to watch from (inclusive). No start_revision is "now". 490 | int64 start_revision = 3; 491 | // progress_notify is set so that the etcd server will periodically send a WatchResponse with 492 | // no events to the new watcher if there are no recent events. It is useful when clients 493 | // wish to recover a disconnected watcher starting from a recent known revision. 494 | // The etcd server may decide how often it will send notifications based on current load. 495 | bool progress_notify = 4; 496 | 497 | enum FilterType { 498 | // filter out put event. 499 | NOPUT = 0; 500 | // filter out delete event. 501 | NODELETE = 1; 502 | } 503 | // filters filter the events at server side before it sends back to the watcher. 504 | repeated FilterType filters = 5; 505 | 506 | // If prev_kv is set, created watcher gets the previous KV before the event happens. 507 | // If the previous KV is already compacted, nothing will be returned. 508 | bool prev_kv = 6; 509 | } 510 | 511 | message WatchCancelRequest { 512 | // watch_id is the watcher id to cancel so that no more events are transmitted. 513 | int64 watch_id = 1; 514 | } 515 | 516 | // Requests a watch stream progress status be sent in the 517 | // watch response stream as soon as possible. 518 | message WatchProgressRequest { 519 | } 520 | 521 | message WatchResponse { 522 | ResponseHeader header = 1; 523 | // watch_id is the ID of the watcher that corresponds to the response. 524 | int64 watch_id = 2; 525 | // created is set to true if the response is for a create watch request. 526 | // The client should record the watch_id and expect to receive events for 527 | // the created watcher from the same stream. 528 | // All events sent to the created watcher will attach with the same watch_id. 529 | bool created = 3; 530 | // canceled is set to true if the response is for a cancel watch request. 531 | // No further events will be sent to the canceled watcher. 532 | bool canceled = 4; 533 | // compact_revision is set to the minimum index if a watcher tries to watch 534 | // at a compacted index. 535 | // 536 | // This happens when creating a watcher at a compacted revision or the watcher cannot 537 | // catch up with the progress of the key-value store. 538 | // 539 | // The client should treat the watcher as canceled and should not try to create any 540 | // watcher with the same start_revision again. 541 | int64 compact_revision = 5; 542 | 543 | // cancel_reason indicates the reason for canceling the watcher. 544 | string cancel_reason = 6; 545 | 546 | repeated mvccpb.Event events = 11; 547 | } 548 | 549 | message LeaseGrantRequest { 550 | // TTL is the advisory time-to-live in seconds. 551 | int64 TTL = 1; 552 | // ID is the requested ID for the lease. If ID is set to 0, the lessor chooses an ID. 553 | int64 ID = 2; 554 | } 555 | 556 | message LeaseGrantResponse { 557 | ResponseHeader header = 1; 558 | // ID is the lease ID for the granted lease. 559 | int64 ID = 2; 560 | // TTL is the server chosen lease time-to-live in seconds. 561 | int64 TTL = 3; 562 | string error = 4; 563 | } 564 | 565 | message LeaseRevokeRequest { 566 | // ID is the lease ID to revoke. When the ID is revoked, all associated keys will be deleted. 567 | int64 ID = 1; 568 | } 569 | 570 | message LeaseRevokeResponse { 571 | ResponseHeader header = 1; 572 | } 573 | 574 | message LeaseKeepAliveRequest { 575 | // ID is the lease ID for the lease to keep alive. 576 | int64 ID = 1; 577 | } 578 | 579 | message LeaseKeepAliveResponse { 580 | ResponseHeader header = 1; 581 | // ID is the lease ID from the keep alive request. 582 | int64 ID = 2; 583 | // TTL is the new time-to-live for the lease. 584 | int64 TTL = 3; 585 | } 586 | 587 | message LeaseTimeToLiveRequest { 588 | // ID is the lease ID for the lease. 589 | int64 ID = 1; 590 | // keys is true to query all the keys attached to this lease. 591 | bool keys = 2; 592 | } 593 | 594 | message LeaseTimeToLiveResponse { 595 | ResponseHeader header = 1; 596 | // ID is the lease ID from the keep alive request. 597 | int64 ID = 2; 598 | // TTL is the remaining TTL in seconds for the lease; the lease will expire in under TTL+1 seconds. 599 | int64 TTL = 3; 600 | // GrantedTTL is the initial granted time in seconds upon lease creation/renewal. 601 | int64 grantedTTL = 4; 602 | // Keys is the list of keys attached to this lease. 603 | repeated bytes keys = 5; 604 | } 605 | 606 | message Member { 607 | // ID is the member ID for this member. 608 | uint64 ID = 1; 609 | // name is the human-readable name of the member. If the member is not started, the name will be an empty string. 610 | string name = 2; 611 | // peerURLs is the list of URLs the member exposes to the cluster for communication. 612 | repeated string peerURLs = 3; 613 | // clientURLs is the list of URLs the member exposes to clients for communication. If the member is not started, clientURLs will be empty. 614 | repeated string clientURLs = 4; 615 | } 616 | 617 | message MemberAddRequest { 618 | // peerURLs is the list of URLs the added member will use to communicate with the cluster. 619 | repeated string peerURLs = 1; 620 | } 621 | 622 | message MemberAddResponse { 623 | ResponseHeader header = 1; 624 | // member is the member information for the added member. 625 | Member member = 2; 626 | // members is a list of all members after adding the new member. 627 | repeated Member members = 3; 628 | } 629 | 630 | message MemberRemoveRequest { 631 | // ID is the member ID of the member to remove. 632 | uint64 ID = 1; 633 | } 634 | 635 | message MemberRemoveResponse { 636 | ResponseHeader header = 1; 637 | // members is a list of all members after removing the member. 638 | repeated Member members = 2; 639 | } 640 | 641 | message MemberUpdateRequest { 642 | // ID is the member ID of the member to update. 643 | uint64 ID = 1; 644 | // peerURLs is the new list of URLs the member will use to communicate with the cluster. 645 | repeated string peerURLs = 2; 646 | } 647 | 648 | message MemberUpdateResponse{ 649 | ResponseHeader header = 1; 650 | // members is a list of all members after updating the member. 651 | repeated Member members = 2; 652 | } 653 | 654 | message MemberListRequest { 655 | } 656 | 657 | message MemberListResponse { 658 | ResponseHeader header = 1; 659 | // members is a list of all members associated with the cluster. 660 | repeated Member members = 2; 661 | } 662 | 663 | message DefragmentRequest { 664 | } 665 | 666 | message DefragmentResponse { 667 | ResponseHeader header = 1; 668 | } 669 | 670 | message MoveLeaderRequest { 671 | // targetID is the node ID for the new leader. 672 | uint64 targetID = 1; 673 | } 674 | 675 | message MoveLeaderResponse { 676 | ResponseHeader header = 1; 677 | } 678 | 679 | enum AlarmType { 680 | NONE = 0; // default, used to query if any alarm is active 681 | NOSPACE = 1; // space quota is exhausted 682 | } 683 | 684 | message AlarmRequest { 685 | enum AlarmAction { 686 | GET = 0; 687 | ACTIVATE = 1; 688 | DEACTIVATE = 2; 689 | } 690 | // action is the kind of alarm request to issue. The action 691 | // may GET alarm statuses, ACTIVATE an alarm, or DEACTIVATE a 692 | // raised alarm. 693 | AlarmAction action = 1; 694 | // memberID is the ID of the member associated with the alarm. If memberID is 0, the 695 | // alarm request covers all members. 696 | uint64 memberID = 2; 697 | // alarm is the type of alarm to consider for this request. 698 | AlarmType alarm = 3; 699 | } 700 | 701 | message AlarmMember { 702 | // memberID is the ID of the member associated with the raised alarm. 703 | uint64 memberID = 1; 704 | // alarm is the type of alarm which has been raised. 705 | AlarmType alarm = 2; 706 | } 707 | 708 | message AlarmResponse { 709 | ResponseHeader header = 1; 710 | // alarms is a list of alarms associated with the alarm request. 711 | repeated AlarmMember alarms = 2; 712 | } 713 | 714 | message StatusRequest { 715 | } 716 | 717 | message StatusResponse { 718 | ResponseHeader header = 1; 719 | // version is the cluster protocol version used by the responding member. 720 | string version = 2; 721 | // dbSize is the size of the backend database, in bytes, of the responding member. 722 | int64 dbSize = 3; 723 | // leader is the member ID which the responding member believes is the current leader. 724 | uint64 leader = 4; 725 | // raftIndex is the current raft index of the responding member. 726 | uint64 raftIndex = 5; 727 | // raftTerm is the current raft term of the responding member. 728 | uint64 raftTerm = 6; 729 | } 730 | 731 | message AuthEnableRequest { 732 | } 733 | 734 | message AuthDisableRequest { 735 | } 736 | 737 | message AuthenticateRequest { 738 | string name = 1; 739 | string password = 2; 740 | } 741 | 742 | message AuthUserAddRequest { 743 | string name = 1; 744 | string password = 2; 745 | } 746 | 747 | message AuthUserGetRequest { 748 | string name = 1; 749 | } 750 | 751 | message AuthUserDeleteRequest { 752 | // name is the name of the user to delete. 753 | string name = 1; 754 | } 755 | 756 | message AuthUserChangePasswordRequest { 757 | // name is the name of the user whose password is being changed. 758 | string name = 1; 759 | // password is the new password for the user. 760 | string password = 2; 761 | } 762 | 763 | message AuthUserGrantRoleRequest { 764 | // user is the name of the user which should be granted a given role. 765 | string user = 1; 766 | // role is the name of the role to grant to the user. 767 | string role = 2; 768 | } 769 | 770 | message AuthUserRevokeRoleRequest { 771 | string name = 1; 772 | string role = 2; 773 | } 774 | 775 | message AuthRoleAddRequest { 776 | // name is the name of the role to add to the authentication system. 777 | string name = 1; 778 | } 779 | 780 | message AuthRoleGetRequest { 781 | string role = 1; 782 | } 783 | 784 | message AuthUserListRequest { 785 | } 786 | 787 | message AuthRoleListRequest { 788 | } 789 | 790 | message AuthRoleDeleteRequest { 791 | string role = 1; 792 | } 793 | 794 | message AuthRoleGrantPermissionRequest { 795 | // name is the name of the role which will be granted the permission. 796 | string name = 1; 797 | // perm is the permission to grant to the role. 798 | authpb.Permission perm = 2; 799 | } 800 | 801 | message AuthRoleRevokePermissionRequest { 802 | string role = 1; 803 | string key = 2; 804 | string range_end = 3; 805 | } 806 | 807 | message AuthEnableResponse { 808 | ResponseHeader header = 1; 809 | } 810 | 811 | message AuthDisableResponse { 812 | ResponseHeader header = 1; 813 | } 814 | 815 | message AuthenticateResponse { 816 | ResponseHeader header = 1; 817 | // token is an authorized token that can be used in succeeding RPCs 818 | string token = 2; 819 | } 820 | 821 | message AuthUserAddResponse { 822 | ResponseHeader header = 1; 823 | } 824 | 825 | message AuthUserGetResponse { 826 | ResponseHeader header = 1; 827 | 828 | repeated string roles = 2; 829 | } 830 | 831 | message AuthUserDeleteResponse { 832 | ResponseHeader header = 1; 833 | } 834 | 835 | message AuthUserChangePasswordResponse { 836 | ResponseHeader header = 1; 837 | } 838 | 839 | message AuthUserGrantRoleResponse { 840 | ResponseHeader header = 1; 841 | } 842 | 843 | message AuthUserRevokeRoleResponse { 844 | ResponseHeader header = 1; 845 | } 846 | 847 | message AuthRoleAddResponse { 848 | ResponseHeader header = 1; 849 | } 850 | 851 | message AuthRoleGetResponse { 852 | ResponseHeader header = 1; 853 | 854 | repeated authpb.Permission perm = 2; 855 | } 856 | 857 | message AuthRoleListResponse { 858 | ResponseHeader header = 1; 859 | 860 | repeated string roles = 2; 861 | } 862 | 863 | message AuthUserListResponse { 864 | ResponseHeader header = 1; 865 | 866 | repeated string users = 2; 867 | } 868 | 869 | message AuthRoleDeleteResponse { 870 | ResponseHeader header = 1; 871 | } 872 | 873 | message AuthRoleGrantPermissionResponse { 874 | ResponseHeader header = 1; 875 | } 876 | 877 | message AuthRoleRevokePermissionResponse { 878 | ResponseHeader header = 1; 879 | } -------------------------------------------------------------------------------- /etcd3/transactions.py: -------------------------------------------------------------------------------- 1 | import etcd3.etcdrpc as etcdrpc 2 | import etcd3.utils as utils 3 | 4 | _OPERATORS = { 5 | etcdrpc.Compare.EQUAL: "==", 6 | etcdrpc.Compare.NOT_EQUAL: "!=", 7 | etcdrpc.Compare.LESS: "<", 8 | etcdrpc.Compare.GREATER: ">" 9 | } 10 | 11 | 12 | class BaseCompare(object): 13 | def __init__(self, key, range_end=None): 14 | self.key = key 15 | self.range_end = range_end 16 | self.value = None 17 | self.op = None 18 | 19 | # TODO check other is of correct type for compare 20 | # Version, Mod and Create can only be ints 21 | def __eq__(self, other): 22 | self.value = other 23 | self.op = etcdrpc.Compare.EQUAL 24 | return self 25 | 26 | def __ne__(self, other): 27 | self.value = other 28 | self.op = etcdrpc.Compare.NOT_EQUAL 29 | return self 30 | 31 | def __lt__(self, other): 32 | self.value = other 33 | self.op = etcdrpc.Compare.LESS 34 | return self 35 | 36 | def __gt__(self, other): 37 | self.value = other 38 | self.op = etcdrpc.Compare.GREATER 39 | return self 40 | 41 | def __repr__(self): 42 | if self.range_end is None: 43 | keys = self.key 44 | else: 45 | keys = "[{}, {})".format(self.key, self.range_end) 46 | return "{}: {} {} '{}'".format(self.__class__, keys, 47 | _OPERATORS.get(self.op), 48 | self.value) 49 | 50 | def build_message(self): 51 | compare = etcdrpc.Compare() 52 | compare.key = utils.to_bytes(self.key) 53 | if self.range_end is not None: 54 | compare.range_end = utils.to_bytes(self.range_end) 55 | 56 | if self.op is None: 57 | raise ValueError('op must be one of =, !=, < or >') 58 | 59 | compare.result = self.op 60 | 61 | self.build_compare(compare) 62 | return compare 63 | 64 | 65 | class Value(BaseCompare): 66 | def build_compare(self, compare): 67 | compare.target = etcdrpc.Compare.VALUE 68 | compare.value = utils.to_bytes(self.value) 69 | 70 | 71 | class Version(BaseCompare): 72 | def build_compare(self, compare): 73 | compare.target = etcdrpc.Compare.VERSION 74 | compare.version = int(self.value) 75 | 76 | 77 | class Create(BaseCompare): 78 | def build_compare(self, compare): 79 | compare.target = etcdrpc.Compare.CREATE 80 | compare.create_revision = int(self.value) 81 | 82 | 83 | class Mod(BaseCompare): 84 | def build_compare(self, compare): 85 | compare.target = etcdrpc.Compare.MOD 86 | compare.mod_revision = int(self.value) 87 | 88 | 89 | class Put(object): 90 | def __init__(self, key, value, lease=None, prev_kv=False): 91 | self.key = key 92 | self.value = value 93 | self.lease = lease 94 | self.prev_kv = prev_kv 95 | 96 | 97 | class Get(object): 98 | def __init__(self, key, range_end=None): 99 | self.key = key 100 | self.range_end = range_end 101 | 102 | 103 | class Delete(object): 104 | def __init__(self, key, range_end=None, prev_kv=False): 105 | self.key = key 106 | self.range_end = range_end 107 | self.prev_kv = prev_kv 108 | 109 | 110 | class Txn(object): 111 | def __init__(self, compare, success=None, failure=None): 112 | self.compare = compare 113 | self.success = success 114 | self.failure = failure 115 | -------------------------------------------------------------------------------- /etcd3/utils.py: -------------------------------------------------------------------------------- 1 | def prefix_range_end(prefix): 2 | """Create a bytestring that can be used as a range_end for a prefix.""" 3 | s = bytearray(prefix) 4 | for i in reversed(range(len(s))): 5 | if s[i] < 0xff: 6 | s[i] = s[i] + 1 7 | break 8 | return bytes(s) 9 | 10 | 11 | def to_bytes(maybe_bytestring): 12 | """ 13 | Encode string to bytes. 14 | 15 | Convenience function to do a simple encode('utf-8') if the input is not 16 | already bytes. Returns the data unmodified if the input is bytes. 17 | """ 18 | if isinstance(maybe_bytestring, bytes): 19 | return maybe_bytestring 20 | else: 21 | return maybe_bytestring.encode('utf-8') 22 | 23 | 24 | def lease_to_id(lease): 25 | """Figure out if the argument is a Lease object, or the lease ID.""" 26 | lease_id = 0 27 | if hasattr(lease, 'id'): 28 | lease_id = lease.id 29 | else: 30 | try: 31 | lease_id = int(lease) 32 | except TypeError: 33 | pass 34 | return lease_id 35 | 36 | 37 | def response_to_event_iterator(response_iterator): 38 | """Convert a watch response iterator to an event iterator.""" 39 | for response in response_iterator: 40 | for event in response.events: 41 | yield event 42 | -------------------------------------------------------------------------------- /etcd3/watch.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import threading 3 | 4 | import grpc 5 | 6 | import six 7 | from six.moves import queue 8 | 9 | import etcd3.etcdrpc as etcdrpc 10 | import etcd3.events as events 11 | import etcd3.exceptions as exceptions 12 | import etcd3.utils as utils 13 | 14 | 15 | _log = logging.getLogger(__name__) 16 | 17 | 18 | class Watch(object): 19 | 20 | def __init__(self, watch_id, iterator=None, etcd_client=None): 21 | self.watch_id = watch_id 22 | self.etcd_client = etcd_client 23 | self.iterator = iterator 24 | 25 | def cancel(self): 26 | self.etcd_client.cancel_watch(self.watch_id) 27 | 28 | def iterator(self): 29 | if self.iterator is not None: 30 | return self.iterator 31 | 32 | raise ValueError('Undefined iterator') 33 | 34 | 35 | class Watcher(object): 36 | 37 | def __init__(self, watchstub, timeout=None, call_credentials=None, 38 | metadata=None): 39 | self.timeout = timeout 40 | self._watch_stub = watchstub 41 | self._credentials = call_credentials 42 | self._metadata = metadata 43 | 44 | self._lock = threading.Lock() 45 | self._request_queue = queue.Queue(maxsize=10) 46 | self._callbacks = {} 47 | self._callback_thread = None 48 | self._new_watch_cond = threading.Condition(lock=self._lock) 49 | self._new_watch = None 50 | self._stopping = False 51 | 52 | def _create_watch_request(self, key, range_end=None, start_revision=None, 53 | progress_notify=False, filters=None, 54 | prev_kv=False): 55 | create_watch = etcdrpc.WatchCreateRequest() 56 | create_watch.key = utils.to_bytes(key) 57 | if range_end is not None: 58 | create_watch.range_end = utils.to_bytes(range_end) 59 | if start_revision is not None: 60 | create_watch.start_revision = start_revision 61 | if progress_notify: 62 | create_watch.progress_notify = progress_notify 63 | if filters is not None: 64 | create_watch.filters.extend(filters) 65 | if prev_kv: 66 | create_watch.prev_kv = prev_kv 67 | return etcdrpc.WatchRequest(create_request=create_watch) 68 | 69 | def add_callback(self, key, callback, range_end=None, start_revision=None, 70 | progress_notify=False, filters=None, prev_kv=False): 71 | rq = self._create_watch_request(key, range_end=range_end, 72 | start_revision=start_revision, 73 | progress_notify=progress_notify, 74 | filters=filters, prev_kv=prev_kv) 75 | 76 | with self._lock: 77 | # Wait for exiting thread to close 78 | if self._stopping: 79 | self._callback_thread.join() 80 | self._callback_thread = None 81 | self._stopping = False 82 | 83 | # Start the callback thread if it is not yet running. 84 | if not self._callback_thread: 85 | thread_name = 'etcd3_watch_%x' % (id(self),) 86 | self._callback_thread = threading.Thread(name=thread_name, 87 | target=self._run) 88 | self._callback_thread.daemon = True 89 | self._callback_thread.start() 90 | 91 | # Only one create watch request can be pending at a time, so if 92 | # there one already, then wait for it to complete first. 93 | while self._new_watch: 94 | self._new_watch_cond.wait() 95 | 96 | # Submit a create watch request. 97 | new_watch = _NewWatch(callback) 98 | self._request_queue.put(rq) 99 | self._new_watch = new_watch 100 | 101 | try: 102 | # Wait for the request to be completed, or timeout. 103 | self._new_watch_cond.wait(timeout=self.timeout) 104 | 105 | # If the request not completed yet, then raise a timeout 106 | # exception. 107 | if new_watch.id is None and new_watch.err is None: 108 | raise exceptions.WatchTimedOut() 109 | 110 | # Raise an exception if the watch request failed. 111 | if new_watch.err: 112 | raise new_watch.err 113 | finally: 114 | # Wake up threads stuck on add_callback call if any. 115 | self._new_watch = None 116 | self._new_watch_cond.notify_all() 117 | 118 | return new_watch.id 119 | 120 | def cancel(self, watch_id): 121 | with self._lock: 122 | callback = self._callbacks.pop(watch_id, None) 123 | if not callback: 124 | return 125 | 126 | self._cancel_no_lock(watch_id) 127 | 128 | def _run(self): 129 | callback_err = None 130 | try: 131 | response_iter = self._watch_stub.Watch( 132 | _new_request_iter(self._request_queue), 133 | credentials=self._credentials, 134 | metadata=self._metadata) 135 | for rs in response_iter: 136 | self._handle_response(rs) 137 | 138 | except grpc.RpcError as err: 139 | callback_err = err 140 | 141 | finally: 142 | with self._lock: 143 | self._stopping = True 144 | if self._new_watch: 145 | self._new_watch.err = callback_err 146 | self._new_watch_cond.notify_all() 147 | 148 | callbacks = self._callbacks 149 | self._callbacks = {} 150 | 151 | # Rotate request queue. This way we can terminate one gRPC 152 | # stream and initiate another one whilst avoiding a race 153 | # between them over requests in the queue. 154 | self._request_queue.put(None) 155 | self._request_queue = queue.Queue(maxsize=10) 156 | 157 | for callback in six.itervalues(callbacks): 158 | _safe_callback(callback, callback_err) 159 | 160 | def _handle_response(self, rs): 161 | with self._lock: 162 | if rs.created: 163 | # If the new watch request has already expired then cancel the 164 | # created watch right away. 165 | if not self._new_watch: 166 | self._cancel_no_lock(rs.watch_id) 167 | return 168 | 169 | if rs.compact_revision != 0: 170 | self._new_watch.err = exceptions.RevisionCompactedError( 171 | rs.compact_revision) 172 | return 173 | 174 | self._callbacks[rs.watch_id] = self._new_watch.callback 175 | self._new_watch.id = rs.watch_id 176 | self._new_watch_cond.notify_all() 177 | 178 | callback = self._callbacks.get(rs.watch_id) 179 | 180 | # Ignore leftovers from canceled watches. 181 | if not callback: 182 | return 183 | 184 | # The watcher can be safely reused, but adding a new event 185 | # to indicate that the revision is already compacted 186 | # requires api change which would break all users of this 187 | # module. So, raising an exception if a watcher is still 188 | # alive. 189 | if rs.compact_revision != 0: 190 | err = exceptions.RevisionCompactedError(rs.compact_revision) 191 | _safe_callback(callback, err) 192 | self.cancel(rs.watch_id) 193 | return 194 | 195 | # Call the callback even when there are no events in the watch 196 | # response so as not to ignore progress notify responses. 197 | if rs.events or not (rs.created or rs.canceled): 198 | new_events = [events.new_event(event) for event in rs.events] 199 | response = WatchResponse(rs.header, new_events) 200 | _safe_callback(callback, response) 201 | 202 | def _cancel_no_lock(self, watch_id): 203 | cancel_watch = etcdrpc.WatchCancelRequest() 204 | cancel_watch.watch_id = watch_id 205 | rq = etcdrpc.WatchRequest(cancel_request=cancel_watch) 206 | self._request_queue.put(rq) 207 | 208 | def close(self): 209 | with self._lock: 210 | if self._callback_thread and not self._stopping: 211 | self._request_queue.put(None) 212 | 213 | 214 | class WatchResponse(object): 215 | 216 | def __init__(self, header, events): 217 | self.header = header 218 | self.events = events 219 | 220 | 221 | class _NewWatch(object): 222 | def __init__(self, callback): 223 | self.callback = callback 224 | self.id = None 225 | self.err = None 226 | 227 | 228 | def _new_request_iter(_request_queue): 229 | while True: 230 | rq = _request_queue.get() 231 | if rq is None: 232 | return 233 | 234 | yield rq 235 | 236 | 237 | def _safe_callback(callback, response_or_err): 238 | try: 239 | callback(response_or_err) 240 | 241 | except Exception: 242 | _log.exception('Watch callback failed') 243 | -------------------------------------------------------------------------------- /requirements/base.in: -------------------------------------------------------------------------------- 1 | grpcio>=1.2.0 2 | protobuf>=3.6.1 3 | -------------------------------------------------------------------------------- /requirements/base.txt: -------------------------------------------------------------------------------- 1 | # SHA1:04cc27ab2e85f48da5bc4b15b5820b244a61578a 2 | # 3 | # This file is autogenerated by pip-compile-multi 4 | # To update, run: 5 | # 6 | # pip-compile-multi 7 | # 8 | grpcio==1.38.0 9 | # via -r requirements/base.in 10 | protobuf==3.17.0 11 | # via -r requirements/base.in 12 | six==1.16.0 13 | # via 14 | # grpcio 15 | # protobuf 16 | -------------------------------------------------------------------------------- /requirements/test.in: -------------------------------------------------------------------------------- 1 | -r base.in 2 | PyYAML>=5.1 3 | Sphinx>=1.8.2 4 | bumpversion>=0.5.3 5 | coverage 6 | flake8-import-order 7 | flake8 8 | grpcio-tools 9 | hypothesis 10 | more-itertools<6 # python2.7 11 | pytest>=4.6.5 12 | pytest-cov 13 | tox>=3.5.3 14 | flake8-docstrings>=1.3.0 15 | pydocstyle<4 # python2.7 16 | mock>=2.0.0 17 | pifpaf 18 | tenacity>=5.0.2 19 | -------------------------------------------------------------------------------- /requirements/test.txt: -------------------------------------------------------------------------------- 1 | # SHA1:925716a635eb5e8d454086bef081ecfa689eff02 2 | # 3 | # This file is autogenerated by pip-compile-multi 4 | # To update, run: 5 | # 6 | # pip-compile-multi 7 | # 8 | -r base.txt 9 | alabaster==0.7.12 10 | # via sphinx 11 | appdirs==1.4.4 12 | # via virtualenv 13 | argparse==1.4.0 14 | # via unittest2 15 | attrs==21.2.0 16 | # via 17 | # hypothesis 18 | # pytest 19 | babel==2.9.1 20 | # via sphinx 21 | bump2version==1.0.1 22 | # via bumpversion 23 | bumpversion==0.6.0 24 | # via -r requirements/test.in 25 | certifi==2020.12.5 26 | # via requests 27 | cffi==1.14.5 28 | # via xattr 29 | chardet==4.0.0 30 | # via requests 31 | click==8.0.1 32 | # via pifpaf 33 | coverage[toml]==5.5 34 | # via 35 | # -r requirements/test.in 36 | # pytest-cov 37 | daiquiri==3.0.0 38 | # via pifpaf 39 | distlib==0.3.1 40 | # via virtualenv 41 | docutils==0.17.1 42 | # via sphinx 43 | extras==1.0.0 44 | # via testtools 45 | filelock==3.0.12 46 | # via 47 | # tox 48 | # virtualenv 49 | fixtures==3.0.0 50 | # via 51 | # pifpaf 52 | # testtools 53 | flake8-docstrings==1.6.0 54 | # via -r requirements/test.in 55 | flake8-import-order==0.18.1 56 | # via -r requirements/test.in 57 | flake8==3.9.2 58 | # via 59 | # -r requirements/test.in 60 | # flake8-docstrings 61 | grpcio-tools==1.38.0 62 | # via -r requirements/test.in 63 | hypothesis==6.13.1 64 | # via -r requirements/test.in 65 | idna==2.10 66 | # via requests 67 | imagesize==1.2.0 68 | # via sphinx 69 | iniconfig==1.1.1 70 | # via pytest 71 | jinja2==3.0.1 72 | # via 73 | # pifpaf 74 | # sphinx 75 | linecache2==1.0.0 76 | # via traceback2 77 | markupsafe==2.0.1 78 | # via jinja2 79 | mccabe==0.6.1 80 | # via flake8 81 | mock==4.0.3 82 | # via -r requirements/test.in 83 | more-itertools==5.0.0 84 | # via -r requirements/test.in 85 | packaging==20.9 86 | # via 87 | # pytest 88 | # sphinx 89 | # tox 90 | pbr==5.6.0 91 | # via 92 | # fixtures 93 | # pifpaf 94 | # testtools 95 | pifpaf==3.1.5 96 | # via -r requirements/test.in 97 | pluggy==0.13.1 98 | # via 99 | # pytest 100 | # tox 101 | psutil==5.8.0 102 | # via pifpaf 103 | py==1.10.0 104 | # via 105 | # pytest 106 | # tox 107 | pycodestyle==2.7.0 108 | # via 109 | # flake8 110 | # flake8-import-order 111 | pycparser==2.20 112 | # via cffi 113 | pydocstyle==3.0.0 114 | # via 115 | # -r requirements/test.in 116 | # flake8-docstrings 117 | pyflakes==2.3.1 118 | # via flake8 119 | pygments==2.9.0 120 | # via sphinx 121 | pyparsing==2.4.7 122 | # via packaging 123 | pytest-cov==2.12.0 124 | # via -r requirements/test.in 125 | pytest==6.2.4 126 | # via 127 | # -r requirements/test.in 128 | # pytest-cov 129 | python-json-logger==2.0.1 130 | # via daiquiri 131 | python-mimeparse==1.6.0 132 | # via testtools 133 | pytz==2021.1 134 | # via babel 135 | pyyaml==5.4.1 136 | # via -r requirements/test.in 137 | requests==2.25.1 138 | # via sphinx 139 | snowballstemmer==2.1.0 140 | # via 141 | # pydocstyle 142 | # sphinx 143 | sortedcontainers==2.4.0 144 | # via hypothesis 145 | sphinx==4.0.2 146 | # via -r requirements/test.in 147 | sphinxcontrib-applehelp==1.0.2 148 | # via sphinx 149 | sphinxcontrib-devhelp==1.0.2 150 | # via sphinx 151 | sphinxcontrib-htmlhelp==1.0.3 152 | # via sphinx 153 | sphinxcontrib-jsmath==1.0.1 154 | # via sphinx 155 | sphinxcontrib-qthelp==1.0.3 156 | # via sphinx 157 | sphinxcontrib-serializinghtml==1.1.4 158 | # via sphinx 159 | tenacity==7.0.0 160 | # via -r requirements/test.in 161 | testtools==2.4.0 162 | # via fixtures 163 | toml==0.10.2 164 | # via 165 | # coverage 166 | # pytest 167 | # tox 168 | tox==3.23.1 169 | # via -r requirements/test.in 170 | traceback2==1.4.0 171 | # via 172 | # testtools 173 | # unittest2 174 | unittest2==1.1.0 175 | # via testtools 176 | urllib3==1.26.4 177 | # via requests 178 | virtualenv==20.4.6 179 | # via tox 180 | xattr==0.9.7 181 | # via pifpaf 182 | 183 | # The following packages are considered to be unsafe in a requirements file: 184 | # setuptools 185 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [bumpversion] 2 | current_version = 0.12.0 3 | commit = True 4 | tag = True 5 | message = Release: {current_version} -> {new_version} 6 | 7 | [bumpversion:file:setup.py] 8 | 9 | [bumpversion:file:etcd3/__init__.py] 10 | 11 | [bdist_wheel] 12 | universal = 1 13 | 14 | [flake8] 15 | exclude = docs 16 | builtins = long 17 | 18 | [coverage:run] 19 | omit = etcd3/etcdrpc/* 20 | 21 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | import re 5 | 6 | from setuptools import setup 7 | 8 | with open('README.rst') as readme_file: 9 | readme = readme_file.read() 10 | 11 | with open('HISTORY.rst') as history_file: 12 | history = history_file.read() 13 | 14 | 15 | def load_reqs(filename): 16 | with open(filename) as reqs_file: 17 | return [ 18 | re.sub('==', '>=', line) for line in reqs_file.readlines() 19 | if not re.match(r'(\s*#|-r)', line) 20 | ] 21 | 22 | 23 | requirements = load_reqs('requirements/base.txt') 24 | test_requirements = load_reqs('requirements/test.txt') 25 | 26 | setup( 27 | name='etcd3', 28 | version='0.12.0', 29 | description="Python client for the etcd3 API", 30 | long_description=readme + '\n\n' + history, 31 | author="Louis Taylor", 32 | author_email='louis@kragniz.eu', 33 | url='https://github.com/kragniz/python-etcd3', 34 | packages=[ 35 | 'etcd3', 36 | 'etcd3.etcdrpc', 37 | ], 38 | package_dir={ 39 | 'etcd3': 'etcd3', 40 | 'etcd3.etcdrpc': 'etcd3/etcdrpc', 41 | }, 42 | include_package_data=True, 43 | install_requires=requirements, 44 | license="Apache Software License 2.0", 45 | zip_safe=False, 46 | keywords='etcd3', 47 | classifiers=[ 48 | 'Development Status :: 2 - Pre-Alpha', 49 | 'Intended Audience :: Developers', 50 | 'License :: OSI Approved :: Apache Software License', 51 | 'Natural Language :: English', 52 | "Programming Language :: Python :: 2", 53 | 'Programming Language :: Python :: 2.7', 54 | 'Programming Language :: Python :: 3', 55 | 'Programming Language :: Python :: 3.4', 56 | 'Programming Language :: Python :: 3.5', 57 | 'Programming Language :: Python :: 3.6', 58 | ], 59 | test_suite='tests', 60 | tests_require=test_requirements 61 | ) 62 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | -------------------------------------------------------------------------------- /tests/ca.crt: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE----- 2 | MIIFXTCCA0WgAwIBAgIJAPc9K2+Wfz9SMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV 3 | BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX 4 | aWRnaXRzIFB0eSBMdGQwHhcNMTYxMTMwMTIxMTEzWhcNMTcxMTMwMTIxMTEzWjBF 5 | MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 6 | ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIIC 7 | CgKCAgEAp4jp17VLzoSuUkP+WP7NH9L0fQWZRjmHntsJKF0VO6LFsacwPNrKN5y2 8 | cJaIyTNSABuagr3B3Lad1b/1/C5h7ADPEAJrPdjCHu9Wj0tsoKchyrIaUVN5Ftc0 9 | gSRhOsHrDVL0Rl6n5f0e6STIe2tGpMO76YeaVvW6s6uNANWYdOg1alk6+YDifAWO 10 | KZj+hI/nyI64lr/JxI1GikVxzIuu77sV8sRWKmpA8bkafUXW9mJ5xLUq33MZ09s6 11 | QR2ViXorrt19Dr7X1AuJRxfHjW8Wyq9/Aah4kRafBz9yWbhJsjUBjzKizXgxi2y5 12 | ocQtn2WhxqK3fZaLFxphgQljrfFEIPbOAHtm6T29vytmuB/AaIgxJ/R2rnMrAwvE 13 | M0tdSuZ5CGRgxyBoE+ZbZi3O6CrVvVJG9a0R6/4d8sdFfGuLP3cNTAjug5J+rzAM 14 | BEpRKDXWhms3ylYFXoffwslpzHpW+U/9qRl8COlAxsoX/EOpIm7kBBDPaf4cRWNL 15 | KJjME9AZDzKmKFs3eMRseDNie8XQERTyCm4jIb3xeQ637Ch315enNpA+e0HAIvC7 16 | Sxo59GKm1JdNIA8J/MHn+og1ze4BU2hZLB/zoA5ijPr3r0aDQx3bx0Tfyh+CwcC0 17 | hKzfwGHlZq2Cku2hhksUKiObxIS9MCo/0hQ+xVyKeW4eX6Qlo80CAwEAAaNQME4w 18 | HQYDVR0OBBYEFHBdQW6D9b4Vwlppr/OoE36kvWWwMB8GA1UdIwQYMBaAFHBdQW6D 19 | 9b4Vwlppr/OoE36kvWWwMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggIB 20 | ACXlTxkaWN4TDnEm3gYNbzw6leJ8lVzSs1ismHaMl2nkq5/SHPOv1reZ2d1rlIcE 21 | Ozxu61wf29utd92VcdSCQA+0lw5wRo/npte1lS7NcwyCNLQNBRfzrVEzxIiEMHYZ 22 | VXN3LChxq60t0g0EkmJ/mc/6AAH/MUchjlNZWVQqx0J0UJVoqNyPKqt8nHNUbf6V 23 | 8sBqJMvunKnaEJNG2F8HTufc6oTpM7jru16z98tEWuQFgUNNstdQAcccNy2ccgPj 24 | tg7scfjk3MeFED89Kan97d5k6fmdqVzTWiXKC2sK+aE9aV4FidHoAEhg9Xoo8CJf 25 | gyrDD5TsI8hjrgzYD0uk8z69iDr41eYG0aNlPT9DSvu+EHnoSo6C1PUL/UoYbGH3 26 | krmrUc6b+JQlXmv+3XHtNHdPWgcTLQGCzuOjkNtjhGIdRR3LAxEa6+3/9kPp34aI 27 | Ne1o/Z9ZDuwqmQKeUhrBh8GzPdsSMuyr52M6RqkOhL09fhmmcZT+Q9g3yvF3dAyP 28 | 9OQvhkyqjr/ET7N3C+EhdcfSgaXZZlE+V1nhxIEO/ljMfChCAyKoF2OFhwLx5TLg 29 | Bn6Jl8kkgO9f07cA3xy12tQr1crgLYtb5EuqUi3SjkdRm2z708rK+L8N1YuoOADt 30 | CBNpyN+KIrvyYU1yqt7lIUB8xY9f+lccyQcHUwk0VCfV 31 | -----END CERTIFICATE----- 32 | -------------------------------------------------------------------------------- /tests/client.crt: -------------------------------------------------------------------------------- 1 | -----BEGIN CERTIFICATE----- 2 | MIIE/jCCAuYCAQEwDQYJKoZIhvcNAQELBQAwRTELMAkGA1UEBhMCQVUxEzARBgNV 3 | BAgMClNvbWUtU3RhdGUxITAfBgNVBAoMGEludGVybmV0IFdpZGdpdHMgUHR5IEx0 4 | ZDAeFw0xNjExMzAxMjExMzNaFw0xNzExMzAxMjExMzNaMEUxCzAJBgNVBAYTAkFV 5 | MRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRz 6 | IFB0eSBMdGQwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+3xYlfCgK 7 | M3GNiHot8dW+LkLDdHhFZA9yvb9fMV5eRahDRZfq4NqL+lv1/POD7//FVJQM2ekD 8 | pd+0GCjFhDdHNKlR+buAu6DDxr0PTAj40XdRS6Gh0EecbuTdINo5ROOMzoxImId+ 9 | 2yicAcdlGqdja+d1JXE2PUu7hRxPQitfw1TYiim9lccv1rwravC3BUcWXkFziQXi 10 | dy/0bmT431+3yr8iElkGKhV7e11de+dRpY6K1LktUqbfEr0qBGXmaMvbWRgSqo8z 11 | h5YgaXSNQb6a9Q5uIooka9hP1JwledlUe4IX5m1AtD+IonU+35+xnY4JuJOq2zQ7 12 | /Lc9gT8xtnfItCDScCG0TviORBC7TpId/DYp0GRY8/mky4vgbkLDqETzyg7UmTbY 13 | imlKXjJ0qIBDRGl3yeSeaUzNqmoLREkKUovDgmwG6Zh+GfVKZAUqwru6/RUAteQL 14 | LQT5/bRrXAokK8h3ACa5iFDXXoQ6CDXp5g8GnnODILngB1+8JvxdIWHwIu+zVLN9 15 | gD7q4WqI62GMe7JoMMNfzvmEPkO427/SLqlYq20g2v3lzNL2MKAtLccEyRtO3C6N 16 | gaVp2pBHKkVSRQq82dWWFV1WRKZEWj0t2PYm1r9JfeXOYYPyqqdSSR6BgFJQoJnK 17 | mWGS4l0xy8FuehZSh7iW3WPbSxnifcKE5QIDAQABMA0GCSqGSIb3DQEBCwUAA4IC 18 | AQBAtYZN3smwrYrFgY/PaDL2U5tZUGPqo/TBHNPO5InRb0o2HDK3H8SNgxblfKpP 19 | KkttJBwQ6HBNNx7DTitXU/Jl7mk6PtQr9OjeKUNipBL/EfnZjcUeK1orYSyxgQuy 20 | QitlZFde/BR8m5KMtDZBo1s9nUxWQej+U09yhsV8R/bZSQWJMGqxVNsde/uRY2hd 21 | g5ebcfJkxCGgM3Fdyf0JE5C8BGoK1fQbFsf2hDK85KEAT6NU3c/d9qRKtSq2X+ZL 22 | fICbhXfxa5W1OIPhq8mJeHu0+z5wgkDdO9ilvebWhJT7bkimA90PDtr4oKkS/Nmg 23 | mOtWLfgvq3YIp5S44wi1q2kGBdkRhIQVhesHfhDEjCzAiALmqx3zX4PO3HAWHMj8 24 | NjD1+YIfPoex7KQBfvYf4eLyAbOOeGMFoAC+VtsFQ9wkiopclgHBB+vsNgik2hLn 25 | ROsUXg1I7HkLK87c2luhyhIq0Iq0ONeB2I6bg2fgZxFCR5X5rjbh5QUBDooYkh+x 26 | aQzH88qovX1qPUvXLAvc4TyNRbcAls7M+Rh4UX+7TgOVnx3Slc5VCIeHRosszh+q 27 | lW1ucb30MoURaa9uv3m0IePNKXFla7YuyB39u0c3hjOnaBakkshfb/AE6TZ3X3dt 28 | h1aTHmQFQQWBAfUpkEK0B0qmlisbtpsWBXu8FJB7HEJVBw== 29 | -----END CERTIFICATE----- 30 | -------------------------------------------------------------------------------- /tests/client.key: -------------------------------------------------------------------------------- 1 | -----BEGIN RSA PRIVATE KEY----- 2 | MIIJKAIBAAKCAgEAvt8WJXwoCjNxjYh6LfHVvi5Cw3R4RWQPcr2/XzFeXkWoQ0WX 3 | 6uDai/pb9fzzg+//xVSUDNnpA6XftBgoxYQ3RzSpUfm7gLugw8a9D0wI+NF3UUuh 4 | odBHnG7k3SDaOUTjjM6MSJiHftsonAHHZRqnY2vndSVxNj1Lu4UcT0IrX8NU2Iop 5 | vZXHL9a8K2rwtwVHFl5Bc4kF4ncv9G5k+N9ft8q/IhJZBioVe3tdXXvnUaWOitS5 6 | LVKm3xK9KgRl5mjL21kYEqqPM4eWIGl0jUG+mvUObiKKJGvYT9ScJXnZVHuCF+Zt 7 | QLQ/iKJ1Pt+fsZ2OCbiTqts0O/y3PYE/MbZ3yLQg0nAhtE74jkQQu06SHfw2KdBk 8 | WPP5pMuL4G5Cw6hE88oO1Jk22IppSl4ydKiAQ0Rpd8nknmlMzapqC0RJClKLw4Js 9 | BumYfhn1SmQFKsK7uv0VALXkCy0E+f20a1wKJCvIdwAmuYhQ116EOgg16eYPBp5z 10 | gyC54AdfvCb8XSFh8CLvs1SzfYA+6uFqiOthjHuyaDDDX875hD5DuNu/0i6pWKtt 11 | INr95czS9jCgLS3HBMkbTtwujYGladqQRypFUkUKvNnVlhVdVkSmRFo9Ldj2Jta/ 12 | SX3lzmGD8qqnUkkegYBSUKCZyplhkuJdMcvBbnoWUoe4lt1j20sZ4n3ChOUCAwEA 13 | AQKCAgApTdFXD0UC4BsEi7IU2y5r10rvTPbx3TNSmykcimxnbh95X0f+teXVQLpB 14 | wxlXRcpNg33+QfGgrnZ4XHoGNBRaPlUdvjMLKo8hjpI/Cy8t2PY/C2TQxWeZvI2e 15 | JIvUtMmhINlOpGM92mMxT1k80cXDSAhwW0fieuU2kRmNNT55nKVxroRV2GGgQpG5 16 | u4yXrnaxqnfzboVtemlQNct3566o8SDnWJ0XLmgzroCHyFZIXtE3zcy8uBQQVdfF 17 | jeoXMDpdHyyMbYhLmF2uaeOk5cM/eSRERQkr1IKc099uZK3ZEFkYnu6pf9f302AX 18 | 2QCPyQ6BoAWkhYcKLCfjEWMBeQD2eAQ3AvSbzf5HNK5CSG6tFW2AJbDynQFhhcf+ 19 | zqF7pSHcJ33MUb3Rn4ucqrT/kjBg7xYQUBN+CcetaZacSSOU+OxocgWGto1URjWx 20 | S4G5FXkjHYbWD8ZF6nTYLpk68qSP6yfAfevEB7Cx/kaHKNfjHmmnTyHyMbS6tdq/ 21 | pcYoX9NpGsMRjbkgyOev6pnIdxFpJfUwr8slfoDpAEwcCgtSOty7RiHvh4jMoXDK 22 | VejL5orLO5kIwFEbKAPDQxIU8lb9eiSGMo83ywdOZB7HdHgh+to1i62M6i76BpU2 23 | hV1WUhxbKOnrBH9mdEYhOXN1Qh4i/LQFLgRIQ1aCpakwWEuuSQKCAQEA4VMaGAfw 24 | yvj0s9GlpGmZkMqtY5aQroCHK4DPbcJv9y4DwfIUZZRZTCDAsPHJLJ3a7YkDjvZt 25 | /bTFxvBdoD5cwzcKd0boFsqy6pF5K7lgoZiaEfcfcKfKa6o0IWRNC2hha/pm0tbo 26 | AHdABRjXSpnK5kGQONstRdO+5nBm3nb4763C/m8kXbpEaJkGCp4H4PH+jsSgud/C 27 | e7EyDg1fl2Dm2DapbqnS6Q1MvNe3z1SFDsIePpWnpgVXAizAFPK4P8o97pPdm1ab 28 | maFMuozwUdB2XkhouhaVy/okmLfZTPm+UvfTzx4vOIAgyQ++jR72cPk3tOG9Uxv1 29 | v7v+IdjBNvRMHwKCAQEA2Ns9ZTZqQqbyku15+t2BflOqk/2vLmdqiasrKihpz61Q 30 | 1UgKZlsP/PPNZgd6pwb+rZyzg8B7WGSHFuueaQRk6dBiSE1u5YEGlNZgz/qMlwGv 31 | +qQ5KwTp/SA9Z5sAmvBYSgu7YQqclCXO9xFks5c2PlqEtxdIUrSEgde7Oec5s8iX 32 | 5LTmmznM7gzFIt8T0Qkg+xMiGVoBllEaUQuPx0nd54Zk0Lk881eFopo9yjAUAZVZ 33 | X/PwVH1EoTXQrMn/6GyR2CSgFP04nFCbnnB8hFWjrW9QncbDqCzJomX7f8LPyuy2 34 | MHrtjnggFxG4nTfms+3ZNvHnhIP3cVzO01WZir/OewKCAQBb+S7ahks8nphL2hRx 35 | 4wTi/EgAMZJHGIGUOVTyKX7Id4jjHqxCtmYo0+mzkE4cnyag9N91+JL3D1X3mV+u 36 | LCZzLMFh5JiRzRVw+AZs2ZNfAspI8QVtV6AhiG0VADsOoAG8MI7OGxjCL+r66aPf 37 | eJ1AlZyICLcXHJJ8v37N6eQ8+UFx1+4RMBoCVAwKQ21V3ZGZtxsgI/zfvnl/EOhn 38 | Aw/XS6CnYjyMEnizUJ5fy9EwL/5mb3HqK53Tbm7NXjHlH+ldvA+l+5kyAYwvTZ4+ 39 | 1wep/oZ5BwUIKMfNaqYRbJPKjAxyK5D80BgR2hJeyjev75pUhBxikzQhmlvmdvKP 40 | OvLjAoIBAGSyaW/2NKF+pGVVoK687MScVTkjM5V4sB/9O6331ip5cG+ZZDrjTilH 41 | jLkz/+BPfzNe8HzdhGknRRN/la9uOu3XtcudKUGpCEVUxt/MmDwGrJDWcTVosr63 42 | mcviTgWkVVsM15XYN50TcWeIzBoYvTd3EOl0BkFhUaZ5Zpccp86z9tRcrDioPmDL 43 | zT7EF4+ZjZcI62yaPuJuBqpblAgWZNR9s5K4cXUNzyASP18DtEaH81h7Db4t7RBu 44 | zNdvdUWYJKEZYLxeWUs8owaPdUJ+AeMnLWgWRARzx62BbaeF7rdr863PZ0Agx8Xu 45 | lKtsqdQjPholejwui1g4oFHCDeo/5sECggEBANo1fsxsCKu3ggr5XDoTC4T553ee 46 | u1stfToapGE37y3h0PSYQx3k/sm8SBDpL9uqjda9q/dPgkcFOFkHztjGN+HboHJT 47 | qRb+s4hJqZUpPLbmbZwTKJ7Zgvi2UBbwKUQBSN5pu+AQDkurcOwUk0qVhGMO3Fiq 48 | 6TnAoAuHbJ5hEYQTGj8y1KEhNVX8aDGJRdcjclM6MUnZgvIiifjugMvoLAsPkQO6 49 | VjHWgUfFLdwnJF0P8Swq1q5Du8ToWVXSTUW1gzWhNNHf8CCx8hWk1u1UEtjPxOyu 50 | n3+Y1T5csBjZ73BtxQStqt23oRwhHePmw7XeeUqjqJVxTMelI/bZ9pFxPpY= 51 | -----END RSA PRIVATE KEY----- 52 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = py39, flake8 3 | skipsdist=True 4 | 5 | [testenv:flake8] 6 | commands=flake8 {posargs} 7 | 8 | [testenv] 9 | passenv = ETCD_ENDPOINT TEST_ETCD_VERSION 10 | setenv = 11 | PYTHONPATH = {toxinidir}:{toxinidir}/etcd3 12 | deps= 13 | -r{toxinidir}/requirements/base.txt 14 | -r{toxinidir}/requirements/test.txt 15 | commands = 16 | pip install -U pip 17 | pifpaf -e PYTHON run etcd --cluster -- py.test --cov=etcd3 --cov-report= --basetemp={envtmpdir} {posargs} 18 | 19 | [testenv:coverage] 20 | deps= 21 | -r{toxinidir}/requirements/base.txt 22 | -r{toxinidir}/requirements/test.txt 23 | commands = py.test --cov=etcd3 tests/ 24 | 25 | [testenv:genproto] 26 | whitelist_externals = sed 27 | deps = grpcio-tools 28 | commands = 29 | sed -i -e '/gogoproto/d' etcd3/proto/rpc.proto 30 | sed -i -e 's/etcd\/mvcc\/mvccpb\/kv.proto/kv.proto/g' etcd3/proto/rpc.proto 31 | sed -i -e 's/etcd\/auth\/authpb\/auth.proto/auth.proto/g' etcd3/proto/rpc.proto 32 | sed -i -e '/google\/api\/annotations.proto/d' etcd3/proto/rpc.proto 33 | sed -i -e '/option (google.api.http)/,+3d' etcd3/proto/rpc.proto 34 | python -m grpc.tools.protoc -Ietcd3/proto \ 35 | --python_out=etcd3/etcdrpc/ \ 36 | --grpc_python_out=etcd3/etcdrpc/ \ 37 | etcd3/proto/rpc.proto etcd3/proto/auth.proto etcd3/proto/kv.proto 38 | sed -i -e 's/import auth_pb2/from etcd3.etcdrpc import auth_pb2/g' etcd3/etcdrpc/rpc_pb2.py 39 | sed -i -e 's/import kv_pb2/from etcd3.etcdrpc import kv_pb2/g' etcd3/etcdrpc/rpc_pb2.py 40 | sed -i -e 's/import rpc_pb2/from etcd3.etcdrpc import rpc_pb2/g' etcd3/etcdrpc/rpc_pb2_grpc.py 41 | 42 | [flake8] 43 | exclude = .venv,.git,.tox,dist,docs,*lib/python*,*egg,build,etcd3/etcdrpc/ 44 | application-import-names = etcd3 45 | max-complexity = 10 46 | # TODO add docstrings for public methods, modules, etc 47 | ignore = D1, W503 48 | --------------------------------------------------------------------------------