├── .github └── workflows │ └── ci.yml ├── .gitignore ├── LICENSE ├── LICENSE_CC0 ├── Makefile ├── README.rst ├── conf.py ├── debug_mode.rst ├── examples ├── aiohttp_client.py ├── async_client_blocking.py ├── async_client_nonblocking.py ├── async_page.py ├── asyncio_deferred.py ├── create_task.py ├── hello_clock.py ├── hello_world.py ├── http_client.py ├── loop_stop.py ├── producer_consumer.py ├── producer_consumer_join.py ├── requirements.txt ├── run_in_thread.py ├── simple_server.py ├── subprocess_command.py ├── subprocess_echo.py ├── synchronous_client.py ├── tcp_echo_client.py ├── tcp_echo_server.py └── twisted_deferred.py ├── getting_help.rst ├── getting_started.rst ├── glossary.rst ├── hello_clock.rst ├── hello_world.rst ├── http_client.rst ├── index.rst ├── make.bat ├── performance.rst ├── producer_consumer.rst ├── requirements.txt ├── subprocess.rst ├── tcp_echo.rst ├── threads.rst ├── twisted.rst ├── webscraper.rst └── why_asyncio.rst /.github/workflows/ci.yml: -------------------------------------------------------------------------------- 1 | name: Tests 2 | 3 | on: 4 | pull_request: 5 | push: 6 | branches: master 7 | 8 | jobs: 9 | test: 10 | name: test 11 | runs-on: ubuntu-latest 12 | 13 | steps: 14 | - uses: actions/checkout@v2 15 | - uses: actions/cache@v1 16 | with: 17 | path: ~/.cache/pip 18 | key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }} 19 | restore-keys: | 20 | ${{ runner.os }}-pip- 21 | - uses: actions/setup-python@v2 22 | with: 23 | python-version: 3.9 24 | - name: Install Dependencies 25 | run: python3 -m pip install -U pip -r requirements.txt 26 | - name: Build Docs 27 | run: sphinx-build -n -W -q -b html -d _build/doctrees . _build/html 28 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.py[cod] 2 | *.swp 3 | build 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Attribution-ShareAlike 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More_considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution-ShareAlike 4.0 International Public 58 | License 59 | 60 | By exercising the Licensed Rights (defined below), You accept and agree 61 | to be bound by the terms and conditions of this Creative Commons 62 | Attribution-ShareAlike 4.0 International Public License ("Public 63 | License"). To the extent this Public License may be interpreted as a 64 | contract, You are granted the Licensed Rights in consideration of Your 65 | acceptance of these terms and conditions, and the Licensor grants You 66 | such rights in consideration of benefits the Licensor receives from 67 | making the Licensed Material available under these terms and 68 | conditions. 69 | 70 | 71 | Section 1 -- Definitions. 72 | 73 | a. Adapted Material means material subject to Copyright and Similar 74 | Rights that is derived from or based upon the Licensed Material 75 | and in which the Licensed Material is translated, altered, 76 | arranged, transformed, or otherwise modified in a manner requiring 77 | permission under the Copyright and Similar Rights held by the 78 | Licensor. For purposes of this Public License, where the Licensed 79 | Material is a musical work, performance, or sound recording, 80 | Adapted Material is always produced where the Licensed Material is 81 | synched in timed relation with a moving image. 82 | 83 | b. Adapter's License means the license You apply to Your Copyright 84 | and Similar Rights in Your contributions to Adapted Material in 85 | accordance with the terms and conditions of this Public License. 86 | 87 | c. BY-SA Compatible License means a license listed at 88 | creativecommons.org/compatiblelicenses, approved by Creative 89 | Commons as essentially the equivalent of this Public License. 90 | 91 | d. Copyright and Similar Rights means copyright and/or similar rights 92 | closely related to copyright including, without limitation, 93 | performance, broadcast, sound recording, and Sui Generis Database 94 | Rights, without regard to how the rights are labeled or 95 | categorized. For purposes of this Public License, the rights 96 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 97 | Rights. 98 | 99 | e. Effective Technological Measures means those measures that, in the 100 | absence of proper authority, may not be circumvented under laws 101 | fulfilling obligations under Article 11 of the WIPO Copyright 102 | Treaty adopted on December 20, 1996, and/or similar international 103 | agreements. 104 | 105 | f. Exceptions and Limitations means fair use, fair dealing, and/or 106 | any other exception or limitation to Copyright and Similar Rights 107 | that applies to Your use of the Licensed Material. 108 | 109 | g. License Elements means the license attributes listed in the name 110 | of a Creative Commons Public License. The License Elements of this 111 | Public License are Attribution and ShareAlike. 112 | 113 | h. Licensed Material means the artistic or literary work, database, 114 | or other material to which the Licensor applied this Public 115 | License. 116 | 117 | i. Licensed Rights means the rights granted to You subject to the 118 | terms and conditions of this Public License, which are limited to 119 | all Copyright and Similar Rights that apply to Your use of the 120 | Licensed Material and that the Licensor has authority to license. 121 | 122 | j. Licensor means the individual(s) or entity(ies) granting rights 123 | under this Public License. 124 | 125 | k. Share means to provide material to the public by any means or 126 | process that requires permission under the Licensed Rights, such 127 | as reproduction, public display, public performance, distribution, 128 | dissemination, communication, or importation, and to make material 129 | available to the public including in ways that members of the 130 | public may access the material from a place and at a time 131 | individually chosen by them. 132 | 133 | l. Sui Generis Database Rights means rights other than copyright 134 | resulting from Directive 96/9/EC of the European Parliament and of 135 | the Council of 11 March 1996 on the legal protection of databases, 136 | as amended and/or succeeded, as well as other essentially 137 | equivalent rights anywhere in the world. 138 | 139 | m. You means the individual or entity exercising the Licensed Rights 140 | under this Public License. Your has a corresponding meaning. 141 | 142 | 143 | Section 2 -- Scope. 144 | 145 | a. License grant. 146 | 147 | 1. Subject to the terms and conditions of this Public License, 148 | the Licensor hereby grants You a worldwide, royalty-free, 149 | non-sublicensable, non-exclusive, irrevocable license to 150 | exercise the Licensed Rights in the Licensed Material to: 151 | 152 | a. reproduce and Share the Licensed Material, in whole or 153 | in part; and 154 | 155 | b. produce, reproduce, and Share Adapted Material. 156 | 157 | 2. Exceptions and Limitations. For the avoidance of doubt, where 158 | Exceptions and Limitations apply to Your use, this Public 159 | License does not apply, and You do not need to comply with 160 | its terms and conditions. 161 | 162 | 3. Term. The term of this Public License is specified in Section 163 | 6(a). 164 | 165 | 4. Media and formats; technical modifications allowed. The 166 | Licensor authorizes You to exercise the Licensed Rights in 167 | all media and formats whether now known or hereafter created, 168 | and to make technical modifications necessary to do so. The 169 | Licensor waives and/or agrees not to assert any right or 170 | authority to forbid You from making technical modifications 171 | necessary to exercise the Licensed Rights, including 172 | technical modifications necessary to circumvent Effective 173 | Technological Measures. For purposes of this Public License, 174 | simply making modifications authorized by this Section 2(a) 175 | (4) never produces Adapted Material. 176 | 177 | 5. Downstream recipients. 178 | 179 | a. Offer from the Licensor -- Licensed Material. Every 180 | recipient of the Licensed Material automatically 181 | receives an offer from the Licensor to exercise the 182 | Licensed Rights under the terms and conditions of this 183 | Public License. 184 | 185 | b. Additional offer from the Licensor -- Adapted Material. 186 | Every recipient of Adapted Material from You 187 | automatically receives an offer from the Licensor to 188 | exercise the Licensed Rights in the Adapted Material 189 | under the conditions of the Adapter's License You apply. 190 | 191 | c. No downstream restrictions. You may not offer or impose 192 | any additional or different terms or conditions on, or 193 | apply any Effective Technological Measures to, the 194 | Licensed Material if doing so restricts exercise of the 195 | Licensed Rights by any recipient of the Licensed 196 | Material. 197 | 198 | 6. No endorsement. Nothing in this Public License constitutes or 199 | may be construed as permission to assert or imply that You 200 | are, or that Your use of the Licensed Material is, connected 201 | with, or sponsored, endorsed, or granted official status by, 202 | the Licensor or others designated to receive attribution as 203 | provided in Section 3(a)(1)(A)(i). 204 | 205 | b. Other rights. 206 | 207 | 1. Moral rights, such as the right of integrity, are not 208 | licensed under this Public License, nor are publicity, 209 | privacy, and/or other similar personality rights; however, to 210 | the extent possible, the Licensor waives and/or agrees not to 211 | assert any such rights held by the Licensor to the limited 212 | extent necessary to allow You to exercise the Licensed 213 | Rights, but not otherwise. 214 | 215 | 2. Patent and trademark rights are not licensed under this 216 | Public License. 217 | 218 | 3. To the extent possible, the Licensor waives any right to 219 | collect royalties from You for the exercise of the Licensed 220 | Rights, whether directly or through a collecting society 221 | under any voluntary or waivable statutory or compulsory 222 | licensing scheme. In all other cases the Licensor expressly 223 | reserves any right to collect such royalties. 224 | 225 | 226 | Section 3 -- License Conditions. 227 | 228 | Your exercise of the Licensed Rights is expressly made subject to the 229 | following conditions. 230 | 231 | a. Attribution. 232 | 233 | 1. If You Share the Licensed Material (including in modified 234 | form), You must: 235 | 236 | a. retain the following if it is supplied by the Licensor 237 | with the Licensed Material: 238 | 239 | i. identification of the creator(s) of the Licensed 240 | Material and any others designated to receive 241 | attribution, in any reasonable manner requested by 242 | the Licensor (including by pseudonym if 243 | designated); 244 | 245 | ii. a copyright notice; 246 | 247 | iii. a notice that refers to this Public License; 248 | 249 | iv. a notice that refers to the disclaimer of 250 | warranties; 251 | 252 | v. a URI or hyperlink to the Licensed Material to the 253 | extent reasonably practicable; 254 | 255 | b. indicate if You modified the Licensed Material and 256 | retain an indication of any previous modifications; and 257 | 258 | c. indicate the Licensed Material is licensed under this 259 | Public License, and include the text of, or the URI or 260 | hyperlink to, this Public License. 261 | 262 | 2. You may satisfy the conditions in Section 3(a)(1) in any 263 | reasonable manner based on the medium, means, and context in 264 | which You Share the Licensed Material. For example, it may be 265 | reasonable to satisfy the conditions by providing a URI or 266 | hyperlink to a resource that includes the required 267 | information. 268 | 269 | 3. If requested by the Licensor, You must remove any of the 270 | information required by Section 3(a)(1)(A) to the extent 271 | reasonably practicable. 272 | 273 | b. ShareAlike. 274 | 275 | In addition to the conditions in Section 3(a), if You Share 276 | Adapted Material You produce, the following conditions also apply. 277 | 278 | 1. The Adapter's License You apply must be a Creative Commons 279 | license with the same License Elements, this version or 280 | later, or a BY-SA Compatible License. 281 | 282 | 2. You must include the text of, or the URI or hyperlink to, the 283 | Adapter's License You apply. You may satisfy this condition 284 | in any reasonable manner based on the medium, means, and 285 | context in which You Share Adapted Material. 286 | 287 | 3. You may not offer or impose any additional or different terms 288 | or conditions on, or apply any Effective Technological 289 | Measures to, Adapted Material that restrict exercise of the 290 | rights granted under the Adapter's License You apply. 291 | 292 | 293 | Section 4 -- Sui Generis Database Rights. 294 | 295 | Where the Licensed Rights include Sui Generis Database Rights that 296 | apply to Your use of the Licensed Material: 297 | 298 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 299 | to extract, reuse, reproduce, and Share all or a substantial 300 | portion of the contents of the database; 301 | 302 | b. if You include all or a substantial portion of the database 303 | contents in a database in which You have Sui Generis Database 304 | Rights, then the database in which You have Sui Generis Database 305 | Rights (but not its individual contents) is Adapted Material, 306 | 307 | including for purposes of Section 3(b); and 308 | c. You must comply with the conditions in Section 3(a) if You Share 309 | all or a substantial portion of the contents of the database. 310 | 311 | For the avoidance of doubt, this Section 4 supplements and does not 312 | replace Your obligations under this Public License where the Licensed 313 | Rights include other Copyright and Similar Rights. 314 | 315 | 316 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 317 | 318 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 319 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 320 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 321 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 322 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 323 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 324 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 325 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 326 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 327 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 328 | 329 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 330 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 331 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 332 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 333 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 334 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 335 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 336 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 337 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 338 | 339 | c. The disclaimer of warranties and limitation of liability provided 340 | above shall be interpreted in a manner that, to the extent 341 | possible, most closely approximates an absolute disclaimer and 342 | waiver of all liability. 343 | 344 | 345 | Section 6 -- Term and Termination. 346 | 347 | a. This Public License applies for the term of the Copyright and 348 | Similar Rights licensed here. However, if You fail to comply with 349 | this Public License, then Your rights under this Public License 350 | terminate automatically. 351 | 352 | b. Where Your right to use the Licensed Material has terminated under 353 | Section 6(a), it reinstates: 354 | 355 | 1. automatically as of the date the violation is cured, provided 356 | it is cured within 30 days of Your discovery of the 357 | violation; or 358 | 359 | 2. upon express reinstatement by the Licensor. 360 | 361 | For the avoidance of doubt, this Section 6(b) does not affect any 362 | right the Licensor may have to seek remedies for Your violations 363 | of this Public License. 364 | 365 | c. For the avoidance of doubt, the Licensor may also offer the 366 | Licensed Material under separate terms or conditions or stop 367 | distributing the Licensed Material at any time; however, doing so 368 | will not terminate this Public License. 369 | 370 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 371 | License. 372 | 373 | 374 | Section 7 -- Other Terms and Conditions. 375 | 376 | a. The Licensor shall not be bound by any additional or different 377 | terms or conditions communicated by You unless expressly agreed. 378 | 379 | b. Any arrangements, understandings, or agreements regarding the 380 | Licensed Material not stated herein are separate from and 381 | independent of the terms and conditions of this Public License. 382 | 383 | 384 | Section 8 -- Interpretation. 385 | 386 | a. For the avoidance of doubt, this Public License does not, and 387 | shall not be interpreted to, reduce, limit, restrict, or impose 388 | conditions on any use of the Licensed Material that could lawfully 389 | be made without permission under this Public License. 390 | 391 | b. To the extent possible, if any provision of this Public License is 392 | deemed unenforceable, it shall be automatically reformed to the 393 | minimum extent necessary to make it enforceable. If the provision 394 | cannot be reformed, it shall be severed from this Public License 395 | without affecting the enforceability of the remaining terms and 396 | conditions. 397 | 398 | c. No term or condition of this Public License will be waived and no 399 | failure to comply consented to unless expressly agreed to by the 400 | Licensor. 401 | 402 | d. Nothing in this Public License constitutes or may be interpreted 403 | as a limitation upon, or waiver of, any privileges and immunities 404 | that apply to the Licensor or You, including from the legal 405 | processes of any jurisdiction or authority. 406 | 407 | 408 | ======================================================================= 409 | 410 | Creative Commons is not a party to its public 411 | licenses. Notwithstanding, Creative Commons may elect to apply one of 412 | its public licenses to material it publishes and in those instances 413 | will be considered the “Licensor.” The text of the Creative Commons 414 | public licenses is dedicated to the public domain under the CC0 Public 415 | Domain Dedication. Except for the limited purpose of indicating that 416 | material is shared under a Creative Commons public license or as 417 | otherwise permitted by the Creative Commons policies published at 418 | creativecommons.org/policies, Creative Commons does not authorize the 419 | use of the trademark "Creative Commons" or any other trademark or logo 420 | of Creative Commons without its prior written consent including, 421 | without limitation, in connection with any unauthorized modifications 422 | to any of its public licenses or any other arrangements, 423 | understandings, or agreements concerning use of licensed material. For 424 | the avoidance of doubt, this paragraph does not form part of the 425 | public licenses. 426 | 427 | Creative Commons may be contacted at creativecommons.org. -------------------------------------------------------------------------------- /LICENSE_CC0: -------------------------------------------------------------------------------- 1 | CC0 1.0 Universal 2 | 3 | Statement of Purpose 4 | 5 | The laws of most jurisdictions throughout the world automatically confer 6 | exclusive Copyright and Related Rights (defined below) upon the creator and 7 | subsequent owner(s) (each and all, an "owner") of an original work of 8 | authorship and/or a database (each, a "Work"). 9 | 10 | Certain owners wish to permanently relinquish those rights to a Work for the 11 | purpose of contributing to a commons of creative, cultural and scientific 12 | works ("Commons") that the public can reliably and without fear of later 13 | claims of infringement build upon, modify, incorporate in other works, reuse 14 | and redistribute as freely as possible in any form whatsoever and for any 15 | purposes, including without limitation commercial purposes. These owners may 16 | contribute to the Commons to promote the ideal of a free culture and the 17 | further production of creative, cultural and scientific works, or to gain 18 | reputation or greater distribution for their Work in part through the use and 19 | efforts of others. 20 | 21 | For these and/or other purposes and motivations, and without any expectation 22 | of additional consideration or compensation, the person associating CC0 with a 23 | Work (the "Affirmer"), to the extent that he or she is an owner of Copyright 24 | and Related Rights in the Work, voluntarily elects to apply CC0 to the Work 25 | and publicly distribute the Work under its terms, with knowledge of his or her 26 | Copyright and Related Rights in the Work and the meaning and intended legal 27 | effect of CC0 on those rights. 28 | 29 | 1. Copyright and Related Rights. A Work made available under CC0 may be 30 | protected by copyright and related or neighboring rights ("Copyright and 31 | Related Rights"). Copyright and Related Rights include, but are not limited 32 | to, the following: 33 | 34 | i. the right to reproduce, adapt, distribute, perform, display, communicate, 35 | and translate a Work; 36 | 37 | ii. moral rights retained by the original author(s) and/or performer(s); 38 | 39 | iii. publicity and privacy rights pertaining to a person's image or likeness 40 | depicted in a Work; 41 | 42 | iv. rights protecting against unfair competition in regards to a Work, 43 | subject to the limitations in paragraph 4(a), below; 44 | 45 | v. rights protecting the extraction, dissemination, use and reuse of data in 46 | a Work; 47 | 48 | vi. database rights (such as those arising under Directive 96/9/EC of the 49 | European Parliament and of the Council of 11 March 1996 on the legal 50 | protection of databases, and under any national implementation thereof, 51 | including any amended or successor version of such directive); and 52 | 53 | vii. other similar, equivalent or corresponding rights throughout the world 54 | based on applicable law or treaty, and any national implementations thereof. 55 | 56 | 2. Waiver. To the greatest extent permitted by, but not in contravention of, 57 | applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and 58 | unconditionally waives, abandons, and surrenders all of Affirmer's Copyright 59 | and Related Rights and associated claims and causes of action, whether now 60 | known or unknown (including existing as well as future claims and causes of 61 | action), in the Work (i) in all territories worldwide, (ii) for the maximum 62 | duration provided by applicable law or treaty (including future time 63 | extensions), (iii) in any current or future medium and for any number of 64 | copies, and (iv) for any purpose whatsoever, including without limitation 65 | commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes 66 | the Waiver for the benefit of each member of the public at large and to the 67 | detriment of Affirmer's heirs and successors, fully intending that such Waiver 68 | shall not be subject to revocation, rescission, cancellation, termination, or 69 | any other legal or equitable action to disrupt the quiet enjoyment of the Work 70 | by the public as contemplated by Affirmer's express Statement of Purpose. 71 | 72 | 3. Public License Fallback. Should any part of the Waiver for any reason be 73 | judged legally invalid or ineffective under applicable law, then the Waiver 74 | shall be preserved to the maximum extent permitted taking into account 75 | Affirmer's express Statement of Purpose. In addition, to the extent the Waiver 76 | is so judged Affirmer hereby grants to each affected person a royalty-free, 77 | non transferable, non sublicensable, non exclusive, irrevocable and 78 | unconditional license to exercise Affirmer's Copyright and Related Rights in 79 | the Work (i) in all territories worldwide, (ii) for the maximum duration 80 | provided by applicable law or treaty (including future time extensions), (iii) 81 | in any current or future medium and for any number of copies, and (iv) for any 82 | purpose whatsoever, including without limitation commercial, advertising or 83 | promotional purposes (the "License"). The License shall be deemed effective as 84 | of the date CC0 was applied by Affirmer to the Work. Should any part of the 85 | License for any reason be judged legally invalid or ineffective under 86 | applicable law, such partial invalidity or ineffectiveness shall not 87 | invalidate the remainder of the License, and in such case Affirmer hereby 88 | affirms that he or she will not (i) exercise any of his or her remaining 89 | Copyright and Related Rights in the Work or (ii) assert any associated claims 90 | and causes of action with respect to the Work, in either case contrary to 91 | Affirmer's express Statement of Purpose. 92 | 93 | 4. Limitations and Disclaimers. 94 | 95 | a. No trademark or patent rights held by Affirmer are waived, abandoned, 96 | surrendered, licensed or otherwise affected by this document. 97 | 98 | b. Affirmer offers the Work as-is and makes no representations or warranties 99 | of any kind concerning the Work, express, implied, statutory or otherwise, 100 | including without limitation warranties of title, merchantability, fitness 101 | for a particular purpose, non infringement, or the absence of latent or 102 | other defects, accuracy, or the present or absence of errors, whether or not 103 | discoverable, all to the greatest extent permissible under applicable law. 104 | 105 | c. Affirmer disclaims responsibility for clearing rights of other persons 106 | that may apply to the Work or any use thereof, including without limitation 107 | any person's Copyright and Related Rights in the Work. Further, Affirmer 108 | disclaims responsibility for obtaining any necessary consents, permissions 109 | or other rights required for any use of the Work. 110 | 111 | d. Affirmer understands and acknowledges that Creative Commons is not a 112 | party to this document and has no duty or obligation with respect to this 113 | CC0 or use of the Work. 114 | 115 | For more information, please see 116 | 117 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = build 9 | 10 | # Internal variables. 11 | PAPEROPT_a4 = -D latex_paper_size=a4 12 | PAPEROPT_letter = -D latex_paper_size=letter 13 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 14 | # the i18n builder cannot share the environment and doctrees with the others 15 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 16 | 17 | .PHONY: help 18 | help: 19 | @echo "Please use \`make ' where is one of" 20 | @echo " html to make standalone HTML files" 21 | @echo " dirhtml to make HTML files named index.html in directories" 22 | @echo " singlehtml to make a single large HTML file" 23 | @echo " pickle to make pickle files" 24 | @echo " json to make JSON files" 25 | @echo " htmlhelp to make HTML files and a HTML help project" 26 | @echo " qthelp to make HTML files and a qthelp project" 27 | @echo " applehelp to make an Apple Help Book" 28 | @echo " devhelp to make HTML files and a Devhelp project" 29 | @echo " epub to make an epub" 30 | @echo " epub3 to make an epub3" 31 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 32 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 33 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 34 | @echo " text to make text files" 35 | @echo " man to make manual pages" 36 | @echo " texinfo to make Texinfo files" 37 | @echo " info to make Texinfo files and run them through makeinfo" 38 | @echo " gettext to make PO message catalogs" 39 | @echo " changes to make an overview of all changed/added/deprecated items" 40 | @echo " xml to make Docutils-native XML files" 41 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 42 | @echo " linkcheck to check all external links for integrity" 43 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 44 | @echo " coverage to run coverage check of the documentation (if enabled)" 45 | @echo " dummy to check syntax errors of document sources" 46 | 47 | .PHONY: clean 48 | clean: 49 | rm -rf $(BUILDDIR)/* 50 | 51 | .PHONY: html 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | .PHONY: dirhtml 58 | dirhtml: 59 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 60 | @echo 61 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 62 | 63 | .PHONY: singlehtml 64 | singlehtml: 65 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 66 | @echo 67 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 68 | 69 | .PHONY: pickle 70 | pickle: 71 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 72 | @echo 73 | @echo "Build finished; now you can process the pickle files." 74 | 75 | .PHONY: json 76 | json: 77 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 78 | @echo 79 | @echo "Build finished; now you can process the JSON files." 80 | 81 | .PHONY: htmlhelp 82 | htmlhelp: 83 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 84 | @echo 85 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 86 | ".hhp project file in $(BUILDDIR)/htmlhelp." 87 | 88 | .PHONY: qthelp 89 | qthelp: 90 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 91 | @echo 92 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 93 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 94 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/AsyncioDocumentation.qhcp" 95 | @echo "To view the help file:" 96 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/AsyncioDocumentation.qhc" 97 | 98 | .PHONY: applehelp 99 | applehelp: 100 | $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp 101 | @echo 102 | @echo "Build finished. The help book is in $(BUILDDIR)/applehelp." 103 | @echo "N.B. You won't be able to view it unless you put it in" \ 104 | "~/Library/Documentation/Help or install it in your application" \ 105 | "bundle." 106 | 107 | .PHONY: devhelp 108 | devhelp: 109 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 110 | @echo 111 | @echo "Build finished." 112 | @echo "To view the help file:" 113 | @echo "# mkdir -p $$HOME/.local/share/devhelp/AsyncioDocumentation" 114 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/AsyncioDocumentation" 115 | @echo "# devhelp" 116 | 117 | .PHONY: epub 118 | epub: 119 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 120 | @echo 121 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 122 | 123 | .PHONY: epub3 124 | epub3: 125 | $(SPHINXBUILD) -b epub3 $(ALLSPHINXOPTS) $(BUILDDIR)/epub3 126 | @echo 127 | @echo "Build finished. The epub3 file is in $(BUILDDIR)/epub3." 128 | 129 | .PHONY: latex 130 | latex: 131 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 132 | @echo 133 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 134 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 135 | "(use \`make latexpdf' here to do that automatically)." 136 | 137 | .PHONY: latexpdf 138 | latexpdf: 139 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 140 | @echo "Running LaTeX files through pdflatex..." 141 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 142 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 143 | 144 | .PHONY: latexpdfja 145 | latexpdfja: 146 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 147 | @echo "Running LaTeX files through platex and dvipdfmx..." 148 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 149 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 150 | 151 | .PHONY: text 152 | text: 153 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 154 | @echo 155 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 156 | 157 | .PHONY: man 158 | man: 159 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 160 | @echo 161 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 162 | 163 | .PHONY: texinfo 164 | texinfo: 165 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 166 | @echo 167 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 168 | @echo "Run \`make' in that directory to run these through makeinfo" \ 169 | "(use \`make info' here to do that automatically)." 170 | 171 | .PHONY: info 172 | info: 173 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 174 | @echo "Running Texinfo files through makeinfo..." 175 | make -C $(BUILDDIR)/texinfo info 176 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 177 | 178 | .PHONY: gettext 179 | gettext: 180 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 181 | @echo 182 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 183 | 184 | .PHONY: changes 185 | changes: 186 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 187 | @echo 188 | @echo "The overview file is in $(BUILDDIR)/changes." 189 | 190 | .PHONY: linkcheck 191 | linkcheck: 192 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 193 | @echo 194 | @echo "Link check complete; look for any errors in the above output " \ 195 | "or in $(BUILDDIR)/linkcheck/output.txt." 196 | 197 | .PHONY: doctest 198 | doctest: 199 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 200 | @echo "Testing of doctests in the sources finished, look at the " \ 201 | "results in $(BUILDDIR)/doctest/output.txt." 202 | 203 | .PHONY: coverage 204 | coverage: 205 | $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage 206 | @echo "Testing of coverage in the sources finished, look at the " \ 207 | "results in $(BUILDDIR)/coverage/python.txt." 208 | 209 | .PHONY: xml 210 | xml: 211 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 212 | @echo 213 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 214 | 215 | .PHONY: pseudoxml 216 | pseudoxml: 217 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 218 | @echo 219 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 220 | 221 | .PHONY: dummy 222 | dummy: 223 | $(SPHINXBUILD) -b dummy $(ALLSPHINXOPTS) $(BUILDDIR)/dummy 224 | @echo 225 | @echo "Build finished. Dummy builder generates no files." 226 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | Asyncio documentation 2 | ===================== 3 | 4 | * Online doc: https://asyncio.readthedocs.io/ 5 | * GitHub: https://github.com/asyncio-docs/asyncio-doc 6 | * AsyncIO documentation is written with `Sphinx `_. 7 | 8 | 9 | Notes to writers 10 | ================ 11 | 12 | Tutorials should use Python 3.5 ``async`` and ``await`` keywords rather than 13 | ``@asyncio.coroutine`` and ``yield from``. 14 | 15 | 16 | Ideas 17 | ===== 18 | 19 | * Advanced section: 20 | 21 | - protocols and transports: as least point to good implementations 22 | - explain how to *test* asyncio applications. `Twisted documentation example 23 | `_ 24 | 25 | How to install Sphinx 26 | ===================== 27 | 28 | Firstly, you need to install the Sphinx tool using the Linux package manager 29 | like apt-get or dnf for example. 30 | 31 | But if you want to install it via `pip `_ , you 32 | can create a virtual environment with the `venv` module of Python 3 :: 33 | 34 | python3 -m venv env 35 | source env/bin/activate 36 | pip install -r requirements.txt 37 | 38 | Once you have installed Sphinx, you can build the documentation. 39 | 40 | How to build the documentation 41 | ============================== 42 | 43 | Install Sphinx using the Linux package manager like apt-get or dnf for example. 44 | Then build the documentation using:: 45 | 46 | make html 47 | 48 | 49 | See also 50 | ======== 51 | 52 | * https://docs.python.org/3/library/asyncio.html 53 | * http://krondo.com/an-introduction-to-asynchronous-programming-and-twisted/ 54 | * https://curio.readthedocs.io/en/latest/tutorial.html 55 | 56 | License 57 | ======= 58 | 59 | All of the code examples in this site are licensed under the `Creative Commons Zero 60 | (CC0) `_ license. 61 | 62 | All other content of this site is licensed under the `Creative Commons Attribution 63 | Share Alike 4.0 (CC-BY-SA) `_ license. 64 | -------------------------------------------------------------------------------- /conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # Asyncio Documentation documentation build configuration file, created by 4 | # sphinx-quickstart on Thu Jul 21 16:59:00 2016. 5 | # 6 | # This file is execfile()d with the current directory set to its 7 | # containing dir. 8 | # 9 | # Note that not all possible configuration values are present in this 10 | # autogenerated file. 11 | # 12 | # All configuration values have a default; values that are commented out 13 | # serve to show the default. 14 | 15 | # If extensions (or modules to document with autodoc) are in another directory, 16 | # add these directories to sys.path here. If the directory is relative to the 17 | # documentation root, use os.path.abspath to make it absolute, like shown here. 18 | # 19 | import os 20 | import sys 21 | sys.path.insert(0, os.path.abspath('.')) 22 | 23 | # -- General configuration ------------------------------------------------ 24 | 25 | # If your documentation needs a minimal Sphinx version, state it here. 26 | # 27 | # needs_sphinx = '1.0' 28 | 29 | # Add any Sphinx extension module names here, as strings. They can be 30 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 31 | # ones. 32 | #extensions = ['alabaster'] 33 | 34 | # Add any paths that contain templates here, relative to this directory. 35 | templates_path = ['_templates'] 36 | 37 | # The suffix(es) of source filenames. 38 | # You can specify multiple suffix as a list of string: 39 | # 40 | # source_suffix = ['.rst', '.md'] 41 | source_suffix = '.rst' 42 | 43 | # The encoding of source files. 44 | # 45 | # source_encoding = 'utf-8-sig' 46 | 47 | # The master toctree document. 48 | master_doc = 'index' 49 | 50 | # General information about the project. 51 | project = u'Asyncio Documentation' 52 | copyright = u'2016, Victor Stinner' 53 | author = u'Victor Stinner' 54 | 55 | # The version info for the project you're documenting, acts as replacement for 56 | # |version| and |release|, also used in various other places throughout the 57 | # built documents. 58 | # 59 | # The short X.Y version. 60 | version = u'0.0' 61 | # The full version, including alpha/beta/rc tags. 62 | release = u'0.0' 63 | 64 | # The language for content autogenerated by Sphinx. Refer to documentation 65 | # for a list of supported languages. 66 | # 67 | # This is also used if you do content translation via gettext catalogs. 68 | # Usually you set "language" from the command line for these cases. 69 | language = None 70 | 71 | # There are two options for replacing |today|: either, you set today to some 72 | # non-false value, then it is used: 73 | # 74 | # today = '' 75 | # 76 | # Else, today_fmt is used as the format for a strftime call. 77 | # 78 | # today_fmt = '%B %d, %Y' 79 | 80 | # List of patterns, relative to source directory, that match files and 81 | # directories to ignore when looking for source files. 82 | # This patterns also effect to html_static_path and html_extra_path 83 | exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] 84 | 85 | # The reST default role (used for this markup: `text`) to use for all 86 | # documents. 87 | # 88 | # default_role = None 89 | 90 | # If true, '()' will be appended to :func: etc. cross-reference text. 91 | # 92 | # add_function_parentheses = True 93 | 94 | # If true, the current module name will be prepended to all description 95 | # unit titles (such as .. function::). 96 | # 97 | # add_module_names = True 98 | 99 | # If true, sectionauthor and moduleauthor directives will be shown in the 100 | # output. They are ignored by default. 101 | # 102 | # show_authors = False 103 | 104 | # The name of the Pygments (syntax highlighting) style to use. 105 | pygments_style = 'sphinx' 106 | 107 | # The default language to highlight source code in. 108 | highlight_language = 'python3' 109 | 110 | # A list of ignored prefixes for module index sorting. 111 | # modindex_common_prefix = [] 112 | 113 | # If true, keep warnings as "system message" paragraphs in the built documents. 114 | # keep_warnings = False 115 | 116 | # If true, `todo` and `todoList` produce output, else they produce nothing. 117 | todo_include_todos = False 118 | 119 | 120 | # -- Options for HTML output ---------------------------------------------- 121 | 122 | # The theme to use for HTML and HTML Help pages. See the documentation for 123 | # a list of builtin themes. 124 | # 125 | import alabaster 126 | html_theme = 'alabaster' 127 | 128 | html_theme_path = [alabaster.get_path()] 129 | extensions = ['alabaster', 'sphinx.ext.intersphinx'] 130 | html_theme = 'alabaster' 131 | html_sidebars = { 132 | '**': [ 133 | 'about.html', 134 | 'navigation.html', 135 | 'relations.html', 136 | 'searchbox.html', 137 | 'donate.html', 138 | 'localtoc.html', 139 | 'sourcelink.html', 140 | ] 141 | } 142 | intersphinx_mapping = {'python': ('https://docs.python.org/3.5', None)} 143 | 144 | # Theme options are theme-specific and customize the look and feel of a theme 145 | # further. For a list of options available for each theme, see the 146 | # documentation. 147 | # 148 | html_theme_options = { 149 | 'github_user': 'asyncio-docs', 150 | 'github_repo': 'asyncio-doc', 151 | 'github_banner': True, 152 | 'github_type': 'star', 153 | } 154 | 155 | # Add any paths that contain custom themes here, relative to this directory. 156 | # html_theme_path = [] 157 | 158 | # The name for this set of Sphinx documents. 159 | # " v documentation" by default. 160 | # 161 | # html_title = u'Asyncio Documentation v0.0' 162 | 163 | # A shorter title for the navigation bar. Default is the same as html_title. 164 | # 165 | # html_short_title = None 166 | 167 | # The name of an image file (relative to this directory) to place at the top 168 | # of the sidebar. 169 | # 170 | # html_logo = None 171 | 172 | # The name of an image file (relative to this directory) to use as a favicon of 173 | # the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 174 | # pixels large. 175 | # 176 | # html_favicon = None 177 | 178 | # Add any paths that contain custom static files (such as style sheets) here, 179 | # relative to this directory. They are copied after the builtin static files, 180 | # so a file named "default.css" will overwrite the builtin "default.css". 181 | html_static_path = ['_build/html/_static'] 182 | 183 | # Add any extra paths that contain custom files (such as robots.txt or 184 | # .htaccess) here, relative to this directory. These files are copied 185 | # directly to the root of the documentation. 186 | # 187 | # html_extra_path = [] 188 | 189 | # If not None, a 'Last updated on:' timestamp is inserted at every page 190 | # bottom, using the given strftime format. 191 | # The empty string is equivalent to '%b %d, %Y'. 192 | # 193 | # html_last_updated_fmt = None 194 | 195 | # If true, SmartyPants will be used to convert quotes and dashes to 196 | # typographically correct entities. 197 | # 198 | # html_use_smartypants = True 199 | 200 | # Custom sidebar templates, maps document names to template names. 201 | # 202 | # html_sidebars = {} 203 | 204 | # Additional templates that should be rendered to pages, maps page names to 205 | # template names. 206 | # 207 | # html_additional_pages = {} 208 | 209 | # If false, no module index is generated. 210 | # 211 | # html_domain_indices = True 212 | 213 | # If false, no index is generated. 214 | # 215 | # html_use_index = True 216 | 217 | # If true, the index is split into individual pages for each letter. 218 | # 219 | # html_split_index = False 220 | 221 | # If true, links to the reST sources are added to the pages. 222 | # 223 | # html_show_sourcelink = True 224 | 225 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 226 | # 227 | # html_show_sphinx = True 228 | 229 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 230 | # 231 | # html_show_copyright = True 232 | 233 | # If true, an OpenSearch description file will be output, and all pages will 234 | # contain a tag referring to it. The value of this option must be the 235 | # base URL from which the finished HTML is served. 236 | # 237 | # html_use_opensearch = '' 238 | 239 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 240 | # html_file_suffix = None 241 | 242 | # Language to be used for generating the HTML full-text search index. 243 | # Sphinx supports the following languages: 244 | # 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja' 245 | # 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr', 'zh' 246 | # 247 | # html_search_language = 'en' 248 | 249 | # A dictionary with options for the search language support, empty by default. 250 | # 'ja' uses this config value. 251 | # 'zh' user can custom change `jieba` dictionary path. 252 | # 253 | # html_search_options = {'type': 'default'} 254 | 255 | # The name of a javascript file (relative to the configuration directory) that 256 | # implements a search results scorer. If empty, the default will be used. 257 | # 258 | # html_search_scorer = 'scorer.js' 259 | 260 | # Output file base name for HTML help builder. 261 | htmlhelp_basename = 'AsyncioDocumentationdoc' 262 | 263 | # -- Options for LaTeX output --------------------------------------------- 264 | 265 | latex_elements = { 266 | # The paper size ('letterpaper' or 'a4paper'). 267 | # 268 | # 'papersize': 'letterpaper', 269 | 270 | # The font size ('10pt', '11pt' or '12pt'). 271 | # 272 | # 'pointsize': '10pt', 273 | 274 | # Additional stuff for the LaTeX preamble. 275 | # 276 | # 'preamble': '', 277 | 278 | # Latex figure (float) alignment 279 | # 280 | # 'figure_align': 'htbp', 281 | } 282 | 283 | # Grouping the document tree into LaTeX files. List of tuples 284 | # (source start file, target name, title, 285 | # author, documentclass [howto, manual, or own class]). 286 | latex_documents = [ 287 | (master_doc, 288 | 'AsyncioDocumentation.tex', 289 | u'Asyncio Documentation Documentation', 290 | # TODO: Add more authors here 291 | u'Victor Stinner \\ Mike Müller', 292 | 'manual'), 293 | ] 294 | 295 | # The name of an image file (relative to this directory) to place at the top of 296 | # the title page. 297 | # 298 | # latex_logo = None 299 | 300 | # For "manual" documents, if this is true, then toplevel headings are parts, 301 | # not chapters. 302 | # 303 | # latex_use_parts = False 304 | 305 | # If true, show page references after internal links. 306 | # 307 | # latex_show_pagerefs = False 308 | 309 | # If true, show URL addresses after external links. 310 | # 311 | # latex_show_urls = False 312 | 313 | # Documents to append as an appendix to all manuals. 314 | # 315 | # latex_appendices = [] 316 | 317 | # If false, no module index is generated. 318 | # 319 | # latex_domain_indices = True 320 | 321 | 322 | # -- Options for manual page output --------------------------------------- 323 | 324 | # One entry per manual page. List of tuples 325 | # (source start file, name, description, authors, manual section). 326 | man_pages = [ 327 | (master_doc, 'asynciodocumentation', u'Asyncio Documentation Documentation', 328 | [author], 1) 329 | ] 330 | 331 | # If true, show URL addresses after external links. 332 | # 333 | # man_show_urls = False 334 | 335 | 336 | # -- Options for Texinfo output ------------------------------------------- 337 | 338 | # Grouping the document tree into Texinfo files. List of tuples 339 | # (source start file, target name, title, author, 340 | # dir menu entry, description, category) 341 | texinfo_documents = [ 342 | (master_doc, 'AsyncioDocumentation', u'Asyncio Documentation Documentation', 343 | author, 'AsyncioDocumentation', 'One line description of project.', 344 | 'Miscellaneous'), 345 | ] 346 | 347 | # Documents to append as an appendix to all manuals. 348 | # 349 | # texinfo_appendices = [] 350 | 351 | # If false, no module index is generated. 352 | # 353 | # texinfo_domain_indices = True 354 | 355 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 356 | # 357 | # texinfo_show_urls = 'footnote' 358 | 359 | # If true, do not generate a @detailmenu in the "Top" node's menu. 360 | # 361 | # texinfo_no_detailmenu = False 362 | -------------------------------------------------------------------------------- /debug_mode.rst: -------------------------------------------------------------------------------- 1 | ++++++++++++++++++ 2 | Asyncio Debug Mode 3 | ++++++++++++++++++ 4 | 5 | * Set the environment variable: ``PYTHONASYNCIODEBUG=1`` 6 | * Configure Python logging: 7 | ``logging.basicConfig(level=logging.ERROR)`` 8 | 9 | See also: `Debug mode of asyncio (ref doc) 10 | `_. 11 | 12 | -------------------------------------------------------------------------------- /examples/aiohttp_client.py: -------------------------------------------------------------------------------- 1 | """aiohttp-based client to retrieve web pages. 2 | """ 3 | 4 | import asyncio 5 | from contextlib import closing 6 | import time 7 | 8 | import aiohttp 9 | 10 | 11 | async def fetch_page(session, host, port=8000, wait=0): 12 | """Get one page. 13 | """ 14 | url = '{}:{}/{}'.format(host, port, wait) 15 | with aiohttp.Timeout(10): 16 | async with session.get(url) as response: 17 | assert response.status == 200 18 | return await response.text() 19 | 20 | 21 | def get_multiple_pages(host, waits, port=8000, show_time=True): 22 | """Get multiple pages. 23 | """ 24 | tasks = [] 25 | pages = [] 26 | start = time.perf_counter() 27 | with closing(asyncio.get_event_loop()) as loop: 28 | with aiohttp.ClientSession(loop=loop) as session: 29 | for wait in waits: 30 | tasks.append(fetch_page(session, host, port, wait)) 31 | pages = loop.run_until_complete(asyncio.gather(*tasks)) 32 | duration = time.perf_counter() - start 33 | sum_waits = sum(waits) 34 | if show_time: 35 | msg = 'It took {:4.2f} seconds for a total waiting time of {:4.2f}.' 36 | print(msg.format(duration, sum_waits)) 37 | return pages 38 | 39 | 40 | if __name__ == '__main__': 41 | 42 | def main(): 43 | """Test it. 44 | """ 45 | pages = get_multiple_pages(host='http://localhost', port='8000', 46 | waits=[1, 5, 3, 2]) 47 | for page in pages: 48 | print(page) 49 | 50 | main() 51 | -------------------------------------------------------------------------------- /examples/async_client_blocking.py: -------------------------------------------------------------------------------- 1 | """Get "web pages. 2 | 3 | Waiting until one pages is download before getting the next." 4 | """ 5 | 6 | import asyncio 7 | from contextlib import closing 8 | import time 9 | 10 | from async_page import get_page 11 | 12 | 13 | def get_multiple_pages(host, port, waits, show_time=True): 14 | """Get multiple pages. 15 | """ 16 | start = time.perf_counter() 17 | pages = [] 18 | with closing(asyncio.get_event_loop()) as loop: 19 | for wait in waits: 20 | pages.append(loop.run_until_complete(get_page(host, port, wait))) 21 | duration = time.perf_counter() - start 22 | sum_waits = sum(waits) 23 | if show_time: 24 | msg = 'It took {:4.2f} seconds for a total waiting time of {:4.2f}.' 25 | print(msg.format(duration, sum_waits)) 26 | return pages 27 | 28 | if __name__ == '__main__': 29 | 30 | def main(): 31 | """Test it. 32 | """ 33 | pages = get_multiple_pages(host='localhost', port='8000', 34 | waits=[1, 5, 3, 2]) 35 | for page in pages: 36 | print(page) 37 | 38 | main() 39 | -------------------------------------------------------------------------------- /examples/async_client_nonblocking.py: -------------------------------------------------------------------------------- 1 | """Get "web pages. 2 | 3 | Waiting until one pages is download before getting the next." 4 | """ 5 | 6 | import asyncio 7 | from contextlib import closing 8 | import time 9 | 10 | from async_page import get_page 11 | 12 | 13 | def get_multiple_pages(host, port, waits, show_time=True): 14 | """Get multiple pages. 15 | """ 16 | start = time.perf_counter() 17 | pages = [] 18 | tasks = [] 19 | with closing(asyncio.get_event_loop()) as loop: 20 | for wait in waits: 21 | tasks.append(get_page(host, port, wait)) 22 | pages = loop.run_until_complete(asyncio.gather(*tasks)) 23 | duration = time.perf_counter() - start 24 | sum_waits = sum(waits) 25 | if show_time: 26 | msg = 'It took {:4.2f} seconds for a total waiting time of {:4.2f}.' 27 | print(msg.format(duration, sum_waits)) 28 | return pages 29 | 30 | if __name__ == '__main__': 31 | 32 | def main(): 33 | """Test it. 34 | """ 35 | pages = get_multiple_pages(host='localhost', port='8000', 36 | waits=[1, 5, 3, 2]) 37 | for page in pages: 38 | print(page) 39 | 40 | main() 41 | -------------------------------------------------------------------------------- /examples/async_page.py: -------------------------------------------------------------------------------- 1 | # file: async_page.py 2 | 3 | """Get a "web page" asynchronously. 4 | """ 5 | 6 | import asyncio 7 | 8 | ENCODING = 'ISO-8859-1' 9 | 10 | 11 | def get_encoding(header): 12 | """Find out encoding. 13 | """ 14 | for line in header: 15 | if line.lstrip().startswith('Content-type'): 16 | for entry in line.split(';'): 17 | if entry.strip().startswith('charset'): 18 | return entry.split('=')[1].strip() 19 | return ENCODING 20 | 21 | 22 | async def get_page(host, port, wait=0): 23 | """Get a "web page" asynchronously. 24 | """ 25 | reader, writer = await asyncio.open_connection(host, port) 26 | writer.write(b'\r\n'.join([ 27 | 'GET /{} HTTP/1.0'.format(wait).encode(ENCODING), 28 | b'Host: %b' % host.encode(ENCODING), 29 | b'Connection: close', 30 | b'', b'' 31 | ])) 32 | header = [] 33 | msg_lines = [] 34 | async for raw_line in reader: 35 | line = raw_line.decode(ENCODING).strip() 36 | if not line.strip(): 37 | break 38 | header.append(line) 39 | encoding = get_encoding(header) 40 | async for raw_line in reader: 41 | line = raw_line.decode(encoding).strip() 42 | msg_lines.append(line) 43 | writer.close() 44 | return '\n'.join(msg_lines) 45 | -------------------------------------------------------------------------------- /examples/asyncio_deferred.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | 4 | async def multiply(x): 5 | result = x * 2 6 | await asyncio.sleep(1) 7 | return result 8 | 9 | 10 | async def steps(x): 11 | result = await multiply(x) 12 | print("result: %s" % result) 13 | 14 | 15 | loop = asyncio.get_event_loop() 16 | coro = steps(5) 17 | loop.run_until_complete(coro) 18 | loop.close() 19 | 20 | -------------------------------------------------------------------------------- /examples/create_task.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | async def say(what, when): 4 | await asyncio.sleep(when) 5 | print(what) 6 | 7 | 8 | loop = asyncio.get_event_loop() 9 | 10 | loop.create_task(say('first hello', 2)) 11 | loop.create_task(say('second hello', 1)) 12 | 13 | loop.run_forever() 14 | loop.close() 15 | -------------------------------------------------------------------------------- /examples/hello_clock.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | 4 | async def print_every_minute(): 5 | "Print minutes for ten minutes" 6 | for i in range(10): 7 | await asyncio.sleep(60) 8 | print(i, 'minute') 9 | 10 | 11 | async def print_every_second(): 12 | "Print seconds for ten minutes" 13 | for i in range(10 * 60): 14 | for i in range(60): 15 | print(i, 's') 16 | await asyncio.sleep(1) 17 | 18 | 19 | loop = asyncio.get_event_loop() 20 | loop.run_until_complete( 21 | asyncio.gather(print_every_minute(), 22 | print_every_second()) 23 | ) 24 | loop.close() 25 | -------------------------------------------------------------------------------- /examples/hello_world.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | async def say(what, when): 4 | await asyncio.sleep(when) 5 | print(what) 6 | 7 | loop = asyncio.get_event_loop() 8 | loop.run_until_complete(say('hello world', 1)) 9 | loop.close() 10 | -------------------------------------------------------------------------------- /examples/http_client.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | import aiohttp 3 | 4 | async def fetch_page(url): 5 | timeout = aiohttp.ClientTimeout(10) 6 | async with aiohttp.ClientSession(loop=loop, timeout=timeout) as session: 7 | async with session.get(url) as response: 8 | assert response.status == 200 9 | return await response.read() 10 | 11 | loop = asyncio.get_event_loop() 12 | content = loop.run_until_complete( 13 | fetch_page('http://python.org')) 14 | print(content) 15 | loop.close() 16 | -------------------------------------------------------------------------------- /examples/loop_stop.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | async def say(what, when): 4 | await asyncio.sleep(when) 5 | print(what) 6 | 7 | async def stop_after(loop, when): 8 | await asyncio.sleep(when) 9 | loop.stop() 10 | 11 | 12 | loop = asyncio.get_event_loop() 13 | 14 | loop.create_task(say('first hello', 2)) 15 | loop.create_task(say('second hello', 1)) 16 | loop.create_task(say('third hello', 4)) 17 | loop.create_task(stop_after(loop, 3)) 18 | 19 | loop.run_forever() 20 | loop.close() 21 | -------------------------------------------------------------------------------- /examples/producer_consumer.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | import random 3 | 4 | 5 | async def produce(queue, n): 6 | for x in range(1, n + 1): 7 | # produce an item 8 | print('producing {}/{}'.format(x, n)) 9 | # simulate i/o operation using sleep 10 | await asyncio.sleep(random.random()) 11 | item = str(x) 12 | # put the item in the queue 13 | await queue.put(item) 14 | 15 | # indicate the producer is done 16 | await queue.put(None) 17 | 18 | 19 | async def consume(queue): 20 | while True: 21 | # wait for an item from the producer 22 | item = await queue.get() 23 | if item is None: 24 | # the producer emits None to indicate that it is done 25 | break 26 | 27 | # process the item 28 | print('consuming item {}...'.format(item)) 29 | # simulate i/o operation using sleep 30 | await asyncio.sleep(random.random()) 31 | 32 | 33 | loop = asyncio.get_event_loop() 34 | queue = asyncio.Queue(loop=loop) 35 | producer_coro = produce(queue, 10) 36 | consumer_coro = consume(queue) 37 | loop.run_until_complete(asyncio.gather(producer_coro, consumer_coro)) 38 | loop.close() 39 | -------------------------------------------------------------------------------- /examples/producer_consumer_join.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | import random 3 | 4 | 5 | async def produce(queue, n): 6 | for x in range(n): 7 | # produce an item 8 | print('producing {}/{}'.format(x, n)) 9 | # simulate i/o operation using sleep 10 | await asyncio.sleep(random.random()) 11 | item = str(x) 12 | # put the item in the queue 13 | await queue.put(item) 14 | 15 | 16 | async def consume(queue): 17 | while True: 18 | # wait for an item from the producer 19 | item = await queue.get() 20 | 21 | # process the item 22 | print('consuming {}...'.format(item)) 23 | # simulate i/o operation using sleep 24 | await asyncio.sleep(random.random()) 25 | 26 | # Notify the queue that the item has been processed 27 | queue.task_done() 28 | 29 | 30 | async def run(n): 31 | queue = asyncio.Queue() 32 | # schedule the consumer 33 | consumer = asyncio.ensure_future(consume(queue)) 34 | # run the producer and wait for completion 35 | await produce(queue, n) 36 | # wait until the consumer has processed all items 37 | await queue.join() 38 | # the consumer is still awaiting for an item, cancel it 39 | consumer.cancel() 40 | 41 | 42 | loop = asyncio.get_event_loop() 43 | loop.run_until_complete(run(10)) 44 | loop.close() 45 | -------------------------------------------------------------------------------- /examples/requirements.txt: -------------------------------------------------------------------------------- 1 | aiohttp 2 | -------------------------------------------------------------------------------- /examples/run_in_thread.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | 4 | def compute_pi(digits): 5 | # implementation 6 | return 3.14 7 | 8 | 9 | async def main(loop): 10 | digits = await loop.run_in_executor(None, compute_pi, 20000) 11 | print("pi: %s" % digits) 12 | 13 | 14 | loop = asyncio.get_event_loop() 15 | loop.run_until_complete(main(loop)) 16 | loop.close() 17 | -------------------------------------------------------------------------------- /examples/simple_server.py: -------------------------------------------------------------------------------- 1 | # file: simple_server.py 2 | 3 | """Simple HTTP server with GET that waits for given seconds. 4 | """ 5 | 6 | from http.server import BaseHTTPRequestHandler, HTTPServer 7 | from socketserver import ThreadingMixIn 8 | import time 9 | 10 | 11 | ENCODING = 'utf-8' 12 | 13 | 14 | class ThreadingHTTPServer(ThreadingMixIn, HTTPServer): 15 | """Simple multi-threaded HTTP server. 16 | """ 17 | pass 18 | 19 | 20 | class MyRequestHandler(BaseHTTPRequestHandler): 21 | """Very simple request handler. Only supports GET. 22 | """ 23 | 24 | def do_GET(self): # pylint: disable=invalid-name 25 | """Respond after seconds given in path. 26 | """ 27 | try: 28 | seconds = float(self.path[1:]) 29 | except ValueError: 30 | seconds = 0.0 31 | if seconds < 0: 32 | seconds = 0.0 33 | text = "Waited for {:4.2f} seconds.\nThat's all.\n" 34 | msg = text.format(seconds).encode(ENCODING) 35 | time.sleep(seconds) 36 | self.send_response(200) 37 | self.send_header("Content-type", 'text/plain; charset=utf-8') 38 | self.send_header("Content-length", str(len(msg))) 39 | self.end_headers() 40 | self.wfile.write(msg) 41 | 42 | 43 | def run(server_class=ThreadingHTTPServer, 44 | handler_class=MyRequestHandler, 45 | port=8000): 46 | """Run the simple server on given port. 47 | """ 48 | server_address = ('', port) 49 | httpd = server_class(server_address, handler_class) 50 | print('Serving from port {} ...'.format(port)) 51 | httpd.serve_forever() 52 | 53 | 54 | if __name__ == '__main__': 55 | run() 56 | -------------------------------------------------------------------------------- /examples/subprocess_command.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | 4 | async def run_command(*args): 5 | # Create subprocess 6 | process = await asyncio.create_subprocess_exec( 7 | *args, 8 | # stdout must a pipe to be accessible as process.stdout 9 | stdout=asyncio.subprocess.PIPE) 10 | # Wait for the subprocess to finish 11 | stdout, stderr = await process.communicate() 12 | # Return stdout 13 | return stdout.decode().strip() 14 | 15 | 16 | loop = asyncio.get_event_loop() 17 | # Gather uname and date commands 18 | commands = asyncio.gather(run_command('uname'), run_command('date')) 19 | # Run the commands 20 | uname, date = loop.run_until_complete(commands) 21 | # Print a report 22 | print('uname: {}, date: {}'.format(uname, date)) 23 | loop.close() 24 | -------------------------------------------------------------------------------- /examples/subprocess_echo.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | 4 | async def echo(msg): 5 | # Run an echo subprocess 6 | process = await asyncio.create_subprocess_exec( 7 | 'cat', 8 | # stdin must a pipe to be accessible as process.stdin 9 | stdin=asyncio.subprocess.PIPE, 10 | # stdout must a pipe to be accessible as process.stdout 11 | stdout=asyncio.subprocess.PIPE) 12 | # Write message 13 | print('Writing {!r} ...'.format(msg)) 14 | process.stdin.write(msg.encode() + b'\n') 15 | # Read reply 16 | data = await process.stdout.readline() 17 | reply = data.decode().strip() 18 | print('Received {!r}'.format(reply)) 19 | # Stop the subprocess 20 | process.terminate() 21 | code = await process.wait() 22 | print('Terminated with code {}'.format(code)) 23 | 24 | 25 | loop = asyncio.get_event_loop() 26 | loop.run_until_complete(echo('hello!')) 27 | loop.close() 28 | -------------------------------------------------------------------------------- /examples/synchronous_client.py: -------------------------------------------------------------------------------- 1 | """Synchronous client to retrieve web pages. 2 | """ 3 | 4 | 5 | from urllib.request import urlopen 6 | import time 7 | 8 | ENCODING = 'ISO-8859-1' 9 | 10 | 11 | def get_encoding(http_response): 12 | """Find out encoding. 13 | """ 14 | content_type = http_response.getheader('Content-type') 15 | for entry in content_type.split(';'): 16 | if entry.strip().startswith('charset'): 17 | return entry.split('=')[1].strip() 18 | return ENCODING 19 | 20 | 21 | def get_page(host, port, wait=0): 22 | """Get one page supplying `wait` time. 23 | 24 | The path will be build with: `host:port/wait` 25 | """ 26 | full_url = '{}:{}/{}'.format(host, port, wait) 27 | with urlopen(full_url) as http_response: 28 | html = http_response.read().decode(get_encoding(http_response)) 29 | return html 30 | 31 | 32 | def get_multiple_pages(host, port, waits, show_time=True): 33 | """Get multiple pages. 34 | """ 35 | start = time.perf_counter() 36 | pages = [get_page(host, port, wait) for wait in waits] 37 | duration = time.perf_counter() - start 38 | sum_waits = sum(waits) 39 | if show_time: 40 | msg = 'It took {:4.2f} seconds for a total waiting time of {:4.2f}.' 41 | print(msg.format(duration, sum_waits)) 42 | return pages 43 | 44 | 45 | if __name__ == '__main__': 46 | 47 | def main(): 48 | """Test it. 49 | """ 50 | pages = get_multiple_pages(host='http://localhost', port='8000', 51 | waits=[1, 5, 3, 2]) 52 | for page in pages: 53 | print(page) 54 | 55 | main() 56 | -------------------------------------------------------------------------------- /examples/tcp_echo_client.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | 4 | async def tcp_echo_client(message, loop): 5 | reader, writer = await asyncio.open_connection('127.0.0.1', 8888, 6 | loop=loop) 7 | 8 | print('Send: %r' % message) 9 | writer.write(message.encode()) 10 | 11 | data = await reader.read(100) 12 | print('Received: %r' % data.decode()) 13 | 14 | print('Close the socket') 15 | writer.close() 16 | 17 | 18 | message = 'Hello World!' 19 | loop = asyncio.get_event_loop() 20 | loop.run_until_complete(tcp_echo_client(message, loop)) 21 | loop.close() 22 | -------------------------------------------------------------------------------- /examples/tcp_echo_server.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | 3 | async def handle_echo(reader, writer): 4 | data = await reader.read(100) 5 | message = data.decode() 6 | addr = writer.get_extra_info('peername') 7 | print("Received %r from %r" % (message, addr)) 8 | 9 | print("Send: %r" % message) 10 | writer.write(data) 11 | await writer.drain() 12 | 13 | print("Close the client socket") 14 | writer.close() 15 | 16 | loop = asyncio.get_event_loop() 17 | coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop) 18 | server = loop.run_until_complete(coro) 19 | 20 | # Serve requests until Ctrl+C is pressed 21 | print('Serving on {}'.format(server.sockets[0].getsockname())) 22 | try: 23 | loop.run_forever() 24 | except KeyboardInterrupt: 25 | pass 26 | 27 | # Close the server 28 | server.close() 29 | loop.run_until_complete(server.wait_closed()) 30 | loop.close() 31 | -------------------------------------------------------------------------------- /examples/twisted_deferred.py: -------------------------------------------------------------------------------- 1 | from twisted.internet import defer 2 | from twisted.internet import reactor 3 | 4 | 5 | def multiply(x): 6 | result = x * 2 7 | d = defer.Deferred() 8 | reactor.callLater(1.0, d.callback, 9 | result) 10 | return d 11 | 12 | 13 | def step1(x): 14 | return multiply(x) 15 | 16 | 17 | def step2(result): 18 | print("result: %s" % result) 19 | 20 | reactor.stop() 21 | 22 | 23 | d = defer.Deferred() 24 | d.addCallback(step1) 25 | d.addCallback(step2) 26 | d.callback(5) 27 | 28 | reactor.run() 29 | 30 | -------------------------------------------------------------------------------- /getting_help.rst: -------------------------------------------------------------------------------- 1 | ++++++++++++ 2 | Getting Help 3 | ++++++++++++ 4 | 5 | 6 | Mailing list 7 | ============ 8 | 9 | * `async-sig `_: 10 | discussions about asynchronous programming in Python (i.e. async/await) 11 | * `python-tulip Google Group 12 | `_: 13 | historical name of the official asyncio mailing list 14 | 15 | 16 | StackOverflow 17 | ============= 18 | 19 | There is an `python-asyncio tag on StackOverflow 20 | `_ where you can 21 | read new questions but also read answers of previous questions. 22 | 23 | 24 | IRC 25 | === 26 | 27 | There is an IRC channel ``#asyncio`` on the `Freenode server 28 | `_. 29 | -------------------------------------------------------------------------------- /getting_started.rst: -------------------------------------------------------------------------------- 1 | +++++++++++++++ 2 | Getting Started 3 | +++++++++++++++ 4 | 5 | Python 3.5 (or higher) only 6 | =========================== 7 | 8 | This documentation is written for Python 3.5 to avail of the new 9 | ``async`` and ``await`` keywords. 10 | 11 | If you have Python 3.5 installed you only need to install ``aiohttp`` 12 | 13 | .. highlight:: shell 14 | 15 | pip install -U aiohttp 16 | 17 | If you don't have Python 3.5 installed yet, you have several options 18 | to install it. 19 | 20 | All platforms with ``conda`` 21 | ---------------------------- 22 | 23 | * Download and install 24 | `Miniconda `_ for our platform. 25 | * Create a new Python 3.5 environment (named ``aio35``, use a different 26 | if you like):: 27 | 28 | conda create -n aio35 python=3.5 29 | .. note:: 30 | ``conda activate`` and ``conda deactivate`` only work on conda 4.6 and later versions. 31 | For conda versions prior to 4.6, run: 32 | 33 | * Windows: ``activate`` or ``deactivate`` 34 | * Linux and macOS: ``source activate`` or ``source deactivate`` 35 | 36 | * Activate it. 37 | $ conda activate aio35 38 | 39 | * Install ``aiohttp``:: 40 | 41 | $(aio35) pip install aiohttp 42 | 43 | Platform specific 44 | ----------------- 45 | 46 | .. would be good to have some word about installing on Windows 47 | 48 | * Windows: The easiest way to use Python 3.5 would be to use a package manager 49 | such as conda. See the installation instructions above. 50 | * Mac OS X: Install `Homebrew `_ and 52 | then type ``brew install python3`` 53 | * Linux: Ubuntu 16.04+ and Arch linux ship with Python 3.5 included. 54 | If you don't have Python 3.5+ on your computer, you can compile it or use 55 | `Pythonz `_. 56 | 57 | 58 | Create a virtual environment to run examples 59 | ============================================ 60 | 61 | If you don't use conda (see above), create a virtual environment:: 62 | 63 | python3 -m venv venv 64 | 65 | .. note:: 66 | Depending on your platform, the Python 3 interpreter could be invoked by 67 | ``python`` instead. This is the case for conda on Windows for example. 68 | 69 | Install ``aiohttp`` in the virtual environment:: 70 | 71 | ./venv/bin/python -m pip install -U aiohttp 72 | 73 | -------------------------------------------------------------------------------- /glossary.rst: -------------------------------------------------------------------------------- 1 | .. _glossary: 2 | 3 | ******** 4 | Glossary 5 | ******** 6 | 7 | .. if you add new entries, keep the alphabetical sorting! 8 | 9 | .. glossary:: 10 | 11 | coroutine 12 | A coroutine is a piece of code that can be paused and resumed. In 13 | contrast to threads which are preemptively multitasked by the operating 14 | system, coroutines multitask cooperatively. I.e. they choose when to 15 | pause (or to use terminology for coroutines before 3.4 - ``yield``) 16 | execution. They can also execute other coroutines. 17 | 18 | event loop 19 | The event loop is the central execution device to launch execution of 20 | coroutines and handle I/O (Network, sub-processes...) 21 | 22 | future 23 | It's like a mailbox where you can subscribe to receive a result when it 24 | will be done. More details in `official future documentation 25 | `_ 26 | 27 | task 28 | It represents the execution of a coroutine and take care the result in a 29 | future. More details in `official task documentation 30 | `_ 31 | 32 | -------------------------------------------------------------------------------- /hello_clock.rst: -------------------------------------------------------------------------------- 1 | Hello Clock 2 | ^^^^^^^^^^^ 3 | 4 | Example illustrating how to schedule two :term:`coroutines ` to run 5 | concurrently. They run for ten minutes, during which the first :term:`coroutine 6 | ` is scheduled to run every second, while the second is scheduled to 7 | run every minute. 8 | 9 | The function `asyncio.gather` is used to schedule both :term:`coroutines 10 | ` at once. 11 | 12 | .. literalinclude:: examples/hello_clock.py 13 | -------------------------------------------------------------------------------- /hello_world.rst: -------------------------------------------------------------------------------- 1 | +++++++++++ 2 | Hello World 3 | +++++++++++ 4 | 5 | This is a series of examples showing the basics of how to write 6 | :term:`coroutines ` and schedule them in the asyncio 7 | :term:`event loop `. 8 | 9 | Simple coroutine 10 | ---------------- 11 | 12 | This example uses the :py:meth:`asyncio.AbstractEventLoop.run_until_complete` 13 | method to schedule a simple function that will wait one second, print 14 | ``hello`` and then finish. 15 | 16 | Because it is launched with :py:meth:`asyncio.AbstractEventLoop.run_until_complete`, 17 | the :term:`event loop ` itself 18 | will terminate once the :term:`coroutine ` is completed. 19 | 20 | .. literalinclude:: examples/hello_world.py 21 | 22 | 23 | Creating tasks 24 | -------------- 25 | 26 | This second example shows how you can schedule multiple :term:`coroutines 27 | ` in the 28 | event loop, and then run the :term:`event loop `. 29 | 30 | Notice that this example will print ``second_hello`` before ``first_hello``, 31 | as the first :term:`task ` scheduled waits longer that the second one 32 | before printing. 33 | 34 | Also note that this example will never terminate, as the :term:`loop ` is asked to 36 | `run_forever`. 37 | 38 | .. literalinclude:: examples/create_task.py 39 | 40 | 41 | Stopping the loop 42 | ----------------- 43 | 44 | This third example adds another :term:`task ` that will stop the 45 | :term:`event loop ` before 46 | all scheduled :term:`tasks ` could execute, which results in a warning. 47 | 48 | .. literalinclude:: examples/loop_stop.py 49 | 50 | Warning:: 51 | 52 | .. highlight:: none 53 | 54 | Task was destroyed but it is pending! 55 | task: 56 | wait_for=> 57 | 58 | -------------------------------------------------------------------------------- /http_client.rst: -------------------------------------------------------------------------------- 1 | +++++++++++++++++++ 2 | HTTP client example 3 | +++++++++++++++++++ 4 | 5 | HTTP client example: 6 | 7 | .. literalinclude:: examples/http_client.py 8 | 9 | For more information, see `aiohttp documentation 10 | `_. 11 | -------------------------------------------------------------------------------- /index.rst: -------------------------------------------------------------------------------- 1 | +++++++++++++++++++++ 2 | Asyncio Documentation 3 | +++++++++++++++++++++ 4 | 5 | Chapter 1: First steps with asyncio 6 | =================================== 7 | 8 | .. toctree:: 9 | :maxdepth: 2 10 | 11 | why_asyncio.rst 12 | getting_started.rst 13 | hello_world.rst 14 | hello_clock.rst 15 | http_client.rst 16 | performance.rst 17 | twisted.rst 18 | getting_help.rst 19 | 20 | 21 | Chapter 2: Advanced topics 22 | ========================== 23 | 24 | .. toctree:: 25 | :maxdepth: 2 26 | 27 | tcp_echo.rst 28 | threads.rst 29 | subprocess.rst 30 | producer_consumer.rst 31 | debug_mode.rst 32 | 33 | Chapter 3: Larger examples 34 | ========================== 35 | 36 | .. toctree:: 37 | :maxdepth: 2 38 | 39 | webscraper.rst 40 | 41 | Indices and tables 42 | ================== 43 | 44 | .. toctree:: 45 | :maxdepth: 2 46 | 47 | glossary.rst 48 | 49 | 50 | See also 51 | ======== 52 | 53 | * `asyncio wiki `_ 54 | * `asyncio Reference Documentation 55 | `_. 56 | * `A Web Crawler With asyncio Coroutines 57 | `_ 58 | by A. Jesse Jiryu Davis and Guido van Rossum 59 | * `Writing Redis in Python with asyncio: Part 1 60 | `_ 61 | by James Saryerwinnie 62 | 63 | 64 | Contributing 65 | ============ 66 | 67 | .. toctree:: 68 | :maxdepth: 2 69 | 70 | README.rst 71 | 72 | License 73 | ======= 74 | 75 | All of the code examples in this site are licensed under the `Creative Commons Zero 76 | (CC0) `_ license. 77 | 78 | All other content of this site is licensed under the `Creative Commons Attribution 79 | Share Alike 4.0 (CC-BY-SA) `_ license. 80 | -------------------------------------------------------------------------------- /make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | REM Command file for Sphinx documentation 4 | 5 | if "%SPHINXBUILD%" == "" ( 6 | set SPHINXBUILD=sphinx-build 7 | ) 8 | set BUILDDIR=build 9 | set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . 10 | set I18NSPHINXOPTS=%SPHINXOPTS% . 11 | if NOT "%PAPER%" == "" ( 12 | set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% 13 | set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% 14 | ) 15 | 16 | if "%1" == "" goto help 17 | 18 | if "%1" == "help" ( 19 | :help 20 | echo.Please use `make ^` where ^ is one of 21 | echo. html to make standalone HTML files 22 | echo. dirhtml to make HTML files named index.html in directories 23 | echo. singlehtml to make a single large HTML file 24 | echo. pickle to make pickle files 25 | echo. json to make JSON files 26 | echo. htmlhelp to make HTML files and a HTML help project 27 | echo. qthelp to make HTML files and a qthelp project 28 | echo. devhelp to make HTML files and a Devhelp project 29 | echo. epub to make an epub 30 | echo. epub3 to make an epub3 31 | echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter 32 | echo. text to make text files 33 | echo. man to make manual pages 34 | echo. texinfo to make Texinfo files 35 | echo. gettext to make PO message catalogs 36 | echo. changes to make an overview over all changed/added/deprecated items 37 | echo. xml to make Docutils-native XML files 38 | echo. pseudoxml to make pseudoxml-XML files for display purposes 39 | echo. linkcheck to check all external links for integrity 40 | echo. doctest to run all doctests embedded in the documentation if enabled 41 | echo. coverage to run coverage check of the documentation if enabled 42 | echo. dummy to check syntax errors of document sources 43 | goto end 44 | ) 45 | 46 | if "%1" == "clean" ( 47 | for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i 48 | del /q /s %BUILDDIR%\* 49 | goto end 50 | ) 51 | 52 | 53 | REM Check if sphinx-build is available and fallback to Python version if any 54 | %SPHINXBUILD% 1>NUL 2>NUL 55 | if errorlevel 9009 goto sphinx_python 56 | goto sphinx_ok 57 | 58 | :sphinx_python 59 | 60 | set SPHINXBUILD=python -m sphinx.__init__ 61 | %SPHINXBUILD% 2> nul 62 | if errorlevel 9009 ( 63 | echo. 64 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 65 | echo.installed, then set the SPHINXBUILD environment variable to point 66 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 67 | echo.may add the Sphinx directory to PATH. 68 | echo. 69 | echo.If you don't have Sphinx installed, grab it from 70 | echo.http://sphinx-doc.org/ 71 | exit /b 1 72 | ) 73 | 74 | :sphinx_ok 75 | 76 | 77 | if "%1" == "html" ( 78 | %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html 79 | if errorlevel 1 exit /b 1 80 | echo. 81 | echo.Build finished. The HTML pages are in %BUILDDIR%/html. 82 | goto end 83 | ) 84 | 85 | if "%1" == "dirhtml" ( 86 | %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml 87 | if errorlevel 1 exit /b 1 88 | echo. 89 | echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. 90 | goto end 91 | ) 92 | 93 | if "%1" == "singlehtml" ( 94 | %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml 95 | if errorlevel 1 exit /b 1 96 | echo. 97 | echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. 98 | goto end 99 | ) 100 | 101 | if "%1" == "pickle" ( 102 | %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle 103 | if errorlevel 1 exit /b 1 104 | echo. 105 | echo.Build finished; now you can process the pickle files. 106 | goto end 107 | ) 108 | 109 | if "%1" == "json" ( 110 | %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json 111 | if errorlevel 1 exit /b 1 112 | echo. 113 | echo.Build finished; now you can process the JSON files. 114 | goto end 115 | ) 116 | 117 | if "%1" == "htmlhelp" ( 118 | %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp 119 | if errorlevel 1 exit /b 1 120 | echo. 121 | echo.Build finished; now you can run HTML Help Workshop with the ^ 122 | .hhp project file in %BUILDDIR%/htmlhelp. 123 | goto end 124 | ) 125 | 126 | if "%1" == "qthelp" ( 127 | %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp 128 | if errorlevel 1 exit /b 1 129 | echo. 130 | echo.Build finished; now you can run "qcollectiongenerator" with the ^ 131 | .qhcp project file in %BUILDDIR%/qthelp, like this: 132 | echo.^> qcollectiongenerator %BUILDDIR%\qthelp\AsyncioDocumentation.qhcp 133 | echo.To view the help file: 134 | echo.^> assistant -collectionFile %BUILDDIR%\qthelp\AsyncioDocumentation.ghc 135 | goto end 136 | ) 137 | 138 | if "%1" == "devhelp" ( 139 | %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp 140 | if errorlevel 1 exit /b 1 141 | echo. 142 | echo.Build finished. 143 | goto end 144 | ) 145 | 146 | if "%1" == "epub" ( 147 | %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub 148 | if errorlevel 1 exit /b 1 149 | echo. 150 | echo.Build finished. The epub file is in %BUILDDIR%/epub. 151 | goto end 152 | ) 153 | 154 | if "%1" == "epub3" ( 155 | %SPHINXBUILD% -b epub3 %ALLSPHINXOPTS% %BUILDDIR%/epub3 156 | if errorlevel 1 exit /b 1 157 | echo. 158 | echo.Build finished. The epub3 file is in %BUILDDIR%/epub3. 159 | goto end 160 | ) 161 | 162 | if "%1" == "latex" ( 163 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 164 | if errorlevel 1 exit /b 1 165 | echo. 166 | echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. 167 | goto end 168 | ) 169 | 170 | if "%1" == "latexpdf" ( 171 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 172 | cd %BUILDDIR%/latex 173 | make all-pdf 174 | cd %~dp0 175 | echo. 176 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 177 | goto end 178 | ) 179 | 180 | if "%1" == "latexpdfja" ( 181 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 182 | cd %BUILDDIR%/latex 183 | make all-pdf-ja 184 | cd %~dp0 185 | echo. 186 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 187 | goto end 188 | ) 189 | 190 | if "%1" == "text" ( 191 | %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text 192 | if errorlevel 1 exit /b 1 193 | echo. 194 | echo.Build finished. The text files are in %BUILDDIR%/text. 195 | goto end 196 | ) 197 | 198 | if "%1" == "man" ( 199 | %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man 200 | if errorlevel 1 exit /b 1 201 | echo. 202 | echo.Build finished. The manual pages are in %BUILDDIR%/man. 203 | goto end 204 | ) 205 | 206 | if "%1" == "texinfo" ( 207 | %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo 208 | if errorlevel 1 exit /b 1 209 | echo. 210 | echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. 211 | goto end 212 | ) 213 | 214 | if "%1" == "gettext" ( 215 | %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale 216 | if errorlevel 1 exit /b 1 217 | echo. 218 | echo.Build finished. The message catalogs are in %BUILDDIR%/locale. 219 | goto end 220 | ) 221 | 222 | if "%1" == "changes" ( 223 | %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes 224 | if errorlevel 1 exit /b 1 225 | echo. 226 | echo.The overview file is in %BUILDDIR%/changes. 227 | goto end 228 | ) 229 | 230 | if "%1" == "linkcheck" ( 231 | %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck 232 | if errorlevel 1 exit /b 1 233 | echo. 234 | echo.Link check complete; look for any errors in the above output ^ 235 | or in %BUILDDIR%/linkcheck/output.txt. 236 | goto end 237 | ) 238 | 239 | if "%1" == "doctest" ( 240 | %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest 241 | if errorlevel 1 exit /b 1 242 | echo. 243 | echo.Testing of doctests in the sources finished, look at the ^ 244 | results in %BUILDDIR%/doctest/output.txt. 245 | goto end 246 | ) 247 | 248 | if "%1" == "coverage" ( 249 | %SPHINXBUILD% -b coverage %ALLSPHINXOPTS% %BUILDDIR%/coverage 250 | if errorlevel 1 exit /b 1 251 | echo. 252 | echo.Testing of coverage in the sources finished, look at the ^ 253 | results in %BUILDDIR%/coverage/python.txt. 254 | goto end 255 | ) 256 | 257 | if "%1" == "xml" ( 258 | %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml 259 | if errorlevel 1 exit /b 1 260 | echo. 261 | echo.Build finished. The XML files are in %BUILDDIR%/xml. 262 | goto end 263 | ) 264 | 265 | if "%1" == "pseudoxml" ( 266 | %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml 267 | if errorlevel 1 exit /b 1 268 | echo. 269 | echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. 270 | goto end 271 | ) 272 | 273 | if "%1" == "dummy" ( 274 | %SPHINXBUILD% -b dummy %ALLSPHINXOPTS% %BUILDDIR%/dummy 275 | if errorlevel 1 exit /b 1 276 | echo. 277 | echo.Build finished. Dummy builder generates no files. 278 | goto end 279 | ) 280 | 281 | :end 282 | -------------------------------------------------------------------------------- /performance.rst: -------------------------------------------------------------------------------- 1 | +++++++++++++++++++ 2 | asyncio performance 3 | +++++++++++++++++++ 4 | 5 | Random notes about tuning asyncio for performance. Performance means two 6 | different terms which might be incompatible: 7 | 8 | * Number of concurrent requests per second 9 | * Request latency in seconds: min/average/max time to complete a request 10 | 11 | 12 | Architecture: Worker processes 13 | ============================== 14 | 15 | Because of its GIL, CPython is basically only able to use 1 CPU. To increase 16 | the number of concurrent requests, one solution is to spawn multiple worker 17 | processes. See for example: 18 | 19 | * `Gunicorn `_ 20 | * `API-Hour `_ 21 | 22 | 23 | Stream limits 24 | ============= 25 | 26 | * `limit parameter of StreamReader/open_connection() 27 | `_ 28 | * `set_write_buffer_limits() low/high water mark on writing for transports 29 | `_ 30 | 31 | aiohttp uses ``set_writer_buffer_limits(0)`` for backpressure support and 32 | implemented their own buffering, see: 33 | 34 | * `aio-libs/aiohttp#1369 `_ 35 | * `Some thoughts on asynchronous API design in a post-async/await world 36 | `_ 37 | (November, 2016) by Nathaniel J. Smith 38 | 39 | 40 | TCP_NODELAY 41 | =========== 42 | 43 | Since Python 3.6, asyncio now sets the ``TCP_NODELAY`` option on newly created 44 | sockets: disable the Nagle algorithm for send coalescing. Disable segment 45 | buffering so data can be sent out to peer as quickly as possible, so this is 46 | typically used to improve network utilisation. 47 | 48 | See `Nagle's algorithm `_. 49 | 50 | TCP_QUICKACK 51 | ============ 52 | 53 | (This option is not used by asyncio by default.) 54 | 55 | The ``TCP_QUICKACK`` option can be used to send out acknowledgements as early 56 | as possible than delayed under some protocol level exchanging, and it's not 57 | stable/permanent, subsequent TCP transactions (which may happen under the hood) 58 | can disregard this option depending on actual protocol level processing or any 59 | actual disagreements between user setting and stack behaviour. 60 | 61 | 62 | Tune the Linux kernel 63 | ===================== 64 | 65 | Linux TCP sysctls: 66 | 67 | * ``/proc/sys/net/ipv4/tcp_mem`` 68 | * ``/proc/sys/net/core/rmem_default`` and ``/proc/sys/net/core/rmem_max``: 69 | The default and maximum amount for the receive socket memory 70 | * ``/proc/sys/net/core/wmem_default`` and ``/proc/sys/net/core/wmem_max``: 71 | The default and maximum amount for the send socket memory 72 | * ``/proc/sys/net/core/optmem_max``: The maximum amount of option memory 73 | buffers 74 | * ``net.ipv4.tcp_no_metrics_save`` 75 | * ``net.core.netdev_max_backlog``: Set maximum number of packets, queued on the 76 | INPUT side, when the interface receives packets faster than kernel can 77 | process them. 78 | -------------------------------------------------------------------------------- /producer_consumer.rst: -------------------------------------------------------------------------------- 1 | +++++++++++++++++ 2 | Producer/consumer 3 | +++++++++++++++++ 4 | 5 | Simple example 6 | ============== 7 | 8 | A simple producer/consumer example, using an `asyncio.Queue 9 | `_: 10 | 11 | .. literalinclude:: examples/producer_consumer.py 12 | 13 | 14 | Using task_done() 15 | ================= 16 | 17 | 18 | A simple producer/consumer example, using `Queue.task_done 19 | `_ 20 | and `Queue.join 21 | `_: 22 | 23 | .. literalinclude:: examples/producer_consumer_join.py 24 | 25 | For more information, see the `asyncio queue documentation 26 | `_. 27 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | sphinx 2 | -------------------------------------------------------------------------------- /subprocess.rst: -------------------------------------------------------------------------------- 1 | ++++++++++ 2 | Subprocess 3 | ++++++++++ 4 | 5 | Run a subprocess and read its output 6 | ==================================== 7 | 8 | A simple example to run commands in a subprocess using 9 | `asyncio.create_subprocess_exec 10 | `_ 11 | and get the output using `process.communicate 12 | `_: 13 | 14 | .. literalinclude:: examples/subprocess_command.py 15 | 16 | 17 | Communicate with a subprocess using standard streams 18 | ==================================================== 19 | 20 | A simple example to communicate with an echo subprocess using `process.stdin 21 | `_ 22 | and `process.stdout 23 | `_: 24 | 25 | .. literalinclude:: examples/subprocess_echo.py 26 | 27 | For more information, see the `asyncio subprocess documentation 28 | `_. 29 | -------------------------------------------------------------------------------- /tcp_echo.rst: -------------------------------------------------------------------------------- 1 | ++++++++++++++++++++++++++ 2 | TCP echo client and server 3 | ++++++++++++++++++++++++++ 4 | 5 | TCP echo client 6 | --------------- 7 | 8 | TCP echo client using streams: 9 | 10 | .. literalinclude:: examples/tcp_echo_client.py 11 | 12 | 13 | TCP echo server 14 | --------------- 15 | 16 | TCP echo server using streams: 17 | 18 | .. literalinclude:: examples/tcp_echo_server.py 19 | -------------------------------------------------------------------------------- /threads.rst: -------------------------------------------------------------------------------- 1 | +++++++ 2 | Threads 3 | +++++++ 4 | 5 | Run slow CPU-intensive or blocking I/O code in a thread: 6 | 7 | .. literalinclude:: examples/run_in_thread.py 8 | 9 | See also: 10 | 11 | * `run_in_executor() documentation 12 | `_ 13 | * `asyncio: Concurrency and multithreading 14 | `_ 15 | -------------------------------------------------------------------------------- /twisted.rst: -------------------------------------------------------------------------------- 1 | ++++++++++++++++++++++++++++++++++++++ 2 | Learn asyncio if you come from Twisted 3 | ++++++++++++++++++++++++++++++++++++++ 4 | 5 | The `Twisted project `_ is probably one of the 6 | oldest libraries that supports asynchronous programming in Python. 7 | It has been used by many programmers to develop a variety of applications. 8 | It supports many network protocols and can be used for many different 9 | types of network programming. 10 | In fact, asyncio was heavily inspired by Twisted. 11 | The expertise of several Twisted developers had been incorporated in 12 | asyncio. 13 | Soon, there will be a version of Twisted that is based on asyncio. 14 | 15 | 16 | 17 | Rosetta Stone 18 | ============= 19 | 20 | This tables shows equivalent concepts in Twisted and asyncio. 21 | 22 | ======================== ================================== 23 | Twisted asyncio 24 | ======================== ================================== 25 | ``Deferred`` ``asyncio.Future`` 26 | ``deferToThread(func)`` ``loop.run_in_executor(None, func)`` 27 | ``@inlineCallbacks`` ``async def`` 28 | ``reactor.run()`` ``loop.run_forever()`` 29 | ======================== ================================== 30 | 31 | 32 | Deferred example 33 | ================ 34 | 35 | This small example shows two equivalent programs, one implemented in Twisted 36 | and one in asyncio. 37 | 38 | +--------------------------------------------------+--------------------------------------------------+ 39 | | Twisted | asyncio | 40 | +==================================================+==================================================+ 41 | | Basic Twisted example using deferred: | Similar example written using asyncio: | 42 | | | | 43 | | .. literalinclude:: examples/twisted_deferred.py | .. literalinclude:: examples/asyncio_deferred.py | 44 | +--------------------------------------------------+--------------------------------------------------+ 45 | 46 | -------------------------------------------------------------------------------- /webscraper.rst: -------------------------------------------------------------------------------- 1 | ++++++++++++ 2 | Web Scraping 3 | ++++++++++++ 4 | 5 | Web scraping means downloading multiple web pages, often from different 6 | servers. 7 | Typically, there is a considerable waiting time between sending a request and 8 | receiving the answer. 9 | Using a client that always waits for the server to answer before sending 10 | the next request, can lead to spending most of time waiting. 11 | Here ``asyncio`` can help to send many requests without waiting for a response 12 | and collecting the answers later. 13 | The following examples show how a synchronous client spends most of the time 14 | waiting and how to use ``asyncio`` to write asynchronous client that 15 | can handle many requests concurrently. 16 | 17 | A Mock Web Server 18 | ----------------- 19 | 20 | This is a very simple web server. (See below for the code.) 21 | Its only purpose is to wait for a given amount of time. 22 | Test it by running it from the command line:: 23 | 24 | .. highlight:: bash 25 | 26 | $ python simple_server.py 27 | 28 | It will answer like this:: 29 | 30 | Serving from port 8000 ... 31 | 32 | Now, open a browser and go to this URL:: 33 | 34 | http://localhost:8000/ 35 | 36 | You should see this text in your browser:: 37 | 38 | Waited for 0.00 seconds. 39 | 40 | Now, add ``2.5`` to the URL:: 41 | 42 | http://localhost:8000/2.5 43 | 44 | After pressing enter, it will take 2.5 seconds until you see this 45 | response:: 46 | 47 | Waited for 2.50 seconds. 48 | 49 | Use different numbers and see how long it takes until the server responds. 50 | 51 | The full implementation looks like this: 52 | 53 | .. literalinclude:: examples/simple_server.py 54 | :language: python 55 | 56 | Let's have a look into the details. 57 | This provides a simple multi-threaded web server: 58 | 59 | .. literalinclude:: examples/simple_server.py 60 | :language: python 61 | :start-after: ENCODING = 'utf-8' 62 | :end-before: class MyRequestHandle 63 | 64 | It uses multiple inheritance. 65 | The mix-in class ``ThreadingMixIn`` provides the multi-threading support and 66 | the class ``HTTPServer`` a basic HTTP server. 67 | 68 | 69 | The request handler only has a ``GET`` method: 70 | 71 | 72 | .. literalinclude:: examples/simple_server.py 73 | :language: python 74 | :start-after: pass 75 | :end-before: def run( 76 | 77 | It takes the last entry in the paths with ``self.path[1:]``, i.e. 78 | our ``2.5``, and tries to convert it into a floating point number. 79 | This will be the time the function is going to sleep, using ``time.sleep()``. 80 | This means waiting 2.5 seconds until it answers. 81 | The rest of the method contains the HTTP header and message. 82 | 83 | A Synchronous Client 84 | -------------------- 85 | 86 | Our first attempt is synchronous. 87 | This is the full implementation: 88 | 89 | .. literalinclude:: examples/synchronous_client.py 90 | 91 | Again, we go through it step-by-step. 92 | 93 | While about 80 % of the websites use ``utf-8`` as encoding 94 | (provided by the default in ``ENCODING``), it is a good idea to actually use 95 | the encoding specified by ``charset``. 96 | This is our helper to find out what the encoding of the page is: 97 | 98 | .. literalinclude:: examples/synchronous_client.py 99 | :language: python 100 | :start-after: ENCODING = 'ISO-8859-1' 101 | :end-before: def get_page 102 | 103 | It falls back to ``ISO-8859-1`` if it cannot find a specification of the 104 | encoding. 105 | 106 | Using ``urllib.request.urlopen()``, retrieving a web page is rather simple. 107 | The response is a bytestring and ``.encode()`` is needed to convert it into a 108 | string: 109 | 110 | .. literalinclude:: examples/synchronous_client.py 111 | :language: python 112 | :start-after: return ENCODING 113 | :end-before: def get_multiple_pages 114 | 115 | Now, we want multiple pages: 116 | 117 | .. literalinclude:: examples/synchronous_client.py 118 | :language: python 119 | :start-after: return html 120 | :end-before: if __name__ == '__main__': 121 | 122 | We just iterate over the waiting times and call ``get_page()`` for all 123 | of them. 124 | The function ``time.perf_counter()`` provides a time stamp. 125 | Taking two time stamps a different points in time and calculating their 126 | difference provides the elapsed run time. 127 | 128 | Finally, we can run our client:: 129 | 130 | .. highlight:: bash 131 | 132 | $ python synchronous_client.py 133 | 134 | and get this output:: 135 | 136 | It took 11.08 seconds for a total waiting time of 11.00. 137 | Waited for 1.00 seconds. 138 | That's all. 139 | 140 | Waited for 5.00 seconds. 141 | That's all. 142 | 143 | Waited for 3.00 seconds. 144 | That's all. 145 | 146 | Waited for 2.00 seconds. 147 | That's all. 148 | 149 | Because we wait for each call to ``get_page()`` to complete, we need to 150 | wait about 11 seconds. 151 | That is the sum of all waiting times. 152 | Let's see if we can do it any better going asynchronously. 153 | 154 | 155 | Getting One Page Asynchronously 156 | ------------------------------- 157 | 158 | This module contains a functions that reads a page asynchronously, 159 | using the new Python 3.5 keywords ``async`` and ``await``: 160 | 161 | .. literalinclude:: examples/async_page.py 162 | 163 | As with the synchronous example, finding out the encoding of the page 164 | is a good idea. 165 | This function helps here by going through the lines of the HTTP header, 166 | which it gets as an argument, searching for ``charset`` and returning its value 167 | if found. 168 | Again, the default encoding is ``ISO-8859-1``: 169 | 170 | .. literalinclude:: examples/async_page.py 171 | :language: python 172 | :start-after: ENCODING = 'ISO-8859-1' 173 | :end-before: async def get_page 174 | 175 | The next function is way more interesting because it actually works 176 | asynchronously: 177 | 178 | .. literalinclude:: examples/async_page.py 179 | :language: python 180 | :start-after: return ENCODING 181 | 182 | The function ``asyncio.open_connection()`` opens a connection to the given URL. 183 | It returns a coroutine. 184 | Using ``await``, which had to be ``yield from`` in Python versions prior 185 | to 3.5, it yields an instance of a ``StreamReader`` and one of a 186 | ``StreamWriter``. 187 | These only work within the event loop. 188 | 189 | Now, we can send a ``GET`` request, suppling our waiting time by 190 | writing to the ``StreamWriter`` instance ``writer``. 191 | The request has to be in bytes. 192 | Therefore, we need to convert our strings in to bytestrings. 193 | 194 | Next, we read header and message from the reader, which is a ``StreamReader`` 195 | instance. 196 | We need to iterate over the reader by using a special or loop for 197 | ``asyncio``: 198 | 199 | .. code-block:: python 200 | 201 | async for raw_line in reader: 202 | 203 | 204 | Header and message are dived by an empty line. 205 | We just stop the iteration as soon as we found an empty line. 206 | Handing the header over too ``get_encoding()`` provides the encoding 207 | of the retrieved page. 208 | The ``.decode()`` method uses this encoding to convert the read bytes 209 | into strings. 210 | After closing the writer, we can return the message lines joined by newline 211 | characters. 212 | 213 | Getting Multiple Pages Asynchronously - Without Time Savings 214 | ------------------------------------------------------------ 215 | 216 | This is our first approach retrieving multiple pages, using our asynchronous 217 | ``get_page()``: 218 | 219 | 220 | .. literalinclude:: examples/async_client_blocking.py 221 | 222 | 223 | The interesting things happen in a few lines in ``get_multiple_pages()`` 224 | (the rest of this function just measures the run time and displays it): 225 | 226 | .. literalinclude:: examples/async_client_blocking.py 227 | :language: python 228 | :start-after: pages = [] 229 | :end-before: duration 230 | 231 | The ``closing`` from the standard library module ``contextlib`` starts 232 | the event loop within a context and closes the loop when leaving the context: 233 | 234 | .. code-block:: python 235 | 236 | with closing(asyncio.get_event_loop()) as loop: 237 | 238 | 239 | The two lines above are equivalent to these five lines: 240 | 241 | .. code-block:: python 242 | 243 | loop = asyncio.get_event_loop(): 244 | try: 245 | 246 | finally: 247 | loop.close() 248 | 249 | We call ``get_page()`` for each page in a loop. 250 | Here we decide to wrap each call in ``loop.run_until_complete()``: 251 | 252 | .. code-block:: python 253 | 254 | for wait in waits: 255 | pages.append(loop.run_until_complete(get_page(host, port, wait))) 256 | 257 | This means, we wait until each pages has been retrieved before asking for 258 | the next. 259 | Let's run it from the command-line to see what happens:: 260 | 261 | .. highlight:: bash 262 | 263 | $ async_client_blocking.py 264 | It took 11.06 seconds for a total waiting time of 11.00. 265 | Waited for 1.00 seconds. 266 | That's all. 267 | Waited for 5.00 seconds. 268 | That's all. 269 | Waited for 3.00 seconds. 270 | That's all. 271 | Waited for 2.00 seconds. 272 | That's all. 273 | 274 | So it still takes about eleven seconds in total. 275 | We made it more complex and did not improve speed. 276 | Let's see if we can do better. 277 | 278 | Getting Multiple Pages Asynchronously - With Time Savings 279 | --------------------------------------------------------- 280 | 281 | We want to take advantage of the asynchronous nature of ``get_page()`` 282 | and save time. 283 | We modify our client to use a list with four instances of 284 | a :term:`task `. 285 | This allows us to send out requests for all pages we want to retrieve without 286 | waiting for the answer before asking for the next page: 287 | 288 | .. literalinclude:: examples/async_client_nonblocking.py 289 | 290 | The interesting part is in this loop: 291 | 292 | .. code-block:: python 293 | 294 | with closing(asyncio.get_event_loop()) as loop: 295 | for wait in waits: 296 | tasks.append(get_page(host, port, wait)) 297 | pages = loop.run_until_complete(asyncio.gather(*tasks)) 298 | 299 | We append all return values of ``get_page()`` to our lits of tasks. 300 | This allows us to send out all request, in our case four, without 301 | waiting for the answers. 302 | After sending all of them, we wait for the answers, using: 303 | 304 | .. code-block:: python 305 | 306 | loop.run_until_complete(asyncio.gather(*tasks)) 307 | 308 | We used ``loop.run_until_complete()`` already for each call to ``get_page()`` 309 | in the previous section. 310 | The difference here is the use of ``asyncio.gather()`` that is called with all 311 | our tasks in the list ``tasks`` as arguments. 312 | The ``asyncio.gather(*tasks)`` means for our example with four list entries: 313 | 314 | .. code-block:: python 315 | 316 | asyncio.gather(tasks[0], tasks[1], tasks[2], tasks[3]) 317 | 318 | So, for a list with 100 tasks it would mean: 319 | 320 | .. code-block:: python 321 | 322 | asyncio.gather(tasks[0], tasks[1], tasks[2], 323 | # 96 more tasks here 324 | tasks[99]) 325 | 326 | 327 | Let's see if we got any faster:: 328 | 329 | .. highlight:: bash 330 | 331 | $ async_client_nonblocking.py 332 | It took 5.08 seconds for a total waiting time of 11.00. 333 | Waited for 1.00 seconds. 334 | That's all. 335 | Waited for 5.00 seconds. 336 | That's all. 337 | Waited for 3.00 seconds. 338 | That's all. 339 | Waited for 2.00 seconds. 340 | That's all. 341 | 342 | Yes! It works. 343 | The total run time is about five seconds. 344 | This is the run time for the longest wait. 345 | Now, we don't have to wait for the sum of ``waits`` but rather for 346 | ``max(waits)``. 347 | 348 | We did quite a bit of work, sending a request and scanning an answer, 349 | including finding out the encoding. 350 | There should be a shorter way as these steps seem to be always necessary for 351 | getting the page content with the right encoding. 352 | Therefore, in the next section, we will have a look at high-level library 353 | ``aiohttp`` that can help to make our code shorter. 354 | 355 | Exercise 356 | ++++++++ 357 | 358 | Add more waiting times to the list ``waits`` and see how this impacts 359 | the run times of the blocking and the non-blocking implementation. 360 | Try (positive) numbers that are all less than five. 361 | Then try numbers greater than five. 362 | 363 | High-Level Approach with ``aiohttp`` 364 | ------------------------------------ 365 | 366 | The library aiohttp_ allows to write HTTP client and server applications, 367 | using a high-level approach. 368 | Install with:: 369 | 370 | .. highlight:: bash 371 | 372 | $ pip install aiohttp 373 | 374 | 375 | .. _aiohttp: https://aiohttp.readthedocs.io/en/stable/ 376 | 377 | The whole program looks like this: 378 | 379 | .. literalinclude:: examples/aiohttp_client.py 380 | 381 | The function to get one page is asynchronous, because of the ``async def``: 382 | 383 | 384 | .. literalinclude:: examples/aiohttp_client.py 385 | :language: python 386 | :start-after: import aiohttp 387 | :end-before: def get_multiple_pages 388 | 389 | The arguments are the same as those for the previous function to retrieve one 390 | page plus the additional argument ``session``. 391 | The first task is to construct the full URL as a string from the given 392 | host, port, and the desired waiting time. 393 | 394 | We use a timeout of 10 seconds. 395 | If it takes longer than the given time to retrieve a page, the programm 396 | throws a ``TimeoutError``. 397 | Therefore, to make this more robust, you might want to catch this error and 398 | handle it appropriately. 399 | 400 | The ``async with`` provides a context manager that gives us a response. 401 | After checking the status being ``200``, which means that all is alright, 402 | we need to ``await`` again to return the body of the page, using the method 403 | ``text()`` on the response. 404 | 405 | This is the interesting part of ``get_multiple_pages()``: 406 | 407 | .. code-block:: python 408 | 409 | with closing(asyncio.get_event_loop()) as loop: 410 | with aiohttp.ClientSession(loop=loop) as session: 411 | for wait in waits: 412 | tasks.append(fetch_page(session, host, port, wait)) 413 | pages = loop.run_until_complete(asyncio.gather(*tasks)) 414 | 415 | It is very similar to the code in the example of the time-saving implementation 416 | with ``asyncio``. 417 | The only difference is the opened client session and handing over this session 418 | to ``fetch_page()`` as the first argument. 419 | 420 | Finally, we run this program:: 421 | 422 | .. highlight:: bash 423 | 424 | $ python aiohttp_client.py 425 | It took 5.04 seconds for a total waiting time of 11.00. 426 | Waited for 1.00 seconds. 427 | That's all. 428 | 429 | Waited for 5.00 seconds. 430 | That's all. 431 | 432 | Waited for 3.00 seconds. 433 | That's all. 434 | 435 | Waited for 2.00 seconds. 436 | That's all. 437 | 438 | It also takes about five seconds and gives the same output as our version 439 | before. 440 | But the implementation for getting a single page is much simpler and takes 441 | care of the encoding and other aspects not mentioned here. 442 | 443 | -------------------------------------------------------------------------------- /why_asyncio.rst: -------------------------------------------------------------------------------- 1 | ++++++++++++++++ 2 | Why use asyncio? 3 | ++++++++++++++++ 4 | 5 | Why asynchronous programming? 6 | ============================= 7 | 8 | asyncio is a library to write asynchronous applications. It is the most 9 | efficient way to implement a network server having to handle many concurrent 10 | users. 11 | 12 | 13 | But gevent and eventlet just work! 14 | ================================== 15 | 16 | Or *Why should I bother with all these extra annoying async and await 17 | keywords?*. 18 | 19 | In short, asyncio adopted a radically different solution for race conditions. 20 | 21 | Parallel computing using threads is hard because of race conditions. Gevent and 22 | eventlet have a similar issue using "green" (lightweight) threads. 23 | 24 | Code written with asyncio is less error-prone: by just looking at the code, it 25 | is possible to identify which parts of the code are under our controls and 26 | where the event loop takes over the control flow and is able to run other tasks 27 | when our task is waiting for something. 28 | 29 | gevent and eventlet are designed to hide the asynchronous programming. For 30 | non-expert, and sometimes even for experts, it is really hard to guess where 31 | the event loop is allowed to suspend the task and run other tasks in 32 | background. It is even worse. A modification in a third party library can 33 | change the behaviour of our code, introduce a new point where the task is 34 | suspended. 35 | 36 | For an example, see the "Ca(sh|che Coherent) Money" section of the `Unyielding 37 | `_ article (by Glyph, 38 | February, 2014). 39 | --------------------------------------------------------------------------------