├── .github ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md └── workflows │ └── ci.yaml ├── .gitignore ├── CHANGELOG.md ├── Cargo.toml ├── LICENSE-APACHE ├── LICENSE-MIT ├── README.md ├── examples └── print_client_debug.rs ├── src ├── config.rs ├── h1 │ ├── mod.rs │ ├── tcp.rs │ └── tls.rs ├── hyper.rs ├── isahc.rs ├── lib.rs ├── native.rs └── wasm.rs └── tests └── test.rs /.github/CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, gender identity and expression, level of 9 | experience, 10 | education, socio-economic status, nationality, personal appearance, race, 11 | religion, or sexual identity and orientation. 12 | 13 | ## Our Standards 14 | 15 | Examples of behavior that contributes to creating a positive environment 16 | include: 17 | 18 | - Using welcoming and inclusive language 19 | - Being respectful of differing viewpoints and experiences 20 | - Gracefully accepting constructive criticism 21 | - Focusing on what is best for the community 22 | - Showing empathy towards other community members 23 | 24 | Examples of unacceptable behavior by participants include: 25 | 26 | - The use of sexualized language or imagery and unwelcome sexual attention or 27 | advances 28 | - Trolling, insulting/derogatory comments, and personal or political attacks 29 | - Public or private harassment 30 | - Publishing others' private information, such as a physical or electronic 31 | address, without explicit permission 32 | - Other conduct which could reasonably be considered inappropriate in a 33 | professional setting 34 | 35 | 36 | ## Our Responsibilities 37 | 38 | Project maintainers are responsible for clarifying the standards of acceptable 39 | behavior and are expected to take appropriate and fair corrective action in 40 | response to any instances of unacceptable behavior. 41 | 42 | Project maintainers have the right and responsibility to remove, edit, or 43 | reject comments, commits, code, wiki edits, issues, and other contributions 44 | that are not aligned to this Code of Conduct, or to ban temporarily or 45 | permanently any contributor for other behaviors that they deem inappropriate, 46 | threatening, offensive, or harmful. 47 | 48 | ## Scope 49 | 50 | This Code of Conduct applies both within project spaces and in public spaces 51 | when an individual is representing the project or its community. Examples of 52 | representing a project or community include using an official project e-mail 53 | address, posting via an official social media account, or acting as an appointed 54 | representative at an online or offline event. Representation of a project may be 55 | further defined and clarified by project maintainers. 56 | 57 | ## Enforcement 58 | 59 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 60 | reported by contacting the project team at yoshuawuyts@gmail.com, or through 61 | IRC. All complaints will be reviewed and investigated and will result in a 62 | response that is deemed necessary and appropriate to the circumstances. The 63 | project team is obligated to maintain confidentiality with regard to the 64 | reporter of an incident. 65 | Further details of specific enforcement policies may be posted separately. 66 | 67 | Project maintainers who do not follow or enforce the Code of Conduct in good 68 | faith may face temporary or permanent repercussions as determined by other 69 | members of the project's leadership. 70 | 71 | ## Attribution 72 | 73 | This Code of Conduct is adapted from the Contributor Covenant, version 1.4, 74 | available at 75 | https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 76 | -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | Contributions include code, documentation, answering user questions, running the 3 | project's infrastructure, and advocating for all types of users. 4 | 5 | The project welcomes all contributions from anyone willing to work in good faith 6 | with other contributors and the community. No contribution is too small and all 7 | contributions are valued. 8 | 9 | This guide explains the process for contributing to the project's GitHub 10 | Repository. 11 | 12 | - [Code of Conduct](#code-of-conduct) 13 | - [Bad Actors](#bad-actors) 14 | 15 | ## Code of Conduct 16 | The project has a [Code of Conduct](./CODE_OF_CONDUCT.md) that *all* 17 | contributors are expected to follow. This code describes the *minimum* behavior 18 | expectations for all contributors. 19 | 20 | As a contributor, how you choose to act and interact towards your 21 | fellow contributors, as well as to the community, will reflect back not only 22 | on yourself but on the project as a whole. The Code of Conduct is designed and 23 | intended, above all else, to help establish a culture within the project that 24 | allows anyone and everyone who wants to contribute to feel safe doing so. 25 | 26 | Should any individual act in any way that is considered in violation of the 27 | [Code of Conduct](./CODE_OF_CONDUCT.md), corrective actions will be taken. It is 28 | possible, however, for any individual to *act* in such a manner that is not in 29 | violation of the strict letter of the Code of Conduct guidelines while still 30 | going completely against the spirit of what that Code is intended to accomplish. 31 | 32 | Open, diverse, and inclusive communities live and die on the basis of trust. 33 | Contributors can disagree with one another so long as they trust that those 34 | disagreements are in good faith and everyone is working towards a common 35 | goal. 36 | 37 | ## Bad Actors 38 | All contributors to tacitly agree to abide by both the letter and 39 | spirit of the [Code of Conduct](./CODE_OF_CONDUCT.md). Failure, or 40 | unwillingness, to do so will result in contributions being respectfully 41 | declined. 42 | 43 | A *bad actor* is someone who repeatedly violates the *spirit* of the Code of 44 | Conduct through consistent failure to self-regulate the way in which they 45 | interact with other contributors in the project. In doing so, bad actors 46 | alienate other contributors, discourage collaboration, and generally reflect 47 | poorly on the project as a whole. 48 | 49 | Being a bad actor may be intentional or unintentional. Typically, unintentional 50 | bad behavior can be easily corrected by being quick to apologize and correct 51 | course *even if you are not entirely convinced you need to*. Giving other 52 | contributors the benefit of the doubt and having a sincere willingness to admit 53 | that you *might* be wrong is critical for any successful open collaboration. 54 | 55 | Don't be a bad actor. 56 | -------------------------------------------------------------------------------- /.github/workflows/ci.yaml: -------------------------------------------------------------------------------- 1 | name: CI 2 | 3 | on: 4 | pull_request: 5 | push: 6 | branches: 7 | - staging 8 | - trying 9 | 10 | jobs: 11 | build_and_test: 12 | name: Build and test 13 | runs-on: ${{ matrix.os }} 14 | strategy: 15 | matrix: 16 | os: [ubuntu-latest, windows-latest, macOS-latest] 17 | backend: ["h1_client,native-tls", hyper_client, curl_client] 18 | 19 | steps: 20 | - uses: actions/checkout@master 21 | 22 | - name: check 23 | run: cargo check --all-targets --workspace --no-default-features --features '${{ matrix.backend }}' 24 | 25 | - name: tests 26 | run: cargo test --all-targets --workspace --no-default-features --features '${{ matrix.backend }}' 27 | 28 | check_no_features: 29 | name: Checking without default features 30 | runs-on: ubuntu-latest 31 | steps: 32 | - uses: actions/checkout@master 33 | 34 | - name: check 35 | run: cargo check --no-default-features 36 | 37 | clippy_fmt_docs: 38 | name: Running clippy & fmt & docs 39 | runs-on: ubuntu-latest 40 | steps: 41 | - uses: actions/checkout@master 42 | 43 | - name: Install nightly toolchain 44 | uses: actions-rs/toolchain@v1 45 | with: 46 | profile: minimal 47 | toolchain: nightly 48 | override: true 49 | components: clippy, rustfmt 50 | 51 | - name: clippy 52 | run: cargo clippy --all-targets --workspace --features=docs 53 | 54 | - name: fmt 55 | run: cargo fmt --all -- --check 56 | 57 | - name: docs 58 | run: cargo doc --no-deps --features=docs 59 | 60 | check_wasm: 61 | name: Check wasm targets 62 | runs-on: ubuntu-latest 63 | 64 | steps: 65 | - uses: actions/checkout@master 66 | 67 | - name: Install nightly with wasm32-unknown-unknown 68 | uses: actions-rs/toolchain@v1 69 | with: 70 | toolchain: nightly 71 | target: wasm32-unknown-unknown 72 | override: true 73 | 74 | - name: check 75 | uses: actions-rs/cargo@v1 76 | with: 77 | command: check 78 | args: --target wasm32-unknown-unknown --no-default-features --features "native_client,wasm_client" 79 | 80 | check_features: 81 | name: Check feature combinations 82 | runs-on: ubuntu-latest 83 | steps: 84 | - uses: actions/checkout@master 85 | 86 | - name: Install cargo-hack 87 | run: cargo install cargo-hack 88 | 89 | - name: Check all feature combinations works properly 90 | # * `--feature-powerset` - run for the feature powerset of the package 91 | # * `--no-dev-deps` - build without dev-dependencies to avoid https://github.com/rust-lang/cargo/issues/4866 92 | # * `--skip docs` - skip `docs` feature 93 | run: cargo hack check --feature-powerset --no-dev-deps --skip docs 94 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | coverage/ 2 | target/ 3 | tmp/ 4 | dist/ 5 | npm-debug.log* 6 | Cargo.lock 7 | .DS_Store 8 | .vscode/ 9 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | All notable changes to surf will be documented in this file. 4 | 5 | The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), 6 | and this project adheres to [Semantic Versioning](https://book.async.rs/overview/stability-guarantees.html). 7 | 8 | ## [Unreleased] 9 | 10 | ## [6.5.3] 11 | 12 | ### Deps 13 | - `h1-client` now uses `dashmap` at version `5.x >`, fixing an unsoundness issue. 14 | 15 | ## [6.5.2] 16 | 17 | ### Deps 18 | - Now only uses `dashmap` for `h1-client`. 19 | 20 | ## [6.5.1] 21 | 22 | Same as 6.5.0 with one change: 23 | 24 | `Config::max_connections_per_host()` is now properly named `Config::set_max_connections_per_host()`. 25 | 26 | ## [6.5.0] 27 | 28 | (Yanked) 29 | 30 | ### Added 31 | - `Config` has been stabilized and is now available by default! 32 | - `wasm_client` support for `Config` (only timeouts). 33 | - `Config::max_connections_per_host` (Supported on `h1_client` and `curl_client`.) 34 | 35 | ### Deprecated 36 | - `H1Client::with_max_connections()` will be superseded by `Config::max_connections_per_host`. 37 | 38 | ## [6.4.1] - 2021-05-19 39 | 40 | ### Docs 41 | - Added `"unstable-config"` to the docs builds. 42 | 43 | ## [6.4.0] - 2021-05-17 44 | 45 | ### Added 46 | - Added a new `unstable-config` feature, which exposes runtime configuration via a new `Config` struct. 47 | 48 | ## [6.3.5] - 2021-03-12 49 | 50 | ### Fixed 51 | - Multiple headers of the same name are now present with any client backend and not just `h1_client`. 52 | - Connection when multiple IPs are present for a hostname not function with the `h1_client` backend. 53 | 54 | ## [6.3.4] - 2021-03-06 55 | 56 | ### Fixed 57 | - `h1_client` connection pools now properly check if connections are still alive before recycling them. 58 | - Like, actually properly this time. 59 | - There is a test now to ensure closed connections don't cause errors. 60 | 61 | ## [6.3.3] - 2021-03-01 62 | 63 | ### Fixed 64 | - `h1_client` connection pools now properly check if connections are still alive before recycling them. 65 | 66 | ## [6.3.2] - 2021-03-01 67 | 68 | _(This was the same thing as 6.3.1 released by git accident.)_ 69 | 70 | ## [6.3.1] - 2021-02-15 71 | 72 | ### Fixed 73 | - Allow http-client to build & run properly when `h1_client` is enabled without either tls option. 74 | - Prefer `rustls` if both tls features are enabled. 75 | 76 | ### Internal 77 | - More exhaustive CI for feature combinations. 78 | 79 | ## [6.3.0] - 2021-02-12 80 | 81 | ### Added 82 | - Connection pooling (HTTP/1.1 `keep-alive`) for `h1_client` (default). 83 | - `native-tls` (default) and `rustls` feature flags. 84 | - Only works with `h1_client`. 85 | - Isahc metrics as a response extension for `curl_client`. 86 | 87 | ### Fixed 88 | - `Box` no longer infinitely recurses. 89 | - `curl_client` now always correctly reads the response body. 90 | - `hyper_client` should now build correctly. 91 | - `WasmClient` fetch from worker scope now works correctly. 92 | 93 | ### Internal 94 | - Improved CI 95 | 96 | ## [6.2.0] - 2020-10-26 97 | 98 | This release implements `HttpClient` for `Box`. 99 | 100 | ### Added 101 | - `impl HttpClient for Box` 102 | 103 | ## [6.1.0] - 2020-10-09 104 | 105 | This release brings improvements for `HyperClient` (`hyper_client` feature). 106 | 107 | ### Added 108 | - `HyperClient` now impls `Default`. 109 | - `HyperClient::from_client(hyper::Client)`. 110 | 111 | ### Changed 112 | - `HyperClient` now re-uses the internal client, allowing connection pooling. 113 | 114 | ## [6.0.0] - 2020-09-25 115 | 116 | This release moves the responsibility of any client sharing to the user. 117 | 118 | ### Changed 119 | - `HttpClient` implementations no longer `impl Clone`. 120 | - The responsibility for sharing is the user's. 121 | - `H1Client` can no longer be instatiated via `H1Client {}`. 122 | - `::new()` should be used. 123 | 124 | ## [5.0.1] - 2020-09-18 125 | 126 | ### Fixed 127 | - Fixed a body stream translation bug in the `hyper_client`. 128 | 129 | ## [5.0.0] - 2020-09-18 130 | 131 | This release includes an optional backend using [hyper.rs](https://hyper.rs/), and uses [async-trait](https://crates.io/crates/async-trait) for `HttpClient`. 132 | 133 | ### Added 134 | - `hyper_client` feature, for using [hyper.rs](https://hyper.rs/) as the client backend. 135 | 136 | ### Changed 137 | - `HttpClient` now uses [async-trait](https://crates.io/crates/async-trait). 138 | - This attribute is also re-exported as `http_client::async_trait`. 139 | 140 | ### Fixed 141 | - Fixed WASM compilation. 142 | - Fixed Isahc (curl) client translation setting duplicate headers incorrectly. 143 | 144 | ## [4.0.0] - 2020-07-09 145 | 146 | This release allows `HttpClient` to be used as a dynamic Trait object. 147 | 148 | - `HttpClient`: removed `Clone` bounds. 149 | - `HttpClient`: removed `Error` type. 150 | 151 | ## [3.0.0] - 2020-05-29 152 | 153 | This patch updates `http-client` to `http-types 2.0.0` and a new version of `async-h1`. 154 | 155 | ### Changes 156 | - http types and async-h1 for 2.0.0 #27 157 | 158 | ## [2.0.0] - 2020-04-17 159 | 160 | ### Added 161 | - Added a new backend: `h1-client` https://github.com/http-rs/http-client/pull/22 162 | 163 | ### Changed 164 | - All types are now based from `hyperium/http` to `http-types` https://github.com/http-rs/http-client/pull/22 165 | -------------------------------------------------------------------------------- /Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "http-client" 3 | version = "6.5.3" 4 | license = "MIT OR Apache-2.0" 5 | repository = "https://github.com/http-rs/http-client" 6 | documentation = "https://docs.rs/http-client" 7 | description = "Types and traits for http clients." 8 | keywords = ["http", "service", "client", "futures", "async"] 9 | categories = ["asynchronous", "web-programming", "web-programming::http-client", "web-programming::websocket"] 10 | authors = [ 11 | "Yoshua Wuyts ", 12 | "dignifiedquire ", 13 | "Jeremiah Senkpiel " 14 | ] 15 | readme = "README.md" 16 | edition = "2018" 17 | 18 | [package.metadata.docs.rs] 19 | features = ["docs"] 20 | rustdoc-args = ["--cfg", "feature=\"docs\""] 21 | 22 | [features] 23 | default = ["h1_client", "native-tls"] 24 | docs = ["h1_client", "curl_client", "wasm_client", "hyper_client"] 25 | 26 | h1_client = ["async-h1", "async-std", "dashmap", "deadpool", "futures"] 27 | native_client = ["curl_client", "wasm_client"] 28 | curl_client = ["isahc", "async-std"] 29 | wasm_client = ["js-sys", "web-sys", "wasm-bindgen", "wasm-bindgen-futures", "futures", "async-std"] 30 | hyper_client = ["hyper", "hyper-tls", "http-types/hyperium_http", "futures-util", "tokio"] 31 | 32 | native-tls = ["async-native-tls"] 33 | rustls = ["async-tls", "rustls_crate"] 34 | 35 | unstable-config = [] # deprecated 36 | 37 | [dependencies] 38 | async-trait = "0.1.37" 39 | http-types = "2.3.0" 40 | log = "0.4.7" 41 | cfg-if = "1.0.0" 42 | 43 | # h1_client 44 | async-h1 = { version = "2.0.0", optional = true } 45 | async-std = { version = "1.6.0", default-features = false, optional = true } 46 | async-native-tls = { version = "0.3.1", optional = true } 47 | dashmap = { version = "5.3.4", optional = true } 48 | deadpool = { version = "0.7.0", optional = true } 49 | futures = { version = "0.3.8", optional = true } 50 | 51 | # h1_client_rustls 52 | async-tls = { version = "0.11", optional = true } 53 | rustls_crate = { version = "0.19", optional = true, package = "rustls" } 54 | 55 | # hyper_client 56 | hyper = { version = "0.13.6", features = ["tcp"], optional = true } 57 | hyper-tls = { version = "0.4.3", optional = true } 58 | futures-util = { version = "0.3.5", features = ["io"], optional = true } 59 | tokio = { version = "0.2", features = ["time"], optional = true } 60 | 61 | # curl_client 62 | [target.'cfg(not(target_arch = "wasm32"))'.dependencies] 63 | isahc = { version = "0.9", optional = true, default-features = false, features = ["http2"] } 64 | 65 | # wasm_client 66 | [target.'cfg(target_arch = "wasm32")'.dependencies] 67 | js-sys = { version = "0.3.25", optional = true } 68 | wasm-bindgen = { version = "0.2.48", optional = true } 69 | wasm-bindgen-futures = { version = "0.4.5", optional = true } 70 | futures = { version = "0.3.1", optional = true } 71 | send_wrapper = { version = "0.6.0", features = ["futures"] } 72 | 73 | [target.'cfg(target_arch = "wasm32")'.dependencies.web-sys] 74 | version = "0.3.25" 75 | optional = true 76 | features = [ 77 | "AbortSignal", 78 | "Headers", 79 | "ObserverCallback", 80 | "ReferrerPolicy", 81 | "Request", 82 | "RequestCache", 83 | "RequestCredentials", 84 | "RequestInit", 85 | "RequestMode", 86 | "RequestRedirect", 87 | "Response", 88 | "Window", 89 | "WorkerGlobalScope", 90 | ] 91 | 92 | [dev-dependencies] 93 | async-std = { version = "1.6.0", features = ["unstable", "attributes"] } 94 | portpicker = "0.1.0" 95 | tide = { version = "0.15.0", default-features = false, features = ["h1-server"] } 96 | tide-rustls = { version = "0.1.4" } 97 | tokio = { version = "0.2.21", features = ["macros"] } 98 | serde = "1.0" 99 | serde_json = "1.0" 100 | mockito = "0.23.3" 101 | 102 | [dev-dependencies.getrandom] 103 | version = "0.2" 104 | features = ["js"] 105 | -------------------------------------------------------------------------------- /LICENSE-APACHE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | Copyright 2019 Yoshua Wuyts 179 | 180 | Licensed under the Apache License, Version 2.0 (the "License"); 181 | you may not use this file except in compliance with the License. 182 | You may obtain a copy of the License at 183 | 184 | http://www.apache.org/licenses/LICENSE-2.0 185 | 186 | Unless required by applicable law or agreed to in writing, software 187 | distributed under the License is distributed on an "AS IS" BASIS, 188 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 189 | See the License for the specific language governing permissions and 190 | limitations under the License. 191 | -------------------------------------------------------------------------------- /LICENSE-MIT: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2019 Yoshua Wuyts 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

http-client

2 |
3 | 4 | Types and traits for http clients. 5 | 6 |
7 | 8 |
9 | 10 |
11 | 12 | 13 | Crates.io version 15 | 16 | 17 | 18 | Download 20 | 21 | 22 | 23 | docs.rs docs 25 | 26 |
27 | 28 | 43 | 44 | ## Installation 45 | 46 | With [cargo add][cargo-add] installed run: 47 | 48 | ```sh 49 | $ cargo add http-client 50 | ``` 51 | 52 | [cargo-add]: https://github.com/killercup/cargo-edit 53 | 54 | ## Safety 55 | 56 | For non-wasm clients, this crate uses ``#![deny(unsafe_code)]`` to ensure everything is implemented in 57 | 100% Safe Rust. 58 | 59 | ## Contributing 60 | 61 | Want to join us? Check out our ["Contributing" guide][contributing] and take a 62 | look at some of these issues: 63 | 64 | - [Issues labeled "good first issue"][good-first-issue] 65 | - [Issues labeled "help wanted"][help-wanted] 66 | 67 | [contributing]: https://github.com/http-rs/http-client/blob/main/.github/CONTRIBUTING.md 68 | [good-first-issue]: https://github.com/http-rs/http-client/labels/good%20first%20issue 69 | [help-wanted]: https://github.com/http-rs/http-client/labels/help%20wanted 70 | 71 | ## License 72 | 73 | 74 | Licensed under either of Apache License, Version 75 | 2.0 or MIT license at your option. 76 | 77 | 78 |
79 | 80 | 81 | Unless you explicitly state otherwise, any contribution intentionally submitted 82 | for inclusion in this crate by you, as defined in the Apache-2.0 license, shall 83 | be dual licensed as above, without any additional terms or conditions. 84 | 85 | -------------------------------------------------------------------------------- /examples/print_client_debug.rs: -------------------------------------------------------------------------------- 1 | use http_client::HttpClient; 2 | use http_types::{Method, Request}; 3 | 4 | #[cfg(any(feature = "h1_client", feature = "docs"))] 5 | use http_client::h1::H1Client as Client; 6 | #[cfg(all(feature = "hyper_client", not(feature = "docs")))] 7 | use http_client::hyper::HyperClient as Client; 8 | #[cfg(all(feature = "curl_client", not(feature = "docs")))] 9 | use http_client::isahc::IsahcClient as Client; 10 | #[cfg(all(feature = "wasm_client", not(feature = "docs")))] 11 | use http_client::wasm::WasmClient as Client; 12 | 13 | #[async_std::main] 14 | async fn main() { 15 | let client = Client::new(); 16 | 17 | let req = Request::new(Method::Get, "http://example.org"); 18 | 19 | client.send(req).await.unwrap(); 20 | 21 | dbg!(client); 22 | } 23 | -------------------------------------------------------------------------------- /src/config.rs: -------------------------------------------------------------------------------- 1 | //! Configuration for `HttpClient`s. 2 | 3 | use std::fmt::Debug; 4 | use std::time::Duration; 5 | 6 | /// Configuration for `HttpClient`s. 7 | #[non_exhaustive] 8 | #[derive(Clone)] 9 | pub struct Config { 10 | /// HTTP/1.1 `keep-alive` (connection pooling). 11 | /// 12 | /// Default: `true`. 13 | /// 14 | /// Note: Does nothing on `wasm_client`. 15 | pub http_keep_alive: bool, 16 | /// TCP `NO_DELAY`. 17 | /// 18 | /// Default: `false`. 19 | /// 20 | /// Note: Does nothing on `wasm_client`. 21 | pub tcp_no_delay: bool, 22 | /// Connection timeout duration. 23 | /// 24 | /// Default: `Some(Duration::from_secs(60))`. 25 | pub timeout: Option, 26 | /// Maximum number of simultaneous connections that this client is allowed to keep open to individual hosts at one time. 27 | /// 28 | /// Default: `50`. 29 | /// This number is based on a few random benchmarks and see whatever gave decent perf vs resource use in Orogene. 30 | /// 31 | /// Note: The behavior of this is different depending on the backend in use. 32 | /// - `h1_client`: `0` is disallowed and asserts as otherwise it would cause a semaphore deadlock. 33 | /// - `curl_client`: `0` allows for limitless connections per host. 34 | /// - `hyper_client`: No effect. Hyper does not support such an option. 35 | /// - `wasm_client`: No effect. Web browsers do not support such an option. 36 | pub max_connections_per_host: usize, 37 | /// TLS Configuration (Rustls) 38 | #[cfg_attr(feature = "docs", doc(cfg(feature = "h1_client")))] 39 | #[cfg(all(feature = "h1_client", feature = "rustls"))] 40 | pub tls_config: Option>, 41 | /// TLS Configuration (Native TLS) 42 | #[cfg_attr(feature = "docs", doc(cfg(feature = "h1_client")))] 43 | #[cfg(all(feature = "h1_client", feature = "native-tls", not(feature = "rustls")))] 44 | pub tls_config: Option>, 45 | } 46 | 47 | impl Debug for Config { 48 | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { 49 | let mut dbg_struct = f.debug_struct("Config"); 50 | dbg_struct 51 | .field("http_keep_alive", &self.http_keep_alive) 52 | .field("tcp_no_delay", &self.tcp_no_delay) 53 | .field("timeout", &self.timeout) 54 | .field("max_connections_per_host", &self.max_connections_per_host); 55 | 56 | #[cfg(all(feature = "h1_client", feature = "rustls"))] 57 | { 58 | if self.tls_config.is_some() { 59 | dbg_struct.field("tls_config", &"Some(rustls::ClientConfig)"); 60 | } else { 61 | dbg_struct.field("tls_config", &"None"); 62 | } 63 | } 64 | #[cfg(all(feature = "h1_client", feature = "native-tls", not(feature = "rustls")))] 65 | { 66 | dbg_struct.field("tls_config", &self.tls_config); 67 | } 68 | 69 | dbg_struct.finish() 70 | } 71 | } 72 | 73 | impl Config { 74 | /// Construct new empty config. 75 | pub fn new() -> Self { 76 | Self { 77 | http_keep_alive: true, 78 | tcp_no_delay: false, 79 | timeout: Some(Duration::from_secs(60)), 80 | max_connections_per_host: 50, 81 | #[cfg(all(feature = "h1_client", any(feature = "rustls", feature = "native-tls")))] 82 | tls_config: None, 83 | } 84 | } 85 | } 86 | 87 | impl Default for Config { 88 | fn default() -> Self { 89 | Self::new() 90 | } 91 | } 92 | 93 | impl Config { 94 | /// Set HTTP/1.1 `keep-alive` (connection pooling). 95 | pub fn set_http_keep_alive(mut self, keep_alive: bool) -> Self { 96 | self.http_keep_alive = keep_alive; 97 | self 98 | } 99 | 100 | /// Set TCP `NO_DELAY`. 101 | pub fn set_tcp_no_delay(mut self, no_delay: bool) -> Self { 102 | self.tcp_no_delay = no_delay; 103 | self 104 | } 105 | 106 | /// Set connection timeout duration. 107 | pub fn set_timeout(mut self, timeout: Option) -> Self { 108 | self.timeout = timeout; 109 | self 110 | } 111 | 112 | /// Set the maximum number of simultaneous connections that this client is allowed to keep open to individual hosts at one time. 113 | pub fn set_max_connections_per_host(mut self, max_connections_per_host: usize) -> Self { 114 | self.max_connections_per_host = max_connections_per_host; 115 | self 116 | } 117 | 118 | /// Set TLS Configuration (Rustls) 119 | #[cfg_attr(feature = "docs", doc(cfg(feature = "h1_client")))] 120 | #[cfg(all(feature = "h1_client", feature = "rustls"))] 121 | pub fn set_tls_config( 122 | mut self, 123 | tls_config: Option>, 124 | ) -> Self { 125 | self.tls_config = tls_config; 126 | self 127 | } 128 | /// Set TLS Configuration (Native TLS) 129 | #[cfg_attr(feature = "docs", doc(cfg(feature = "h1_client")))] 130 | #[cfg(all(feature = "h1_client", feature = "native-tls", not(feature = "rustls")))] 131 | pub fn set_tls_config( 132 | mut self, 133 | tls_config: Option>, 134 | ) -> Self { 135 | self.tls_config = tls_config; 136 | self 137 | } 138 | } 139 | -------------------------------------------------------------------------------- /src/h1/mod.rs: -------------------------------------------------------------------------------- 1 | //! http-client implementation for async-h1, with connection pooling ("Keep-Alive"). 2 | 3 | use std::convert::{Infallible, TryFrom}; 4 | use std::fmt::Debug; 5 | use std::net::SocketAddr; 6 | use std::sync::Arc; 7 | 8 | use async_h1::client; 9 | use async_std::net::TcpStream; 10 | use dashmap::DashMap; 11 | use deadpool::managed::Pool; 12 | use http_types::StatusCode; 13 | 14 | cfg_if::cfg_if! { 15 | if #[cfg(feature = "rustls")] { 16 | use async_tls::client::TlsStream; 17 | } else if #[cfg(feature = "native-tls")] { 18 | use async_native_tls::TlsStream; 19 | } 20 | } 21 | 22 | use crate::Config; 23 | 24 | use super::{async_trait, Error, HttpClient, Request, Response}; 25 | 26 | mod tcp; 27 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 28 | mod tls; 29 | 30 | use tcp::{TcpConnWrapper, TcpConnection}; 31 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 32 | use tls::{TlsConnWrapper, TlsConnection}; 33 | 34 | type HttpPool = DashMap>; 35 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 36 | type HttpsPool = DashMap, Error>>; 37 | 38 | /// async-h1 based HTTP Client, with connection pooling ("Keep-Alive"). 39 | pub struct H1Client { 40 | http_pools: HttpPool, 41 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 42 | https_pools: HttpsPool, 43 | config: Arc, 44 | } 45 | 46 | impl Debug for H1Client { 47 | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { 48 | let https_pools = if cfg!(any(feature = "native-tls", feature = "rustls")) { 49 | self.http_pools 50 | .iter() 51 | .map(|pool| { 52 | let status = pool.status(); 53 | format!( 54 | "Connections: {}, Available: {}, Max: {}", 55 | status.size, status.available, status.max_size 56 | ) 57 | }) 58 | .collect::>() 59 | } else { 60 | vec![] 61 | }; 62 | 63 | f.debug_struct("H1Client") 64 | .field( 65 | "http_pools", 66 | &self 67 | .http_pools 68 | .iter() 69 | .map(|pool| { 70 | let status = pool.status(); 71 | format!( 72 | "Connections: {}, Available: {}, Max: {}", 73 | status.size, status.available, status.max_size 74 | ) 75 | }) 76 | .collect::>(), 77 | ) 78 | .field("https_pools", &https_pools) 79 | .field("config", &self.config) 80 | .finish() 81 | } 82 | } 83 | 84 | impl Default for H1Client { 85 | fn default() -> Self { 86 | Self::new() 87 | } 88 | } 89 | 90 | impl H1Client { 91 | /// Create a new instance. 92 | pub fn new() -> Self { 93 | Self { 94 | http_pools: DashMap::new(), 95 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 96 | https_pools: DashMap::new(), 97 | config: Arc::new(Config::default()), 98 | } 99 | } 100 | 101 | /// Create a new instance. 102 | #[deprecated( 103 | since = "6.5.0", 104 | note = "This function is misnamed. Prefer `Config::max_connections_per_host` instead." 105 | )] 106 | pub fn with_max_connections(max: usize) -> Self { 107 | #[cfg(features = "h1_client")] 108 | assert!(max > 0, "max_connections_per_host with h1_client must be greater than zero or it will deadlock!"); 109 | 110 | let config = Config { 111 | max_connections_per_host: max, 112 | ..Default::default() 113 | }; 114 | 115 | Self { 116 | http_pools: DashMap::new(), 117 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 118 | https_pools: DashMap::new(), 119 | config: Arc::new(config), 120 | } 121 | } 122 | } 123 | 124 | #[async_trait] 125 | impl HttpClient for H1Client { 126 | async fn send(&self, mut req: Request) -> Result { 127 | req.insert_header("Connection", "keep-alive"); 128 | 129 | // Insert host 130 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 131 | let host = req 132 | .url() 133 | .host_str() 134 | .ok_or_else(|| Error::from_str(StatusCode::BadRequest, "missing hostname"))? 135 | .to_string(); 136 | 137 | let scheme = req.url().scheme(); 138 | if scheme != "http" 139 | && (scheme != "https" || cfg!(not(any(feature = "native-tls", feature = "rustls")))) 140 | { 141 | return Err(Error::from_str( 142 | StatusCode::BadRequest, 143 | format!("invalid url scheme '{}'", scheme), 144 | )); 145 | } 146 | 147 | let addrs = req.url().socket_addrs(|| match req.url().scheme() { 148 | "http" => Some(80), 149 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 150 | "https" => Some(443), 151 | _ => None, 152 | })?; 153 | 154 | log::trace!("> Scheme: {}", scheme); 155 | 156 | let max_addrs_idx = addrs.len() - 1; 157 | for (idx, addr) in addrs.into_iter().enumerate() { 158 | let has_another_addr = idx != max_addrs_idx; 159 | 160 | if !self.config.http_keep_alive { 161 | match scheme { 162 | "http" => { 163 | let stream = async_std::net::TcpStream::connect(addr).await?; 164 | req.set_peer_addr(stream.peer_addr().ok()); 165 | req.set_local_addr(stream.local_addr().ok()); 166 | let tcp_conn = client::connect(stream, req); 167 | return if let Some(timeout) = self.config.timeout { 168 | async_std::future::timeout(timeout, tcp_conn).await? 169 | } else { 170 | tcp_conn.await 171 | }; 172 | } 173 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 174 | "https" => { 175 | let raw_stream = async_std::net::TcpStream::connect(addr).await?; 176 | req.set_peer_addr(raw_stream.peer_addr().ok()); 177 | req.set_local_addr(raw_stream.local_addr().ok()); 178 | let tls_stream = tls::add_tls(&host, raw_stream, &self.config).await?; 179 | let tsl_conn = client::connect(tls_stream, req); 180 | return if let Some(timeout) = self.config.timeout { 181 | async_std::future::timeout(timeout, tsl_conn).await? 182 | } else { 183 | tsl_conn.await 184 | }; 185 | } 186 | _ => unreachable!(), 187 | } 188 | } 189 | 190 | match scheme { 191 | "http" => { 192 | let pool_ref = if let Some(pool_ref) = self.http_pools.get(&addr) { 193 | pool_ref 194 | } else { 195 | let manager = TcpConnection::new(addr, self.config.clone()); 196 | let pool = Pool::::new( 197 | manager, 198 | self.config.max_connections_per_host, 199 | ); 200 | self.http_pools.insert(addr, pool); 201 | self.http_pools.get(&addr).unwrap() 202 | }; 203 | 204 | // Deadlocks are prevented by cloning an inner pool Arc and dropping the original locking reference before we await. 205 | let pool = pool_ref.clone(); 206 | std::mem::drop(pool_ref); 207 | 208 | let stream = match pool.get().await { 209 | Ok(s) => s, 210 | Err(_) if has_another_addr => continue, 211 | Err(e) => return Err(Error::from_str(400, e.to_string())), 212 | }; 213 | 214 | req.set_peer_addr(stream.peer_addr().ok()); 215 | req.set_local_addr(stream.local_addr().ok()); 216 | 217 | let tcp_conn = client::connect(TcpConnWrapper::new(stream), req); 218 | return if let Some(timeout) = self.config.timeout { 219 | async_std::future::timeout(timeout, tcp_conn).await? 220 | } else { 221 | tcp_conn.await 222 | }; 223 | } 224 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 225 | "https" => { 226 | let pool_ref = if let Some(pool_ref) = self.https_pools.get(&addr) { 227 | pool_ref 228 | } else { 229 | let manager = TlsConnection::new(host.clone(), addr, self.config.clone()); 230 | let pool = Pool::, Error>::new( 231 | manager, 232 | self.config.max_connections_per_host, 233 | ); 234 | self.https_pools.insert(addr, pool); 235 | self.https_pools.get(&addr).unwrap() 236 | }; 237 | 238 | // Deadlocks are prevented by cloning an inner pool Arc and dropping the original locking reference before we await. 239 | let pool = pool_ref.clone(); 240 | std::mem::drop(pool_ref); 241 | 242 | let stream = match pool.get().await { 243 | Ok(s) => s, 244 | Err(_) if has_another_addr => continue, 245 | Err(e) => return Err(Error::from_str(400, e.to_string())), 246 | }; 247 | 248 | req.set_peer_addr(stream.get_ref().peer_addr().ok()); 249 | req.set_local_addr(stream.get_ref().local_addr().ok()); 250 | 251 | let tls_conn = client::connect(TlsConnWrapper::new(stream), req); 252 | return if let Some(timeout) = self.config.timeout { 253 | async_std::future::timeout(timeout, tls_conn).await? 254 | } else { 255 | tls_conn.await 256 | }; 257 | } 258 | _ => unreachable!(), 259 | } 260 | } 261 | 262 | Err(Error::from_str( 263 | StatusCode::BadRequest, 264 | "missing valid address", 265 | )) 266 | } 267 | 268 | /// Override the existing configuration with new configuration. 269 | /// 270 | /// Config options may not impact existing connections. 271 | fn set_config(&mut self, config: Config) -> http_types::Result<()> { 272 | #[cfg(features = "h1_client")] 273 | assert!(config.max_connections_per_host > 0, "max_connections_per_host with h1_client must be greater than zero or it will deadlock!"); 274 | 275 | self.config = Arc::new(config); 276 | 277 | Ok(()) 278 | } 279 | 280 | /// Get the current configuration. 281 | fn config(&self) -> &Config { 282 | &*self.config 283 | } 284 | } 285 | 286 | impl TryFrom for H1Client { 287 | type Error = Infallible; 288 | 289 | fn try_from(config: Config) -> Result { 290 | #[cfg(features = "h1_client")] 291 | assert!(config.max_connections_per_host > 0, "max_connections_per_host with h1_client must be greater than zero or it will deadlock!"); 292 | 293 | Ok(Self { 294 | http_pools: DashMap::new(), 295 | #[cfg(any(feature = "native-tls", feature = "rustls"))] 296 | https_pools: DashMap::new(), 297 | config: Arc::new(config), 298 | }) 299 | } 300 | } 301 | 302 | #[cfg(test)] 303 | mod tests { 304 | use super::*; 305 | use async_std::prelude::*; 306 | use async_std::task; 307 | use http_types::url::Url; 308 | use http_types::Result; 309 | use std::time::Duration; 310 | 311 | fn build_test_request(url: Url) -> Request { 312 | let mut req = Request::new(http_types::Method::Post, url); 313 | req.set_body("hello"); 314 | req.append_header("test", "value"); 315 | req 316 | } 317 | 318 | #[async_std::test] 319 | async fn basic_functionality() -> Result<()> { 320 | let port = portpicker::pick_unused_port().unwrap(); 321 | let mut app = tide::new(); 322 | app.at("/").all(|mut r: tide::Request<()>| async move { 323 | let mut response = tide::Response::new(http_types::StatusCode::Ok); 324 | response.set_body(r.body_bytes().await.unwrap()); 325 | Ok(response) 326 | }); 327 | 328 | let server = task::spawn(async move { 329 | app.listen(("localhost", port)).await?; 330 | Result::Ok(()) 331 | }); 332 | 333 | let client = task::spawn(async move { 334 | task::sleep(Duration::from_millis(100)).await; 335 | let request = 336 | build_test_request(Url::parse(&format!("http://localhost:{}/", port)).unwrap()); 337 | let mut response: Response = H1Client::new().send(request).await?; 338 | assert_eq!(response.body_string().await.unwrap(), "hello"); 339 | Ok(()) 340 | }); 341 | 342 | server.race(client).await?; 343 | 344 | Ok(()) 345 | } 346 | 347 | #[async_std::test] 348 | async fn https_functionality() -> Result<()> { 349 | task::sleep(Duration::from_millis(100)).await; 350 | // Send a POST request to https://httpbin.org/post 351 | // The result should be a JSon string similar to what you get with: 352 | // curl -X POST "https://httpbin.org/post" -H "accept: application/json" -H "Content-Type: text/plain;charset=utf-8" -d "hello" 353 | let request = build_test_request(Url::parse("https://httpbin.org/post").unwrap()); 354 | let mut response: Response = H1Client::new().send(request).await?; 355 | let json_val: serde_json::value::Value = 356 | serde_json::from_str(&response.body_string().await.unwrap())?; 357 | assert_eq!(*json_val.get("data").unwrap(), serde_json::json!("hello")); 358 | Ok(()) 359 | } 360 | } 361 | -------------------------------------------------------------------------------- /src/h1/tcp.rs: -------------------------------------------------------------------------------- 1 | use std::net::SocketAddr; 2 | use std::pin::Pin; 3 | use std::sync::Arc; 4 | 5 | use async_std::net::TcpStream; 6 | use async_trait::async_trait; 7 | use deadpool::managed::{Manager, Object, RecycleResult}; 8 | use futures::io::{AsyncRead, AsyncWrite}; 9 | use futures::task::{Context, Poll}; 10 | 11 | use crate::Config; 12 | 13 | #[derive(Clone)] 14 | #[cfg_attr(not(feature = "rustls"), derive(std::fmt::Debug))] 15 | pub(crate) struct TcpConnection { 16 | addr: SocketAddr, 17 | config: Arc, 18 | } 19 | 20 | impl TcpConnection { 21 | pub(crate) fn new(addr: SocketAddr, config: Arc) -> Self { 22 | Self { addr, config } 23 | } 24 | } 25 | 26 | pub(crate) struct TcpConnWrapper { 27 | conn: Object, 28 | } 29 | impl TcpConnWrapper { 30 | pub(crate) fn new(conn: Object) -> Self { 31 | Self { conn } 32 | } 33 | } 34 | 35 | impl AsyncRead for TcpConnWrapper { 36 | fn poll_read( 37 | mut self: Pin<&mut Self>, 38 | cx: &mut Context<'_>, 39 | buf: &mut [u8], 40 | ) -> Poll> { 41 | Pin::new(&mut *self.conn).poll_read(cx, buf) 42 | } 43 | } 44 | 45 | impl AsyncWrite for TcpConnWrapper { 46 | fn poll_write( 47 | mut self: Pin<&mut Self>, 48 | cx: &mut Context<'_>, 49 | buf: &[u8], 50 | ) -> Poll> { 51 | Pin::new(&mut *self.conn).poll_write(cx, buf) 52 | } 53 | 54 | fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { 55 | Pin::new(&mut *self.conn).poll_flush(cx) 56 | } 57 | 58 | fn poll_close(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { 59 | Pin::new(&mut *self.conn).poll_close(cx) 60 | } 61 | } 62 | 63 | #[async_trait] 64 | impl Manager for TcpConnection { 65 | async fn create(&self) -> Result { 66 | let tcp_stream = TcpStream::connect(self.addr).await?; 67 | 68 | tcp_stream.set_nodelay(self.config.tcp_no_delay)?; 69 | 70 | Ok(tcp_stream) 71 | } 72 | 73 | async fn recycle(&self, conn: &mut TcpStream) -> RecycleResult { 74 | let mut buf = [0; 4]; 75 | let mut cx = Context::from_waker(futures::task::noop_waker_ref()); 76 | 77 | conn.set_nodelay(self.config.tcp_no_delay)?; 78 | 79 | match Pin::new(conn).poll_read(&mut cx, &mut buf) { 80 | Poll::Ready(Err(error)) => Err(error), 81 | Poll::Ready(Ok(bytes)) if bytes == 0 => Err(std::io::Error::new( 82 | std::io::ErrorKind::UnexpectedEof, 83 | "connection appeared to be closed (EoF)", 84 | )), 85 | _ => Ok(()), 86 | }?; 87 | Ok(()) 88 | } 89 | } 90 | -------------------------------------------------------------------------------- /src/h1/tls.rs: -------------------------------------------------------------------------------- 1 | use std::net::SocketAddr; 2 | use std::pin::Pin; 3 | use std::sync::Arc; 4 | 5 | use async_std::net::TcpStream; 6 | use async_trait::async_trait; 7 | use deadpool::managed::{Manager, Object, RecycleResult}; 8 | use futures::io::{AsyncRead, AsyncWrite}; 9 | use futures::task::{Context, Poll}; 10 | 11 | cfg_if::cfg_if! { 12 | if #[cfg(feature = "rustls")] { 13 | use async_tls::client::TlsStream; 14 | } else if #[cfg(feature = "native-tls")] { 15 | use async_native_tls::TlsStream; 16 | } 17 | } 18 | 19 | use crate::{Config, Error}; 20 | 21 | #[derive(Clone)] 22 | #[cfg_attr(not(feature = "rustls"), derive(std::fmt::Debug))] 23 | pub(crate) struct TlsConnection { 24 | host: String, 25 | addr: SocketAddr, 26 | config: Arc, 27 | } 28 | 29 | impl TlsConnection { 30 | pub(crate) fn new(host: String, addr: SocketAddr, config: Arc) -> Self { 31 | Self { host, addr, config } 32 | } 33 | } 34 | 35 | pub(crate) struct TlsConnWrapper { 36 | conn: Object, Error>, 37 | } 38 | impl TlsConnWrapper { 39 | pub(crate) fn new(conn: Object, Error>) -> Self { 40 | Self { conn } 41 | } 42 | } 43 | 44 | impl AsyncRead for TlsConnWrapper { 45 | fn poll_read( 46 | mut self: Pin<&mut Self>, 47 | cx: &mut Context<'_>, 48 | buf: &mut [u8], 49 | ) -> Poll> { 50 | Pin::new(&mut *self.conn).poll_read(cx, buf) 51 | } 52 | } 53 | 54 | impl AsyncWrite for TlsConnWrapper { 55 | fn poll_write( 56 | mut self: Pin<&mut Self>, 57 | cx: &mut Context<'_>, 58 | buf: &[u8], 59 | ) -> Poll> { 60 | Pin::new(&mut *self.conn).poll_write(cx, buf) 61 | } 62 | 63 | fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { 64 | Pin::new(&mut *self.conn).poll_flush(cx) 65 | } 66 | 67 | fn poll_close(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { 68 | Pin::new(&mut *self.conn).poll_close(cx) 69 | } 70 | } 71 | 72 | #[async_trait] 73 | impl Manager, Error> for TlsConnection { 74 | async fn create(&self) -> Result, Error> { 75 | let raw_stream = async_std::net::TcpStream::connect(self.addr).await?; 76 | 77 | raw_stream.set_nodelay(self.config.tcp_no_delay)?; 78 | 79 | let tls_stream = add_tls(&self.host, raw_stream, &self.config).await?; 80 | Ok(tls_stream) 81 | } 82 | 83 | async fn recycle(&self, conn: &mut TlsStream) -> RecycleResult { 84 | let mut buf = [0; 4]; 85 | let mut cx = Context::from_waker(futures::task::noop_waker_ref()); 86 | 87 | conn.get_ref() 88 | .set_nodelay(self.config.tcp_no_delay) 89 | .map_err(Error::from)?; 90 | 91 | match Pin::new(conn).poll_read(&mut cx, &mut buf) { 92 | Poll::Ready(Err(error)) => Err(error), 93 | Poll::Ready(Ok(bytes)) if bytes == 0 => Err(std::io::Error::new( 94 | std::io::ErrorKind::UnexpectedEof, 95 | "connection appeared to be closed (EoF)", 96 | )), 97 | _ => Ok(()), 98 | } 99 | .map_err(Error::from)?; 100 | 101 | Ok(()) 102 | } 103 | } 104 | 105 | cfg_if::cfg_if! { 106 | if #[cfg(feature = "rustls")] { 107 | #[allow(unused_variables)] 108 | pub(crate) async fn add_tls(host: &str, stream: TcpStream, config: &Config) -> Result, std::io::Error> { 109 | let connector = if let Some(tls_config) = config.tls_config.as_ref().cloned() { 110 | tls_config.into() 111 | } else { 112 | async_tls::TlsConnector::default() 113 | }; 114 | 115 | connector.connect(host, stream).await 116 | } 117 | } else if #[cfg(feature = "native-tls")] { 118 | #[allow(unused_variables)] 119 | pub(crate) async fn add_tls( 120 | host: &str, 121 | stream: TcpStream, 122 | config: &Config, 123 | ) -> Result, async_native_tls::Error> { 124 | let connector = config.tls_config.as_ref().cloned().unwrap_or_default(); 125 | 126 | connector.connect(host, stream).await 127 | } 128 | } 129 | } 130 | -------------------------------------------------------------------------------- /src/hyper.rs: -------------------------------------------------------------------------------- 1 | //! http-client implementation for reqwest 2 | 3 | use std::convert::{Infallible, TryFrom}; 4 | use std::fmt::Debug; 5 | use std::io; 6 | use std::str::FromStr; 7 | 8 | use futures_util::stream::TryStreamExt; 9 | use http_types::headers::{HeaderName, HeaderValue}; 10 | use http_types::StatusCode; 11 | use hyper::body::HttpBody; 12 | use hyper::client::connect::Connect; 13 | use hyper_tls::HttpsConnector; 14 | 15 | use crate::Config; 16 | 17 | use super::{async_trait, Error, HttpClient, Request, Response}; 18 | 19 | type HyperRequest = hyper::Request; 20 | 21 | // Avoid leaking Hyper generics into HttpClient by hiding it behind a dynamic trait object pointer. 22 | trait HyperClientObject: Debug + Send + Sync + 'static { 23 | fn dyn_request(&self, req: hyper::Request) -> hyper::client::ResponseFuture; 24 | } 25 | 26 | impl HyperClientObject for hyper::Client { 27 | fn dyn_request(&self, req: HyperRequest) -> hyper::client::ResponseFuture { 28 | self.request(req) 29 | } 30 | } 31 | 32 | /// Hyper-based HTTP Client. 33 | #[derive(Debug)] 34 | pub struct HyperClient { 35 | client: Box, 36 | config: Config, 37 | } 38 | 39 | impl HyperClient { 40 | /// Create a new client instance. 41 | pub fn new() -> Self { 42 | let https = HttpsConnector::new(); 43 | let client = hyper::Client::builder().build(https); 44 | 45 | Self { 46 | client: Box::new(client), 47 | config: Config::default(), 48 | } 49 | } 50 | 51 | /// Create from externally initialized and configured client. 52 | pub fn from_client(client: hyper::Client) -> Self 53 | where 54 | C: Clone + Connect + Debug + Send + Sync + 'static, 55 | { 56 | Self { 57 | client: Box::new(client), 58 | config: Config::default(), 59 | } 60 | } 61 | } 62 | 63 | impl Default for HyperClient { 64 | fn default() -> Self { 65 | Self::new() 66 | } 67 | } 68 | 69 | #[async_trait] 70 | impl HttpClient for HyperClient { 71 | async fn send(&self, req: Request) -> Result { 72 | let req = HyperHttpRequest::try_from(req).await?.into_inner(); 73 | 74 | let conn_fut = self.client.dyn_request(req); 75 | let response = if let Some(timeout) = self.config.timeout { 76 | match tokio::time::timeout(timeout, conn_fut).await { 77 | Err(_elapsed) => Err(Error::from_str(400, "Client timed out")), 78 | Ok(Ok(try_res)) => Ok(try_res), 79 | Ok(Err(e)) => Err(e.into()), 80 | }? 81 | } else { 82 | conn_fut.await? 83 | }; 84 | 85 | let res = HttpTypesResponse::try_from(response).await?.into_inner(); 86 | Ok(res) 87 | } 88 | 89 | /// Override the existing configuration with new configuration. 90 | /// 91 | /// Config options may not impact existing connections. 92 | fn set_config(&mut self, config: Config) -> http_types::Result<()> { 93 | let connector = HttpsConnector::new(); 94 | let mut builder = hyper::Client::builder(); 95 | 96 | if !config.http_keep_alive { 97 | builder.pool_max_idle_per_host(1); 98 | } 99 | 100 | self.client = Box::new(builder.build(connector)); 101 | self.config = config; 102 | 103 | Ok(()) 104 | } 105 | 106 | /// Get the current configuration. 107 | fn config(&self) -> &Config { 108 | &self.config 109 | } 110 | } 111 | 112 | impl TryFrom for HyperClient { 113 | type Error = Infallible; 114 | 115 | fn try_from(config: Config) -> Result { 116 | let connector = HttpsConnector::new(); 117 | let mut builder = hyper::Client::builder(); 118 | 119 | if !config.http_keep_alive { 120 | builder.pool_max_idle_per_host(1); 121 | } 122 | 123 | Ok(Self { 124 | client: Box::new(builder.build(connector)), 125 | config, 126 | }) 127 | } 128 | } 129 | 130 | struct HyperHttpRequest(HyperRequest); 131 | 132 | impl HyperHttpRequest { 133 | async fn try_from(mut value: Request) -> Result { 134 | // UNWRAP: This unwrap is unjustified in `http-types`, need to check if it's actually safe. 135 | let uri = hyper::Uri::try_from(&format!("{}", value.url())).unwrap(); 136 | 137 | // `HyperClient` depends on the scheme being either "http" or "https" 138 | match uri.scheme_str() { 139 | Some("http") | Some("https") => (), 140 | _ => return Err(Error::from_str(StatusCode::BadRequest, "invalid scheme")), 141 | }; 142 | 143 | let mut request = hyper::Request::builder(); 144 | 145 | // UNWRAP: Default builder is safe 146 | let req_headers = request.headers_mut().unwrap(); 147 | for (name, values) in &value { 148 | // UNWRAP: http-types and http have equivalent validation rules 149 | let name = hyper::header::HeaderName::from_str(name.as_str()).unwrap(); 150 | 151 | for value in values.iter() { 152 | // UNWRAP: http-types and http have equivalent validation rules 153 | let value = 154 | hyper::header::HeaderValue::from_bytes(value.as_str().as_bytes()).unwrap(); 155 | req_headers.append(&name, value); 156 | } 157 | } 158 | 159 | let body = value.body_bytes().await?; 160 | let body = hyper::Body::from(body); 161 | 162 | let request = request 163 | .method(value.method()) 164 | .version(value.version().map(|v| v.into()).unwrap_or_default()) 165 | .uri(uri) 166 | .body(body)?; 167 | 168 | Ok(HyperHttpRequest(request)) 169 | } 170 | 171 | fn into_inner(self) -> hyper::Request { 172 | self.0 173 | } 174 | } 175 | 176 | struct HttpTypesResponse(Response); 177 | 178 | impl HttpTypesResponse { 179 | async fn try_from(value: hyper::Response) -> Result { 180 | let (parts, body) = value.into_parts(); 181 | 182 | let size_hint = body.size_hint().upper().map(|s| s as usize); 183 | let body = body.map_err(|err| io::Error::new(io::ErrorKind::Other, err.to_string())); 184 | let body = http_types::Body::from_reader(body.into_async_read(), size_hint); 185 | 186 | let mut res = Response::new(parts.status); 187 | res.set_version(Some(parts.version.into())); 188 | 189 | for (name, value) in parts.headers { 190 | let value = value.as_bytes().to_owned(); 191 | let value = HeaderValue::from_bytes(value)?; 192 | 193 | if let Some(name) = name { 194 | let name = name.as_str(); 195 | let name = HeaderName::from_str(name)?; 196 | res.append_header(name, value); 197 | } 198 | } 199 | 200 | res.set_body(body); 201 | Ok(HttpTypesResponse(res)) 202 | } 203 | 204 | fn into_inner(self) -> Response { 205 | self.0 206 | } 207 | } 208 | 209 | #[cfg(test)] 210 | mod tests { 211 | use crate::{Error, HttpClient}; 212 | use http_types::{Method, Request, Url}; 213 | use hyper::service::{make_service_fn, service_fn}; 214 | use std::time::Duration; 215 | use tokio::sync::oneshot::channel; 216 | 217 | use super::HyperClient; 218 | 219 | async fn echo( 220 | req: hyper::Request, 221 | ) -> Result, hyper::Error> { 222 | Ok(hyper::Response::new(req.into_body())) 223 | } 224 | 225 | #[tokio::test] 226 | async fn basic_functionality() { 227 | let (send, recv) = channel::<()>(); 228 | 229 | let recv = async move { recv.await.unwrap_or(()) }; 230 | 231 | let addr = ([127, 0, 0, 1], portpicker::pick_unused_port().unwrap()).into(); 232 | let service = make_service_fn(|_| async { Ok::<_, hyper::Error>(service_fn(echo)) }); 233 | let server = hyper::Server::bind(&addr) 234 | .serve(service) 235 | .with_graceful_shutdown(recv); 236 | 237 | let client = HyperClient::new(); 238 | let url = Url::parse(&format!("http://localhost:{}", addr.port())).unwrap(); 239 | let mut req = Request::new(Method::Get, url); 240 | req.set_body("hello"); 241 | 242 | let client = async move { 243 | tokio::time::delay_for(Duration::from_millis(100)).await; 244 | let mut resp = client.send(req).await?; 245 | send.send(()).unwrap(); 246 | assert_eq!(resp.body_string().await?, "hello"); 247 | 248 | Result::<(), Error>::Ok(()) 249 | }; 250 | 251 | let (client_res, server_res) = tokio::join!(client, server); 252 | assert!(client_res.is_ok()); 253 | assert!(server_res.is_ok()); 254 | } 255 | } 256 | -------------------------------------------------------------------------------- /src/isahc.rs: -------------------------------------------------------------------------------- 1 | //! http-client implementation for isahc 2 | 3 | use std::convert::TryFrom; 4 | 5 | use async_std::io::BufReader; 6 | use isahc::config::Configurable; 7 | use isahc::{http, ResponseExt}; 8 | 9 | use crate::Config; 10 | 11 | use super::{async_trait, Body, Error, HttpClient, Request, Response}; 12 | 13 | /// Curl-based HTTP Client. 14 | #[derive(Debug)] 15 | pub struct IsahcClient { 16 | client: isahc::HttpClient, 17 | config: Config, 18 | } 19 | 20 | impl Default for IsahcClient { 21 | fn default() -> Self { 22 | Self::new() 23 | } 24 | } 25 | 26 | impl IsahcClient { 27 | /// Create a new instance. 28 | pub fn new() -> Self { 29 | Self::from_client(isahc::HttpClient::new().unwrap()) 30 | } 31 | 32 | /// Create from externally initialized and configured client. 33 | pub fn from_client(client: isahc::HttpClient) -> Self { 34 | Self { 35 | client, 36 | config: Config::default(), 37 | } 38 | } 39 | } 40 | 41 | #[async_trait] 42 | impl HttpClient for IsahcClient { 43 | async fn send(&self, mut req: Request) -> Result { 44 | let mut builder = http::Request::builder() 45 | .uri(req.url().as_str()) 46 | .method(http::Method::from_bytes(req.method().to_string().as_bytes()).unwrap()); 47 | 48 | for (name, value) in req.iter() { 49 | builder = builder.header(name.as_str(), value.as_str()); 50 | } 51 | 52 | let body = req.take_body(); 53 | let body = match body.len() { 54 | Some(len) => isahc::Body::from_reader_sized(body, len as u64), 55 | None => isahc::Body::from_reader(body), 56 | }; 57 | 58 | let request = builder.body(body).unwrap(); 59 | let res = self.client.send_async(request).await.map_err(Error::from)?; 60 | let maybe_metrics = res.metrics().cloned(); 61 | let (parts, body) = res.into_parts(); 62 | let body = Body::from_reader(BufReader::new(body), None); 63 | let mut response = http_types::Response::new(parts.status.as_u16()); 64 | for (name, value) in &parts.headers { 65 | response.append_header(name.as_str(), value.to_str().unwrap()); 66 | } 67 | 68 | if let Some(metrics) = maybe_metrics { 69 | response.ext_mut().insert(metrics); 70 | } 71 | 72 | response.set_body(body); 73 | Ok(response) 74 | } 75 | 76 | /// Override the existing configuration with new configuration. 77 | /// 78 | /// Config options may not impact existing connections. 79 | fn set_config(&mut self, config: Config) -> http_types::Result<()> { 80 | let mut builder = 81 | isahc::HttpClient::builder().max_connections_per_host(config.max_connections_per_host); 82 | 83 | if !config.http_keep_alive { 84 | builder = builder.connection_cache_size(0); 85 | } 86 | if config.tcp_no_delay { 87 | builder = builder.tcp_nodelay(); 88 | } 89 | if let Some(timeout) = config.timeout { 90 | builder = builder.timeout(timeout); 91 | } 92 | 93 | self.client = builder.build()?; 94 | self.config = config; 95 | 96 | Ok(()) 97 | } 98 | 99 | /// Get the current configuration. 100 | fn config(&self) -> &Config { 101 | &self.config 102 | } 103 | } 104 | 105 | impl TryFrom for IsahcClient { 106 | type Error = isahc::Error; 107 | 108 | fn try_from(config: Config) -> Result { 109 | let mut builder = isahc::HttpClient::builder(); 110 | 111 | if !config.http_keep_alive { 112 | builder = builder.connection_cache_size(0); 113 | } 114 | if config.tcp_no_delay { 115 | builder = builder.tcp_nodelay(); 116 | } 117 | if let Some(timeout) = config.timeout { 118 | builder = builder.timeout(timeout); 119 | } 120 | 121 | Ok(Self { 122 | client: builder.build()?, 123 | config, 124 | }) 125 | } 126 | } 127 | 128 | #[cfg(test)] 129 | mod tests { 130 | use super::*; 131 | use async_std::prelude::*; 132 | use async_std::task; 133 | use http_types::url::Url; 134 | use http_types::Result; 135 | use std::time::Duration; 136 | 137 | fn build_test_request(url: Url) -> Request { 138 | let mut req = Request::new(http_types::Method::Post, url); 139 | req.set_body("hello"); 140 | req.append_header("test", "value"); 141 | req 142 | } 143 | 144 | #[async_std::test] 145 | async fn basic_functionality() -> Result<()> { 146 | let port = portpicker::pick_unused_port().unwrap(); 147 | let mut app = tide::new(); 148 | app.at("/").all(|mut r: tide::Request<()>| async move { 149 | let mut response = tide::Response::new(http_types::StatusCode::Ok); 150 | response.set_body(r.body_bytes().await.unwrap()); 151 | Ok(response) 152 | }); 153 | 154 | let server = task::spawn(async move { 155 | app.listen(("localhost", port)).await?; 156 | Result::Ok(()) 157 | }); 158 | 159 | let client = task::spawn(async move { 160 | task::sleep(Duration::from_millis(100)).await; 161 | let request = 162 | build_test_request(Url::parse(&format!("http://localhost:{}/", port)).unwrap()); 163 | let mut response: Response = IsahcClient::new().send(request).await?; 164 | assert_eq!(response.body_string().await.unwrap(), "hello"); 165 | Ok(()) 166 | }); 167 | 168 | server.race(client).await?; 169 | 170 | Ok(()) 171 | } 172 | } 173 | -------------------------------------------------------------------------------- /src/lib.rs: -------------------------------------------------------------------------------- 1 | //! Types and traits for http clients. 2 | //! 3 | //! This crate has been extracted from `surf`'s internals, but can be used by any http client impl. 4 | //! The purpose of this crate is to provide a unified interface for multiple HTTP client backends, 5 | //! so that they can be abstracted over without doing extra work. 6 | 7 | #![forbid(future_incompatible, rust_2018_idioms)] 8 | #![deny(missing_debug_implementations, nonstandard_style)] 9 | #![warn(missing_docs, missing_doc_code_examples, unreachable_pub)] 10 | #![cfg_attr(feature = "docs", feature(doc_cfg))] 11 | // Forbid `unsafe` for the native & curl features, but allow it (for now) under the WASM backend 12 | #![cfg_attr( 13 | not(all(feature = "wasm_client", target_arch = "wasm32")), 14 | forbid(unsafe_code) 15 | )] 16 | 17 | mod config; 18 | pub use config::Config; 19 | 20 | #[cfg_attr(feature = "docs", doc(cfg(feature = "curl_client")))] 21 | #[cfg(all(feature = "curl_client", not(target_arch = "wasm32")))] 22 | pub mod isahc; 23 | 24 | #[cfg_attr(feature = "docs", doc(cfg(feature = "wasm_client")))] 25 | #[cfg(all(feature = "wasm_client", target_arch = "wasm32"))] 26 | pub mod wasm; 27 | 28 | #[cfg_attr(feature = "docs", doc(cfg(feature = "native_client")))] 29 | #[cfg(any(feature = "curl_client", feature = "wasm_client"))] 30 | pub mod native; 31 | 32 | #[cfg_attr(feature = "docs", doc(cfg(feature = "h1_client")))] 33 | #[cfg_attr(feature = "docs", doc(cfg(feature = "default")))] 34 | #[cfg(any(feature = "h1_client", feature = "h1_client_rustls"))] 35 | pub mod h1; 36 | 37 | #[cfg_attr(feature = "docs", doc(cfg(feature = "hyper_client")))] 38 | #[cfg(feature = "hyper_client")] 39 | pub mod hyper; 40 | 41 | /// An HTTP Request type with a streaming body. 42 | pub type Request = http_types::Request; 43 | 44 | /// An HTTP Response type with a streaming body. 45 | pub type Response = http_types::Response; 46 | 47 | pub use async_trait::async_trait; 48 | pub use http_types; 49 | 50 | /// An abstract HTTP client. 51 | /// 52 | /// __note that this is only exposed for use in middleware. Building new backing clients is not 53 | /// recommended yet. Once it is we'll likely publish a new `http_client` crate, and re-export this 54 | /// trait from there together with all existing HTTP client implementations.__ 55 | /// 56 | /// ## Spawning new request from middleware 57 | /// 58 | /// When threading the trait through a layer of middleware, the middleware must be able to perform 59 | /// new requests. In order to enable this efficiently an `HttpClient` instance may want to be passed 60 | /// though middleware for one of its own requests, and in order to do so should be wrapped in an 61 | /// `Rc`/`Arc` to enable reference cloning. 62 | #[async_trait] 63 | pub trait HttpClient: std::fmt::Debug + Unpin + Send + Sync + 'static { 64 | /// Perform a request. 65 | async fn send(&self, req: Request) -> Result; 66 | 67 | /// Override the existing configuration with new configuration. 68 | /// 69 | /// Config options may not impact existing connections. 70 | fn set_config(&mut self, _config: Config) -> http_types::Result<()> { 71 | unimplemented!( 72 | "{} has not implemented `HttpClient::set_config()`", 73 | type_name_of(self) 74 | ) 75 | } 76 | 77 | /// Get the current configuration. 78 | fn config(&self) -> &Config { 79 | unimplemented!( 80 | "{} has not implemented `HttpClient::config()`", 81 | type_name_of(self) 82 | ) 83 | } 84 | } 85 | 86 | fn type_name_of(_val: &T) -> &'static str { 87 | std::any::type_name::() 88 | } 89 | 90 | /// The raw body of an http request or response. 91 | pub type Body = http_types::Body; 92 | 93 | /// Error type. 94 | pub type Error = http_types::Error; 95 | 96 | #[async_trait] 97 | impl HttpClient for Box { 98 | async fn send(&self, req: Request) -> http_types::Result { 99 | self.as_ref().send(req).await 100 | } 101 | 102 | fn set_config(&mut self, config: Config) -> http_types::Result<()> { 103 | self.as_mut().set_config(config) 104 | } 105 | 106 | fn config(&self) -> &Config { 107 | self.as_ref().config() 108 | } 109 | } 110 | -------------------------------------------------------------------------------- /src/native.rs: -------------------------------------------------------------------------------- 1 | //! http-client implementation for curl + fetch 2 | 3 | #[cfg(all(feature = "curl_client", not(target_arch = "wasm32")))] 4 | pub use super::isahc::IsahcClient as NativeClient; 5 | 6 | #[cfg(all(feature = "wasm_client", target_arch = "wasm32"))] 7 | pub use super::wasm::WasmClient as NativeClient; 8 | -------------------------------------------------------------------------------- /src/wasm.rs: -------------------------------------------------------------------------------- 1 | //! http-client implementation for fetch 2 | 3 | use std::convert::{Infallible, TryFrom}; 4 | use std::pin::Pin; 5 | 6 | use futures::prelude::*; 7 | use send_wrapper::SendWrapper; 8 | 9 | use crate::Config; 10 | 11 | use super::{http_types::Headers, Body, Error, HttpClient, Request, Response}; 12 | 13 | /// WebAssembly HTTP Client. 14 | #[derive(Debug)] 15 | pub struct WasmClient { 16 | config: Config, 17 | } 18 | 19 | impl WasmClient { 20 | /// Create a new instance. 21 | pub fn new() -> Self { 22 | Self { 23 | config: Config::default(), 24 | } 25 | } 26 | } 27 | 28 | impl Default for WasmClient { 29 | fn default() -> Self { 30 | Self::new() 31 | } 32 | } 33 | 34 | impl HttpClient for WasmClient { 35 | fn send<'a, 'async_trait>( 36 | &'a self, 37 | req: Request, 38 | ) -> Pin> + Send + 'async_trait>> 39 | where 40 | 'a: 'async_trait, 41 | Self: 'async_trait, 42 | { 43 | let config = self.config.clone(); 44 | 45 | wrap_send(async move { 46 | let req: fetch::Request = fetch::Request::new(req).await?; 47 | let conn = req.send(); 48 | let mut res = if let Some(timeout) = config.timeout { 49 | async_std::future::timeout(timeout, conn).await?? 50 | } else { 51 | conn.await? 52 | }; 53 | 54 | let body = res.body_bytes(); 55 | let mut response = 56 | Response::new(http_types::StatusCode::try_from(res.status()).unwrap()); 57 | response.set_body(Body::from(body)); 58 | for (name, value) in res.headers() { 59 | let name: http_types::headers::HeaderName = name.parse().unwrap(); 60 | response.append_header(&name, value); 61 | } 62 | 63 | Ok(response) 64 | }) 65 | } 66 | 67 | /// Override the existing configuration with new configuration. 68 | /// 69 | /// Config options may not impact existing connections. 70 | fn set_config(&mut self, config: Config) -> http_types::Result<()> { 71 | self.config = config; 72 | 73 | Ok(()) 74 | } 75 | 76 | /// Get the current configuration. 77 | fn config(&self) -> &Config { 78 | &self.config 79 | } 80 | } 81 | 82 | impl TryFrom for WasmClient { 83 | type Error = Infallible; 84 | 85 | fn try_from(config: Config) -> Result { 86 | Ok(Self { config }) 87 | } 88 | } 89 | 90 | // This should not panic because WASM doesn't have threads yet. Once WASM supports threads 91 | // we can use a thread to park the blocking implementation until it's been completed. 92 | fn wrap_send(f: Fut) -> Pin + Send + Sync + 'static>> 93 | where 94 | Fut: Future + 'static, 95 | { 96 | Box::pin(SendWrapper::new(f)) 97 | } 98 | 99 | mod fetch { 100 | use js_sys::{Array, ArrayBuffer, Reflect, Uint8Array}; 101 | use wasm_bindgen::{prelude::*, JsCast}; 102 | use wasm_bindgen_futures::JsFuture; 103 | use web_sys::{RequestInit, Window, WorkerGlobalScope}; 104 | 105 | use std::iter::{IntoIterator, Iterator}; 106 | use std::pin::Pin; 107 | 108 | use http_types::StatusCode; 109 | 110 | use crate::Error; 111 | 112 | enum WindowOrWorker { 113 | Window(Window), 114 | Worker(WorkerGlobalScope), 115 | } 116 | 117 | impl WindowOrWorker { 118 | fn new() -> Self { 119 | #[wasm_bindgen] 120 | extern "C" { 121 | type Global; 122 | 123 | #[wasm_bindgen(method, getter, js_name = Window)] 124 | fn window(this: &Global) -> JsValue; 125 | 126 | #[wasm_bindgen(method, getter, js_name = WorkerGlobalScope)] 127 | fn worker(this: &Global) -> JsValue; 128 | } 129 | 130 | let global: Global = js_sys::global().unchecked_into(); 131 | 132 | if !global.window().is_undefined() { 133 | Self::Window(global.unchecked_into()) 134 | } else if !global.worker().is_undefined() { 135 | Self::Worker(global.unchecked_into()) 136 | } else { 137 | panic!("Only supported in a browser or web worker"); 138 | } 139 | } 140 | } 141 | 142 | /// Create a new fetch request. 143 | 144 | /// An HTTP Fetch Request. 145 | pub(crate) struct Request { 146 | request: web_sys::Request, 147 | /// This field stores the body of the request to ensure it stays allocated as long as the request needs it. 148 | #[allow(dead_code)] 149 | body_buf: Pin>, 150 | } 151 | 152 | impl Request { 153 | /// Create a new instance. 154 | pub(crate) async fn new(mut req: super::Request) -> Result { 155 | // create a fetch request initaliser 156 | let mut init = RequestInit::new(); 157 | 158 | // set the fetch method 159 | init.method(req.method().as_ref()); 160 | 161 | let uri = req.url().to_string(); 162 | let body = req.take_body(); 163 | 164 | // convert the body into a uint8 array 165 | // needs to be pinned and retained inside the Request because the Uint8Array passed to 166 | // js is just a portal into WASM linear memory, and if the underlying data is moved the 167 | // js ref will become silently invalid 168 | let body_buf = body.into_bytes().await.map_err(|_| { 169 | Error::from_str(StatusCode::BadRequest, "could not read body into a buffer") 170 | })?; 171 | let body_pinned = Pin::new(body_buf); 172 | if body_pinned.len() > 0 { 173 | let uint_8_array = unsafe { js_sys::Uint8Array::view(&body_pinned) }; 174 | init.body(Some(&uint_8_array)); 175 | } 176 | 177 | let request = web_sys::Request::new_with_str_and_init(&uri, &init).map_err(|e| { 178 | Error::from_str( 179 | StatusCode::BadRequest, 180 | format!("failed to create request: {:?}", e), 181 | ) 182 | })?; 183 | 184 | // add any fetch headers 185 | let headers: &mut super::Headers = req.as_mut(); 186 | for (name, value) in headers.iter() { 187 | let name = name.as_str(); 188 | let value = value.as_str(); 189 | 190 | request.headers().set(name, value).map_err(|_| { 191 | Error::from_str( 192 | StatusCode::BadRequest, 193 | format!("could not add header: {} = {}", name, value), 194 | ) 195 | })?; 196 | } 197 | 198 | Ok(Self { 199 | request, 200 | body_buf: body_pinned, 201 | }) 202 | } 203 | 204 | /// Submit a request 205 | // TODO(yoshuawuyts): turn this into a `Future` impl on `Request` instead. 206 | pub(crate) async fn send(self) -> Result { 207 | // Send the request. 208 | let scope = WindowOrWorker::new(); 209 | let promise = match scope { 210 | WindowOrWorker::Window(window) => window.fetch_with_request(&self.request), 211 | WindowOrWorker::Worker(worker) => worker.fetch_with_request(&self.request), 212 | }; 213 | let resp = JsFuture::from(promise) 214 | .await 215 | .map_err(|e| Error::from_str(StatusCode::BadRequest, format!("{:?}", e)))?; 216 | 217 | debug_assert!(resp.is_instance_of::()); 218 | let res: web_sys::Response = resp.dyn_into().unwrap(); 219 | 220 | // Get the response body. 221 | let promise = res.array_buffer().unwrap(); 222 | let resp = JsFuture::from(promise).await.unwrap(); 223 | debug_assert!(resp.is_instance_of::()); 224 | let buf: ArrayBuffer = resp.dyn_into().unwrap(); 225 | let slice = Uint8Array::new(&buf); 226 | let mut body: Vec = vec![0; slice.length() as usize]; 227 | slice.copy_to(&mut body); 228 | 229 | Ok(Response::new(res, body)) 230 | } 231 | } 232 | 233 | /// An HTTP Fetch Response. 234 | pub(crate) struct Response { 235 | res: web_sys::Response, 236 | body: Option>, 237 | } 238 | 239 | impl Response { 240 | fn new(res: web_sys::Response, body: Vec) -> Self { 241 | Self { 242 | res, 243 | body: Some(body), 244 | } 245 | } 246 | 247 | /// Access the HTTP headers. 248 | pub(crate) fn headers(&self) -> Headers { 249 | Headers { 250 | headers: self.res.headers(), 251 | } 252 | } 253 | 254 | /// Get the request body as a byte vector. 255 | /// 256 | /// Returns an empty vector if the body has already been consumed. 257 | pub(crate) fn body_bytes(&mut self) -> Vec { 258 | self.body.take().unwrap_or_else(|| vec![]) 259 | } 260 | 261 | /// Get the HTTP return status code. 262 | pub(crate) fn status(&self) -> u16 { 263 | self.res.status() 264 | } 265 | } 266 | 267 | /// HTTP Headers. 268 | pub(crate) struct Headers { 269 | headers: web_sys::Headers, 270 | } 271 | 272 | impl IntoIterator for Headers { 273 | type Item = (String, String); 274 | type IntoIter = HeadersIter; 275 | 276 | fn into_iter(self) -> Self::IntoIter { 277 | HeadersIter { 278 | iter: js_sys::try_iter(&self.headers).unwrap().unwrap(), 279 | } 280 | } 281 | } 282 | 283 | /// HTTP Headers Iterator. 284 | pub(crate) struct HeadersIter { 285 | iter: js_sys::IntoIter, 286 | } 287 | 288 | impl Iterator for HeadersIter { 289 | type Item = (String, String); 290 | 291 | fn next(&mut self) -> Option { 292 | let pair = self.iter.next()?; 293 | 294 | let array: Array = pair.unwrap().into(); 295 | let vals = array.values(); 296 | 297 | let prop = String::from("value").into(); 298 | let key = Reflect::get(&vals.next().unwrap(), &prop).unwrap(); 299 | let value = Reflect::get(&vals.next().unwrap(), &prop).unwrap(); 300 | 301 | Some(( 302 | key.as_string().to_owned().unwrap(), 303 | value.as_string().to_owned().unwrap(), 304 | )) 305 | } 306 | } 307 | } 308 | -------------------------------------------------------------------------------- /tests/test.rs: -------------------------------------------------------------------------------- 1 | use mockito::mock; 2 | 3 | use http_client::HttpClient; 4 | use http_types::{Body, Request, Response, Url}; 5 | 6 | use cfg_if::cfg_if; 7 | 8 | cfg_if! { 9 | if #[cfg(not(feature = "hyper_client"))] { 10 | use async_std::test as atest; 11 | } else { 12 | use tokio::test as atest; 13 | } 14 | } 15 | 16 | cfg_if! { 17 | if #[cfg(feature = "curl_client")] { 18 | use http_client::isahc::IsahcClient as DefaultClient; 19 | } else if #[cfg(feature = "wasm_client")] { 20 | use http_client::wasm::WasmClient as DefaultClient; 21 | } else if #[cfg(any(feature = "h1_client", feature = "h1_client_rustls"))] { 22 | use http_client::h1::H1Client as DefaultClient; 23 | } else if #[cfg(feature = "hyper_client")] { 24 | use http_client::hyper::HyperClient as DefaultClient; 25 | } 26 | } 27 | 28 | #[atest] 29 | async fn post_json() -> Result<(), http_types::Error> { 30 | #[derive(serde::Deserialize, serde::Serialize)] 31 | struct Cat { 32 | name: String, 33 | } 34 | 35 | let cat = Cat { 36 | name: "Chashu".to_string(), 37 | }; 38 | 39 | let m = mock("POST", "/") 40 | .with_status(200) 41 | .match_body(&serde_json::to_string(&cat)?[..]) 42 | .with_body(&serde_json::to_string(&cat)?[..]) 43 | .create(); 44 | let mut req = Request::new( 45 | http_types::Method::Post, 46 | Url::parse(&mockito::server_url()).unwrap(), 47 | ); 48 | req.append_header("Accept", "application/json"); 49 | req.set_body(Body::from_json(&cat)?); 50 | let res: Response = DefaultClient::new().send(req).await?; 51 | m.assert(); 52 | assert_eq!(res.status(), http_types::StatusCode::Ok); 53 | Ok(()) 54 | } 55 | 56 | #[atest] 57 | async fn get_json() -> Result<(), http_types::Error> { 58 | #[derive(serde::Deserialize)] 59 | struct Message { 60 | message: String, 61 | } 62 | let m = mock("GET", "/") 63 | .with_status(200) 64 | .with_body(r#"{"message": "hello, world!"}"#) 65 | .create(); 66 | let req = Request::new( 67 | http_types::Method::Get, 68 | Url::parse(&mockito::server_url()).unwrap(), 69 | ); 70 | let mut res: Response = DefaultClient::new().send(req).await?; 71 | let msg: Message = serde_json::from_str(&res.body_string().await?)?; 72 | m.assert(); 73 | assert_eq!(msg.message, "hello, world!"); 74 | Ok(()) 75 | } 76 | 77 | #[atest] 78 | async fn get_google() -> Result<(), http_types::Error> { 79 | let url = "https://www.google.com"; 80 | let req = Request::new(http_types::Method::Get, Url::parse(url).unwrap()); 81 | let mut res: Response = DefaultClient::new().send(req).await?; 82 | assert_eq!(res.status(), http_types::StatusCode::Ok); 83 | 84 | let msg = res.body_bytes().await?; 85 | let msg = String::from_utf8_lossy(&msg); 86 | println!("recieved: '{}'", msg); 87 | assert!(msg.contains("")); 88 | assert!(msg.contains("Google")); 89 | assert!(msg.contains("")); 90 | assert!(msg.contains("")); 91 | assert!(msg.contains("")); 92 | assert!(msg.contains("")); 93 | 94 | assert!(msg.contains("")); 96 | assert!(msg.contains("")); 97 | 98 | Ok(()) 99 | } 100 | 101 | #[atest] 102 | async fn get_github() -> Result<(), http_types::Error> { 103 | let url = "https://raw.githubusercontent.com/http-rs/surf/6627d9fc15437aea3c0a69e0b620ae7769ea6765/LICENSE-MIT"; 104 | let req = Request::new(http_types::Method::Get, Url::parse(url).unwrap()); 105 | let mut res: Response = DefaultClient::new().send(req).await?; 106 | assert_eq!(res.status(), http_types::StatusCode::Ok, "{:?}", &res); 107 | 108 | let msg = res.body_string().await?; 109 | 110 | assert_eq!( 111 | msg, 112 | "The MIT License (MIT) 113 | 114 | Copyright (c) 2019 Yoshua Wuyts 115 | 116 | Permission is hereby granted, free of charge, to any person obtaining a copy 117 | of this software and associated documentation files (the \"Software\"), to deal 118 | in the Software without restriction, including without limitation the rights 119 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 120 | copies of the Software, and to permit persons to whom the Software is 121 | furnished to do so, subject to the following conditions: 122 | 123 | The above copyright notice and this permission notice shall be included in all 124 | copies or substantial portions of the Software. 125 | 126 | THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 127 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 128 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 129 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 130 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 131 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 132 | SOFTWARE. 133 | " 134 | ); 135 | 136 | Ok(()) 137 | } 138 | 139 | #[atest] 140 | async fn keep_alive() { 141 | let _mock_guard = mockito::mock("GET", "/report") 142 | .with_status(200) 143 | .expect_at_least(2) 144 | .create(); 145 | 146 | let client = DefaultClient::new(); 147 | let url: Url = format!("{}/report", mockito::server_url()).parse().unwrap(); 148 | let req = Request::new(http_types::Method::Get, url); 149 | client.send(req.clone()).await.unwrap(); 150 | client.send(req.clone()).await.unwrap(); 151 | } 152 | 153 | #[atest] 154 | async fn fallback_to_ipv4() { 155 | let client = DefaultClient::new(); 156 | let _mock_guard = mock("GET", "/") 157 | .with_status(200) 158 | .expect_at_least(2) 159 | .create(); 160 | 161 | // Kips the initial "http://127.0.0.1:" to get only the port number 162 | let mock_port = &mockito::server_url()[17..]; 163 | 164 | let url = &format!("http://localhost:{}", mock_port); 165 | let req = Request::new(http_types::Method::Get, Url::parse(url).unwrap()); 166 | client.send(req.clone()).await.unwrap(); 167 | } 168 | --------------------------------------------------------------------------------