├── .github └── FUNDING.yml ├── .gitignore ├── .travis.yml ├── CHANGELOG.md ├── LICENSE ├── MANIFEST.in ├── README.md ├── aws_requests_auth ├── __init__.py ├── aws_auth.py ├── boto_utils.py └── tests │ ├── __init__.py │ ├── test_aws_auth.py │ └── test_boto_utils.py ├── requirements.txt ├── setup.cfg └── setup.py /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | custom: https://pragprog.com/titles/dmpython/intuitive-python/ 2 | ko_fi: davidmuller 3 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | *.py[co] 3 | *.egg-info 4 | MANIFEST 5 | dist/ 6 | build/ 7 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "2.7" 4 | - "3.5" 5 | - "3.6" 6 | - "3.7" 7 | - "3.8" 8 | 9 | # command to install dependencies 10 | install: pip install -r requirements.txt 11 | 12 | # command to run tests 13 | script: python -m unittest discover aws_requests_auth 14 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | Changelog (aws-requests-auth) 2 | ================== 3 | 4 | 0.4.3 5 | ------------------ 6 | - Also publish the package as a wheel 7 | - Contributed by @kgaughan: https://github.com/DavidMuller/aws-requests-auth/issues/54 8 | 9 | 0.4.2 10 | ------------------ 11 | - Add x-amz-content-sha256 header to request 12 | - Contributed by @samuelsh: https://github.com/DavidMuller/aws-requests-auth/pull/37 13 | 14 | 15 | 0.4.1 16 | ------------------ 17 | - Allow utf-8 encoding failures for python2 on the request body for hashing 18 | - Contributed by @bigjust: https://github.com/DavidMuller/aws-requests-auth/pull/30 19 | 20 | 21 | 0.4.0 22 | ------------------ 23 | - Add `BotoAWSRequestsAuth` convenience class which automatically gathers (and refreshes) AWS credentials using botocore 24 | - Contributed by @tobiasmcnulty: https://github.com/DavidMuller/aws-requests-auth/pull/29 25 | 26 | 27 | 0.3.3 28 | ------------------ 29 | - Add classifiers to the pypi distribution 30 | 31 | 32 | 0.3.2 33 | ------------------ 34 | - Add convenience methods for dynamically pulling AWS credentials via boto3 35 | - Thanks to @schobster: https://github.com/DavidMuller/aws-requests-auth/pull/22 36 | 37 | 38 | 0.3.1 39 | ------------------ 40 | - Patch encoding error on python 3.6 41 | - See https://github.com/DavidMuller/aws-requests-auth/pull/21 42 | 43 | 44 | 0.3.0 45 | ------------------ 46 | - Add python3 support -- thanks to @jlaine, and @anantasty 47 | - See https://github.com/DavidMuller/aws-requests-auth/pull/16 48 | 49 | 0.2.5 50 | ------------------ 51 | - Stop urlencoding query params in get_canonical_querystring(). The urlencoding in get_canonical_querystring() was causing "double encoding issues" because elasticsearch-py already apperas to urlencode query params 52 | - If you are using a client other than elasticsearch-py, you will need to be sure that your client urlecondes your query params before they are passed to the `AWSRequests` auth class 53 | - See https://github.com/DavidMuller/aws-requests-auth/pull/13 for more details 54 | 55 | 0.2.4 56 | ------------------ 57 | - Add support for [AWS STS](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html) using the `aws_token` keyword argument to `AWSRequestsAuth` 58 | - See [issue #9](https://github.com/DavidMuller/aws-requests-auth/issues/9) and [PR #11](https://github.com/DavidMuller/aws-requests-auth/pull/11 for) for additional details 59 | 60 | 0.2.3 61 | ------------------ 62 | - Fix handling of multiple query parameters 63 | - For example, the two `pretty=True` query paramaters in the following url 64 | `http://search-service-foobar.us-east-1.es.amazonaws.com?pretty=True&pretty=True` 65 | are now handled properly 66 | - see https://github.com/DavidMuller/aws-requests-auth/pull/7 67 | 68 | 69 | 0.2.2 70 | ------------------ 71 | - Update url quoting for canonical uri and canonical query string 72 | 73 | 74 | 0.2.1 75 | ------------------ 76 | - Fix bug where cannonical uri and query string was not url encoded appropriately for the signing process 77 | 78 | 79 | 0.2.0 80 | ------------------ 81 | - Fix typos of `aws_secret_access_key` : https://github.com/DavidMuller/aws-requests-auth/pull/1 82 | - This is a breaking change. The `AWSRequestsAuth` constructor now expects the kwarg `aws_secret_access_key` (instead of the incorrectly spelled `aws_secret_acces_key`). 83 | 84 | 85 | 0.1.0 86 | ------------------ 87 | Initial release 88 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) David Muller. 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without modification, 5 | are permitted provided that the following conditions are met: 6 | 7 | 1. Redistributions of source code must retain the above copyright notice, 8 | this list of conditions and the following disclaimer. 9 | 10 | 2. Redistributions in binary form must reproduce the above copyright 11 | notice, this list of conditions and the following disclaimer in the 12 | documentation and/or other materials provided with the distribution. 13 | 14 | 3. The names of its contributors may not be used to endorse or promote 15 | products derived from this software without specific prior written 16 | permission. 17 | 18 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 19 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 20 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 21 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR 22 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 23 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 24 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 25 | ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 26 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 27 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.md 2 | include LICENSE 3 | include CHANGELOG.md 4 | include MANIFEST.in 5 | recursive-include aws_requests_auth * 6 | recursive-exclude * __pycache__ 7 | recursive-exclude * *.py[co] -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Build Status](https://travis-ci.org/DavidMuller/aws-requests-auth.svg?branch=master)](https://travis-ci.org/DavidMuller/aws-requests-auth) 2 | 3 | # AWS Signature Version 4 Signing Process with python requests 4 | 5 | This package allows you to authenticate to AWS with Amazon's [signature version 4 signing process](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html) with the python [requests](https://requests.readthedocs.io/en/master/) library. 6 | 7 | Tested with both python `2.7` and `3`. 8 | 9 | (Conceivably, the authentication class is flexible enough to be used with any AWS service, but it was initially created to interface with AWS Elasticsearch instances.) 10 | 11 | # Installation 12 | 13 | ``` 14 | pip install aws-requests-auth 15 | ``` 16 | 17 | # Usage 18 | 19 | ```python 20 | import requests 21 | from aws_requests_auth.aws_auth import AWSRequestsAuth 22 | 23 | # let's talk to our AWS Elasticsearch cluster 24 | auth = AWSRequestsAuth(aws_access_key='YOURKEY', 25 | aws_secret_access_key='YOURSECRET', 26 | aws_host='search-service-foobar.us-east-1.es.amazonaws.com', 27 | aws_region='us-east-1', 28 | aws_service='es') 29 | 30 | response = requests.get('http://search-service-foobar.us-east-1.es.amazonaws.com', 31 | auth=auth) 32 | print response.content 33 | 34 | { 35 | "status" : 200, 36 | "name" : "Stevie Hunter", 37 | "cluster_name" : "elasticsearch", 38 | "version" : { 39 | "number" : "1.5.2", 40 | etc.... 41 | }, 42 | "tagline" : "You Know, for Search" 43 | } 44 | ``` 45 | 46 | ## Support 47 | 48 | If this piece of software brought value to you/your organization... 49 | 50 | Check out my new book [**Intuitive Python: Productive Development for Projects that Last**](https://pragprog.com/titles/dmpython/intuitive-python/) 51 | 52 |

53 | 54 | 55 | 56 |

57 | 58 | 59 | > Developers power their projects with Python because it emphasizes readability, ease of use, and access to a meticulously maintained set of packages and tools. The language itself continues to improve with every release: writing in Python is full of possibility. But to maintain a successful Python project, you need to know more than just the language. You need tooling and instincts to help you make the most out of what’s available to you. Use this book as your guide to help you hone your skills and sculpt a Python project that can stand the test of time. 60 | 61 | ## elasticsearch-py Client Usage Example 62 | 63 | It's possible to inject the `AWSRequestsAuth` class directly into the [elasticsearch-py](https://elasticsearch-py.readthedocs.org/en/master/) library so you can talk to your Amazon AWS cluster directly through the elasticsearch-py client. 64 | 65 | ```python 66 | from aws_requests_auth.aws_auth import AWSRequestsAuth 67 | from elasticsearch import Elasticsearch, RequestsHttpConnection 68 | 69 | es_host = 'search-service-foobar.us-east-1.es.amazonaws.com' 70 | auth = AWSRequestsAuth(aws_access_key='YOURKEY', 71 | aws_secret_access_key='YOURSECRET', 72 | aws_host=es_host, 73 | aws_region='us-east-1', 74 | aws_service='es') 75 | 76 | # use the requests connection_class and pass in our custom auth class 77 | es_client = Elasticsearch(host=es_host, 78 | port=80, 79 | connection_class=RequestsHttpConnection, 80 | http_auth=auth) 81 | print es_client.info() 82 | ``` 83 | 84 | ## Temporary Security Credentials 85 | If you are using [AWS STS](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html) to grant temporary access to your Elasticsearch resource, you can use the `aws_token` keyword argument to include your credentials in `AWSRequestsAuth`. See [issue #9](https://github.com/DavidMuller/aws-requests-auth/issues/9) and [PR #11](https://github.com/DavidMuller/aws-requests-auth/pull/11) for additional details. 86 | 87 | ## AWS Lambda Quickstart Example 88 | If you are using an AWS lamba to talk to your Elasticsearch cluster and you've assigned an IAM role to your lambda function that allows the lambda to communicate with your Elasticserach cluster, you can instantiate an instance of AWSRequestsAuth by reading your credentials from environment variables: 89 | ```python 90 | import os 91 | from aws_requests_auth.aws_auth import AWSRequestsAuth 92 | 93 | def lambda_handler(event, context): 94 | auth = AWSRequestsAuth(aws_access_key=os.environ['AWS_ACCESS_KEY_ID'], 95 | aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'], 96 | aws_token=os.environ['AWS_SESSION_TOKEN'], 97 | aws_host='search-service-foobar.us-east-1.es.amazonaws.com', 98 | aws_region='us-east-1', 99 | aws_service='es') 100 | print 'My lambda finished executing' 101 | ``` 102 | `'AWS_ACCESS_KEY_ID'`, `'AWS_SECRET_ACCESS_KEY'`, `'AWS_SESSION_TOKEN'` are [reserved environment variables in AWS lambdas](https://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html#lambda-environment-variables). 103 | 104 | ## Using Boto To Automatically Gather AWS Credentials 105 | `botocore` (the core functionality of `boto3`) is not a strict requirement of `aws-requests-auth`, but we do provide some convenience methods if you'd like to use `botocore` to automatically retrieve your AWS credentials for you. 106 | 107 | `botocore` [can dynamically pull AWS credentials from environment variables, AWS config files, IAM Role, 108 | and other locations](http://boto3.readthedocs.io/en/latest/guide/configuration.html#configuring-credentials). Dynamic credential fetching can come in handy if you need to run a program leveraging `aws-requests-auth` in several places where you may authenticate in different manners. For example, you may rely on a `.aws/credentials` file when running on your local machine, but use an IAM role when running your program in a docker container in the cloud. 109 | 110 | To take advantage of these conveniences, and help you authenticate wherever `botocore` finds AWS credentials, you can import the `boto_utils` file and initialize `BotoAWSRequestsAuth` as follows: 111 | 112 | ```python 113 | # note that this line will fail if you do not have botocore installed 114 | # botocore installation instructions available here: 115 | # https://boto3.readthedocs.io/en/latest/guide/quickstart.html#installation 116 | from aws_requests_auth.boto_utils import BotoAWSRequestsAuth 117 | 118 | auth = BotoAWSRequestsAuth(aws_host='search-service-foobar.us-east-1.es.amazonaws.com', 119 | aws_region='us-east-1', 120 | aws_service='es') 121 | ``` 122 | 123 | Credentials are only accessed when needed at runtime, and they will be refreshed using the underlying methods in `botocore` if needed. 124 | 125 | 126 | ## AWS API Gateway example with IAM authentication and Boto automatic credentials 127 | 128 | If you are using AWS API Gateway with IAM authentication 129 | ([ref](https://aws.amazon.com/premiumsupport/knowledge-center/iam-authentication-api-gateway/)), 130 | here's how to sign an HTTP request using automatic AWS credentials with boto 131 | 132 | ```python 133 | from aws_requests_auth.boto_utils import BotoAWSRequestsAuth 134 | auth = BotoAWSRequestsAuth(aws_host='api.example.com', 135 | aws_region='us-east-1', 136 | aws_service='execute-api') 137 | 138 | import requests 139 | response = requests.post('https://api.example.com/test', json={"foo": "bar"}, auth=auth) 140 | ``` 141 | -------------------------------------------------------------------------------- /aws_requests_auth/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DavidMuller/aws-requests-auth/2e1dd0f37e3815c417c3b0630215a77aab5af617/aws_requests_auth/__init__.py -------------------------------------------------------------------------------- /aws_requests_auth/aws_auth.py: -------------------------------------------------------------------------------- 1 | import hmac 2 | import hashlib 3 | import datetime 4 | 5 | try: 6 | # python 2 7 | from urllib import quote 8 | from urlparse import urlparse 9 | except ImportError: 10 | # python 3 11 | from urllib.parse import quote, urlparse 12 | 13 | import requests 14 | 15 | 16 | def sign(key, msg): 17 | """ 18 | Copied from https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html 19 | """ 20 | return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest() 21 | 22 | 23 | def getSignatureKey(key, dateStamp, regionName, serviceName): 24 | """ 25 | Copied from https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html 26 | """ 27 | kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp) 28 | kRegion = sign(kDate, regionName) 29 | kService = sign(kRegion, serviceName) 30 | kSigning = sign(kService, 'aws4_request') 31 | return kSigning 32 | 33 | 34 | class AWSRequestsAuth(requests.auth.AuthBase): 35 | """ 36 | Auth class that allows us to connect to AWS services 37 | via Amazon's signature version 4 signing process 38 | 39 | Adapted from https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html 40 | """ 41 | 42 | def __init__(self, 43 | aws_access_key, 44 | aws_secret_access_key, 45 | aws_host, 46 | aws_region, 47 | aws_service, 48 | aws_token=None): 49 | """ 50 | Example usage for talking to an AWS Elasticsearch Service: 51 | 52 | AWSRequestsAuth(aws_access_key='YOURKEY', 53 | aws_secret_access_key='YOURSECRET', 54 | aws_host='search-service-foobar.us-east-1.es.amazonaws.com', 55 | aws_region='us-east-1', 56 | aws_service='es', 57 | aws_token='...') 58 | 59 | The aws_token is optional and is used only if you are using STS 60 | temporary credentials. 61 | """ 62 | self.aws_access_key = aws_access_key 63 | self.aws_secret_access_key = aws_secret_access_key 64 | self.aws_host = aws_host 65 | self.aws_region = aws_region 66 | self.service = aws_service 67 | self.aws_token = aws_token 68 | 69 | def __call__(self, r): 70 | """ 71 | Adds the authorization headers required by Amazon's signature 72 | version 4 signing process to the request. 73 | 74 | Adapted from https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html 75 | """ 76 | aws_headers = self.get_aws_request_headers_handler(r) 77 | r.headers.update(aws_headers) 78 | return r 79 | 80 | def get_aws_request_headers_handler(self, r): 81 | """ 82 | Override get_aws_request_headers_handler() if you have a 83 | subclass that needs to call get_aws_request_headers() with 84 | an arbitrary set of AWS credentials. The default implementation 85 | calls get_aws_request_headers() with self.aws_access_key, 86 | self.aws_secret_access_key, and self.aws_token 87 | """ 88 | return self.get_aws_request_headers(r=r, 89 | aws_access_key=self.aws_access_key, 90 | aws_secret_access_key=self.aws_secret_access_key, 91 | aws_token=self.aws_token) 92 | 93 | def get_aws_request_headers(self, r, aws_access_key, aws_secret_access_key, aws_token): 94 | """ 95 | Returns a dictionary containing the necessary headers for Amazon's 96 | signature version 4 signing process. An example return value might 97 | look like 98 | 99 | { 100 | 'Authorization': 'AWS4-HMAC-SHA256 Credential=YOURKEY/20160618/us-east-1/es/aws4_request, ' 101 | 'SignedHeaders=host;x-amz-date, ' 102 | 'Signature=ca0a856286efce2a4bd96a978ca6c8966057e53184776c0685169d08abd74739', 103 | 'x-amz-date': '20160618T220405Z', 104 | } 105 | """ 106 | # Create a date for headers and the credential string 107 | t = datetime.datetime.utcnow() 108 | amzdate = t.strftime('%Y%m%dT%H%M%SZ') 109 | datestamp = t.strftime('%Y%m%d') # Date w/o time for credential_scope 110 | 111 | canonical_uri = AWSRequestsAuth.get_canonical_path(r) 112 | 113 | canonical_querystring = AWSRequestsAuth.get_canonical_querystring(r) 114 | 115 | # Create the canonical headers and signed headers. Header names 116 | # and value must be trimmed and lowercase, and sorted in ASCII order. 117 | # Note that there is a trailing \n. 118 | canonical_headers = ('host:' + self.aws_host + '\n' + 119 | 'x-amz-date:' + amzdate + '\n') 120 | if aws_token: 121 | canonical_headers += 'x-amz-security-token:' + aws_token + '\n' 122 | 123 | # Create the list of signed headers. This lists the headers 124 | # in the canonical_headers list, delimited with ";" and in alpha order. 125 | # Note: The request can include any headers; canonical_headers and 126 | # signed_headers lists those that you want to be included in the 127 | # hash of the request. "Host" and "x-amz-date" are always required. 128 | signed_headers = 'host;x-amz-date' 129 | if aws_token: 130 | signed_headers += ';x-amz-security-token' 131 | 132 | # Create payload hash (hash of the request body content). For GET 133 | # requests, the payload is an empty string (''). 134 | body = r.body if r.body else bytes() 135 | try: 136 | body = body.encode('utf-8') 137 | except (AttributeError, UnicodeDecodeError): 138 | # On py2, if unicode characters in present in `body`, 139 | # encode() throws UnicodeDecodeError, but we can safely 140 | # pass unencoded `body` to execute hexdigest(). 141 | # 142 | # For py3, encode() will execute successfully regardless 143 | # of the presence of unicode data 144 | body = body 145 | 146 | payload_hash = hashlib.sha256(body).hexdigest() 147 | 148 | # Combine elements to create create canonical request 149 | canonical_request = (r.method + '\n' + canonical_uri + '\n' + 150 | canonical_querystring + '\n' + canonical_headers + 151 | '\n' + signed_headers + '\n' + payload_hash) 152 | 153 | # Match the algorithm to the hashing algorithm you use, either SHA-1 or 154 | # SHA-256 (recommended) 155 | algorithm = 'AWS4-HMAC-SHA256' 156 | credential_scope = (datestamp + '/' + self.aws_region + '/' + 157 | self.service + '/' + 'aws4_request') 158 | string_to_sign = (algorithm + '\n' + amzdate + '\n' + credential_scope + 159 | '\n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()) 160 | 161 | # Create the signing key using the function defined above. 162 | signing_key = getSignatureKey(aws_secret_access_key, 163 | datestamp, 164 | self.aws_region, 165 | self.service) 166 | 167 | # Sign the string_to_sign using the signing_key 168 | string_to_sign_utf8 = string_to_sign.encode('utf-8') 169 | signature = hmac.new(signing_key, 170 | string_to_sign_utf8, 171 | hashlib.sha256).hexdigest() 172 | 173 | # The signing information can be either in a query string value or in 174 | # a header named Authorization. This code shows how to use a header. 175 | # Create authorization header and add to request headers 176 | authorization_header = (algorithm + ' ' + 'Credential=' + aws_access_key + 177 | '/' + credential_scope + ', ' + 'SignedHeaders=' + 178 | signed_headers + ', ' + 'Signature=' + signature) 179 | 180 | headers = { 181 | 'Authorization': authorization_header, 182 | 'x-amz-date': amzdate, 183 | 'x-amz-content-sha256': payload_hash 184 | } 185 | if aws_token: 186 | headers['X-Amz-Security-Token'] = aws_token 187 | return headers 188 | 189 | @classmethod 190 | def get_canonical_path(cls, r): 191 | """ 192 | Create canonical URI--the part of the URI from domain to query 193 | string (use '/' if no path) 194 | """ 195 | parsedurl = urlparse(r.url) 196 | 197 | # safe chars adapted from boto's use of urllib.parse.quote 198 | # https://github.com/boto/boto/blob/d9e5cfe900e1a58717e393c76a6e3580305f217a/boto/auth.py#L393 199 | return quote(parsedurl.path if parsedurl.path else '/', safe='/-_.~') 200 | 201 | @classmethod 202 | def get_canonical_querystring(cls, r): 203 | """ 204 | Create the canonical query string. According to AWS, by the 205 | end of this function our query string values must 206 | be URL-encoded (space=%20) and the parameters must be sorted 207 | by name. 208 | 209 | This method assumes that the query params in `r` are *already* 210 | url encoded. If they are not url encoded by the time they make 211 | it to this function, AWS may complain that the signature for your 212 | request is incorrect. 213 | 214 | It appears elasticsearc-py url encodes query paramaters on its own: 215 | https://github.com/elastic/elasticsearch-py/blob/5dfd6985e5d32ea353d2b37d01c2521b2089ac2b/elasticsearch/connection/http_requests.py#L64 216 | 217 | If you are using a different client than elasticsearch-py, it 218 | will be your responsibility to urleconde your query params before 219 | this method is called. 220 | """ 221 | canonical_querystring = '' 222 | 223 | parsedurl = urlparse(r.url) 224 | querystring_sorted = '&'.join(sorted(parsedurl.query.split('&'))) 225 | 226 | for query_param in querystring_sorted.split('&'): 227 | key_val_split = query_param.split('=', 1) 228 | 229 | key = key_val_split[0] 230 | if len(key_val_split) > 1: 231 | val = key_val_split[1] 232 | else: 233 | val = '' 234 | 235 | if key: 236 | if canonical_querystring: 237 | canonical_querystring += "&" 238 | canonical_querystring += u'='.join([key, val]) 239 | 240 | return canonical_querystring 241 | -------------------------------------------------------------------------------- /aws_requests_auth/boto_utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Functions in this file are included as a convenience for working with AWSRequestsAuth. 3 | External libraries, like boto, that this file imports are not a strict requirement for the 4 | aws-requests-auth package. 5 | """ 6 | 7 | from botocore.session import Session 8 | 9 | from .aws_auth import AWSRequestsAuth 10 | 11 | 12 | def get_credentials(credentials_obj=None): 13 | """ 14 | Interacts with boto to retrieve AWS credentials, and returns a dictionary of 15 | kwargs to be used in AWSRequestsAuth. boto automatically pulls AWS credentials from 16 | a variety of sources including but not limited to credentials files and IAM role. 17 | AWS credentials are pulled in the order listed here: 18 | http://boto3.readthedocs.io/en/latest/guide/configuration.html#configuring-credentials 19 | """ 20 | if credentials_obj is None: 21 | credentials_obj = Session().get_credentials() 22 | # use get_frozen_credentials to avoid the race condition where one or more 23 | # properties may be refreshed and the other(s) not refreshed 24 | frozen_credentials = credentials_obj.get_frozen_credentials() 25 | return { 26 | 'aws_access_key': frozen_credentials.access_key, 27 | 'aws_secret_access_key': frozen_credentials.secret_key, 28 | 'aws_token': frozen_credentials.token, 29 | } 30 | 31 | 32 | class BotoAWSRequestsAuth(AWSRequestsAuth): 33 | 34 | def __init__(self, aws_host, aws_region, aws_service): 35 | """ 36 | Example usage for talking to an AWS Elasticsearch Service: 37 | 38 | BotoAWSRequestsAuth(aws_host='search-service-foobar.us-east-1.es.amazonaws.com', 39 | aws_region='us-east-1', 40 | aws_service='es') 41 | 42 | The aws_access_key, aws_secret_access_key, and aws_token are discovered 43 | automatically from the environment, in the order described here: 44 | http://boto3.readthedocs.io/en/latest/guide/configuration.html#configuring-credentials 45 | """ 46 | super(BotoAWSRequestsAuth, self).__init__(None, None, aws_host, aws_region, aws_service) 47 | self._refreshable_credentials = Session().get_credentials() 48 | 49 | def get_aws_request_headers_handler(self, r): 50 | # provide credentials explicitly during each __call__, to take advantage 51 | # of botocore's underlying logic to refresh expired credentials 52 | credentials = get_credentials(self._refreshable_credentials) 53 | return self.get_aws_request_headers(r, **credentials) 54 | -------------------------------------------------------------------------------- /aws_requests_auth/tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DavidMuller/aws-requests-auth/2e1dd0f37e3815c417c3b0630215a77aab5af617/aws_requests_auth/tests/__init__.py -------------------------------------------------------------------------------- /aws_requests_auth/tests/test_aws_auth.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | import hashlib 3 | import mock 4 | import sys 5 | import unittest 6 | 7 | from aws_requests_auth.aws_auth import AWSRequestsAuth 8 | 9 | 10 | class TestAWSRequestsAuth(unittest.TestCase): 11 | """ 12 | Tests for AWSRequestsAuth 13 | """ 14 | 15 | def test_no_query_params(self): 16 | """ 17 | Assert we generate the 'correct' cannonical query string 18 | and canonical path for a request with no query params 19 | 20 | Correct is relative here b/c 'correct' simply means what 21 | the AWS Elasticsearch service expects 22 | """ 23 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/' 24 | mock_request = mock.Mock() 25 | mock_request.url = url 26 | self.assertEqual('/', AWSRequestsAuth.get_canonical_path(mock_request)) 27 | self.assertEqual('', AWSRequestsAuth.get_canonical_querystring(mock_request)) 28 | 29 | def test_characters_escaped_in_path(self): 30 | """ 31 | Assert we generate the 'correct' cannonical query string 32 | and path a request with characters that need to be escaped 33 | """ 34 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/+foo.*/_stats' 35 | mock_request = mock.Mock() 36 | mock_request.url = url 37 | self.assertEqual('/%2Bfoo.%2A/_stats', AWSRequestsAuth.get_canonical_path(mock_request)) 38 | self.assertEqual('', AWSRequestsAuth.get_canonical_querystring(mock_request)) 39 | 40 | def test_path_with_querystring(self): 41 | """ 42 | Assert we generate the 'correct' cannonical query string 43 | and path for request that includes a query stirng 44 | """ 45 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/my_index/?pretty=True' 46 | mock_request = mock.Mock() 47 | mock_request.url = url 48 | self.assertEqual('/my_index/', AWSRequestsAuth.get_canonical_path(mock_request)) 49 | self.assertEqual('pretty=True', AWSRequestsAuth.get_canonical_querystring(mock_request)) 50 | 51 | def test_multiple_get_params(self): 52 | """ 53 | Assert we generate the 'correct' cannonical query string 54 | for request that includes more than one query parameter 55 | """ 56 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/index/type/_search?scroll=5m&search_type=scan' 57 | mock_request = mock.Mock() 58 | mock_request.url = url 59 | self.assertEqual('scroll=5m&search_type=scan', AWSRequestsAuth.get_canonical_querystring(mock_request)) 60 | 61 | def test_post_request_with_get_param(self): 62 | """ 63 | Assert we generate the 'correct' cannonical query string 64 | for a post request that includes GET-parameters 65 | """ 66 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/index/type/1/_update?version=1' 67 | mock_request = mock.Mock() 68 | mock_request.url = url 69 | mock_request.method = "POST" 70 | self.assertEqual('version=1', AWSRequestsAuth.get_canonical_querystring(mock_request)) 71 | 72 | def test_auth_for_get(self): 73 | auth = AWSRequestsAuth(aws_access_key='YOURKEY', 74 | aws_secret_access_key='YOURSECRET', 75 | aws_host='search-foo.us-east-1.es.amazonaws.com', 76 | aws_region='us-east-1', 77 | aws_service='es') 78 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/' 79 | mock_request = mock.Mock() 80 | mock_request.url = url 81 | mock_request.method = "GET" 82 | mock_request.body = None 83 | mock_request.headers = {} 84 | 85 | frozen_datetime = datetime.datetime(2016, 6, 18, 22, 4, 5) 86 | with mock.patch('datetime.datetime') as mock_datetime: 87 | mock_datetime.utcnow.return_value = frozen_datetime 88 | auth(mock_request) 89 | self.assertEqual({ 90 | 'Authorization': 'AWS4-HMAC-SHA256 Credential=YOURKEY/20160618/us-east-1/es/aws4_request, ' 91 | 'SignedHeaders=host;x-amz-date, ' 92 | 'Signature=ca0a856286efce2a4bd96a978ca6c8966057e53184776c0685169d08abd74739', 93 | 'x-amz-date': '20160618T220405Z', 94 | 'x-amz-content-sha256': hashlib.sha256(b'').hexdigest(), 95 | 96 | }, mock_request.headers) 97 | 98 | def test_auth_for_post(self): 99 | auth = AWSRequestsAuth(aws_access_key='YOURKEY', 100 | aws_secret_access_key='YOURSECRET', 101 | aws_host='search-foo.us-east-1.es.amazonaws.com', 102 | aws_region='us-east-1', 103 | aws_service='es') 104 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/' 105 | mock_request = mock.Mock() 106 | mock_request.url = url 107 | mock_request.method = "POST" 108 | mock_request.body = b'foo=bar' 109 | mock_request.headers = { 110 | 'Content-Type': 'application/x-www-form-urlencoded', 111 | } 112 | 113 | frozen_datetime = datetime.datetime(2016, 6, 18, 22, 4, 5) 114 | with mock.patch('datetime.datetime') as mock_datetime: 115 | mock_datetime.utcnow.return_value = frozen_datetime 116 | auth(mock_request) 117 | self.assertEqual({ 118 | 'Authorization': 'AWS4-HMAC-SHA256 Credential=YOURKEY/20160618/us-east-1/es/aws4_request, ' 119 | 'SignedHeaders=host;x-amz-date, ' 120 | 'Signature=a6fd88e5f5c43e005482894001d9b05b43f6710e96be6098bcfcfccdeb8ed812', 121 | 'Content-Type': 'application/x-www-form-urlencoded', 122 | 'x-amz-date': '20160618T220405Z', 123 | 'x-amz-content-sha256': hashlib.sha256(mock_request.body).hexdigest(), 124 | 125 | }, mock_request.headers) 126 | 127 | def test_auth_for_post_with_str_body(self): 128 | auth = AWSRequestsAuth(aws_access_key='YOURKEY', 129 | aws_secret_access_key='YOURSECRET', 130 | aws_host='search-foo.us-east-1.es.amazonaws.com', 131 | aws_region='us-east-1', 132 | aws_service='es') 133 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/' 134 | mock_request = mock.Mock() 135 | mock_request.url = url 136 | mock_request.method = "POST" 137 | mock_request.body = 'foo=bar' 138 | mock_request.headers = { 139 | 'Content-Type': 'application/x-www-form-urlencoded', 140 | } 141 | 142 | frozen_datetime = datetime.datetime(2016, 6, 18, 22, 4, 5) 143 | with mock.patch('datetime.datetime') as mock_datetime: 144 | mock_datetime.utcnow.return_value = frozen_datetime 145 | auth(mock_request) 146 | self.assertEqual({ 147 | 'Authorization': 'AWS4-HMAC-SHA256 Credential=YOURKEY/20160618/us-east-1/es/aws4_request, ' 148 | 'SignedHeaders=host;x-amz-date, ' 149 | 'Signature=a6fd88e5f5c43e005482894001d9b05b43f6710e96be6098bcfcfccdeb8ed812', 150 | 'Content-Type': 'application/x-www-form-urlencoded', 151 | 'x-amz-date': '20160618T220405Z', 152 | 'x-amz-content-sha256': hashlib.sha256(mock_request.body.encode()).hexdigest(), 153 | 154 | }, mock_request.headers) 155 | 156 | @unittest.skipIf( 157 | int(sys.version[0]) > 2, 158 | 'python3 produces a different hash that we\'re comparing.', 159 | ) 160 | def test_auth_for_post_with_unicode_body_python2(self): 161 | auth = AWSRequestsAuth(aws_access_key='YOURKEY', 162 | aws_secret_access_key='YOURSECRET', 163 | aws_host='search-foo.us-east-1.es.amazonaws.com', 164 | aws_region='us-east-1', 165 | aws_service='es') 166 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/' 167 | mock_request = mock.Mock() 168 | mock_request.url = url 169 | mock_request.method = "POST" 170 | mock_request.body = 'foo=bar\xc3' 171 | mock_request.headers = { 172 | 'Content-Type': 'application/x-www-form-urlencoded', 173 | } 174 | 175 | frozen_datetime = datetime.datetime(2016, 6, 18, 22, 4, 5) 176 | with mock.patch('datetime.datetime') as mock_datetime: 177 | mock_datetime.utcnow.return_value = frozen_datetime 178 | auth(mock_request) 179 | 180 | self.assertEqual({ 181 | 'Authorization': 'AWS4-HMAC-SHA256 Credential=YOURKEY/20160618/us-east-1/es/aws4_request, ' 182 | 'SignedHeaders=host;x-amz-date, ' 183 | 'Signature=88046be72423b267de5e7e604aaffb2c5668c3fd9022ef4aac8287b82ab71124', 184 | 'Content-Type': 'application/x-www-form-urlencoded', 185 | 'x-amz-date': '20160618T220405Z', 186 | 'x-amz-content-sha256': hashlib.sha256(mock_request.body).hexdigest(), 187 | 188 | }, mock_request.headers) 189 | 190 | @unittest.skipIf( 191 | int(sys.version[0]) < 3, 192 | 'python3 produces a different hash that we\'re comparing.' 193 | ) 194 | def test_auth_for_post_with_unicode_body_python3(self): 195 | auth = AWSRequestsAuth(aws_access_key='YOURKEY', 196 | aws_secret_access_key='YOURSECRET', 197 | aws_host='search-foo.us-east-1.es.amazonaws.com', 198 | aws_region='us-east-1', 199 | aws_service='es') 200 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/' 201 | mock_request = mock.Mock() 202 | mock_request.url = url 203 | mock_request.method = "POST" 204 | mock_request.body = 'foo=bar\xc3' 205 | mock_request.headers = { 206 | 'Content-Type': 'application/x-www-form-urlencoded', 207 | } 208 | 209 | frozen_datetime = datetime.datetime(2016, 6, 18, 22, 4, 5) 210 | with mock.patch('datetime.datetime') as mock_datetime: 211 | mock_datetime.utcnow.return_value = frozen_datetime 212 | auth(mock_request) 213 | 214 | self.assertEqual({ 215 | 'Authorization': 'AWS4-HMAC-SHA256 Credential=YOURKEY/20160618/us-east-1/es/aws4_request, ' 216 | 'SignedHeaders=host;x-amz-date, ' 217 | 'Signature=0836dae4bce95c1bcdbd3751c84c0c7e589ba7c81331bab92d0e1acb94adcdd9', 218 | 'Content-Type': 'application/x-www-form-urlencoded', 219 | 'x-amz-date': '20160618T220405Z', 220 | 'x-amz-content-sha256': hashlib.sha256(mock_request.body.encode()).hexdigest(), 221 | 222 | }, mock_request.headers) 223 | -------------------------------------------------------------------------------- /aws_requests_auth/tests/test_boto_utils.py: -------------------------------------------------------------------------------- 1 | import datetime 2 | import hashlib 3 | import os 4 | import unittest 5 | 6 | import mock 7 | 8 | from aws_requests_auth.boto_utils import BotoAWSRequestsAuth, get_credentials 9 | 10 | 11 | class TestBotoUtils(unittest.TestCase): 12 | """ 13 | Tests for boto_utils module. 14 | """ 15 | 16 | def setUp(self): 17 | self.saved_env = {} 18 | for var in ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY', 'AWS_SESSION_TOKEN']: 19 | self.saved_env[var] = os.environ.get(var) 20 | os.environ[var] = 'test-%s' % var 21 | 22 | def tearDown(self): 23 | for k, v in self.saved_env.items(): 24 | if v is None: 25 | os.environ.pop(k) 26 | else: 27 | os.environ[k] = v 28 | 29 | def test_get_credentials(self): 30 | creds = get_credentials() # botocore should discover these from os.environ 31 | self.assertEqual(creds['aws_access_key'], 'test-AWS_ACCESS_KEY_ID') 32 | self.assertEqual(creds['aws_secret_access_key'], 'test-AWS_SECRET_ACCESS_KEY') 33 | self.assertEqual(creds['aws_token'], 'test-AWS_SESSION_TOKEN') 34 | 35 | def test_boto_class(self): 36 | boto_auth_inst = BotoAWSRequestsAuth( 37 | aws_host='search-foo.us-east-1.es.amazonaws.com', 38 | aws_region='us-east-1', 39 | aws_service='es', 40 | ) 41 | url = 'http://search-foo.us-east-1.es.amazonaws.com:80/' 42 | mock_request = mock.Mock() 43 | mock_request.url = url 44 | mock_request.method = "GET" 45 | mock_request.body = None 46 | mock_request.headers = {} 47 | 48 | frozen_datetime = datetime.datetime(2016, 6, 18, 22, 4, 5) 49 | with mock.patch('datetime.datetime') as mock_datetime: 50 | mock_datetime.utcnow.return_value = frozen_datetime 51 | boto_auth_inst(mock_request) 52 | self.assertEqual({ 53 | 'Authorization': 'AWS4-HMAC-SHA256 Credential=test-AWS_ACCESS_KEY_ID/20160618/us-east-1/es/aws4_request, ' 54 | 'SignedHeaders=host;x-amz-date;x-amz-security-token, ' 55 | 'Signature=9d35f096395c7aa5061e69aca897417dd41bb8fb01a465bb78343624f8f123bf', 56 | 'x-amz-date': '20160618T220405Z', 57 | 'X-Amz-Security-Token': 'test-AWS_SESSION_TOKEN', 58 | 'x-amz-content-sha256': hashlib.sha256(b'').hexdigest(), 59 | 60 | }, mock_request.headers) 61 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | mock==1.3.0 2 | requests==2.20.0 3 | botocore==1.7.8 4 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [bdist_wheel] 2 | universal = 1 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | 4 | setup( 5 | name='aws-requests-auth', 6 | version='0.4.3', 7 | author='David Muller', 8 | author_email='davehmuller@gmail.com', 9 | packages=['aws_requests_auth'], 10 | url='https://github.com/davidmuller/aws-requests-auth', 11 | description='AWS signature version 4 signing process for the python requests module', 12 | long_description='See https://github.com/davidmuller/aws-requests-auth for installation and usage instructions.', 13 | install_requires=['requests>=0.14.0'], 14 | classifiers=[ 15 | 'License :: OSI Approved :: BSD License', 16 | 'Programming Language :: Python', 17 | 'Programming Language :: Python :: 3', 18 | ] 19 | ) 20 | --------------------------------------------------------------------------------