├── .github └── workflows │ └── python.yaml ├── .gitignore ├── .travis.yml ├── README.md ├── dpla ├── __init__.py ├── api.py └── settings.py ├── license.txt ├── pyproject.toml ├── requirements.txt ├── setup.cfg ├── setup.py └── tests.py /.github/workflows/python.yaml: -------------------------------------------------------------------------------- 1 | name: Python package 2 | 3 | on: [push] 4 | 5 | jobs: 6 | build: 7 | 8 | runs-on: ubuntu-latest 9 | strategy: 10 | matrix: 11 | python-version: ["3.7", "3.8", "3.9", "3.10"] 12 | 13 | steps: 14 | - uses: actions/checkout@v3 15 | - name: Set up Python ${{ matrix.python-version }} 16 | uses: actions/setup-python@v4 17 | with: 18 | python-version: ${{ matrix.python-version }} 19 | - name: Install dependencies 20 | run: | 21 | python -m pip install --upgrade pip 22 | pip install black 23 | if [ -f requirements.txt ]; then pip install -r requirements.txt; fi 24 | - name: Lint with black 25 | run: | 26 | black . 27 | - name: Run tests 28 | run: | 29 | python tests.py 30 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.egg 3 | *.egg-info 4 | dist/ 5 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - "2.7" 4 | - "3.3" 5 | - 3.4 6 | - 3.5 7 | 8 | script: python setup.py test --verbose 9 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![Build status](https://github.com/bibliotechy/dpyla/actions/workflows/python.yaml/badge.svg) 2 | 3 | # DPyLA - A Python client for the DPLA API 4 | #### under active development! 5 | 6 | [The DPLA](http://dp.la) (Digital Public Library of America) is an aggregated 7 | digital library, archive and museum collections. What really makes it stand 8 | out is its awesome API. This python library is a wrapper around that API, 9 | making it easier to interact with. 10 | 11 | Tested and working with Python 3.7, 3.8, 3.9, 3.10. 12 | 13 | ### Dependencies 14 | Depends on the awesome [Requests package](http://www.python-requests.org/en/latest/) 15 | 16 | ####Getting started 17 | 18 | First, install the module: 19 | 20 | ```bash 21 | pip install dpla 22 | ``` 23 | 24 | Then fire up your fave python interpreter and: 25 | 26 | ```python 27 | >>> from dpla.api import DPLA 28 | ``` 29 | 30 | Then create the dpla object with your DPLA API key. 31 | 32 | ```python 33 | >>> dpla = DPLA('your-key-here') 34 | ``` 35 | 36 | If you don't have a DPLA API key yet, you can 37 | [follow their instructions](http://dp.la/info/developers/codex/policies/#get-a-key) 38 | or simply run the following: 39 | 40 | ```python 41 | >>> DPLA.new_key("your.email.address@here.com") 42 | ``` 43 | 44 | And then check your email. 45 | 46 | 47 | Now, let's create your first search: 48 | 49 | ```python 50 | >>> result = dpla.search('chicken') 51 | ``` 52 | 53 | Records returned are in `result.items`: 54 | 55 | ```python 56 | >>> result.items[0] #gets you a multidimensional dictionary of the first result. Much omitted below for brevity. 57 | {u'@context': {u'@vocab': u'http://purl.org/dc/terms/', 58 | # ... 59 | u'@id': u'http://dp.la/api/items/bc944ed8339050bbbcf25f3838895a03', 60 | u'_id': u'kentucky--http://kdl.kyvl.org/catalog/xt7sf7664q86_1755_1', 61 | # ... 62 | u'hasView': {u'@id': u'http://nyx.uky.edu/dips/xt7sf7664q86/data/1/016_0006_p/016_0006/016_0006.jpg'}, 63 | # ... 64 | u'sourceResource': {u'collection': [], 65 | u'creator': u'University of Kentucky', 66 | u'language': [{u'iso639_3': u'eng', u'name': u'English'}], 67 | u'stateLocatedIn': [{u'name': u'University of Kentucky'}], 68 | u'subject': [{u'name': u'Agriculture-United States'}, 69 | {u'name': u'Animal Culture-United States'}, 70 | {u'name': u'Photographs of animals'}, 71 | {u'name': u'Photographs of livestock'}], 72 | u'title': u'Chicken'}} 73 | ``` 74 | 75 | You can also find out how many records were found matching that criteria: 76 | 77 | ```python 78 | >>> result.count # 79 | 925 80 | ``` 81 | 82 | But you don't have all 925 records. Unless you tell it otherwise, DPLA API sets a limit of ten records returned. 83 | 84 | ```python 85 | >>> results.limit 86 | 10 # 87 | ``` 88 | 89 | But if you want more, it's easy. Just do this: 90 | 91 | ```python 92 | >>> result = dpla.search(q="chicken", page_size=100) 93 | >>> result.limit 94 | 100 # More records, YAY! 95 | ``` 96 | 97 | You can also use the `all_records()` helper method to get back a generator that allows you to iterate through all of the records in your result list. 98 | 99 | ```python 100 | >>> result = dpla.search(q="chicken", page_size=100) 101 | >>> result.count 102 | 925 103 | >>> for item in result.all_records(): 104 | print(item["sourceResource"]["title"]) 105 | "Chicken" 106 | "Chicken and cow" 107 | "Chicken and pig" 108 | # ...(922 more titles) 109 | "Last of the Chicken records" 110 | ``` 111 | 112 | ### Fetch item(s) by ID 113 | If you have the id of of a record you want to retrieve, or a series of IDs, you can use the `fetch_by_id` method. Just pass an array of IDs and it will return all fields for those records. 114 | 115 | ```python 116 | >>> result = dpla.fetch_by_id(["93583acc6425f8172b7b506f44a32121"]) 117 | >>> result.items[0]["@id"] 118 | 'http://dp.la/api/items/93583acc6425f8172b7b506f44a32121' 119 | ``` 120 | 121 | Or multiple IDs: 122 | 123 | ```python 124 | >>> ids = ["93583acc6425f8172b7b506f44a32121","fe47a8b71de4c136fe115a19ead13e4d" ] 125 | >>> result = dpla.fetch_by_id(ids) 126 | >>> result.count 127 | 2 128 | ``` 129 | 130 | ### More Options 131 | 132 | The DPLA gives you a lot of options for tailoring your search to get back exactly what you want. DPyLA helps make creating those fine grained searches easier (easier than creating your own 250-charcter url anyway!) 133 | 134 | #### Query 135 | 136 | A standard keyword query that searches across all fields. 137 | Just enter a string with your search terms. If combining with other search parameters, make sure it is the first param passed. 138 | 139 | ```python 140 | >>> result = dpla.search("chicken") 141 | >>> result = dpla.search("chicken man") 142 | >>> result = dpla.search("chicken", fields=["sourceResource.title"]) 143 | ``` 144 | 145 | #### Search within specific fields 146 | 147 | You can search within specific fields to narrow your search. 148 | Pass a dictionary of key / value pairs to the `searchFields` parameter, where field names are the keys and search values are the value. 149 | 150 | ```python 151 | >>> fields = {"sourceResource.title" : "crime", "sourceResource.spatial.state" : "Illinois"} 152 | >>> result = dpla.search(searchFields=fields) 153 | ``` 154 | 155 | #### Return Fields 156 | You can also choose what fields should be included with returned records, so you only get back what you need. 157 | Pass a list or tuple of filed names to the `fields` parameter 158 | 159 | ```python 160 | result = dpla.search("chicken", fields=["sourceResource.title"]) 161 | >>> result.items[0] 162 | {u'sourceResource.title': u'Chicken'} 163 | ``` 164 | 165 | #### Facets 166 | Get back a list of the most common terms within a field for this set of results. See the [DPLA facet docs](http://dp.la/info/developers/codex/requests/#faceting) for more info. 167 | 168 | ```python 169 | >>> result = dpla.search("chicken", facets=["sourceResource.subject"]) 170 | >>> result.facets[0] 171 | {u'sourceResource.subject.name': {u'_type': u'terms', 172 | u'missing': 151, 173 | u'other': 3043, 174 | u'terms': [{u'count': 88, u'term': u'Poultry'}, 175 | {u'count': 77, u'term': u'Social Life and Customs'}, 176 | {u'count': 64, u'term': u'Agriculture'}, 177 | {u'count': 60, u'term': u'People'}, 178 | {u'count': 53, u'term': u'Chickens'}, 179 | {u'count': 51, u'term': u'Restaurants'}, 180 | {u'count': 51, u'term': u'Ethnology'}, 181 | {u'count': 41, u'term': u'Domestic Animals'}, 182 | {u'count': 39, u'term': u'Customs'}, 183 | {u'count': 32, u'term': u'Festivals'}, 184 | # .... 185 | 186 | ``` 187 | 188 | #### Spatial Facet 189 | You can also facet by distance from a set of geo-coordinates. It requires extra work in the search url, so it is a seperate parameter. 190 | Pass a length 2 list of `[lat, lng]` to the parameter spatial_facet: 191 | 192 | ```python 193 | >>> result = dpla.search("chicken", spatial_facet=[37,-48]) 194 | >>> result.facets[0] 195 | {u'sourceResource.spatial.coordinates': {u'_type': u'geo_distance', 196 | u'ranges': [{u'from': 1200.0, 197 | u'max': 1296.205781266114, 198 | u'mean': 1277.6015482976388, 199 | u'min': 1265.9189942665018, 200 | u'to': 1300.0, 201 | u'total': 6388.007741488194, 202 | u'total_count': 5}, 203 | # ... 204 | ]} 205 | ``` 206 | 207 | ### Facet Size 208 | Normally, asking for facets will return A LOT OF FACETS! If you only want a few, this is for you. 209 | Pass an int to the parameter `facet_size`. 210 | 211 | ```python 212 | >>> result = dpla.search("chicken", facets=["sourceResource.subject"], facet_size=2) 213 | >>> result.facets[1] 214 | {u'sourceResource.subject.name': {u'_type': u'terms', 215 | u'missing': 151, 216 | u'other': 4097, 217 | u'terms': [{u'count': 69, u'term': u'Poultry'}, 218 | {u'count': 47, u'term': u'Social Life and Customs'}], 219 | u'total': 4213}} 220 | ``` 221 | 222 | #### Sort 223 | How do you want the records sorted? Pass a string field name to the `sort` parameter. 224 | 225 | ```python 226 | >>> result = dpla.search("chicken", sort="sourceResource.title") 227 | ``` 228 | 229 | #### Spatial Sort 230 | You can also sort by distance from a geo-coordinate. Pass the length 2 tuple of `[lat, lng]` to the `spatial_sort` parameter. 231 | 232 | ``` 233 | >>> result = dpla.search("chicken", spatial_sort=[37, -48]) 234 | ``` 235 | 236 | #### Page size 237 | By Default, only ten records are returned per request. To increase that number, pass an integer (or string integer) to the `page_size` parameter. The upper limit is 100 per request. 238 | 239 | ```python 240 | >>> result = dpla.search(q="chicken", page_size=100) 241 | ``` 242 | 243 | #### Page 244 | 245 | If that’s not enough, you can get the next ten items incrementing the page parameter (it’s one-indexed). 246 | 247 | ```python 248 | >>> result = dpla.search(q="chicken", page_size=100, page=2) 249 | ``` 250 | 251 | ## Limitations 252 | This project is still in its infancy. It ain't purrfect. 253 | 254 | * Right now, the client only does items search. Collections search ~~and individual item fetch~~ to come eventually. 255 | * It does not do a great job catching exceptions, so be warned! 256 | * Test coverage is limited. 257 | 258 | ## License 259 | 260 | GPLV2. 261 | See [license.txt](license.txt) 262 | -------------------------------------------------------------------------------- /dpla/__init__.py: -------------------------------------------------------------------------------- 1 | __author__ = "chad" 2 | -------------------------------------------------------------------------------- /dpla/api.py: -------------------------------------------------------------------------------- 1 | from re import match 2 | from requests import get, post 3 | from requests.compat import urlencode 4 | from dpla.settings import searchable_fields 5 | 6 | 7 | class DPLA(object): 8 | def __init__(self, api_key=None): 9 | 10 | if api_key is not None: 11 | self.api_key = api_key 12 | else: 13 | raise ValueError("DPLA API requires an api key.") 14 | if len(self.api_key) != 32: 15 | raise ValueError( 16 | "The DPLA key is not in the required format. PLease check it again" 17 | ) 18 | 19 | @staticmethod 20 | def new_key(email_address): 21 | if not match(r"[^@]+@[^@]+\.[^@]+", email_address): 22 | print("Hmmm...That doesn't look like an email address. Please check") 23 | return 24 | else: 25 | r = post("https://api.dp.la/v2/api_key/" + email_address) 26 | if r.status_code == 201: 27 | print(r.content) 28 | else: 29 | print("Hmmm...there seems to have been an error.") 30 | 31 | def fetch_by_id(self, id=None, **kwargs): 32 | if not id: 33 | raise ValueError("No id provided to fetch") 34 | kwargs["id"] = id 35 | kwargs["key"] = self.api_key 36 | request = Request(**kwargs) 37 | return Results(get(request.url).json(), request, self) 38 | 39 | def search(self, q=None, search_type="items", **kwargs): 40 | """ 41 | Builds and performs an item search. 42 | 43 | query -- a simple search query. Boolean Search operators allowed. 44 | type -- determines what type of search to perform, items or collections 45 | **kwargs -- The DPLA API has many possible parameters that can be passed. 46 | Pass parameters as kwarg key=value pairs. Some options include: 47 | search in specific fields -- key = searchFields , value = dictionary of fieldname(s) and search values 48 | Value is searched for only in the specified field. 49 | Multiple fields / search terms can be listed 50 | List of available fields is at https://dp.la/info/developers/codex/responses/field-reference/ 51 | return fields -- "fields" as key, list of field names as value 52 | Only the values of specified fields will be returned. 53 | If no fields are passed, values for all fields will be returned. 54 | facets -- 'facets' as they key, list of field names as value 55 | Returns a list of the most common values for that field 56 | spatial facet -- key = "facet_spatial" 2 item list consisting of of Lat , Long. 57 | Will return list of common distances from that geo-coordinate. 58 | facet limit -- "facet_limit" as key, number as value 59 | Number of facets to display (for each field?) 60 | sort -- "sort" as key , list of fieldnames as value. 61 | Results are sorted by these fields 62 | spatial sort -- "spatial_sort" as key, 2 item list consisting of of Lat , Long. as value 63 | Sort by distance from an geo-coordinate 64 | 65 | """ 66 | if not q and not kwargs: 67 | raise ValueError("You have not entered any search criteria") 68 | if not search_type: 69 | raise ValueError("Search type must be items or collections") 70 | kwargs["search_type"] = search_type 71 | if q: 72 | kwargs["query"] = q 73 | kwargs["key"] = self.api_key 74 | 75 | request = Request(**kwargs) 76 | response = get(request.url) 77 | if not response.ok: 78 | raise Exception(response.content) 79 | return Results(response.json(), request, self) 80 | 81 | 82 | class Request(object): 83 | def __init__(self, search_type="items", **kwargs): 84 | self.params = kwargs 85 | # Build individual url fragments for different search criteria 86 | url_parts = [] 87 | self.base_url = "https://api.dp.la/v2/" 88 | self.api_key = kwargs.get("key", "") 89 | if kwargs.get("id"): 90 | iid = ",".join(kwargs["id"]) 91 | else: 92 | iid = "" 93 | if kwargs.get("query"): 94 | url_parts.append(self._singleValueFormatter("q", kwargs["query"])) 95 | if kwargs.get("searchFields"): 96 | url_parts.append(self._searchFieldsFormatter(kwargs["searchFields"])) 97 | if kwargs.get("fields"): 98 | url_parts.append(self._multiValueFormatter("fields", kwargs["fields"])) 99 | if kwargs.get("facets") and not kwargs.get("spatial_facet"): 100 | url_parts.append(self._multiValueFormatter("facets", kwargs["facets"])) 101 | if kwargs.get("spatial_facet"): 102 | url_parts.append(self._facetSpatialFormatter(kwargs["spatial_facet"])) 103 | if kwargs.get("facet_size"): 104 | url_parts.append( 105 | self._singleValueFormatter("facet_size", kwargs["facet_size"]) 106 | ) 107 | if kwargs.get("sort") and not kwargs.get("spatial_sort"): 108 | url_parts.append(self._singleValueFormatter("sort_by", kwargs["sort"])) 109 | if kwargs.get("spatial_sort"): 110 | url_parts.append( 111 | self._singleValueFormatter( 112 | "sort_by_pin", "{},{}".format(*kwargs["spatial_sort"]) 113 | ) 114 | ) 115 | url_parts.append( 116 | self._singleValueFormatter( 117 | "sort_by", "sourceResource.spatial.coordinates" 118 | ) 119 | ) 120 | if kwargs.get("page_size"): 121 | url_parts.append( 122 | self._singleValueFormatter("page_size", kwargs["page_size"]) 123 | ) 124 | if kwargs.get("page"): 125 | url_parts.append(self._singleValueFormatter("page", kwargs["page"])) 126 | # Now string all the chunks together 127 | self.url = self._buildUrl(search_type, url_parts, iid) 128 | 129 | def _singleValueFormatter(self, param_name, value): 130 | """ 131 | Creates an encoded URL fragment for parameters that contain only a single value 132 | 133 | """ 134 | return urlencode({param_name: value}) 135 | 136 | def _multiValueFormatter(self, param_name, values): 137 | """ 138 | Creates an encoded URL fragment for parameters that may contain multiple values. 139 | 140 | """ 141 | return urlencode({param_name: ",".join(values)}) 142 | 143 | def _searchFieldsFormatter(self, searchFields): 144 | """ 145 | Creates an encoded URL fragment for searching for a value within a specific field. 146 | If multiple fields are specified, a single string is returned 147 | 148 | """ 149 | sf = [ 150 | urlencode({k: v}) for k, v in searchFields.items() if k in searchable_fields 151 | ] 152 | return "&".join(sf) 153 | 154 | def _facetSpatialFormatter(self, spatial_facet): 155 | coords = "sourceResource.spatial.coordinates:{}:{}".format(*spatial_facet) 156 | return urlencode({"facets": coords}) 157 | 158 | def _buildUrl(self, search_type, url_parts=None, id=None): 159 | url = self.base_url + search_type 160 | url_parts = url_parts or [] 161 | 162 | if id: 163 | url += "/" + id + "?" 164 | else: 165 | url += "?" 166 | if search_type == "items": 167 | url += "&".join(url_parts) 168 | if url_parts: 169 | url += "&api_key=" + self.api_key 170 | else: 171 | url += "api_key=" + self.api_key 172 | print(url) 173 | return url 174 | 175 | 176 | class Results(object): 177 | def __init__(self, response, request, dplaObject): 178 | self.dpla = dplaObject 179 | self.request = request 180 | self.count = response.get("count", None) 181 | self.limit = response.get("limit", None) 182 | self.start = response.get("start", None) 183 | self.items = [doc for doc in response["docs"]] 184 | if response.get("facets", None): 185 | self.facets = [{k: v} for k, v in response["facets"].iteritems()] 186 | 187 | def next_page(self): 188 | params = self.request.params 189 | params["page"] = int((self.start - 1) / self.limit) + 2 190 | next_response = self.dpla.search(**params) 191 | self.start = next_response.start 192 | self.items = next_response.items 193 | 194 | def all_records(self): 195 | for i in range(self.count): 196 | yield self.items[i - self.start] 197 | if not i < self.start + self.limit - 1: 198 | self.next_page() 199 | -------------------------------------------------------------------------------- /dpla/settings.py: -------------------------------------------------------------------------------- 1 | ## Defines searchable fields so that we don' try to search fields that aren't real 2 | searchable_fields = ( 3 | "dataProvider", 4 | "hasView", 5 | "hasView.format", 6 | "hasView.rights", 7 | "ingestDate", 8 | "ingestType", 9 | "isShownAt", 10 | "isShownAt.@id", 11 | "isShownAt.format", 12 | "isShownAt.rights", 13 | "object", 14 | "object.@id", 15 | "object.format", 16 | "object.rights", 17 | "originalRecord", 18 | "provider", 19 | "provider.@id", 20 | "provider.name", 21 | "sourceResource", 22 | "sourceResource.collection", 23 | "sourceResource.collection.@id", 24 | "sourceResource.contributor", 25 | "sourceResource.creator", 26 | "sourceResource.date", 27 | "sourceResource.date.begin", 28 | "sourceResource.date.displayDate", 29 | "sourceResource.date.end", 30 | "sourceResource.description", 31 | "sourceResource.extent", 32 | "sourceResource.format", 33 | "sourceResource.identifier", 34 | "sourceResource.language", 35 | "sourceResource.language.name", 36 | "sourceResource.language.iso639", 37 | "sourceResource.physicalMedium", 38 | "sourceResource.publisher", 39 | "sourceResource.rights", 40 | "sourceResource.spatial", 41 | "sourceResource.spatial.coordinates", 42 | "sourceResource.spatial.city", 43 | "sourceResource.spatial.county", 44 | "sourceResource.spatial.distance", 45 | "sourceResource.spatial.iso3166-2", 46 | "sourceResource.spatial.name", 47 | "sourceResource.spatial.region", 48 | "sourceResource.spatial.state", 49 | "sourceResource.stateLocatedIn.name", 50 | "sourceResource.stateLocatedIn.iso3166-2", 51 | "sourceResource.subject", 52 | "sourceResource.subject.@id", 53 | "sourceResource.subject.@type", 54 | "sourceResource.subject.name", 55 | "sourceResource.temporal", 56 | "sourceResource.temporal.begin", 57 | "sourceResource.temporal.end", 58 | "sourceResource.title", 59 | "sourceResource.type", 60 | ) 61 | -------------------------------------------------------------------------------- /license.txt: -------------------------------------------------------------------------------- 1 | 2 | GNU GENERAL PUBLIC LICENSE 3 | Version 2, June 1991 4 | 5 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., [http://fsf.org/] 6 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 7 | Everyone is permitted to copy and distribute verbatim copies 8 | of this license document, but changing it is not allowed. 9 | 10 | Preamble 11 | 12 | The licenses for most software are designed to take away your 13 | freedom to share and change it. By contrast, the GNU General Public 14 | License is intended to guarantee your freedom to share and change free 15 | software--to make sure the software is free for all its users. This 16 | General Public License applies to most of the Free Software 17 | Foundation's software and to any other program whose authors commit to 18 | using it. (Some other Free Software Foundation software is covered by 19 | the GNU Lesser General Public License instead.) You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | this service if you wish), that you receive source code or can get it 26 | if you want it, that you can change the software or use pieces of it 27 | in new free programs; and that you know you can do these things. 28 | 29 | To protect your rights, we need to make restrictions that forbid 30 | anyone to deny you these rights or to ask you to surrender the rights. 31 | These restrictions translate to certain responsibilities for you if you 32 | distribute copies of the software, or if you modify it. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must give the recipients all the rights that 36 | you have. You must make sure that they, too, receive or can get the 37 | source code. And you must show them these terms so they know their 38 | rights. 39 | 40 | We protect your rights with two steps: (1) copyright the software, and 41 | (2) offer you this license which gives you legal permission to copy, 42 | distribute and/or modify the software. 43 | 44 | Also, for each author's protection and ours, we want to make certain 45 | that everyone understands that there is no warranty for this free 46 | software. If the software is modified by someone else and passed on, we 47 | want its recipients to know that what they have is not the original, so 48 | that any problems introduced by others will not reflect on the original 49 | authors' reputations. 50 | 51 | Finally, any free program is threatened constantly by software 52 | patents. We wish to avoid the danger that redistributors of a free 53 | program will individually obtain patent licenses, in effect making the 54 | program proprietary. To prevent this, we have made it clear that any 55 | patent must be licensed for everyone's free use or not licensed at all. 56 | 57 | The precise terms and conditions for copying, distribution and 58 | modification follow. 59 | 60 | GNU GENERAL PUBLIC LICENSE 61 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 62 | 63 | 0. This License applies to any program or other work which contains 64 | a notice placed by the copyright holder saying it may be distributed 65 | under the terms of this General Public License. The "Program", below, 66 | refers to any such program or work, and a "work based on the Program" 67 | means either the Program or any derivative work under copyright law: 68 | that is to say, a work containing the Program or a portion of it, 69 | either verbatim or with modifications and/or translated into another 70 | language. (Hereinafter, translation is included without limitation in 71 | the term "modification".) Each licensee is addressed as "you". 72 | 73 | Activities other than copying, distribution and modification are not 74 | covered by this License; they are outside its scope. The act of 75 | running the Program is not restricted, and the output from the Program 76 | is covered only if its contents constitute a work based on the 77 | Program (independent of having been made by running the Program). 78 | Whether that is true depends on what the Program does. 79 | 80 | 1. You may copy and distribute verbatim copies of the Program's 81 | source code as you receive it, in any medium, provided that you 82 | conspicuously and appropriately publish on each copy an appropriate 83 | copyright notice and disclaimer of warranty; keep intact all the 84 | notices that refer to this License and to the absence of any warranty; 85 | and give any other recipients of the Program a copy of this License 86 | along with the Program. 87 | 88 | You may charge a fee for the physical act of transferring a copy, and 89 | you may at your option offer warranty protection in exchange for a fee. 90 | 91 | 2. You may modify your copy or copies of the Program or any portion 92 | of it, thus forming a work based on the Program, and copy and 93 | distribute such modifications or work under the terms of Section 1 94 | above, provided that you also meet all of these conditions: 95 | 96 | a) You must cause the modified files to carry prominent notices 97 | stating that you changed the files and the date of any change. 98 | 99 | b) You must cause any work that you distribute or publish, that in 100 | whole or in part contains or is derived from the Program or any 101 | part thereof, to be licensed as a whole at no charge to all third 102 | parties under the terms of this License. 103 | 104 | c) If the modified program normally reads commands interactively 105 | when run, you must cause it, when started running for such 106 | interactive use in the most ordinary way, to print or display an 107 | announcement including an appropriate copyright notice and a 108 | notice that there is no warranty (or else, saying that you provide 109 | a warranty) and that users may redistribute the program under 110 | these conditions, and telling the user how to view a copy of this 111 | License. (Exception: if the Program itself is interactive but 112 | does not normally print such an announcement, your work based on 113 | the Program is not required to print an announcement.) 114 | 115 | These requirements apply to the modified work as a whole. If 116 | identifiable sections of that work are not derived from the Program, 117 | and can be reasonably considered independent and separate works in 118 | themselves, then this License, and its terms, do not apply to those 119 | sections when you distribute them as separate works. But when you 120 | distribute the same sections as part of a whole which is a work based 121 | on the Program, the distribution of the whole must be on the terms of 122 | this License, whose permissions for other licensees extend to the 123 | entire whole, and thus to each and every part regardless of who wrote it. 124 | 125 | Thus, it is not the intent of this section to claim rights or contest 126 | your rights to work written entirely by you; rather, the intent is to 127 | exercise the right to control the distribution of derivative or 128 | collective works based on the Program. 129 | 130 | In addition, mere aggregation of another work not based on the Program 131 | with the Program (or with a work based on the Program) on a volume of 132 | a storage or distribution medium does not bring the other work under 133 | the scope of this License. 134 | 135 | 3. You may copy and distribute the Program (or a work based on it, 136 | under Section 2) in object code or executable form under the terms of 137 | Sections 1 and 2 above provided that you also do one of the following: 138 | 139 | a) Accompany it with the complete corresponding machine-readable 140 | source code, which must be distributed under the terms of Sections 141 | 1 and 2 above on a medium customarily used for software interchange; or, 142 | 143 | b) Accompany it with a written offer, valid for at least three 144 | years, to give any third party, for a charge no more than your 145 | cost of physically performing source distribution, a complete 146 | machine-readable copy of the corresponding source code, to be 147 | distributed under the terms of Sections 1 and 2 above on a medium 148 | customarily used for software interchange; or, 149 | 150 | c) Accompany it with the information you received as to the offer 151 | to distribute corresponding source code. (This alternative is 152 | allowed only for noncommercial distribution and only if you 153 | received the program in object code or executable form with such 154 | an offer, in accord with Subsection b above.) 155 | 156 | The source code for a work means the preferred form of the work for 157 | making modifications to it. For an executable work, complete source 158 | code means all the source code for all modules it contains, plus any 159 | associated interface definition files, plus the scripts used to 160 | control compilation and installation of the executable. However, as a 161 | special exception, the source code distributed need not include 162 | anything that is normally distributed (in either source or binary 163 | form) with the major components (compiler, kernel, and so on) of the 164 | operating system on which the executable runs, unless that component 165 | itself accompanies the executable. 166 | 167 | If distribution of executable or object code is made by offering 168 | access to copy from a designated place, then offering equivalent 169 | access to copy the source code from the same place counts as 170 | distribution of the source code, even though third parties are not 171 | compelled to copy the source along with the object code. 172 | 173 | 4. You may not copy, modify, sublicense, or distribute the Program 174 | except as expressly provided under this License. Any attempt 175 | otherwise to copy, modify, sublicense or distribute the Program is 176 | void, and will automatically terminate your rights under this License. 177 | However, parties who have received copies, or rights, from you under 178 | this License will not have their licenses terminated so long as such 179 | parties remain in full compliance. 180 | 181 | 5. You are not required to accept this License, since you have not 182 | signed it. However, nothing else grants you permission to modify or 183 | distribute the Program or its derivative works. These actions are 184 | prohibited by law if you do not accept this License. Therefore, by 185 | modifying or distributing the Program (or any work based on the 186 | Program), you indicate your acceptance of this License to do so, and 187 | all its terms and conditions for copying, distributing or modifying 188 | the Program or works based on it. 189 | 190 | 6. Each time you redistribute the Program (or any work based on the 191 | Program), the recipient automatically receives a license from the 192 | original licensor to copy, distribute or modify the Program subject to 193 | these terms and conditions. You may not impose any further 194 | restrictions on the recipients' exercise of the rights granted herein. 195 | You are not responsible for enforcing compliance by third parties to 196 | this License. 197 | 198 | 7. If, as a consequence of a court judgment or allegation of patent 199 | infringement or for any other reason (not limited to patent issues), 200 | conditions are imposed on you (whether by court order, agreement or 201 | otherwise) that contradict the conditions of this License, they do not 202 | excuse you from the conditions of this License. If you cannot 203 | distribute so as to satisfy simultaneously your obligations under this 204 | License and any other pertinent obligations, then as a consequence you 205 | may not distribute the Program at all. For example, if a patent 206 | license would not permit royalty-free redistribution of the Program by 207 | all those who receive copies directly or indirectly through you, then 208 | the only way you could satisfy both it and this License would be to 209 | refrain entirely from distribution of the Program. 210 | 211 | If any portion of this section is held invalid or unenforceable under 212 | any particular circumstance, the balance of the section is intended to 213 | apply and the section as a whole is intended to apply in other 214 | circumstances. 215 | 216 | It is not the purpose of this section to induce you to infringe any 217 | patents or other property right claims or to contest validity of any 218 | such claims; this section has the sole purpose of protecting the 219 | integrity of the free software distribution system, which is 220 | implemented by public license practices. Many people have made 221 | generous contributions to the wide range of software distributed 222 | through that system in reliance on consistent application of that 223 | system; it is up to the author/donor to decide if he or she is willing 224 | to distribute software through any other system and a licensee cannot 225 | impose that choice. 226 | 227 | This section is intended to make thoroughly clear what is believed to 228 | be a consequence of the rest of this License. 229 | 230 | 8. If the distribution and/or use of the Program is restricted in 231 | certain countries either by patents or by copyrighted interfaces, the 232 | original copyright holder who places the Program under this License 233 | may add an explicit geographical distribution limitation excluding 234 | those countries, so that distribution is permitted only in or among 235 | countries not thus excluded. In such case, this License incorporates 236 | the limitation as if written in the body of this License. 237 | 238 | 9. The Free Software Foundation may publish revised and/or new versions 239 | of the General Public License from time to time. Such new versions will 240 | be similar in spirit to the present version, but may differ in detail to 241 | address new problems or concerns. 242 | 243 | Each version is given a distinguishing version number. If the Program 244 | specifies a version number of this License which applies to it and "any 245 | later version", you have the option of following the terms and conditions 246 | either of that version or of any later version published by the Free 247 | Software Foundation. If the Program does not specify a version number of 248 | this License, you may choose any version ever published by the Free Software 249 | Foundation. 250 | 251 | 10. If you wish to incorporate parts of the Program into other free 252 | programs whose distribution conditions are different, write to the author 253 | to ask for permission. For software which is copyrighted by the Free 254 | Software Foundation, write to the Free Software Foundation; we sometimes 255 | make exceptions for this. Our decision will be guided by the two goals 256 | of preserving the free status of all derivatives of our free software and 257 | of promoting the sharing and reuse of software generally. 258 | 259 | NO WARRANTY 260 | 261 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 262 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 263 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 264 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 265 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 266 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 267 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 268 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 269 | REPAIR OR CORRECTION. 270 | 271 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 272 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 273 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 274 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 275 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 276 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 277 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 278 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 279 | POSSIBILITY OF SUCH DAMAGES. 280 | 281 | END OF TERMS AND CONDITIONS 282 | 283 | How to Apply These Terms to Your New Programs 284 | 285 | If you develop a new program, and you want it to be of the greatest 286 | possible use to the public, the best way to achieve this is to make it 287 | free software which everyone can redistribute and change under these terms. 288 | 289 | To do so, attach the following notices to the program. It is safest 290 | to attach them to the start of each source file to most effectively 291 | convey the exclusion of warranty; and each file should have at least 292 | the "copyright" line and a pointer to where the full notice is found. 293 | 294 | {description} 295 | Copyright (C) {year} {fullname} 296 | 297 | This program is free software; you can redistribute it and/or modify 298 | it under the terms of the GNU General Public License as published by 299 | the Free Software Foundation; either version 2 of the License, or 300 | (at your option) any later version. 301 | 302 | This program is distributed in the hope that it will be useful, 303 | but WITHOUT ANY WARRANTY; without even the implied warranty of 304 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 305 | GNU General Public License for more details. 306 | 307 | You should have received a copy of the GNU General Public License along 308 | with this program; if not, write to the Free Software Foundation, Inc., 309 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 310 | 311 | Also add information on how to contact you by electronic and paper mail. 312 | 313 | If the program is interactive, make it output a short notice like this 314 | when it starts in an interactive mode: 315 | 316 | Gnomovision version 69, Copyright (C) year name of author 317 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 318 | This is free software, and you are welcome to redistribute it 319 | under certain conditions; type `show c' for details. 320 | 321 | The hypothetical commands `show w' and `show c' should show the appropriate 322 | parts of the General Public License. Of course, the commands you use may 323 | be called something other than `show w' and `show c'; they could even be 324 | mouse-clicks or menu items--whatever suits your program. 325 | 326 | You should also get your employer (if you work as a programmer) or your 327 | school, if any, to sign a "copyright disclaimer" for the program, if 328 | necessary. Here is a sample; alter the names: 329 | 330 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 331 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 332 | 333 | {signature of Ty Coon}, 1 April 1989 334 | Ty Coon, President of Vice 335 | 336 | This General Public License does not permit incorporating your program into 337 | proprietary programs. If your program is a subroutine library, you may 338 | consider it more useful to permit linking proprietary applications with the 339 | library. If this is what you want to do, use the GNU Lesser General 340 | Public License instead of this License. 341 | 342 | 343 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [project] 2 | name = "dpla" 3 | version = "0.5" 4 | authors = [ 5 | { name="Chad Nelson", email="chadbnelson@gmail.com" }, 6 | ] 7 | description = "A client for the Digital Public Library of America (DPLA) API" 8 | readme = "README.md" 9 | requires-python = ">=3.7" 10 | classifiers = [ 11 | "Programming Language :: Python :: 3", 12 | "Development Status :: 5 - Production/Stable", 13 | "Operating System :: OS Independent", 14 | ] 15 | 16 | [project.urls] 17 | "Homepage" = "https://github.com/bibliotechy/dpyla" 18 | "Bug Tracker" = "https://github.com/bibliotechy/dpyla/issues" 19 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | requests>=2.0.0 2 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | description-file = README.md -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup 2 | 3 | setup( 4 | name="dpla", 5 | packages=["dpla"], # this must be the same as the name above 6 | version="0.5", 7 | description="A python client for the DPLA API", 8 | author="Chad Nelson", 9 | author_email="chadbnelson@gmail.com", 10 | url="https://github.com/bibliotechy/DPyLA", # use the URL to the github repo 11 | download_url="https://github.com/bibliotechy/DPyLA/releases/latest", 12 | keywords=["libraries", "DPLA", "museums"], # arbitrary keywords 13 | test_suite="tests.py", 14 | classifiers=[], 15 | install_requires=["requests"], 16 | ) 17 | -------------------------------------------------------------------------------- /tests.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | from dpla.api import * 3 | 4 | 5 | class testDPyLAClass(unittest.TestCase): 6 | def test_api_key_passed_as_parameter(self): 7 | key = "YourDPLAApiKeyGoesHereShouldBe32" 8 | dpla = DPLA(key) 9 | message = "Valid key passed as param should be set as attribute" 10 | self.assertEqual(dpla.api_key, key, message) 11 | 12 | def test_missing_api_key(self): 13 | with self.assertRaises(ValueError): 14 | DPLA() 15 | 16 | def test_invalid_api_key(self): 17 | with self.assertRaises(ValueError): 18 | DPLA("shortstring") 19 | 20 | 21 | class DPyLARequest(unittest.TestCase): 22 | def setUp(self): 23 | self.r = Request() 24 | self.query = self.r._singleValueFormatter("q", "chicken") 25 | self.multiword_query = self.r._singleValueFormatter("q", "chicken man") 26 | self.multivalue_fields = self.r._multiValueFormatter( 27 | "fields", ["sourceResource.title", "sourceResource.spatial.state"] 28 | ) 29 | self.search_fields = self.r._searchFieldsFormatter( 30 | {"sourceResource.title": "Chicago", "sourceResource.subject": "Food"} 31 | ) 32 | 33 | def test_single_value_formatter(self): 34 | expected = "q=chicken" 35 | self.assertEqual( 36 | self.query, expected, "Single word single values are formattted correctly" 37 | ) 38 | expected = "q=chicken+man" 39 | self.assertEqual( 40 | self.multiword_query, 41 | expected, 42 | "Multi word single values are formattted correctly", 43 | ) 44 | 45 | def test_multivalue_fields_formatter(self): 46 | expected = "fields=sourceResource.title%2CsourceResource.spatial.state" 47 | self.assertEqual( 48 | self.multivalue_fields, expected, "Return fields url fragment are correct" 49 | ) 50 | 51 | def test_search_field_formatter(self): 52 | expected = ("sourceResource.title=Chicago", "sourceResource.subject=Food") 53 | for expect in expected: 54 | self.assertIn( 55 | expect, 56 | self.search_fields, 57 | "Search specific fields url fragments are correct", 58 | ) 59 | 60 | def test_spatial_facet_formatter(self): 61 | request = self.r._facetSpatialFormatter([37, -48]) 62 | expected = "facets=sourceResource.spatial.coordinates%3A37%3A-48" 63 | self.assertEqual(request, expected, "Spatial facets url fragments are correct") 64 | 65 | def test_build_url(self): 66 | url_parts = [] 67 | 68 | url_parts.append(self.query) 69 | expected = "https://api.dp.la/v2/items?q=chicken&api_key=" 70 | url = self.r._buildUrl("items", url_parts) 71 | 72 | self.assertEqual( 73 | url, expected, "Single parameter item search url is constructed correctly" 74 | ) 75 | 76 | url_parts.append(self.multivalue_fields) 77 | url = self.r._buildUrl("items", url_parts) 78 | expected = "https://api.dp.la/v2/items?q=chicken&fields=sourceResource.title%2CsourceResource.spatial.state&api_key=" 79 | self.assertEqual( 80 | url, expected, "Two parameter item search url is constructed correctly" 81 | ) 82 | 83 | url_parts.append(self.search_fields) 84 | url = self.r._buildUrl("items", url_parts) 85 | 86 | expected = ( 87 | "q=chicken", 88 | "sourceResource.title=Chicago", 89 | "sourceResource.subject=Food", 90 | "api_key=", 91 | "fields=sourceResource.title%2CsourceResource.spatial.state", 92 | ) 93 | for expect in expected: 94 | self.assertIn( 95 | expect, url, "Three parameter item search url is constructed correctly" 96 | ) 97 | 98 | id = "93583acc6425f8172b7b506f44a32121" 99 | url = self.r._buildUrl("items", id=id) 100 | expected = ( 101 | "https://api.dp.la/v2/items/93583acc6425f8172b7b506f44a32121?api_key=" 102 | ) 103 | self.assertEqual(url, expected) 104 | 105 | multiple_ids = ( 106 | "fe47a8b71de4c136fe115a19ead13e4d,93583acc6425f8172b7b506f44a32121" 107 | ) 108 | url = self.r._buildUrl("items", id=multiple_ids) 109 | expected = "https://api.dp.la/v2/items/fe47a8b71de4c136fe115a19ead13e4d,93583acc6425f8172b7b506f44a32121?api_key=" 110 | self.assertEqual(url, expected) 111 | 112 | 113 | class DPyLASearch(unittest.TestCase): 114 | def setUp(self): 115 | self.dpla = DPLA("3e5a0654e663ce2c99497c7efe01e5ef") 116 | 117 | def test_search_simple(self): 118 | results = self.dpla.search("chickens") 119 | self.assertGreaterEqual( 120 | results.count, 0, "Results count should contain at least one result" 121 | ) 122 | 123 | 124 | class DPyLAResults(unittest.TestCase): 125 | def setUp(self): 126 | self.dpla = DPLA("3e5a0654e663ce2c99497c7efe01e5ef") 127 | 128 | def test_all_records_method(self): 129 | results = self.dpla.search( 130 | "chicken man new old", fields=["sourceResource.title"] 131 | ) 132 | count = 0 133 | for i in results.all_records(): 134 | count += 1 135 | self.assertEqual( 136 | results.count, 137 | count, 138 | "Iterates through %i records, but %i are in result" 139 | % (count, results.count), 140 | ) 141 | 142 | 143 | if __name__ == "__main__": 144 | unittest.main() 145 | --------------------------------------------------------------------------------