├── .travis.yml ├── LICENSE ├── README.rst ├── duckduckpy ├── __init__.py ├── api.py ├── core.py ├── exception.py └── utils.py ├── requirements.txt ├── setup.py └── tests.py /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - 2.7 4 | - 3.4 5 | - 3.5 6 | - 3.6 7 | - 3.7 8 | - 3.8 9 | install: 10 | - pip install -r requirements.txt 11 | - pip install coveralls 12 | script: 13 | coverage run --source=duckduckpy setup.py test 14 | after_success: 15 | coveralls 16 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | Copyright (c) 2015 Ivan Kliuk 3 | 4 | Permission is hereby granted, free of charge, to any person obtaining a copy 5 | of this software and associated documentation files (the "Software"), to deal 6 | in the Software without restriction, including without limitation the rights 7 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | copies of the Software, and to permit persons to whom the Software is 9 | furnished to do so, subject to the following conditions: 10 | 11 | The above copyright notice and this permission notice shall be included in 12 | all copies or substantial portions of the Software. 13 | 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 15 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 16 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 17 | IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, 18 | DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR 19 | OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE 20 | OR OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | DuckDuckPy 2 | ========== 3 | 4 | |package| |travis-ci| |coveralls| 5 | 6 | `DuckDuckPy `_ is a Python 7 | library for querying `DuckDuckGo API `_ and 8 | render results either to Python dictionary or namedtuple. 9 | 10 | Features 11 | -------- 12 | 13 | * Uses standard library only 14 | * Works on Python 2.6+ and 3.3+ 15 | * Unit test coverage 100% 16 | * SSL and unicode support 17 | * Licensed under MIT license 18 | 19 | Installation 20 | ------------ 21 | 22 | You can install DuckDuckPy either via the `Python Package Index (PyPI) `_ or 23 | from source. 24 | 25 | To install using ``pip``: 26 | 27 | .. code:: bash 28 | 29 | $ pip install duckduckpy 30 | 31 | To install using ``easy_install``: 32 | 33 | .. code:: bash 34 | 35 | $ easy_install duckduckpy 36 | 37 | To install from sources you can download the latest version of DuckDuckPy 38 | either from `PyPI `_ or 39 | `GitHub `_, extract archive contents and 40 | run following command from the source directory: 41 | 42 | .. code:: bash 43 | 44 | $ python setup.py install 45 | 46 | Latest upstream version can be installed directly from the git repository: 47 | 48 | .. code:: bash 49 | 50 | $ pip install git+https://github.com/ivankliuk/duckduckpy.git 51 | 52 | API description 53 | --------------- 54 | 55 | .. code-block:: python 56 | 57 | query(query_string, secure=False, container=u'namedtuple', verbose=False, 58 | user_agent=u'duckduckpy 0.2', no_redirect=False, no_html=False, 59 | skip_disambig=False) 60 | 61 | Generates and sends a query to DuckDuckGo API. 62 | 63 | **Arguments:** 64 | 65 | +---------------+-------------------------------------------------------------+ 66 | | query_string | Query to be passed to DuckDuckGo API. | 67 | +---------------+-------------------------------------------------------------+ 68 | | secure | Use secure SSL/TLS connection. Default - False. | 69 | | | Syntactic sugar is secure_query function which is passed | 70 | | | the same parameters. | 71 | +---------------+-------------------------------------------------------------+ 72 | | container | Indicates how dict-like objects are serialized. There are | 73 | | | two possible options: namedtuple and dict. If 'namedtuple' | 74 | | | is passed the objects will be serialized to namedtuple | 75 | | | instance of certain class. If 'dict' is passed the objects | 76 | | | won't be deserialized. Default value: 'namedtuple'. | 77 | +---------------+-------------------------------------------------------------+ 78 | | verbose | Don't raise any exception if error occurs. | 79 | | | Default value: False. | 80 | +---------------+-------------------------------------------------------------+ 81 | | user_agent | User-Agent header of HTTP requests to DuckDuckGo API. | 82 | | | Default value: 'duckduckpy 0.2' | 83 | +---------------+-------------------------------------------------------------+ 84 | | no_redirect | Skip HTTP redirects (for !bang commands). | 85 | | | Default value: False. | 86 | +---------------+-------------------------------------------------------------+ 87 | | no_html | Remove HTML from text, e.g. bold and italics. | 88 | | | Default value: False. | 89 | +---------------+-------------------------------------------------------------+ 90 | | skip_disambig | Skip disambiguation (D) Type. Default value: False. | 91 | +---------------+-------------------------------------------------------------+ 92 | | lang | Override "us-en" language & region. Default - None. | 93 | +---------------+-------------------------------------------------------------+ 94 | 95 | **Raises:** 96 | 97 | +--------------------------+--------------------------------------------------+ 98 | | DuckDuckDeserializeError | JSON serialization failed. | 99 | +--------------------------+--------------------------------------------------+ 100 | | DuckDuckConnectionError | Something went wrong with httplib operation. | 101 | +--------------------------+--------------------------------------------------+ 102 | | DuckDuckArgumentError | Passed argument is wrong. | 103 | +--------------------------+--------------------------------------------------+ 104 | 105 | **Returns:** 106 | 107 | Container depends on container parameter. Each field in the response is 108 | converted to the so-called snake case. 109 | 110 | Usage 111 | ----- 112 | 113 | .. code-block:: python 114 | 115 | >>> from duckduckpy import query 116 | >>> response = query('Python') # namedtuple is used as a container 117 | >>> response 118 | Response(redirect=u'', definition=u'', image_width=0, ...} 119 | >>> type(response) 120 | 121 | >>> response.related_topics[0] 122 | Result(first_url=u'https://duckduckgo.com/Python', text=...) 123 | >>> type(response.related_topics[0]) 124 | 125 | 126 | >>> response = query('Python', container='dict') # dict as the container 127 | >>> type(response) 128 | 129 | >>> response 130 | {u'abstract': u'', u'results': [], u'image_is_logo': 0, ...} 131 | >>> type(response['related_topics'][0]) 132 | 133 | >>> response['related_topics'][0] 134 | {u'first_url': u'https://duckduckgo.com/Python', u'text': ...} 135 | 136 | .. |package| image:: https://badge.fury.io/py/duckduckpy.svg 137 | :target: http://badge.fury.io/py/duckduckpy 138 | :alt: PyPI package 139 | .. |travis-ci| image:: https://travis-ci.org/ivankliuk/duckduckpy.svg?branch=master 140 | :target: https://travis-ci.org/ivankliuk/duckduckpy 141 | :alt: CI Status 142 | .. |coveralls| image:: https://coveralls.io/repos/ivankliuk/duckduckpy/badge.svg?branch=master 143 | :target: https://coveralls.io/r/ivankliuk/duckduckpy?branch=master 144 | :alt: Coverage 145 | -------------------------------------------------------------------------------- /duckduckpy/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | """Library for querying the instant answer API of DuckDuckGo search engine.""" 4 | 5 | __version__ = 0.2 6 | __author__ = 'Ivan Kliuk' 7 | __email__ = 'ivan.kliuk@gmail.com' 8 | __license__ = 'MIT' 9 | __url__ = 'https://github.com/ivankliuk/duckduckpy/' 10 | __all__ = ['query', 'secure_query'] 11 | 12 | 13 | from duckduckpy.core import query 14 | from duckduckpy.core import secure_query 15 | -------------------------------------------------------------------------------- /duckduckpy/api.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # The MIT License (MIT) 4 | # Copyright (c) 2015 Ivan Kliuk 5 | # 6 | # Permission is hereby granted, free of charge, to any person obtaining a copy 7 | # of this software and associated documentation files (the "Software"), to deal 8 | # in the Software without restriction, including without limitation the rights 9 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | # copies of the Software, and to permit persons to whom the Software is 11 | # furnished to do so, subject to the following conditions: 12 | # 13 | # The above copyright notice and this permission notice shall be included in 14 | # all copies or substantial portions of the Software. 15 | # 16 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | # IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, 20 | # DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR 21 | # OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE 22 | # OR OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | from __future__ import unicode_literals 25 | 26 | from . import __version__ 27 | from .utils import camel_to_snake_case_set 28 | 29 | from collections import namedtuple 30 | 31 | SERVER_HOST = 'api.duckduckgo.com' 32 | USER_AGENT = 'duckduckpy {0}'.format(__version__) 33 | 34 | ICON_KEYS = set(['URL', 'Width', 'Height']) 35 | RESULT_KEYS = set(['FirstURL', 'Icon', 'Result', 'Text']) 36 | RELATED_TOPIC_KEYS = set(['Name', 'Topics']) 37 | RESPONSE_KEYS = set([ 38 | 'Redirect', 'Definition', 'ImageWidth', 'Infobox', 'RelatedTopics', 39 | 'ImageHeight', 'Heading', 'Answer', 'AbstractText', 'Type', 'ImageIsLogo', 40 | 'DefinitionSource', 'AbstractURL', 'Abstract', 'DefinitionURL', 'Results', 41 | 'Entity', 'AnswerType', 'AbstractSource', 'Image', 'meta']) 42 | Icon = namedtuple('Icon', camel_to_snake_case_set(ICON_KEYS)) 43 | Result = namedtuple('Result', camel_to_snake_case_set(RESULT_KEYS)) 44 | RelatedTopic = namedtuple('RelatedTopic', 45 | camel_to_snake_case_set(RELATED_TOPIC_KEYS)) 46 | Response = namedtuple('Response', camel_to_snake_case_set(RESPONSE_KEYS)) 47 | -------------------------------------------------------------------------------- /duckduckpy/core.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # The MIT License (MIT) 4 | # Copyright (c) 2015 Ivan Kliuk 5 | # 6 | # Permission is hereby granted, free of charge, to any person obtaining a copy 7 | # of this software and associated documentation files (the "Software"), to deal 8 | # in the Software without restriction, including without limitation the rights 9 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | # copies of the Software, and to permit persons to whom the Software is 11 | # furnished to do so, subject to the following conditions: 12 | # 13 | # The above copyright notice and this permission notice shall be included in 14 | # all copies or substantial portions of the Software. 15 | # 16 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | # IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, 20 | # DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR 21 | # OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE 22 | # OR OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | from __future__ import unicode_literals 25 | 26 | from . import api 27 | from . import exception as exc 28 | from .utils import camel_to_snake_case 29 | from .utils import is_python2 30 | from .utils import decoder 31 | 32 | import functools 33 | import json 34 | import socket 35 | 36 | # Python 2/3 compatibility. 37 | if is_python2(): 38 | import httplib as http_client 39 | from urllib import urlencode 40 | else: 41 | import http.client as http_client 42 | from urllib.parse import urlencode 43 | 44 | 45 | class Hook(object): 46 | """A hook for dict-objects serialization.""" 47 | containers = ['namedtuple', 'dict'] 48 | 49 | def __new__(cls, container, verbose=False): 50 | if container not in cls.containers: 51 | if verbose: 52 | raise exc.DuckDuckDeserializeError( 53 | "Deserialization container '{0}'" 54 | " is not found".format(container)) 55 | return None 56 | return super(Hook, cls).__new__(cls) 57 | 58 | def __init__(self, container, verbose=False): 59 | self._container = container 60 | self._verbose = verbose 61 | 62 | def _camel_to_snake_case(self): 63 | keys = set(self.dict_object.keys()) 64 | for key in keys: 65 | val = self.dict_object.pop(key) 66 | self.dict_object[camel_to_snake_case(key)] = val 67 | 68 | def serialize(self, class_name): 69 | self._camel_to_snake_case() 70 | if self._container == 'namedtuple': 71 | namedtuple_class = getattr(api, class_name) 72 | return namedtuple_class(**self.dict_object) 73 | if self._container == 'dict': 74 | return self.dict_object 75 | 76 | def __call__(self, dict_object): 77 | keys = set(dict_object.keys()) 78 | self.dict_object = dict_object 79 | if not keys: 80 | return {} 81 | if keys == api.ICON_KEYS: 82 | return self.serialize('Icon') 83 | elif keys == api.RESULT_KEYS: 84 | return self.serialize('Result') 85 | elif keys == api.RELATED_TOPIC_KEYS: 86 | return self.serialize('RelatedTopic') 87 | elif keys == api.RESPONSE_KEYS: 88 | return self.serialize('Response') 89 | 90 | # Leave 'meta' object as is. 91 | uppercase_keys = list(filter(lambda k: k[0].isupper(), keys)) 92 | if not self._verbose or not uppercase_keys: 93 | return dict_object 94 | raise exc.DuckDuckDeserializeError( 95 | "Unable to deserialize dict to an object") 96 | 97 | 98 | def url_assembler(query_string, no_redirect=0, no_html=0, skip_disambig=0, lang=None): 99 | """Assembler of parameters for building request query. 100 | 101 | Args: 102 | query_string: Query to be passed to DuckDuckGo API. 103 | no_redirect: Skip HTTP redirects (for !bang commands). Default - False. 104 | no_html: Remove HTML from text, e.g. bold and italics. Default - False. 105 | skip_disambig: Skip disambiguation (D) Type. Default - False. 106 | lang: Override "us-en" language & region. Default - None. 107 | 108 | Returns: 109 | A “percent-encoded” string which is used as a part of the query. 110 | """ 111 | params = [('q', query_string.encode("utf-8")), ('format', 'json')] 112 | 113 | if no_redirect: 114 | params.append(('no_redirect', 1)) 115 | if no_html: 116 | params.append(('no_html', 1)) 117 | if skip_disambig: 118 | params.append(('skip_disambig', 1)) 119 | if lang: 120 | params.append(('kl', lang)) 121 | 122 | return '/?' + urlencode(params) 123 | 124 | 125 | def query(query_string, secure=False, container='namedtuple', verbose=False, 126 | user_agent=api.USER_AGENT, no_redirect=False, no_html=False, 127 | skip_disambig=False, lang=None): 128 | """ 129 | Generates and sends a query to DuckDuckGo API. 130 | 131 | Args: 132 | query_string: Query to be passed to DuckDuckGo API. 133 | secure: Use secure SSL/TLS connection. Default - False. 134 | Syntactic sugar is secure_query function which is passed the same 135 | parameters. 136 | container: Indicates how dict-like objects are serialized. There are 137 | two possible options: namedtuple and dict. If 'namedtuple' is passed 138 | the objects will be serialized to namedtuple instance of certain 139 | class. If 'dict' is passed the objects won't be deserialized. 140 | Default value: 'namedtuple'. 141 | verbose: Don't raise any exception if error occurs. 142 | Default value: False. 143 | user_agent: User-Agent header of HTTP requests to DuckDuckGo API. 144 | Default value: 'duckduckpy 0.2' 145 | no_redirect: Skip HTTP redirects (for !bang commands). 146 | Default value: False. 147 | no_html: Remove HTML from text, e.g. bold and italics. 148 | Default value: False. 149 | skip_disambig: Skip disambiguation (D) Type. Default value: False. 150 | 151 | lang: Override "us-en" language & region. Default value: None 152 | See https://duckduckgo.com/params 153 | 154 | Raises: 155 | DuckDuckDeserializeError: JSON serialization failed. 156 | DuckDuckConnectionError: Something went wrong with client operation. 157 | DuckDuckArgumentError: Passed argument is wrong. 158 | 159 | Returns: 160 | Container depends on container parameter. Each field in the response is 161 | converted to the so-called snake case. 162 | 163 | Usage: 164 | >>> import duckduckpy 165 | >>># Namedtuple is used as a container: 166 | >>> response = duckduckpy.query('Python') 167 | >>> response 168 | Response(redirect=u'', definition=u'', image_width=0, ...} 169 | >>> type(response) 170 | 171 | >>> response.related_topics[0] 172 | Result(first_url=u'https://duckduckgo.com/Python', text=...) 173 | >>> type(response.related_topics[0]) 174 | 175 | 176 | >>># Dict is used as a container: 177 | >>> response = duckduckpy.query('Python', container='dict') 178 | >>> type(response) 179 | 180 | >>> response 181 | {u'abstract': u'', u'results': [], u'image_is_logo': 0, ...} 182 | >>> type(response['related_topics'][0]) 183 | 184 | >>> response['related_topics'][0] 185 | {u'first_url': u'https://duckduckgo.com/Python', u'text': ...} 186 | """ 187 | if container not in Hook.containers: 188 | raise exc.DuckDuckArgumentError( 189 | "Argument 'container' must be one of the values: " 190 | "{0}".format(', '.join(Hook.containers))) 191 | 192 | headers = {"User-Agent": user_agent} 193 | url = url_assembler( 194 | query_string, 195 | no_redirect=no_redirect, 196 | no_html=no_html, 197 | skip_disambig=skip_disambig, 198 | lang=lang) 199 | 200 | if secure: 201 | conn = http_client.HTTPSConnection(api.SERVER_HOST) 202 | else: 203 | conn = http_client.HTTPConnection(api.SERVER_HOST) 204 | 205 | try: 206 | conn.request("GET", url, "", headers) 207 | resp = conn.getresponse() 208 | data = decoder(resp.read()) 209 | except socket.gaierror as e: 210 | raise exc.DuckDuckConnectionError(e.strerror) 211 | finally: 212 | conn.close() 213 | 214 | hook = Hook(container, verbose=verbose) 215 | try: 216 | obj = json.loads(data, object_hook=hook) 217 | except ValueError: 218 | raise exc.DuckDuckDeserializeError( 219 | "Unable to deserialize response to an object") 220 | 221 | return obj 222 | 223 | 224 | secure_query = functools.partial(query, secure=True) 225 | -------------------------------------------------------------------------------- /duckduckpy/exception.py: -------------------------------------------------------------------------------- 1 | # The MIT License (MIT) 2 | # Copyright (c) 2015 Ivan Kliuk 3 | # 4 | # Permission is hereby granted, free of charge, to any person obtaining a copy 5 | # of this software and associated documentation files (the "Software"), to deal 6 | # in the Software without restriction, including without limitation the rights 7 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 8 | # copies of the Software, and to permit persons to whom the Software is 9 | # furnished to do so, subject to the following conditions: 10 | # 11 | # The above copyright notice and this permission notice shall be included in 12 | # all copies or substantial portions of the Software. 13 | # 14 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 15 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 16 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 17 | # IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, 18 | # DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR 19 | # OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE 20 | # OR OTHER DEALINGS IN THE SOFTWARE. 21 | 22 | 23 | class DuckDuckException(Exception): 24 | """Base exception class 25 | """ 26 | pass 27 | 28 | 29 | class DuckDuckDeserializeError(DuckDuckException): 30 | """JSON serialization exception 31 | """ 32 | pass 33 | 34 | 35 | class DuckDuckConnectionError(DuckDuckException): 36 | """A wrapper around httplib exceptions. Raised when something went wrong 37 | with httplib operation. 38 | """ 39 | pass 40 | 41 | 42 | class DuckDuckArgumentError(DuckDuckException): 43 | """Indicates that argument is wrong 44 | """ 45 | pass 46 | -------------------------------------------------------------------------------- /duckduckpy/utils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # The MIT License (MIT) 4 | # Copyright (c) 2015 Ivan Kliuk 5 | # 6 | # Permission is hereby granted, free of charge, to any person obtaining a copy 7 | # of this software and associated documentation files (the "Software"), to deal 8 | # in the Software without restriction, including without limitation the rights 9 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 10 | # copies of the Software, and to permit persons to whom the Software is 11 | # furnished to do so, subject to the following conditions: 12 | # 13 | # The above copyright notice and this permission notice shall be included in 14 | # all copies or substantial portions of the Software. 15 | # 16 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 17 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 18 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 19 | # IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, 20 | # DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR 21 | # OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE 22 | # OR OTHER DEALINGS IN THE SOFTWARE. 23 | 24 | from __future__ import unicode_literals 25 | 26 | import re 27 | import sys 28 | 29 | _1 = re.compile(r'(.)([A-Z][a-z]+)') 30 | _2 = re.compile('([a-z0-9])([A-Z])') 31 | 32 | 33 | def is_python2(): 34 | """Checks whether Python major version is 2.""" 35 | return sys.version_info[0] == 2 36 | 37 | 38 | def camel_to_snake_case(string): 39 | """Converts 'string' presented in camel case to snake case. 40 | 41 | e.g.: CamelCase => snake_case 42 | """ 43 | s = _1.sub(r'\1_\2', string) 44 | return _2.sub(r'\1_\2', s).lower() 45 | 46 | 47 | def camel_to_snake_case_set(seq): 48 | """Converts sequence to the snake case and returns the result as set.""" 49 | return set(map(camel_to_snake_case, seq)) 50 | 51 | 52 | def decoder(obj): 53 | """Decodes 'bytes' object to UTF-8.""" 54 | if isinstance(obj, bytes): 55 | return obj.decode("utf-8") 56 | return obj 57 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | mock==1.0.1 2 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import duckduckpy 2 | from setuptools import setup 3 | 4 | with open('README.rst', 'r') as f: 5 | long_description = f.read() 6 | 7 | setup(name='duckduckpy', 8 | version=duckduckpy.__version__, 9 | packages=['duckduckpy'], 10 | description=duckduckpy.__doc__, 11 | author=duckduckpy.__author__, 12 | author_email=duckduckpy.__email__, 13 | license=duckduckpy.__license__, 14 | url=duckduckpy.__url__, 15 | download_url='https://github.com/ivankliuk/duckduckpy/tarball/0.2', 16 | long_description=long_description, 17 | platforms=['any'], 18 | keywords=["duckduckgo"], 19 | classifiers=[ 20 | "Development Status :: 2 - Pre-Alpha", 21 | "Intended Audience :: Developers", 22 | "License :: OSI Approved :: MIT License", 23 | "Operating System :: OS Independent", 24 | 'Programming Language :: Python', 25 | 'Programming Language :: Python :: 2', 26 | 'Programming Language :: Python :: 2.6', 27 | 'Programming Language :: Python :: 2.7', 28 | 'Programming Language :: Python :: 3', 29 | 'Programming Language :: Python :: 3.3', 30 | 'Programming Language :: Python :: 3.4', 31 | 'Programming Language :: Python :: 3.5', 32 | "Topic :: Internet :: WWW/HTTP :: Indexing/Search"], 33 | zip_safe=True, 34 | test_suite="tests") 35 | -------------------------------------------------------------------------------- /tests.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | from __future__ import unicode_literals 4 | import unittest 5 | from collections import Iterable 6 | from io import StringIO 7 | import mock 8 | import socket 9 | 10 | from duckduckpy.core import api 11 | from duckduckpy.core import Hook 12 | from duckduckpy.core import query 13 | from duckduckpy.core import secure_query 14 | from duckduckpy.core import url_assembler 15 | import duckduckpy.exception as exc 16 | from duckduckpy.utils import camel_to_snake_case 17 | from duckduckpy.utils import is_python2 18 | 19 | 20 | class TestHook(unittest.TestCase): 21 | def test_non_existent_hook(self): 22 | self.assertTrue(Hook(1) is None) 23 | 24 | def test_hook_instance_returned(self): 25 | hook = Hook(Hook.containers[0]) 26 | self.assertTrue(isinstance(hook, Hook)) 27 | 28 | def test_containers_exist(self): 29 | self.assertTrue(isinstance(Hook.containers, Iterable)) 30 | self.assertTrue(len(Hook.containers) > 0) 31 | 32 | def test_no_object_found(self): 33 | obj = { 34 | 'URL': 'www.test.url.com', 35 | 'SomeThing': 'icon', 36 | 'Result': 20, 37 | 'Text': 'Example of a text'} 38 | expected = { 39 | 'URL': 'www.test.url.com', 40 | 'SomeThing': 'icon', 41 | 'Result': 20, 42 | 'Text': 'Example of a text'} 43 | hook = Hook('dict') 44 | actual_dict = hook(obj) 45 | self.assertEqual(actual_dict, expected) 46 | 47 | 48 | class TestHookExceptions(unittest.TestCase): 49 | def test_non_existent_hook_verbose(self): 50 | self.assertRaises(exc.DuckDuckDeserializeError, Hook, 1, verbose=True) 51 | 52 | def test_no_object_found_verbose(self): 53 | obj = { 54 | 'URL': 'www.test.url.com', 55 | 'SomeThing': 'icon', 56 | 'Result': 20, 57 | 'Text': 'Example of a text'} 58 | hook = Hook('dict', verbose=True) 59 | self.assertRaises(exc.DuckDuckDeserializeError, hook, obj) 60 | 61 | 62 | class TestCamelToSnakeCase(unittest.TestCase): 63 | def assertConverted(self, camel_case_string, snake_case_string): 64 | self.assertEqual(snake_case_string, 65 | camel_to_snake_case(camel_case_string)) 66 | 67 | def test_conversion_one_word(self): 68 | self.assertConverted('Camel', 'camel') 69 | 70 | def test_conversion_two_words(self): 71 | self.assertConverted('CamelCase', 'camel_case') 72 | 73 | def test_conversion_three_words(self): 74 | self.assertConverted('CamelCaseToo', 'camel_case_too') 75 | 76 | def test_conversion_three_mixed(self): 77 | self.assertConverted('CamelCase_Underscore', 'camel_case__underscore') 78 | 79 | def test_conversion_with_numbers(self): 80 | self.assertConverted('Camel2CaseToo3', 'camel2_case_too3') 81 | 82 | def test_conversion_equal(self): 83 | self.assertConverted('camel_case', 'camel_case') 84 | 85 | def test_conversion_mixed_case(self): 86 | self.assertConverted('getHTTPResponseCode', 'get_http_response_code') 87 | 88 | def test_conversion_mixed_case_with_numbers(self): 89 | self.assertConverted('get2HTTPResponseCode', 'get2_http_response_code') 90 | 91 | def test_conversion_uppercase_one_after_another(self): 92 | self.assertConverted('HTTPResponseCode', 'http_response_code') 93 | 94 | 95 | class TestURLAssembler(unittest.TestCase): 96 | def test_simple(self): 97 | expected = "/?q=test+query&format=json" 98 | self.assertEqual(url_assembler("test query"), expected) 99 | 100 | def test_cyrillic(self): 101 | expected = ( 102 | "/?q=%D0%A1%D0%BB%D0%B0%D0%B2%D0%B0+%D0%" 103 | "A3%D0%BA%D1%80%D0%B0%D1%97%D0%BD%D1%96&format=json") 104 | url = url_assembler('Слава Україні') 105 | self.assertEqual(url, expected) 106 | 107 | def test_no_redirect(self): 108 | expected = "/?q=test+query&format=json&no_redirect=1" 109 | url = url_assembler("test query", no_redirect=True) 110 | self.assertEqual(url, expected) 111 | 112 | def test_no_html(self): 113 | expected = "/?q=test+query&format=json&no_html=1" 114 | url = url_assembler("test query", no_html=True) 115 | self.assertEqual(url, expected) 116 | 117 | def test_skip_disambig(self): 118 | expected = "/?q=test+query&format=json&skip_disambig=1" 119 | url = url_assembler("test query", skip_disambig=True) 120 | self.assertEqual(url, expected) 121 | 122 | def test_all_options_are_on(self): 123 | expected = ("/?q=test+query&format=json&no_redirect=1" 124 | "&no_html=1&skip_disambig=1") 125 | url = url_assembler("test query", 126 | no_redirect=True, no_html=True, skip_disambig=True) 127 | self.assertEqual(url, expected) 128 | 129 | def test_language_region(self): 130 | expected = ("/?q=test+query&format=json&kl=ru-ru") 131 | url = url_assembler("test query", lang="ru-ru") 132 | self.assertEqual(url, expected) 133 | 134 | 135 | @mock.patch('duckduckpy.core.http_client.HTTPConnection.request') 136 | class TestQuery(unittest.TestCase): 137 | origin = r""" 138 | { 139 | "Abstract": "", 140 | "AbstractSource": "Wikipedia", 141 | "AbstractText": "", 142 | "AbstractURL": "https://en.wikipedia.org/wiki/Python", 143 | "Answer": "", 144 | "AnswerType": "", 145 | "Definition": "", 146 | "DefinitionSource": "", 147 | "DefinitionURL": "", 148 | "Entity": "", 149 | "Heading": "Python", 150 | "Image": "", 151 | "ImageHeight": 0, 152 | "ImageIsLogo": 0, 153 | "ImageWidth": 0, 154 | "Infobox": {}, 155 | "Redirect": "", 156 | "RelatedTopics": [ 157 | { 158 | "FirstURL": "https://duckduckgo.com/Python_(programming", 159 | "Icon": { 160 | "Height": "", 161 | "URL": "https://duckduckgo.com/i/7eec482b.png", 162 | "Width": "" 163 | }, 164 | "Result": "Mo", 175 | "Text": "Monty PythonA British surreal comedy group who" 176 | }, 177 | { 178 | "Name": "Ancient Greece", 179 | "Topics": [ 180 | { 181 | "FirstURL": "https://duckduckgo.com/Python_(programming", 182 | "Icon": { 183 | "Height": "", 184 | "URL": "https://duckduckgo.com/i/7eec482b.png", 185 | "Width": "" 186 | }, 187 | "Result": "Mo", 198 | "Text": "Monty PythonA British surreal comedy group who" 199 | } 200 | ] 201 | } 202 | ], 203 | "Results": [], 204 | "Type": "D", 205 | "meta": { 206 | "attribution": null, 207 | "blockgroup": null, 208 | "created_date": null, 209 | "description": "Wikipedia", 210 | "designer": null, 211 | "dev_date": null, 212 | "dev_milestone": "live", 213 | "developer": [ 214 | { 215 | "name": "DDG Team", 216 | "type": "ddg", 217 | "url": "http://www.duckduckhack.com" 218 | } 219 | ], 220 | "example_query": "nikola tesla", 221 | "id": "wikipedia_fathead", 222 | "is_stackexchange": null, 223 | "js_callback_name": "wikipedia", 224 | "live_date": null, 225 | "maintainer": { 226 | "github": "duckduckgo" 227 | }, 228 | "name": "Wikipedia", 229 | "perl_module": "DDG::Fathead::Wikipedia", 230 | "producer": null, 231 | "production_state": "online", 232 | "repo": "fathead", 233 | "signal_from": "wikipedia_fathead", 234 | "src_domain": "en.wikipedia.org", 235 | "src_id": 1, 236 | "src_name": "Wikipedia", 237 | "src_options": { 238 | "directory": "", 239 | "is_fanon": 0, 240 | "is_mediawiki": 1, 241 | "is_wikipedia": 1, 242 | "language": "en", 243 | "min_abstract_length": "20", 244 | "skip_abstract": 0, 245 | "skip_abstract_paren": 0, 246 | "skip_end": "0", 247 | "skip_icon": 0, 248 | "skip_image_name": 0, 249 | "skip_qr": "", 250 | "source_skip": "", 251 | "src_info": "" 252 | }, 253 | "src_url": null, 254 | "status": "live", 255 | "tab": "About", 256 | "topic": [ 257 | "productivity" 258 | ], 259 | "unsafe": 0 260 | } 261 | }""" 262 | icon1 = {'height': '', 263 | 'url': 'https://duckduckgo.com/i/7eec482b.png', 264 | 'width': ''} 265 | 266 | icon2 = {'height': '', 267 | 'url': 'https://duckduckgo.com/i/4eec9e83.jpg', 268 | 'width': ''} 269 | 270 | result1 = { 271 | 'first_url': 'https://duckduckgo.com/Python_(programming', 272 | 'icon': icon1, 273 | 'result': 'Mo', 280 | 'text': "Monty PythonA British surreal comedy group who"} 281 | 282 | related_topic = { 283 | 'name': "Ancient Greece", 284 | 'topics': [] 285 | } 286 | 287 | meta = { 288 | "attribution": None, 289 | "blockgroup": None, 290 | "created_date": None, 291 | "description": "Wikipedia", 292 | "designer": None, 293 | "dev_date": None, 294 | "dev_milestone": "live", 295 | "developer": [ 296 | { 297 | "name": "DDG Team", 298 | "type": "ddg", 299 | "url": "http://www.duckduckhack.com" 300 | } 301 | ], 302 | "example_query": "nikola tesla", 303 | "id": "wikipedia_fathead", 304 | "is_stackexchange": None, 305 | "js_callback_name": "wikipedia", 306 | "live_date": None, 307 | "maintainer": { 308 | "github": "duckduckgo" 309 | }, 310 | "name": "Wikipedia", 311 | "perl_module": "DDG::Fathead::Wikipedia", 312 | "producer": None, 313 | "production_state": "online", 314 | "repo": "fathead", 315 | "signal_from": "wikipedia_fathead", 316 | "src_domain": "en.wikipedia.org", 317 | "src_id": 1, 318 | "src_name": "Wikipedia", 319 | "src_options": { 320 | "directory": "", 321 | "is_fanon": 0, 322 | "is_mediawiki": 1, 323 | "is_wikipedia": 1, 324 | "language": "en", 325 | "min_abstract_length": "20", 326 | "skip_abstract": 0, 327 | "skip_abstract_paren": 0, 328 | "skip_end": "0", 329 | "skip_icon": 0, 330 | "skip_image_name": 0, 331 | "skip_qr": "", 332 | "source_skip": "", 333 | "src_info": "" 334 | }, 335 | "src_url": None, 336 | "status": "live", 337 | "tab": "About", 338 | "topic": [ 339 | "productivity" 340 | ], 341 | "unsafe": 0 342 | } 343 | response = { 344 | 'abstract': '', 345 | 'abstract_source': 'Wikipedia', 346 | 'abstract_text': '', 347 | 'abstract_url': 'https://en.wikipedia.org/wiki/Python', 348 | 'answer': '', 349 | 'answer_type': '', 350 | 'definition': '', 351 | 'definition_source': '', 352 | 'definition_url': '', 353 | 'entity': '', 354 | 'heading': 'Python', 355 | 'image': '', 356 | 'image_height': 0, 357 | 'image_is_logo': 0, 358 | 'image_width': 0, 359 | 'infobox': {}, 360 | 'redirect': '', 361 | 'related_topics': [], 362 | 'results': [], 363 | 'type': 'D', 364 | 'meta': meta} 365 | 366 | @mock.patch('json.loads') 367 | @mock.patch('duckduckpy.core.http_client.HTTPConnection') 368 | def test_http_connection_used(self, conn, *args): 369 | query('anything', secure=False) 370 | conn.assert_called_once_with(api.SERVER_HOST) 371 | 372 | @mock.patch('duckduckpy.core.http_client.HTTPConnection.getresponse', 373 | return_value=StringIO(origin)) 374 | def test_smoke_dict(self, *args): 375 | self.related_topic['topics'] = [self.result1, self.result2] 376 | self.response['related_topics'] = [ 377 | self.result1, self.result2, self.related_topic] 378 | resp = query('python', container='dict') 379 | self.assertTrue(resp == self.response) 380 | 381 | @mock.patch('duckduckpy.core.http_client.HTTPConnection.getresponse', 382 | return_value=StringIO(origin)) 383 | def test_smoke_namedtuple(self, *args): 384 | self.result1['icon'] = api.Icon(**self.icon1) 385 | self.result2['icon'] = api.Icon(**self.icon2) 386 | 387 | self.related_topic['topics'] = [ 388 | api.Result(**self.result1), 389 | api.Result(**self.result2)] 390 | 391 | self.response['related_topics'] = [ 392 | api.Result(**self.result1), 393 | api.Result(**self.result2), 394 | api.RelatedTopic(**self.related_topic)] 395 | 396 | expected = api.Response(**self.response) 397 | resp = query('python') 398 | self.assertEqual(resp, expected) 399 | 400 | @mock.patch('duckduckpy.core.http_client.HTTPConnection.getresponse', 401 | return_value=mock.Mock(read=lambda: b"{}")) 402 | def test_python3_utf8_decode(self, *args): 403 | # Not relevant to Python 2. 404 | if not is_python2(): 405 | query('python', container='dict') 406 | 407 | @mock.patch('duckduckpy.core.http_client.HTTPConnection.getresponse', 408 | return_value=StringIO("[1, \"x\", true]")) 409 | def test_json_response_as_list(self, *args): 410 | res = query('anything!') 411 | self.assertEqual(res, [1, 'x', True]) 412 | 413 | @mock.patch('duckduckpy.core.http_client.HTTPConnection.getresponse', 414 | return_value=StringIO("Not JSON")) 415 | def test_not_json_response(self, *args): 416 | self.assertRaises(exc.DuckDuckDeserializeError, query, 'anything!') 417 | 418 | 419 | class TestSecureQuery(unittest.TestCase): 420 | @mock.patch('json.loads') 421 | @mock.patch('duckduckpy.core.http_client.HTTPSConnection') 422 | def test_https_connection_used(self, conn, *args): 423 | query('anything', secure=True) 424 | conn.assert_called_once_with(api.SERVER_HOST) 425 | 426 | @mock.patch('json.loads') 427 | @mock.patch('duckduckpy.core.http_client.HTTPSConnection') 428 | def test_shortcut_https_connection_used(self, conn, *args): 429 | secure_query('anything', secure=True) 430 | conn.assert_called_once_with(api.SERVER_HOST) 431 | 432 | 433 | class TestQueryExceptions(unittest.TestCase): 434 | @mock.patch('duckduckpy.core.http_client.HTTPConnection.getresponse', 435 | side_effect=exc.DuckDuckArgumentError) 436 | def test_argument_error(self, *args): 437 | self.assertRaises( 438 | exc.DuckDuckArgumentError, query, '', container='non-existent') 439 | 440 | @mock.patch('duckduckpy.core.http_client.HTTPConnection.getresponse', 441 | side_effect=socket.gaierror) 442 | def test_connection_error(self, *args): 443 | self.assertRaises(exc.DuckDuckConnectionError, query, 'anything!') 444 | 445 | 446 | if __name__ == '__main__': 447 | unittest.main() 448 | --------------------------------------------------------------------------------