├── .gitignore ├── .travis.yml ├── AUTHORS.rst ├── CONTRIBUTING.rst ├── HISTORY.rst ├── LICENSE ├── MANIFEST.in ├── Makefile ├── README.rst ├── conftest.py ├── docs ├── Makefile ├── authors.rst ├── commands.rst ├── conf.py ├── contributing.rst ├── history.rst ├── index.rst ├── installation.rst ├── make.bat ├── mappings.rst ├── readme.rst ├── requirements.txt ├── usage.rst └── views.rst ├── elasticstack ├── __init__.py ├── apps.py ├── backends.py ├── fields.py ├── forms.py ├── management │ ├── __init__.py │ └── commands │ │ ├── __init__.py │ │ ├── show_document.py │ │ └── show_mapping.py ├── models.py ├── utils.py └── views.py ├── requirements-test.txt ├── requirements.txt ├── setup.py ├── tests ├── __init__.py ├── test_backends.py ├── test_fields.py ├── test_forms.py └── test_views.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | *.py[cod] 2 | .cache 3 | 4 | # C extensions 5 | *.so 6 | 7 | # Packages 8 | *.egg 9 | *.egg-info 10 | dist 11 | build 12 | eggs 13 | parts 14 | bin 15 | var 16 | sdist 17 | develop-eggs 18 | .installed.cfg 19 | lib 20 | lib64 21 | 22 | # Installer logs 23 | pip-log.txt 24 | 25 | # Unit test / coverage reports 26 | .coverage 27 | .tox 28 | nosetests.xml 29 | 30 | # Translations 31 | *.mo 32 | 33 | # Mr Developer 34 | .mr.developer.cfg 35 | .project 36 | .pydevproject 37 | 38 | # Complexity 39 | output/*.html 40 | output/*/index.html 41 | 42 | # Sphinx 43 | docs/_build 44 | 45 | # IDE/Editor 46 | .idea 47 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 2.7 3 | env: 4 | - TOX_ENV=flake8 5 | - TOX_ENV=py27-django18-haystack24 6 | - TOX_ENV=py27-django19-haystack24 7 | - TOX_ENV=py27-django110-haystack24 8 | - TOX_ENV=py34-django18-haystack24 9 | - TOX_ENV=py34-django19-haystack24 10 | - TOX_ENV=py34-django110-haystack24 11 | install: 12 | - pip install tox 13 | script: 14 | - tox -e $TOX_ENV 15 | sudo: false 16 | -------------------------------------------------------------------------------- /AUTHORS.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | Credits 3 | ======= 4 | 5 | Author 6 | ------ 7 | 8 | * Ben Lopatin @bennylope 9 | 10 | Contributors 11 | ------------ 12 | 13 | * Basil Shubin @bashu 14 | * Joe Stump @joestump 15 | * Mario César @mariocesar 16 | * Martin Svoboda @martinsvoboda 17 | * Mike Brown @m3brown 18 | * Seizan Shimazaki @seizans 19 | * @rjor2 20 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Contributing 3 | ============ 4 | 5 | Contributions are welcome, and they are greatly appreciated! Every 6 | little bit helps, and credit will always be given. 7 | 8 | You can contribute in many ways: 9 | 10 | Types of Contributions 11 | ---------------------- 12 | 13 | Report Bugs 14 | ~~~~~~~~~~~ 15 | 16 | Report bugs at https://github.com/bennylope/elasticstack/issues. 17 | 18 | If you are reporting a bug, please include: 19 | 20 | * Your operating system name and version. 21 | * Any details about your local setup that might be helpful in troubleshooting. 22 | * Detailed steps to reproduce the bug. 23 | 24 | Fix Bugs 25 | ~~~~~~~~ 26 | 27 | Look through the GitHub issues for bugs. Anything tagged with "bug" 28 | is open to whoever wants to implement it. 29 | 30 | Implement Features 31 | ~~~~~~~~~~~~~~~~~~ 32 | 33 | Look through the GitHub issues for features. Anything tagged with "feature" 34 | is open to whoever wants to implement it. 35 | 36 | Write Documentation 37 | ~~~~~~~~~~~~~~~~~~~ 38 | 39 | elasticstack could always use more documentation, whether as part of the 40 | official elasticstack docs, in docstrings, or even on the web in blog posts, 41 | articles, and such. 42 | 43 | Submit Feedback 44 | ~~~~~~~~~~~~~~~ 45 | 46 | The best way to send feedback is to file an issue at https://github.com/bennylope/elasticstack/issues. 47 | 48 | If you are proposing a feature: 49 | 50 | * Explain in detail how it would work. 51 | * Keep the scope as narrow as possible, to make it easier to implement. 52 | * Remember that this is a volunteer-driven project, and that contributions 53 | are welcome :) 54 | 55 | Get Started! 56 | ------------ 57 | 58 | Ready to contribute? Here's how to set up `elasticstack` for local development. 59 | 60 | 1. Fork the `elasticstack` repo on GitHub. 61 | 2. Clone your fork locally:: 62 | 63 | $ git clone git@github.com:your_name_here/elasticstack.git 64 | 65 | 3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:: 66 | 67 | $ mkvirtualenv elasticstack 68 | $ cd elasticstack/ 69 | $ python setup.py develop 70 | $ pip install -r requirements-test.txt 71 | 72 | 4. Create a branch for local development:: 73 | 74 | $ git checkout -b name-of-your-bugfix-or-feature 75 | 76 | Now you can make your changes locally. 77 | 78 | 5. When you're done making changes, check that your changes pass flake8 and the 79 | tests, including testing other Python versions with tox:: 80 | 81 | $ flake8 elasticstack tests 82 | $ tox 83 | 84 | To get flake8 and tox, just pip install them into your virtualenv. 85 | 86 | 6. Commit your changes and push your branch to GitHub:: 87 | 88 | $ git add . 89 | $ git commit -m "Your detailed description of your changes." 90 | $ git push origin name-of-your-bugfix-or-feature 91 | 92 | 7. Submit a pull request through the GitHub website. 93 | 94 | Pull Request Guidelines 95 | ----------------------- 96 | 97 | Before you submit a pull request, check that it meets these guidelines: 98 | 99 | 1. The pull request should include tests. 100 | 2. If the pull request adds functionality, the docs should be updated. Put 101 | your new functionality into a function with a docstring, and add the 102 | feature to the list in README.rst. 103 | 3. The pull request should work for Python 2.6, 2.7, and 3.3, and for PyPy. Check 104 | https://travis-ci.org/bennylope/elasticstack/pull_requests 105 | and make sure that the tests pass for all supported Python versions. 106 | 107 | Tips 108 | ---- 109 | 110 | To run a subset of tests:: 111 | 112 | $ py.test tests/test_backends.py 113 | -------------------------------------------------------------------------------- /HISTORY.rst: -------------------------------------------------------------------------------- 1 | .. :changelog: 2 | 3 | History 4 | ------- 5 | 6 | 0.5.0 (2017-03-17) 7 | ^^^^^^^^^^^^^^^^^^ 8 | 9 | * Replace deprecated option_list in commands with add_arguments method 10 | * Update Django versions in tox config and docs 11 | 12 | 0.4.1 (2016-05-05) 13 | ^^^^^^^^^^^^^^^^^^ 14 | 15 | * Fix encoding issue in installation. In at least one known environment/Python3 16 | combination an encoding issue prevented installation of the package. 17 | 18 | 0.4.0 (2016-01-28) 19 | ^^^^^^^^^^^^^^^^^^ 20 | 21 | * Allow changing search settings on an index-by-index basis 22 | 23 | 0.3.0 (2015-12-31) 24 | ^^^^^^^^^^^^^^^^^^ 25 | 26 | * Set default analyzer for ngram fields 27 | 28 | 0.2.0 (2015-09-29) 29 | ^^^^^^^^^^^^^^^^^^ 30 | 31 | * Switch to py.test 32 | * Tests against Django 1.8, 1.9 33 | * Drop pyelasticsearch requirement for installation 34 | 35 | 0.1.1 (2015-01-13) 36 | ^^^^^^^^^^^^^^^^^^ 37 | 38 | * Bug fix in show_document management command 39 | 40 | 0.1.0 (2014-11-24) 41 | ^^^^^^^^^^^^^^^^^^ 42 | 43 | * Major structural changes 44 | * Bugfix for configurable search fields 45 | 46 | 0.0.6 (2013-10-04) 47 | ^^^^^^^^^^^^^^^^^^ 48 | 49 | * Require pyelasticsearch for installation 50 | 51 | 0.0.5 (2013-09-28) 52 | ^^^^^^^^^^^^^^^^^^ 53 | 54 | * Fixed reference to old method 55 | 56 | 0.0.4 (2013-09-28) 57 | ^^^^^^^^^^^^^^^^^^ 58 | 59 | * Search form can search using specified field name 60 | * Added management command to output mapping for an individual 61 | document 62 | 63 | 0.0.3 (2013-09-28) 64 | ^^^^^^^^^^^^^^^^^^ 65 | 66 | * Added default analyzer setting 67 | 68 | 0.0.2 (2013-09-28) 69 | ^^^^^^^^^^^^^^^^^^ 70 | 71 | * Packaging fix 72 | 73 | 0.0.1 (2013-09-28) 74 | ^^^^^^^^^^^^^^^^^^ 75 | 76 | * Initial release 77 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2014, Ben Lopatin 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | 7 | Redistributions of source code must retain the above copyright notice, this 8 | list of conditions and the following disclaimer. Redistributions in binary 9 | form must reproduce the above copyright notice, this list of conditions and the 10 | following disclaimer in the documentation and/or other materials provided with 11 | the distribution 12 | 13 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 17 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 19 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 20 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 21 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 22 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include AUTHORS.rst 2 | include CONTRIBUTING.rst 3 | include HISTORY.rst 4 | include LICENSE 5 | include README.rst 6 | recursive-include elasticstack *.html *.png *.gif *js *jpg *jpeg *svg *py -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | .PHONY: clean-pyc clean-build docs clean 2 | 3 | TEST_FLAGS=--verbose 4 | COVER_FLAGS=--cov=elasticstack 5 | 6 | help: 7 | @echo "install - install all requirements including for testing" 8 | @echo "install-quite - same as install but pipes all output to /dev/null" 9 | @echo "clean - remove all artifacts" 10 | @echo "clean-build - remove build artifacts" 11 | @echo "clean-pyc - remove Python file artifacts" 12 | @echo "clean-test - remove test and coverage artifacts" 13 | @echo "clean-test-all - remove all test-related artifacts including tox" 14 | @echo "lint - check style with flake8" 15 | @echo "test - run tests quickly with the default Python" 16 | @echo "test-coverage - run tests with coverage report" 17 | @echo "test-all - run tests on every Python version with tox" 18 | @echo "check - run all necessary steps to check validity of project" 19 | @echo "release - package and upload a release" 20 | @echo "dist - package" 21 | 22 | install: 23 | pip install -r requirements-dev.txt 24 | 25 | install-quiet: 26 | pip install -r requirements-dev.txt > /dev/null 27 | 28 | clean: clean-build clean-pyc clean-test-all 29 | 30 | clean-build: 31 | @rm -rf build/ 32 | @rm -rf dist/ 33 | @rm -rf *.egg-info 34 | 35 | clean-pyc: 36 | -@find . -name '*.pyc' -follow -print0 | xargs -0 rm -f &> /dev/null 37 | -@find . -name '*.pyo' -follow -print0 | xargs -0 rm -f &> /dev/null 38 | -@find . -name '__pycache__' -type d -follow -print0 | xargs -0 rm -rf &> /dev/null 39 | 40 | clean-test: 41 | rm -rf .coverage coverage* 42 | rm -rf tests/.coverage test/coverage* 43 | rm -rf htmlcov/ 44 | 45 | clean-test-all: clean-test 46 | rm -rf .tox/ 47 | 48 | lint: 49 | flake8 elasticstack 50 | 51 | test: 52 | py.test ${TEST_FLAGS} 53 | 54 | test-coverage: clean-test 55 | -py.test ${COVER_FLAGS} ${TEST_FLAGS} 56 | @exit_code=$? 57 | @-coverage html 58 | @exit ${exit_code} 59 | 60 | test-all: 61 | tox 62 | 63 | check: clean-build clean-pyc clean-test lint test-coverage 64 | 65 | build: clean ## Create distribution files for release 66 | python setup.py sdist bdist_wheel 67 | 68 | release: build 69 | python setup.py check -r -s 70 | twine upload dist/* 71 | 72 | sdist: clean 73 | python setup.py sdist 74 | ls -l dist 75 | 76 | docs: 77 | $(MAKE) -C docs clean 78 | $(MAKE) -C docs html 79 | open docs/_build/html/index.html 80 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ############ 2 | elasticstack 3 | ############ 4 | 5 | .. image:: https://badge.fury.io/py/elasticstack.svg 6 | :target: http://badge.fury.io/py/elasticstack 7 | 8 | .. image:: https://travis-ci.org/bennylope/elasticstack.svg?branch=master 9 | :target: https://travis-ci.org/bennylope/elasticstack 10 | 11 | .. image:: https://pypip.in/d/elasticstack/badge.svg 12 | :target: https://crate.io/packages/elasticstack?version=latest 13 | 14 | :Version: 0.5.0 15 | :Author: Ben Lopatin (http://benlopatin.com) 16 | 17 | Configurable indexing and other extras for Haystack (with ElasticSearch 18 | biases). 19 | 20 | Full documentation is on `Read the Docs `_. 21 | 22 | Requirements 23 | ============ 24 | 25 | * `Django `_: tested against Django 1.8, and 1.9 26 | * `Haystack `_: tested against Haystack 2.4.0, 27 | it should work with any combination of Haystack and Django that work 28 | * `ElasticSearch `_: presumably any newish 29 | version will do, however the only version tested against so far is 0.19.x 30 | 31 | Features and goals 32 | ================== 33 | 34 | Some of these features are backend agnostic but most features have 35 | ElasticSearch in mind. 36 | 37 | For more background see the blog post `Stretching Haystack's ElasticSearch Backend `_. 38 | 39 | Global configurable index mapping 40 | --------------------------------- 41 | 42 | The search mapping provided by Haystack's ElasticSearch backend includes brief 43 | but sensible defaults for nGram analysis. You can globaly add change these settings or 44 | add your own mappings by providing a mapping dictionary using 45 | `ELASTICSEARCH_INDEX_SETTINGS` in your settings file. This example takes the 46 | default mapping and adds a synonym analyzer:: 47 | 48 | ELASTICSEARCH_INDEX_SETTINGS = { 49 | 'settings': { 50 | "analysis": { 51 | "analyzer": { 52 | "synonym_analyzer" : { 53 | "type": "custom", 54 | "tokenizer" : "standard", 55 | "filter" : ["synonym"] 56 | }, 57 | "ngram_analyzer": { 58 | "type": "custom", 59 | "tokenizer": "lowercase", 60 | "filter": ["haystack_ngram", "synonym"] 61 | }, 62 | "edgengram_analyzer": { 63 | "type": "custom", 64 | "tokenizer": "lowercase", 65 | "filter": ["haystack_edgengram"] 66 | } 67 | }, 68 | "tokenizer": { 69 | "haystack_ngram_tokenizer": { 70 | "type": "nGram", 71 | "min_gram": 3, 72 | "max_gram": 15, 73 | }, 74 | "haystack_edgengram_tokenizer": { 75 | "type": "edgeNGram", 76 | "min_gram": 2, 77 | "max_gram": 15, 78 | "side": "front" 79 | } 80 | }, 81 | "filter": { 82 | "haystack_ngram": { 83 | "type": "nGram", 84 | "min_gram": 3, 85 | "max_gram": 15 86 | }, 87 | "haystack_edgengram": { 88 | "type": "edgeNGram", 89 | "min_gram": 2, 90 | "max_gram": 15 91 | }, 92 | "synonym" : { 93 | "type" : "synonym", 94 | "ignore_case": "true", 95 | "synonyms_path" : "synonyms.txt" 96 | } 97 | } 98 | } 99 | } 100 | } 101 | 102 | The synonym filter is ready for your index, but will go unused yet. 103 | 104 | Before your new analyzer can be used you will need to change your Haystack engine and rebuild/update 105 | your index. In your `settings.py` modify `HAYSTACK_CONNECTIONS` accordingly:: 106 | 107 | HAYSTACK_CONNECTIONS = { 108 | 'default': { 109 | 'ENGINE': 'elasticstack.backends.ConfigurableElasticSearchEngine', 110 | 'URL': env_var('HAYSTACK_URL', 'http://127.0.0.1:9200/'), 111 | 'INDEX_NAME': 'haystack', 112 | }, 113 | } 114 | 115 | The default analyzer for non-nGram fields in Haystack's ElasticSearch backend 116 | is the `snowball analyzer `_. 117 | A perfectly good analyzer but not necessarily what you need. It's also language 118 | specific (English by default). 119 | 120 | Specify your analyzer with `ELASTICSEARCH_DEFAULT_ANALYZER` in your settings 121 | file:: 122 | 123 | ELASTICSEARCH_DEFAULT_ANALYZER = 'synonym_analyzer' 124 | 125 | Now all your analyzed fields, except for nGram fields, will be analyzed using 126 | `synonym_analyzer`. 127 | 128 | If you want to specify a custom search_analyzer for nGram/EdgeNgram fields, 129 | define it with the `ELASTICSEARCH_DEFAULT_NGRAM_SEARCH_ANALYZER` settings:: 130 | 131 | ELASTICSEARCH_DEFAULT_NGRAM_SEARCH_ANALYZER = 'standard' 132 | 133 | Configurable index mapping per index 134 | ------------------------------------ 135 | 136 | Alternatively you can configure index mapping per index. This is usefull for multilanguage index settup. 137 | In this case `HAYSTACK_CONNECTION` contains key `SETTINGS_NAME` have to match with name in `ELASTICSEARCH_INDEX_SETTINGS`:: 138 | 139 | 140 | HAYSTACK_CONNECTIONS = { 141 | 'default': { 142 | 'ENGINE': 'elasticstack.backends.ConfigurableElasticSearchEngine', 143 | 'URL': env_var('HAYSTACK_URL', 'http://127.0.0.1:9200/'), 144 | 'INDEX_NAME': 'haystack', 145 | 'SETTINGS_NAME': 'cs', 146 | 'DEFAULT_ANALYZER': 'czech_hunspell', 147 | 'DEFAULT_NGRAM_SEARCH_ANALYZER': 'standard', 148 | }, 149 | } 150 | 151 | ELASTICSEARCH_INDEX_SETTINGS = { 152 | 'cs': { 153 | "settings": { 154 | "analysis": { 155 | "analyzer": { 156 | "czech_hunspell": { 157 | "type": "custom", 158 | "tokenizer": "standard", 159 | "filter": ["stopwords_CZ", "lowercase", "hunspell_CZ", "stopwords_CZ", "remove_duplicities"] 160 | } 161 | }, 162 | "filter": { 163 | "stopwords_CZ": { 164 | "type": "stop", 165 | "stopwords": ["právě", "že", "test", "_czech_"], 166 | "ignore_case": True 167 | }, 168 | "hunspell_CZ": { 169 | "type": "hunspell", 170 | "locale": "cs_CZ", 171 | "dedup": True, 172 | "recursion_level": 0 173 | }, 174 | "remove_duplicities": { 175 | "type": "unique", 176 | "only_on_same_position": True 177 | }, 178 | } 179 | } 180 | } 181 | }, 182 | } 183 | 184 | 185 | Field based analysis 186 | -------------------- 187 | 188 | Even with a new default analyzer you may want to change this on a field by 189 | field basis as fits your needs. To do so, use the fields from 190 | `elasticstack.fields` to specify your analyzer with the `analyzer` keyword 191 | argument:: 192 | 193 | from haystack import indexes 194 | from elasticstack.fields import CharField 195 | from myapp.models import MyContent 196 | 197 | class MyContentIndex(indexes.SearchIndex, indexes.Indexable): 198 | text = CharField(document=True, use_template=True, 199 | analyzer='synonym_analyzer') 200 | 201 | def get_model(self): 202 | return MyContent 203 | 204 | 205 | Django CBV style views 206 | ---------------------- 207 | 208 | Haystacks's class based views predate the inclusion of CBVs into the Django 209 | core and so the paradigms are different. This makes it harder to impossible to 210 | make use of view mixins. 211 | 212 | The bundled `SearchView` and `FacetedSearchView` classes are based on 213 | `django.views.generic.edit.FormView` using the `SearchMixin` and 214 | `FacetedSearchMixin`, respectively. The `SearchMixin` provides the necessary 215 | search related attributes and overloads the form processing methods to execute 216 | the search. 217 | 218 | The `SearchMixin` adds a few search specific attributes: 219 | 220 | * `load_all` - a Boolean value for `specifying database lookups `_ 221 | * `queryset` - a default `SearchQuerySet`. Defaults to `EmtpySearchQuerySet` 222 | * `search_field` - the name of the form field used for the query. This is added 223 | to allow for views which may have more than one search form. Defaults to `q`. 224 | 225 | .. note:: 226 | The `SearchMixin` uses the attribute named `queryset` for the resultant 227 | `SearchQuerySet`. Naming this attribute `searchqueryset` would make more 228 | sense semantically and hew closer to Haystack's naming convention, however 229 | by using the `queryset` attribute shared by other Django view mixins it is 230 | relatively easy to combine search functionality with other mixins and 231 | views. 232 | 233 | Management commands 234 | ------------------- 235 | 236 | show_mapping 237 | ^^^^^^^^^^^^ 238 | 239 | Make a change and wonder why your results don't look as expected? The 240 | management command `show_mapping` will print the current mapping for 241 | your defined search index(es). At the least it may show that you've simply 242 | forgotten to update your index with new mappings:: 243 | 244 | python manage.py show_mapping 245 | 246 | By default this will display the `existing_mapping` which shows the index, 247 | document type, and document properties.:: 248 | 249 | { 250 | "haystack": { 251 | "modelresult": { 252 | "properties": { 253 | "is_active": { 254 | "type": "boolean" 255 | }, 256 | "text": { 257 | "type": "string" 258 | }, 259 | "published": { 260 | "type": "date", 261 | "format": "dateOptionalTime" 262 | } 263 | } 264 | } 265 | } 266 | } 267 | 268 | If you provide the `--detail` flag this will return only the field mappings but 269 | including additional details, such as boost levels and field-specific 270 | analyzers.:: 271 | 272 | { 273 | "is_active": { 274 | "index": "not_analyzed", 275 | "boost": 1, 276 | "store": "yes", 277 | "type": "boolean" 278 | }, 279 | "text": { 280 | "index": "analyzed", 281 | "term_vector": "with_positions_offsets", 282 | "type": "string", 283 | "analyzer": "custom_analyzer", 284 | "boost": 1, 285 | "store": "yes" 286 | }, 287 | "pub_date": { 288 | "index": "analyzed", 289 | "boost": 1, 290 | "store": "yes", 291 | "type": "date" 292 | } 293 | } 294 | 295 | show_document 296 | ^^^^^^^^^^^^^ 297 | 298 | Provided the name of an indexed model and a key it generates and prints the 299 | generated document for this object:: 300 | 301 | python manage.py show_document myapp.MyModel 19181 302 | 303 | The JSON document will be formatted with 'pretty' indenting. 304 | 305 | Stability, docs, and tests 306 | ========================== 307 | 308 | The form, view, and backend functionality in this project is considered stable. 309 | Test coverage is not substantial, but is run against Django 1.8 through Django 310 | 1.10 on Python 2.7, 3.4, and 3.5. 311 | 312 | Why not add this stuff to Haystack? 313 | ----------------------------------- 314 | 315 | This project first aims to solve problems related specifically to working with 316 | ElasticSearch. Haystack is 1) backend agnostic (a good thing), 2) needs to 317 | support existing codebases, and 3) not my project. Most importantly, adding 318 | these features through a separate Django app means providing them without 319 | needing to fork Haystack. Hopefully some of the features here, once finalized 320 | and tested, will be suitable to add to Haystack. 321 | -------------------------------------------------------------------------------- /conftest.py: -------------------------------------------------------------------------------- 1 | """ 2 | Configuration file for py.test 3 | """ 4 | 5 | import django 6 | 7 | 8 | def pytest_configure(): 9 | from django.conf import settings 10 | settings.configure( 11 | DEBUG=True, 12 | USE_TZ=True, 13 | DATABASES={ 14 | "default": { 15 | "ENGINE": "django.db.backends.sqlite3", 16 | "NAME": "test.sqlite3", 17 | } 18 | }, 19 | ROOT_URLCONF="elasticstack.urls", 20 | INSTALLED_APPS=[ 21 | "django.contrib.auth", 22 | "django.contrib.contenttypes", 23 | "django.contrib.sites", 24 | "haystack", 25 | "elasticstack", 26 | ], 27 | HAYSTACK_CONNECTIONS={ 28 | 'default': { 29 | 'ENGINE': 'elasticstack.backends.ConfigurableElasticSearchEngine', 30 | 'URL': 'http://127.0.0.1:9200/', 31 | 'INDEX_NAME': 'haystack', 32 | }, 33 | }, 34 | MIDDLEWARE_CLASSES=(), 35 | SITE_ID=1, 36 | ) 37 | try: 38 | django.setup() 39 | except AttributeError: 40 | pass 41 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # User-friendly check for sphinx-build 11 | ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) 12 | $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) 13 | endif 14 | 15 | # Internal variables. 16 | PAPEROPT_a4 = -D latex_paper_size=a4 17 | PAPEROPT_letter = -D latex_paper_size=letter 18 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 19 | # the i18n builder cannot share the environment and doctrees with the others 20 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 21 | 22 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 23 | 24 | help: 25 | @echo "Please use \`make ' where is one of" 26 | @echo " html to make standalone HTML files" 27 | @echo " dirhtml to make HTML files named index.html in directories" 28 | @echo " singlehtml to make a single large HTML file" 29 | @echo " pickle to make pickle files" 30 | @echo " json to make JSON files" 31 | @echo " htmlhelp to make HTML files and a HTML help project" 32 | @echo " qthelp to make HTML files and a qthelp project" 33 | @echo " devhelp to make HTML files and a Devhelp project" 34 | @echo " epub to make an epub" 35 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 36 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 37 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 38 | @echo " text to make text files" 39 | @echo " man to make manual pages" 40 | @echo " texinfo to make Texinfo files" 41 | @echo " info to make Texinfo files and run them through makeinfo" 42 | @echo " gettext to make PO message catalogs" 43 | @echo " changes to make an overview of all changed/added/deprecated items" 44 | @echo " xml to make Docutils-native XML files" 45 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 46 | @echo " linkcheck to check all external links for integrity" 47 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 48 | 49 | clean: 50 | rm -rf $(BUILDDIR)/* 51 | 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | dirhtml: 58 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 59 | @echo 60 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 61 | 62 | singlehtml: 63 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 64 | @echo 65 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 66 | 67 | pickle: 68 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 69 | @echo 70 | @echo "Build finished; now you can process the pickle files." 71 | 72 | json: 73 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 74 | @echo 75 | @echo "Build finished; now you can process the JSON files." 76 | 77 | htmlhelp: 78 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 79 | @echo 80 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 81 | ".hhp project file in $(BUILDDIR)/htmlhelp." 82 | 83 | qthelp: 84 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 85 | @echo 86 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 87 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 88 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/complexity.qhcp" 89 | @echo "To view the help file:" 90 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/complexity.qhc" 91 | 92 | devhelp: 93 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 94 | @echo 95 | @echo "Build finished." 96 | @echo "To view the help file:" 97 | @echo "# mkdir -p $$HOME/.local/share/devhelp/complexity" 98 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/complexity" 99 | @echo "# devhelp" 100 | 101 | epub: 102 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 103 | @echo 104 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 105 | 106 | latex: 107 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 108 | @echo 109 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 110 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 111 | "(use \`make latexpdf' here to do that automatically)." 112 | 113 | latexpdf: 114 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 115 | @echo "Running LaTeX files through pdflatex..." 116 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 117 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 118 | 119 | latexpdfja: 120 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 121 | @echo "Running LaTeX files through platex and dvipdfmx..." 122 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 123 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 124 | 125 | text: 126 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 127 | @echo 128 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 129 | 130 | man: 131 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 132 | @echo 133 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 134 | 135 | texinfo: 136 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 137 | @echo 138 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 139 | @echo "Run \`make' in that directory to run these through makeinfo" \ 140 | "(use \`make info' here to do that automatically)." 141 | 142 | info: 143 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 144 | @echo "Running Texinfo files through makeinfo..." 145 | make -C $(BUILDDIR)/texinfo info 146 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 147 | 148 | gettext: 149 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 150 | @echo 151 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 152 | 153 | changes: 154 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 155 | @echo 156 | @echo "The overview file is in $(BUILDDIR)/changes." 157 | 158 | linkcheck: 159 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 160 | @echo 161 | @echo "Link check complete; look for any errors in the above output " \ 162 | "or in $(BUILDDIR)/linkcheck/output.txt." 163 | 164 | doctest: 165 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 166 | @echo "Testing of doctests in the sources finished, look at the " \ 167 | "results in $(BUILDDIR)/doctest/output.txt." 168 | 169 | xml: 170 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 171 | @echo 172 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 173 | 174 | pseudoxml: 175 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 176 | @echo 177 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." -------------------------------------------------------------------------------- /docs/authors.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../AUTHORS.rst -------------------------------------------------------------------------------- /docs/commands.rst: -------------------------------------------------------------------------------- 1 | =================== 2 | Management commands 3 | =================== 4 | 5 | The extra management commands are small tools to help in diagnosing problems 6 | with unexpected search results, by showing you how your data is actually mapped 7 | for ElasticSearch and how a specific model instance (with a matching 8 | `SearchIndex` class) is mapped as an example. 9 | 10 | show_mapping 11 | ============ 12 | 13 | Make a change and wonder why your results don't look as expected? The 14 | management command `show_mapping` will print the current mapping for 15 | your defined search index(es). At the least it may show that you've simply 16 | forgotten to update your index with new mappings:: 17 | 18 | python manage.py show_mapping 19 | 20 | By default this will display the `existing_mapping` which shows the index, 21 | document type, and document properties.:: 22 | 23 | { 24 | "haystack": { 25 | "modelresult": { 26 | "properties": { 27 | "is_active": { 28 | "type": "boolean" 29 | }, 30 | "text": { 31 | "type": "string" 32 | }, 33 | "published": { 34 | "type": "date", 35 | "format": "dateOptionalTime" 36 | } 37 | } 38 | } 39 | } 40 | } 41 | 42 | If you provide the `--detail` flag this will return only the field mappings but 43 | including additional details, such as boost levels and field-specific 44 | analyzers.:: 45 | 46 | { 47 | "is_active": { 48 | "index": "not_analyzed", 49 | "boost": 1, 50 | "store": "yes", 51 | "type": "boolean" 52 | }, 53 | "text": { 54 | "index": "analyzed", 55 | "term_vector": "with_positions_offsets", 56 | "type": "string", 57 | "analyzer": "custom_analyzer", 58 | "boost": 1, 59 | "store": "yes" 60 | }, 61 | "pub_date": { 62 | "index": "analyzed", 63 | "boost": 1, 64 | "store": "yes", 65 | "type": "date" 66 | } 67 | } 68 | 69 | show_document 70 | ============= 71 | 72 | Provided the name of an indexed model and a key it generates and prints the 73 | generated document for this object:: 74 | 75 | python manage.py show_document myapp.MyModel 19181 76 | 77 | The JSON document will be formatted with 'pretty' indenting. 78 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # complexity documentation build configuration file, created by 4 | # sphinx-quickstart on Tue Jul 9 22:26:36 2013. 5 | # 6 | # This file is execfile()d with the current directory set to its containing dir. 7 | # 8 | # Note that not all possible configuration values are present in this 9 | # autogenerated file. 10 | # 11 | # All configuration values have a default; values that are commented out 12 | # serve to show the default. 13 | 14 | import sys, os 15 | 16 | # If extensions (or modules to document with autodoc) are in another directory, 17 | # add these directories to sys.path here. If the directory is relative to the 18 | # documentation root, use os.path.abspath to make it absolute, like shown here. 19 | #sys.path.insert(0, os.path.abspath('.')) 20 | 21 | cwd = os.getcwd() 22 | parent = os.path.dirname(cwd) 23 | sys.path.append(parent) 24 | 25 | import elasticstack 26 | 27 | # -- General configuration ----------------------------------------------------- 28 | 29 | # If your documentation needs a minimal Sphinx version, state it here. 30 | #needs_sphinx = '1.0' 31 | 32 | # Add any Sphinx extension module names here, as strings. They can be extensions 33 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 34 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] 35 | 36 | # Add any paths that contain templates here, relative to this directory. 37 | templates_path = ['_templates'] 38 | 39 | # The suffix of source filenames. 40 | source_suffix = '.rst' 41 | 42 | # The encoding of source files. 43 | #source_encoding = 'utf-8-sig' 44 | 45 | # The master toctree document. 46 | master_doc = 'index' 47 | 48 | # General information about the project. 49 | project = u'elasticstack' 50 | copyright = u'2014, Ben Lopatin' 51 | 52 | # The version info for the project you're documenting, acts as replacement for 53 | # |version| and |release|, also used in various other places throughout the 54 | # built documents. 55 | # 56 | # The short X.Y version. 57 | version = elasticstack.__version__ 58 | # The full version, including alpha/beta/rc tags. 59 | release = elasticstack.__version__ 60 | 61 | # The language for content autogenerated by Sphinx. Refer to documentation 62 | # for a list of supported languages. 63 | #language = None 64 | 65 | # There are two options for replacing |today|: either, you set today to some 66 | # non-false value, then it is used: 67 | #today = '' 68 | # Else, today_fmt is used as the format for a strftime call. 69 | #today_fmt = '%B %d, %Y' 70 | 71 | # List of patterns, relative to source directory, that match files and 72 | # directories to ignore when looking for source files. 73 | exclude_patterns = ['_build'] 74 | 75 | # The reST default role (used for this markup: `text`) to use for all documents. 76 | #default_role = None 77 | 78 | # If true, '()' will be appended to :func: etc. cross-reference text. 79 | #add_function_parentheses = True 80 | 81 | # If true, the current module name will be prepended to all description 82 | # unit titles (such as .. function::). 83 | #add_module_names = True 84 | 85 | # If true, sectionauthor and moduleauthor directives will be shown in the 86 | # output. They are ignored by default. 87 | #show_authors = False 88 | 89 | # The name of the Pygments (syntax highlighting) style to use. 90 | pygments_style = 'sphinx' 91 | 92 | # A list of ignored prefixes for module index sorting. 93 | #modindex_common_prefix = [] 94 | 95 | # If true, keep warnings as "system message" paragraphs in the built documents. 96 | #keep_warnings = False 97 | 98 | 99 | # -- Options for HTML output --------------------------------------------------- 100 | 101 | # The theme to use for HTML and HTML Help pages. See the documentation for 102 | # a list of builtin themes. 103 | html_theme = 'default' 104 | 105 | # Theme options are theme-specific and customize the look and feel of a theme 106 | # further. For a list of options available for each theme, see the 107 | # documentation. 108 | #html_theme_options = {} 109 | 110 | # Add any paths that contain custom themes here, relative to this directory. 111 | #html_theme_path = [] 112 | 113 | # The name for this set of Sphinx documents. If None, it defaults to 114 | # " v documentation". 115 | #html_title = None 116 | 117 | # A shorter title for the navigation bar. Default is the same as html_title. 118 | #html_short_title = None 119 | 120 | # The name of an image file (relative to this directory) to place at the top 121 | # of the sidebar. 122 | #html_logo = None 123 | 124 | # The name of an image file (within the static path) to use as favicon of the 125 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 126 | # pixels large. 127 | #html_favicon = None 128 | 129 | # Add any paths that contain custom static files (such as style sheets) here, 130 | # relative to this directory. They are copied after the builtin static files, 131 | # so a file named "default.css" will overwrite the builtin "default.css". 132 | html_static_path = ['_static'] 133 | 134 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 135 | # using the given strftime format. 136 | #html_last_updated_fmt = '%b %d, %Y' 137 | 138 | # If true, SmartyPants will be used to convert quotes and dashes to 139 | # typographically correct entities. 140 | #html_use_smartypants = True 141 | 142 | # Custom sidebar templates, maps document names to template names. 143 | #html_sidebars = {} 144 | 145 | # Additional templates that should be rendered to pages, maps page names to 146 | # template names. 147 | #html_additional_pages = {} 148 | 149 | # If false, no module index is generated. 150 | #html_domain_indices = True 151 | 152 | # If false, no index is generated. 153 | #html_use_index = True 154 | 155 | # If true, the index is split into individual pages for each letter. 156 | #html_split_index = False 157 | 158 | # If true, links to the reST sources are added to the pages. 159 | #html_show_sourcelink = True 160 | 161 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 162 | #html_show_sphinx = True 163 | 164 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 165 | #html_show_copyright = True 166 | 167 | # If true, an OpenSearch description file will be output, and all pages will 168 | # contain a tag referring to it. The value of this option must be the 169 | # base URL from which the finished HTML is served. 170 | #html_use_opensearch = '' 171 | 172 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 173 | #html_file_suffix = None 174 | 175 | # Output file base name for HTML help builder. 176 | htmlhelp_basename = 'elasticstackdoc' 177 | 178 | 179 | # -- Options for LaTeX output -------------------------------------------------- 180 | 181 | latex_elements = { 182 | # The paper size ('letterpaper' or 'a4paper'). 183 | #'papersize': 'letterpaper', 184 | 185 | # The font size ('10pt', '11pt' or '12pt'). 186 | #'pointsize': '10pt', 187 | 188 | # Additional stuff for the LaTeX preamble. 189 | #'preamble': '', 190 | } 191 | 192 | # Grouping the document tree into LaTeX files. List of tuples 193 | # (source start file, target name, title, author, documentclass [howto/manual]). 194 | latex_documents = [ 195 | ('index', 'elasticstack.tex', u'elasticstack Documentation', 196 | u'Ben Lopatin', 'manual'), 197 | ] 198 | 199 | # The name of an image file (relative to this directory) to place at the top of 200 | # the title page. 201 | #latex_logo = None 202 | 203 | # For "manual" documents, if this is true, then toplevel headings are parts, 204 | # not chapters. 205 | #latex_use_parts = False 206 | 207 | # If true, show page references after internal links. 208 | #latex_show_pagerefs = False 209 | 210 | # If true, show URL addresses after external links. 211 | #latex_show_urls = False 212 | 213 | # Documents to append as an appendix to all manuals. 214 | #latex_appendices = [] 215 | 216 | # If false, no module index is generated. 217 | #latex_domain_indices = True 218 | 219 | 220 | # -- Options for manual page output -------------------------------------------- 221 | 222 | # One entry per manual page. List of tuples 223 | # (source start file, name, description, authors, manual section). 224 | man_pages = [ 225 | ('index', 'elasticstack', u'elasticstack Documentation', 226 | [u'Ben Lopatin'], 1) 227 | ] 228 | 229 | # If true, show URL addresses after external links. 230 | #man_show_urls = False 231 | 232 | 233 | # -- Options for Texinfo output ------------------------------------------------ 234 | 235 | # Grouping the document tree into Texinfo files. List of tuples 236 | # (source start file, target name, title, author, 237 | # dir menu entry, description, category) 238 | texinfo_documents = [ 239 | ('index', 'elasticstack', u'elasticstack Documentation', 240 | u'Ben Lopatin', 'elasticstack', 'One line description of project.', 241 | 'Miscellaneous'), 242 | ] 243 | 244 | # Documents to append as an appendix to all manuals. 245 | #texinfo_appendices = [] 246 | 247 | # If false, no module index is generated. 248 | #texinfo_domain_indices = True 249 | 250 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 251 | #texinfo_show_urls = 'footnote' 252 | 253 | # If true, do not generate a @detailmenu in the "Top" node's menu. 254 | #texinfo_no_detailmenu = False -------------------------------------------------------------------------------- /docs/contributing.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CONTRIBUTING.rst -------------------------------------------------------------------------------- /docs/history.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../HISTORY.rst -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | elasticstack: Haystack tools for ElasticSearch 2 | ============================================== 3 | 4 | `Django `_ is the web framework for 5 | perfectionists with deadlines. 6 | 7 | `ElasticSearch `_ is a Lucene based search 8 | engine and distributed data store with a JSON interface. 9 | 10 | `Haystack `_ is the fastest 11 | way to map Django project models to a search index and search your site. 12 | 13 | elasticstack is a set of ElasticSearch specific helpers for Haystack-based 14 | projects. 15 | 16 | Contents: 17 | 18 | .. toctree:: 19 | :maxdepth: 2 20 | 21 | readme 22 | installation 23 | mappings 24 | views 25 | commands 26 | contributing 27 | authors 28 | history 29 | -------------------------------------------------------------------------------- /docs/installation.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Installation 3 | ============ 4 | 5 | Base installation 6 | ================= 7 | 8 | Installation is straightforward. With your `virtualenv 9 | `_ activated, use pip to install:: 10 | 11 | $ pip install elasticstack 12 | 13 | Then add `elasticstack` to your Django project's `INSTALLED_APPS`:: 14 | 15 | INSTALLED_APPS = ( 16 | "django.contrib.auth", 17 | "django.contrib.contenttypes", 18 | "django.contrib.sites", 19 | "haystack", 20 | "elasticstack", 21 | ), 22 | 23 | Adding the app to your `INSTALLED_APPS` is necessary to make the management 24 | commands available. 25 | 26 | Haystack connection settings 27 | ============================ 28 | 29 | In order to use the configurable ElasticSearch indexing settings you will need 30 | to make sure that you're using the project defined backend. Change this:: 31 | 32 | HAYSTACK_CONNECTIONS = { 33 | 'default': { 34 | 'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine', 35 | 'URL': 'http://127.0.0.1:9200/', 36 | 'INDEX_NAME': 'haystack', 37 | }, 38 | } 39 | 40 | To this:: 41 | 42 | HAYSTACK_CONNECTIONS = { 43 | 'default': { 44 | 'ENGINE': 'elasticstack.backends.ConfigurableElasticSearchEngine', 45 | 'URL': 'http://127.0.0.1:9200/', 46 | 'INDEX_NAME': 'haystack', 47 | }, 48 | } 49 | 50 | For a full explanation of why and how to customize your index settings, see the :doc:`/mappings` documentation. 51 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | REM Command file for Sphinx documentation 4 | 5 | if "%SPHINXBUILD%" == "" ( 6 | set SPHINXBUILD=sphinx-build 7 | ) 8 | set BUILDDIR=_build 9 | set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . 10 | set I18NSPHINXOPTS=%SPHINXOPTS% . 11 | if NOT "%PAPER%" == "" ( 12 | set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% 13 | set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% 14 | ) 15 | 16 | if "%1" == "" goto help 17 | 18 | if "%1" == "help" ( 19 | :help 20 | echo.Please use `make ^` where ^ is one of 21 | echo. html to make standalone HTML files 22 | echo. dirhtml to make HTML files named index.html in directories 23 | echo. singlehtml to make a single large HTML file 24 | echo. pickle to make pickle files 25 | echo. json to make JSON files 26 | echo. htmlhelp to make HTML files and a HTML help project 27 | echo. qthelp to make HTML files and a qthelp project 28 | echo. devhelp to make HTML files and a Devhelp project 29 | echo. epub to make an epub 30 | echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter 31 | echo. text to make text files 32 | echo. man to make manual pages 33 | echo. texinfo to make Texinfo files 34 | echo. gettext to make PO message catalogs 35 | echo. changes to make an overview over all changed/added/deprecated items 36 | echo. xml to make Docutils-native XML files 37 | echo. pseudoxml to make pseudoxml-XML files for display purposes 38 | echo. linkcheck to check all external links for integrity 39 | echo. doctest to run all doctests embedded in the documentation if enabled 40 | goto end 41 | ) 42 | 43 | if "%1" == "clean" ( 44 | for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i 45 | del /q /s %BUILDDIR%\* 46 | goto end 47 | ) 48 | 49 | 50 | %SPHINXBUILD% 2> nul 51 | if errorlevel 9009 ( 52 | echo. 53 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 54 | echo.installed, then set the SPHINXBUILD environment variable to point 55 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 56 | echo.may add the Sphinx directory to PATH. 57 | echo. 58 | echo.If you don't have Sphinx installed, grab it from 59 | echo.http://sphinx-doc.org/ 60 | exit /b 1 61 | ) 62 | 63 | if "%1" == "html" ( 64 | %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html 65 | if errorlevel 1 exit /b 1 66 | echo. 67 | echo.Build finished. The HTML pages are in %BUILDDIR%/html. 68 | goto end 69 | ) 70 | 71 | if "%1" == "dirhtml" ( 72 | %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml 73 | if errorlevel 1 exit /b 1 74 | echo. 75 | echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. 76 | goto end 77 | ) 78 | 79 | if "%1" == "singlehtml" ( 80 | %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml 81 | if errorlevel 1 exit /b 1 82 | echo. 83 | echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. 84 | goto end 85 | ) 86 | 87 | if "%1" == "pickle" ( 88 | %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle 89 | if errorlevel 1 exit /b 1 90 | echo. 91 | echo.Build finished; now you can process the pickle files. 92 | goto end 93 | ) 94 | 95 | if "%1" == "json" ( 96 | %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json 97 | if errorlevel 1 exit /b 1 98 | echo. 99 | echo.Build finished; now you can process the JSON files. 100 | goto end 101 | ) 102 | 103 | if "%1" == "htmlhelp" ( 104 | %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp 105 | if errorlevel 1 exit /b 1 106 | echo. 107 | echo.Build finished; now you can run HTML Help Workshop with the ^ 108 | .hhp project file in %BUILDDIR%/htmlhelp. 109 | goto end 110 | ) 111 | 112 | if "%1" == "qthelp" ( 113 | %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp 114 | if errorlevel 1 exit /b 1 115 | echo. 116 | echo.Build finished; now you can run "qcollectiongenerator" with the ^ 117 | .qhcp project file in %BUILDDIR%/qthelp, like this: 118 | echo.^> qcollectiongenerator %BUILDDIR%\qthelp\complexity.qhcp 119 | echo.To view the help file: 120 | echo.^> assistant -collectionFile %BUILDDIR%\qthelp\complexity.ghc 121 | goto end 122 | ) 123 | 124 | if "%1" == "devhelp" ( 125 | %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp 126 | if errorlevel 1 exit /b 1 127 | echo. 128 | echo.Build finished. 129 | goto end 130 | ) 131 | 132 | if "%1" == "epub" ( 133 | %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub 134 | if errorlevel 1 exit /b 1 135 | echo. 136 | echo.Build finished. The epub file is in %BUILDDIR%/epub. 137 | goto end 138 | ) 139 | 140 | if "%1" == "latex" ( 141 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 142 | if errorlevel 1 exit /b 1 143 | echo. 144 | echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. 145 | goto end 146 | ) 147 | 148 | if "%1" == "latexpdf" ( 149 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 150 | cd %BUILDDIR%/latex 151 | make all-pdf 152 | cd %BUILDDIR%/.. 153 | echo. 154 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 155 | goto end 156 | ) 157 | 158 | if "%1" == "latexpdfja" ( 159 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 160 | cd %BUILDDIR%/latex 161 | make all-pdf-ja 162 | cd %BUILDDIR%/.. 163 | echo. 164 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 165 | goto end 166 | ) 167 | 168 | if "%1" == "text" ( 169 | %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text 170 | if errorlevel 1 exit /b 1 171 | echo. 172 | echo.Build finished. The text files are in %BUILDDIR%/text. 173 | goto end 174 | ) 175 | 176 | if "%1" == "man" ( 177 | %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man 178 | if errorlevel 1 exit /b 1 179 | echo. 180 | echo.Build finished. The manual pages are in %BUILDDIR%/man. 181 | goto end 182 | ) 183 | 184 | if "%1" == "texinfo" ( 185 | %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo 186 | if errorlevel 1 exit /b 1 187 | echo. 188 | echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. 189 | goto end 190 | ) 191 | 192 | if "%1" == "gettext" ( 193 | %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale 194 | if errorlevel 1 exit /b 1 195 | echo. 196 | echo.Build finished. The message catalogs are in %BUILDDIR%/locale. 197 | goto end 198 | ) 199 | 200 | if "%1" == "changes" ( 201 | %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes 202 | if errorlevel 1 exit /b 1 203 | echo. 204 | echo.The overview file is in %BUILDDIR%/changes. 205 | goto end 206 | ) 207 | 208 | if "%1" == "linkcheck" ( 209 | %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck 210 | if errorlevel 1 exit /b 1 211 | echo. 212 | echo.Link check complete; look for any errors in the above output ^ 213 | or in %BUILDDIR%/linkcheck/output.txt. 214 | goto end 215 | ) 216 | 217 | if "%1" == "doctest" ( 218 | %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest 219 | if errorlevel 1 exit /b 1 220 | echo. 221 | echo.Testing of doctests in the sources finished, look at the ^ 222 | results in %BUILDDIR%/doctest/output.txt. 223 | goto end 224 | ) 225 | 226 | if "%1" == "xml" ( 227 | %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml 228 | if errorlevel 1 exit /b 1 229 | echo. 230 | echo.Build finished. The XML files are in %BUILDDIR%/xml. 231 | goto end 232 | ) 233 | 234 | if "%1" == "pseudoxml" ( 235 | %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml 236 | if errorlevel 1 exit /b 1 237 | echo. 238 | echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. 239 | goto end 240 | ) 241 | 242 | :end -------------------------------------------------------------------------------- /docs/mappings.rst: -------------------------------------------------------------------------------- 1 | ========================== 2 | Configurable index mapping 3 | ========================== 4 | 5 | ElasticSearch gives you fine grained control over how your indexed content is 6 | analyzed, from choosing between `built-in analyzers 7 | `_, 8 | choosing options for built-in analyzers, and creating your own from existing 9 | tokenizers and filters. 10 | 11 | .. note:: 12 | An analyzer is a combination of a tokenizer and one or more text filters. 13 | The tokenizer is responsible for breaking apart the text into individual 14 | "tokens", which could be words or pieces of words. The filters are 15 | responsible for transforming and removing tokens from the indexed content, 16 | e.g. making all text lowercase, removing common words, indexing synonyms, 17 | etc. 18 | 19 | The default ElasticSearch backend in Haystack doesn't expose any of this 20 | configuration however. The search mapping provided by this backend maps 21 | non-nGram text fields to the `snowball analyzer 22 | `_. 23 | This is a pretty good default for English, but may not meet your requirements 24 | and won't work well for non-English languages. 25 | 26 | The elasticstack backend takes advantage of the Haystack backend's structure to 27 | make it relatively simple to override and extend the search mapping in your 28 | project. 29 | 30 | elasticstack lets you manage your index mapping in three ways: 31 | 32 | 1. Changing the default analyzer 33 | 2. Specifying an analyzer for an individual `SearchIndex` field 34 | 3. Specifying a complete search mapping including custom analyzers 35 | 36 | Haystack configuration 37 | ====================== 38 | 39 | First, you'll need to ensure that you're using the elasticstack backend, not 40 | Haystack's. Your `HAYSTACK_CONNECTIONS` should look something like this, so 41 | that the `ENGINE` value for any defined search index is using the elasticstack 42 | search engine class.:: 43 | 44 | HAYSTACK_CONNECTIONS = { 45 | 'default': { 46 | 'ENGINE': 'elasticstack.backends.ConfigurableElasticSearchEngine', 47 | 'URL': 'http://127.0.0.1:9200/', 48 | 'INDEX_NAME': 'haystack', 49 | }, 50 | } 51 | 52 | And of course make sure you've followed the instructions for `installing 53 | Haystack `_ and 54 | your ElasticSearch instance. 55 | 56 | .. important:: 57 | All of the options described here depend on this configurable search engine 58 | backend. 59 | 60 | 61 | Chaning the default analyzer 62 | ============================ 63 | 64 | Haystack will map the `snowball` analyzer to non-nGram text content by default. 65 | 66 | You can specify an alternate analyzer using the 67 | `ELASTICSEARCH_DEFAULT_ANALYZER` setting in your `settings.py` file:: 68 | 69 | ELASTICSEARCH_DEFAULT_ANALYZER = 'stop' 70 | 71 | Any field that would have been analyzed with the `snowball` analyzer will now 72 | use the `stop 73 | `_ 74 | analyzer. 75 | 76 | 77 | Choosing a field-specific analyzer 78 | ================================== 79 | 80 | Even with a new default analyzer you may want to change this on a field by 81 | field basis as fits your needs. To do so, use the fields from 82 | `elasticstack.fields` to specify your analyzer with the `analyzer` keyword 83 | argument:: 84 | 85 | from haystack import indexes 86 | from haystack.fields import CharField as BaseCharField 87 | from elasticstack.fields import CharField 88 | from myapp.models import MyContent 89 | 90 | class MyContentIndex(indexes.SearchIndex, indexes.Indexable): 91 | text = CharField(document=True, use_template=True, 92 | analyzer='stop') 93 | body = BaseCharField(use_template=True) 94 | 95 | def get_model(self): 96 | return MyContent 97 | 98 | Now the `text` field will be indexed using the `stop` analyzer, and the `body` 99 | field will be indexed using the default analyzer. 100 | 101 | .. attention:: 102 | 103 | Using a configurable field without specifying an analyzer will raise a 104 | `ValueError`. 105 | 106 | 107 | Custom analyzers and additional configuration 108 | ============================================= 109 | 110 | If instead you need to configure an analyzer, define your own, or in any way 111 | further customize the search mapping, you can customize the base `analysis 112 | settings 113 | `_ 114 | for your index. 115 | 116 | You do this by creating a dictionary of analysis settings in your `settings.py` 117 | file for the `ELASTICSEARCH_INDEX_SETTINGS` setting. 118 | 119 | This example takes the default mapping and adds a synonym analyzer. 120 | 121 | .. code-block:: python 122 | :linenos: 123 | 124 | ELASTICSEARCH_INDEX_SETTINGS = { 125 | 'settings': { 126 | "analysis": { 127 | "analyzer": { 128 | "synonym_analyzer" : { 129 | "type": "custom", 130 | "tokenizer" : "standard", 131 | "filter" : ["synonym"] 132 | }, 133 | "ngram_analyzer": { 134 | "type": "custom", 135 | "tokenizer": "lowercase", 136 | "filter": ["haystack_ngram", "synonym"] 137 | }, 138 | "edgengram_analyzer": { 139 | "type": "custom", 140 | "tokenizer": "lowercase", 141 | "filter": ["haystack_edgengram"] 142 | } 143 | }, 144 | "tokenizer": { 145 | "haystack_ngram_tokenizer": { 146 | "type": "nGram", 147 | "min_gram": 3, 148 | "max_gram": 15, 149 | }, 150 | "haystack_edgengram_tokenizer": { 151 | "type": "edgeNGram", 152 | "min_gram": 2, 153 | "max_gram": 15, 154 | "side": "front" 155 | } 156 | }, 157 | "filter": { 158 | "haystack_ngram": { 159 | "type": "nGram", 160 | "min_gram": 3, 161 | "max_gram": 15 162 | }, 163 | "haystack_edgengram": { 164 | "type": "edgeNGram", 165 | "min_gram": 2, 166 | "max_gram": 15 167 | }, 168 | "synonym" : { 169 | "type" : "synonym", 170 | "ignore_case": "true", 171 | "synonyms_path" : "synonyms.txt" 172 | } 173 | } 174 | } 175 | } 176 | } 177 | 178 | The two additions to this mapping are the `synonym_analyzer` at line 5 and the 179 | `synonym` filter at line 45. 180 | 181 | Adding this mapping in and of itself does nothing more than make your new 182 | analyzer available. To use it you either need to change your 183 | `ELASTICSEARCH_DEFAULT_ANALYZER` or specify the analyzer in the search index field. 184 | 185 | 186 | Custom analyzers and index settings per index 187 | ============================================= 188 | 189 | Global configurable index mapping is great when all your indexes share same configuration. 190 | In case of multiple language index configuration you need set settings per index. 191 | In following we show how to configure application for two language separated indexes (czech and italian):: 192 | 193 | 194 | HAYSTACK_CONNECTIONS = { 195 | 'default': { 196 | 'ENGINE': 'elasticstack.backends.ConfigurableElasticSearchEngine', 197 | 'URL': 'http://127.0.0.1:9200/', 198 | 'INDEX_NAME': 'default', 199 | 'SETTINGS_NAME': 'default', 200 | 'DEFAULT_ANALYZER': 'snowball', 201 | }, 202 | 'default_cs': { 203 | 'ENGINE': 'elasticstack.backends.ConfigurableElasticSearchEngine', 204 | 'URL': 'http://127.0.0.1:9200/', 205 | 'INDEX_NAME': 'default_cs', 206 | 'SETTINGS_NAME': 'cs', 207 | 'DEFAULT_ANALYZER': 'czech_hunspell', 208 | }, 209 | 'default_it': { 210 | 'ENGINE': 'elasticstack.backends.ConfigurableElasticSearchEngine', 211 | 'URL': 'http://127.0.0.1:9200/', 212 | 'INDEX_NAME': 'default_it', 213 | 'SETTINGS_NAME': 'default', 214 | 'DEFAULT_ANALYZER': 'italian', 215 | }, 216 | } 217 | 218 | ELASTICSEARCH_INDEX_SETTINGS = { 219 | 'cs': { 220 | "settings": { 221 | "analysis": { 222 | "analyzer": { 223 | "czech_hunspell": { 224 | "type": "custom", 225 | "tokenizer": "standard", 226 | "filter": ["stopwords_CZ", "lowercase", "hunspell_CZ", "stopwords_CZ", "remove_duplicities"] 227 | } 228 | }, 229 | "filter": { 230 | "stopwords_CZ": { 231 | "type": "stop", 232 | "stopwords": ["_czech_"], 233 | "ignore_case": True 234 | }, 235 | "hunspell_CZ": { 236 | "type": "hunspell", 237 | "locale": "cs_CZ", 238 | "dedup": True, 239 | "recursion_level": 0 240 | }, 241 | "remove_duplicities": { 242 | "type": "unique", 243 | "only_on_same_position": True 244 | }, 245 | } 246 | } 247 | } 248 | }, 249 | } 250 | 251 | .. note:: 252 | Czech configures hunspell dictionary. For this example you need to 253 | `install it `_ 254 | 255 | 256 | Realizing custom changes 257 | ======================== 258 | 259 | Even with all of these changes you won't notice any difference in your queries 260 | until you've reindexed your content. The mappings for your search index define 261 | how that content is handled when it goes into the index; it does nothing for 262 | content already there. 263 | -------------------------------------------------------------------------------- /docs/readme.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../README.rst -------------------------------------------------------------------------------- /docs/requirements.txt: -------------------------------------------------------------------------------- 1 | Sphinx>=1.2.0 2 | -------------------------------------------------------------------------------- /docs/usage.rst: -------------------------------------------------------------------------------- 1 | ======== 2 | Usage 3 | ======== 4 | 5 | To use elasticstack in a project:: 6 | 7 | import elasticstack -------------------------------------------------------------------------------- /docs/views.rst: -------------------------------------------------------------------------------- 1 | ====================== 2 | Django CBV style views 3 | ====================== 4 | 5 | Haystacks's class based views predate the inclusion of CBVs into the Django 6 | core and so the paradigms are different. This makes it harder to impossible to 7 | make use of view mixins. 8 | 9 | The bundled `SearchView` and `FacetedSearchView` classes are based on 10 | `django.views.generic.edit.FormView` using the `SearchMixin` and 11 | `FacetedSearchMixin`, respectively. The `SearchMixin` provides the necessary 12 | search related attributes and overloads the form processing methods to execute 13 | the search. 14 | 15 | The `SearchMixin` adds a few search specific attributes: 16 | 17 | * `load_all` - a Boolean value for `specifying database lookups `_ 18 | * `queryset` - a default `SearchQuerySet`. Defaults to `EmtpySearchQuerySet` 19 | * `search_field` - the name of the form field used for the query. This is added 20 | to allow for views which may have more than one search form. Defaults to `q`. 21 | 22 | .. note:: 23 | The `SearchMixin` uses the attribute named `queryset` for the resultant 24 | `SearchQuerySet`. Naming this attribute `searchqueryset` would make more 25 | sense semantically and hew closer to Haystack's naming convention, however 26 | by using the `queryset` attribute shared by other Django view mixins it is 27 | relatively easy to combine search functionality with other mixins and 28 | views. 29 | -------------------------------------------------------------------------------- /elasticstack/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # Copyright (c) 2014-2015, Ben Lopatin 4 | # All rights reserved. 5 | 6 | # Redistribution and use in source and binary forms, with or without 7 | # modification, are permitted provided that the following conditions are met: 8 | 9 | # Redistributions of source code must retain the above copyright notice, this 10 | # list of conditions and the following disclaimer. Redistributions in binary 11 | # form must reproduce the above copyright notice, this list of conditions and the 12 | # following disclaimer in the documentation and/or other materials provided with 13 | # the distribution 14 | 15 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 16 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 19 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 21 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 22 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 23 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 24 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 25 | 26 | __title__ = "elasticstack" 27 | __author__ = "Ben Lopatin" 28 | __version__ = "0.5.0" 29 | 30 | default_app_config = "elasticstack.apps.ElasticstackConfig" 31 | -------------------------------------------------------------------------------- /elasticstack/apps.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | # Copyright (c) 2014-2015, Ben Lopatin 4 | # All rights reserved. 5 | 6 | # Redistribution and use in source and binary forms, with or without 7 | # modification, are permitted provided that the following conditions are met: 8 | 9 | # Redistributions of source code must retain the above copyright notice, this 10 | # list of conditions and the following disclaimer. Redistributions in binary 11 | # form must reproduce the above copyright notice, this list of conditions and the 12 | # following disclaimer in the documentation and/or other materials provided with 13 | # the distribution 14 | 15 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 16 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 17 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 18 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 19 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 21 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 22 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 23 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 24 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 25 | 26 | from django.apps import AppConfig 27 | 28 | 29 | class ElasticstackConfig(AppConfig): 30 | name = "elasticstack" 31 | verbose_name = "elasticstack" 32 | -------------------------------------------------------------------------------- /elasticstack/backends.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2014-2015, Ben Lopatin 2 | # All rights reserved. 3 | 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are met: 6 | 7 | # Redistributions of source code must retain the above copyright notice, this 8 | # list of conditions and the following disclaimer. Redistributions in binary 9 | # form must reproduce the above copyright notice, this list of conditions and the 10 | # following disclaimer in the documentation and/or other materials provided with 11 | # the distribution 12 | 13 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 17 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 19 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 20 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 21 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 22 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | 24 | from django.conf import settings 25 | from django.core.exceptions import ImproperlyConfigured 26 | from haystack.backends.elasticsearch_backend import ( 27 | ElasticsearchSearchBackend, ElasticsearchSearchEngine 28 | ) 29 | 30 | 31 | class ConfigurableElasticBackend(ElasticsearchSearchBackend): 32 | """ 33 | Extends the Haystack ElasticSearch backend to allow configuration of index 34 | mappings and field-by-field analyzers. 35 | """ 36 | DEFAULT_ANALYZER = "snowball" 37 | DEFAULT_NGRAM_SEARCH_ANALYZER = None 38 | 39 | def __init__(self, connection_alias, **connection_options): # noqa 40 | super(ConfigurableElasticBackend, self).__init__( 41 | connection_alias, **connection_options 42 | ) 43 | 44 | # user index settings 45 | 46 | global_settings_dict = getattr(settings, "ELASTICSEARCH_INDEX_SETTINGS", None) 47 | if global_settings_dict: 48 | if ( 49 | "settings" in global_settings_dict 50 | and "SETTINGS_NAME" in connection_options 51 | ): 52 | raise ImproperlyConfigured( 53 | "You cannot specify ELASTICSEARCH_INDEX_SETTINGS['settings'] in settings " 54 | "and also 'SETTINGS_NAME' in your index connection '%s'. " 55 | "Use only one configuration way." % connection_alias 56 | ) 57 | 58 | user_settings = None 59 | if "settings" in global_settings_dict: 60 | user_settings = getattr(settings, "ELASTICSEARCH_INDEX_SETTINGS", None) 61 | if "SETTINGS_NAME" in connection_options: 62 | settings_name = connection_options.get("SETTINGS_NAME", None) 63 | if settings_name not in global_settings_dict: 64 | raise ImproperlyConfigured( 65 | "'SETTINGS_NAME' '%s' is missing in ELASTICSEARCH_INDEX_SETTINGS dict." 66 | % settings_name 67 | ) 68 | 69 | user_settings = global_settings_dict.get(settings_name) 70 | 71 | if user_settings: 72 | setattr(self, "DEFAULT_SETTINGS", user_settings) 73 | 74 | # user settings of analyzers 75 | 76 | if ( 77 | hasattr(settings, "ELASTICSEARCH_DEFAULT_ANALYZER") 78 | and "DEFAULT_ANALYZER" in connection_options 79 | ): 80 | raise ImproperlyConfigured( 81 | "You cannot specify ELASTICSEARCH_DEFAULT_ANALYZER in settings " 82 | "and also 'DEFAULT_ANALYZER' in your index connection '%s'. " 83 | "Use only one configuration way." % connection_alias 84 | ) 85 | 86 | if ( 87 | hasattr(settings, "ELASTICSEARCH_DEFAULT_NGRAM_SEARCH_ANALYZER") 88 | and "DEFAULT_NGRAM_SEARCH_ANALYZER" in connection_options 89 | ): 90 | raise ImproperlyConfigured( 91 | "You cannot specify ELASTICSEARCH_DEFAULT_NGRAM_SEARCH_ANALYZER in settings " 92 | "and also 'DEFAULT_NGRAM_SEARCH_ANALYZER' in your index connection '%s'. " 93 | "Use only one configuration way." % connection_alias 94 | ) 95 | 96 | user_analyzer = getattr( 97 | settings, "ELASTICSEARCH_DEFAULT_ANALYZER", None 98 | ) or connection_options.get( 99 | "DEFAULT_ANALYZER", None 100 | ) 101 | ngram_search_analyzer = getattr( 102 | settings, "ELASTICSEARCH_DEFAULT_NGRAM_SEARCH_ANALYZER", None 103 | ) or connection_options.get( 104 | "DEFAULT_NGRAM_SEARCH_ANALYZER", None 105 | ) 106 | if user_analyzer: 107 | setattr(self, "DEFAULT_ANALYZER", user_analyzer) 108 | if ngram_search_analyzer: 109 | setattr(self, "DEFAULT_NGRAM_SEARCH_ANALYZER", ngram_search_analyzer) 110 | 111 | def build_schema(self, fields): 112 | content_field_name, mapping = super( 113 | ConfigurableElasticBackend, self 114 | ).build_schema( 115 | fields 116 | ) 117 | 118 | for field_name, field_class in fields.items(): 119 | field_mapping = mapping[field_class.index_fieldname] 120 | 121 | if field_mapping["type"] == "string" and field_class.indexed: 122 | if ( 123 | not hasattr(field_class, "facet_for") 124 | and not field_class.field_type in ("ngram", "edge_ngram") 125 | ): 126 | field_mapping["analyzer"] = getattr( 127 | field_class, "analyzer", self.DEFAULT_ANALYZER 128 | ) 129 | if ( 130 | not hasattr(field_class, "facet_for") 131 | and field_class.field_type in ("ngram", "edge_ngram") 132 | and self.DEFAULT_NGRAM_SEARCH_ANALYZER 133 | ): 134 | field_mapping["search_analyzer"] = getattr( 135 | field_class, 136 | "search_analyzer", 137 | self.DEFAULT_NGRAM_SEARCH_ANALYZER, 138 | ) 139 | mapping.update({field_class.index_fieldname: field_mapping}) 140 | return (content_field_name, mapping) 141 | 142 | 143 | class ConfigurableElasticSearchEngine(ElasticsearchSearchEngine): 144 | backend = ConfigurableElasticBackend 145 | -------------------------------------------------------------------------------- /elasticstack/fields.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2014-2015, Ben Lopatin 2 | # All rights reserved. 3 | 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are met: 6 | 7 | # Redistributions of source code must retain the above copyright notice, this 8 | # list of conditions and the following disclaimer. Redistributions in binary 9 | # form must reproduce the above copyright notice, this list of conditions and the 10 | # following disclaimer in the documentation and/or other materials provided with 11 | # the distribution 12 | 13 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 17 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 19 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 20 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 21 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 22 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | 24 | from haystack.fields import ( 25 | CharField as BaseCharField, 26 | LocationField as BaseLocationField, 27 | NgramField as BaseNgramField, 28 | EdgeNgramField as BaseEdgeNgramField, 29 | IntegerField as BaseIntegerField, 30 | FloatField as BaseFloatField, 31 | DecimalField as BaseDecimalField, 32 | BooleanField as BaseBooleanField, 33 | DateField as BaseDateField, 34 | DateTimeField as BaseDateTimeField, 35 | MultiValueField as BaseMultiValueField, 36 | FacetField as BaseFacetField, 37 | ) 38 | 39 | 40 | class ConfigurableFieldMixin(object): 41 | """ 42 | A mixin which allows specifying the analyzer on a per field basis. 43 | """ 44 | 45 | def __init__(self, **kwargs): 46 | self.analyzer = kwargs.pop("analyzer", None) 47 | self.search_analyzer = kwargs.pop("search_analyzer", None) 48 | if self.analyzer is None: 49 | raise ValueError("Configurable fields must have an analyzer type") 50 | 51 | super(ConfigurableFieldMixin, self).__init__(**kwargs) 52 | 53 | 54 | class CharField(ConfigurableFieldMixin, BaseCharField): 55 | pass 56 | 57 | 58 | class LocationField(ConfigurableFieldMixin, BaseLocationField): 59 | pass 60 | 61 | 62 | class NgramField(ConfigurableFieldMixin, BaseNgramField): 63 | pass 64 | 65 | 66 | class EdgeNgramField(ConfigurableFieldMixin, BaseEdgeNgramField): 67 | pass 68 | 69 | 70 | class IntegerField(ConfigurableFieldMixin, BaseIntegerField): 71 | pass 72 | 73 | 74 | class FloatField(ConfigurableFieldMixin, BaseFloatField): 75 | pass 76 | 77 | 78 | class DecimalField(ConfigurableFieldMixin, BaseDecimalField): 79 | pass 80 | 81 | 82 | class BooleanField(ConfigurableFieldMixin, BaseBooleanField): 83 | pass 84 | 85 | 86 | class DateField(ConfigurableFieldMixin, BaseDateField): 87 | pass 88 | 89 | 90 | class DateTimeField(ConfigurableFieldMixin, BaseDateTimeField): 91 | pass 92 | 93 | 94 | class MultiValueField(ConfigurableFieldMixin, BaseMultiValueField): 95 | pass 96 | 97 | 98 | class FacetField(ConfigurableFieldMixin, BaseFacetField): 99 | pass 100 | 101 | 102 | class FacetCharField(FacetField, CharField): 103 | pass 104 | 105 | 106 | class FacetIntegerField(FacetField, IntegerField): 107 | pass 108 | 109 | 110 | class FacetFloatField(FacetField, FloatField): 111 | pass 112 | 113 | 114 | class FacetDecimalField(FacetField, DecimalField): 115 | pass 116 | 117 | 118 | class FacetBooleanField(FacetField, BooleanField): 119 | pass 120 | 121 | 122 | class FacetDateField(FacetField, DateField): 123 | pass 124 | 125 | 126 | class FacetDateTimeField(FacetField, DateTimeField): 127 | pass 128 | 129 | 130 | class FacetMultiValueField(FacetField, MultiValueField): 131 | pass 132 | -------------------------------------------------------------------------------- /elasticstack/forms.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2014-2015, Ben Lopatin 2 | # All rights reserved. 3 | 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are met: 6 | 7 | # Redistributions of source code must retain the above copyright notice, this 8 | # list of conditions and the following disclaimer. Redistributions in binary 9 | # form must reproduce the above copyright notice, this list of conditions and the 10 | # following disclaimer in the documentation and/or other materials provided with 11 | # the distribution 12 | 13 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 17 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 19 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 20 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 21 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 22 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | 24 | from django import forms 25 | from django.utils.translation import ugettext_lazy as _ 26 | 27 | from haystack.query import SearchQuerySet 28 | 29 | 30 | class SearchForm(forms.Form): 31 | """ 32 | A search form that does not require the use of a specifically named `q` 33 | field for search. 34 | 35 | Another field can be substituted provided that it is identified using the 36 | `search_field_name` attribute. 37 | """ 38 | q = forms.CharField(label=_("Search")) 39 | 40 | search_field_name = "q" 41 | 42 | def __init__(self, *args, **kwargs): 43 | self.searchqueryset = kwargs.pop("searchqueryset", SearchQuerySet()) 44 | self.load_all = kwargs.pop("load_all", False) 45 | super(SearchForm, self).__init__(*args, **kwargs) 46 | if self.search_field_name != "q": 47 | self.fields.pop("q") 48 | 49 | def search(self): 50 | if not self.is_valid(): 51 | return self.no_query_found() 52 | 53 | if not self.cleaned_data.get(self.search_field_name): 54 | return self.no_query_found() 55 | 56 | sqs = self.searchqueryset.auto_query(self.cleaned_data[self.search_field_name]) 57 | 58 | if self.load_all: 59 | sqs = sqs.load_all() 60 | 61 | return sqs 62 | 63 | def get_suggestion(self): 64 | if not self.is_valid(): 65 | return None 66 | 67 | return self.searchqueryset.spelling_suggestion( 68 | self.cleaned_data[self.search_field_name] 69 | ) 70 | -------------------------------------------------------------------------------- /elasticstack/management/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bennylope/elasticstack/8d1e99489815de6346fbf6720c9df5176546288e/elasticstack/management/__init__.py -------------------------------------------------------------------------------- /elasticstack/management/commands/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bennylope/elasticstack/8d1e99489815de6346fbf6720c9df5176546288e/elasticstack/management/commands/__init__.py -------------------------------------------------------------------------------- /elasticstack/management/commands/show_document.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2014-2015, Ben Lopatin 2 | # All rights reserved. 3 | 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are met: 6 | 7 | # Redistributions of source code must retain the above copyright notice, this 8 | # list of conditions and the following disclaimer. Redistributions in binary 9 | # form must reproduce the above copyright notice, this list of conditions and the 10 | # following disclaimer in the documentation and/or other materials provided with 11 | # the distribution 12 | 13 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 17 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 19 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 20 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 21 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 22 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | 24 | import json 25 | 26 | from django.core.management.base import BaseCommand 27 | 28 | from ...utils import prepare_object, get_model 29 | 30 | 31 | class Command(BaseCommand): 32 | 33 | help = "Prints the indexing document generated for a model object." "\nUse dotted path name for model and the primary key." 34 | 35 | def add_arguments(self, parser): 36 | parser.add_argument( 37 | "--using", 38 | action="store", 39 | dest="using", 40 | default="default", 41 | help="The Haystack backend to use", 42 | ) 43 | 44 | def handle(self, *args, **options): 45 | try: 46 | label, pk = args 47 | except (IndexError, ValueError): 48 | self.stderr.write("Provide the model name and primary key") 49 | exit(1) 50 | 51 | app_label, model_name = label.split(".") 52 | model = get_model(app_label, model_name) 53 | 54 | obj = model.objects.get(pk=pk) 55 | doc_json = prepare_object(obj, options.get("using")) 56 | self.stdout.write(json.dumps(doc_json, indent=4)) 57 | -------------------------------------------------------------------------------- /elasticstack/management/commands/show_mapping.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2014-2015, Ben Lopatin 2 | # All rights reserved. 3 | 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are met: 6 | 7 | # Redistributions of source code must retain the above copyright notice, this 8 | # list of conditions and the following disclaimer. Redistributions in binary 9 | # form must reproduce the above copyright notice, this list of conditions and the 10 | # following disclaimer in the documentation and/or other materials provided with 11 | # the distribution 12 | 13 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 17 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 19 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 20 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 21 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 22 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | 24 | import json 25 | 26 | from django.conf import settings 27 | from django.core.management.base import BaseCommand 28 | 29 | from haystack import connections 30 | 31 | 32 | class Command(BaseCommand): 33 | 34 | help = "Prints the search mapping for specififed connections." "\nDefaults to all connections in settings." 35 | 36 | def add_arguments(self, parser): 37 | parser.add_argument( 38 | "--detail", 39 | action="store_true", 40 | dest="detail", 41 | default=False, 42 | help="Display mapping details, including analyzers and boost levels.", 43 | ) 44 | 45 | def handle(self, *args, **options): 46 | backends = args if args else settings.HAYSTACK_CONNECTIONS.keys() 47 | for backend in backends: 48 | engine = connections[backend].get_backend() 49 | unified_index = connections[backend].get_unified_index() 50 | content_field_name, field_mapping = engine.build_schema( 51 | unified_index.all_searchfields() 52 | ) 53 | engine.setup() 54 | 55 | if options.get("detail"): 56 | mapping = field_mapping 57 | else: 58 | mapping = engine.existing_mapping 59 | 60 | self.stdout.write("{0}\n{1}\n".format(backend, "-" * len(backend))) 61 | self.stdout.write(json.dumps(mapping, indent=4)) 62 | -------------------------------------------------------------------------------- /elasticstack/models.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bennylope/elasticstack/8d1e99489815de6346fbf6720c9df5176546288e/elasticstack/models.py -------------------------------------------------------------------------------- /elasticstack/utils.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2014-2015, Ben Lopatin 2 | # All rights reserved. 3 | 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are met: 6 | 7 | # Redistributions of source code must retain the above copyright notice, this 8 | # list of conditions and the following disclaimer. Redistributions in binary 9 | # form must reproduce the above copyright notice, this list of conditions and the 10 | # following disclaimer in the documentation and/or other materials provided with 11 | # the distribution 12 | 13 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 17 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 19 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 20 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 21 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 22 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | 24 | from haystack import connections 25 | from importlib import import_module 26 | 27 | 28 | def prepare_object(obj, using="default"): 29 | """ 30 | Returns a Python dictionary representation of the given object, expected to 31 | be a Model object with an associated SearchIndex. The optional argument 32 | `using` specifies the backend to use from the Haystack connections list. 33 | """ 34 | model = obj.__class__ 35 | unified_index = connections[using].get_unified_index() 36 | index = unified_index.get_index(model) 37 | prepped_data = index.full_prepare(obj) 38 | final_data = {} 39 | for key, value in prepped_data.items(): 40 | final_data[key] = connections[using].get_backend()._from_python(value) 41 | return final_data 42 | 43 | 44 | def get_model(app_label, model_name): 45 | """ 46 | Fetches a Django model using the app registry. 47 | 48 | This doesn't require that an app with the given app label exists, which 49 | makes it safe to call when the registry is being populated. All other 50 | methods to access models might raise an exception about the registry not 51 | being ready yet. 52 | 53 | Raises LookupError if model isn't found. 54 | """ 55 | try: 56 | from django.apps import apps 57 | from django.core.exceptions import AppRegistryNotReady 58 | except ImportError: 59 | # Django < 1.7 60 | from django.db import models 61 | return models.get_model(app_label, model_name) 62 | 63 | try: 64 | return apps.get_model(app_label, model_name) 65 | 66 | except AppRegistryNotReady: 67 | if apps.apps_ready and not apps.models_ready: 68 | # If this function is called while `apps.populate()` is 69 | # loading models, ensure that the module that defines the 70 | # target model has been imported and try looking the model up 71 | # in the app registry. This effectively emulates 72 | # `from path.to.app.models import Model` where we use 73 | # `Model = get_model('app', 'Model')` instead. 74 | app_config = apps.get_app_config(app_label) 75 | # `app_config.import_models()` cannot be used here because it 76 | # would interfere with `apps.populate()`. 77 | import_module("%s.%s" % (app_config.name, "models")) 78 | # In order to account for case-insensitivity of model_name, 79 | # look up the model through a private API of the app registry. 80 | return apps.get_registered_model(app_label, model_name) 81 | 82 | else: 83 | # This must be a different case (e.g. the model really doesn't 84 | # exist). We just re-raise the exception. 85 | raise 86 | -------------------------------------------------------------------------------- /elasticstack/views.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2014-2015, Ben Lopatin 2 | # All rights reserved. 3 | 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are met: 6 | 7 | # Redistributions of source code must retain the above copyright notice, this 8 | # list of conditions and the following disclaimer. Redistributions in binary 9 | # form must reproduce the above copyright notice, this list of conditions and the 10 | # following disclaimer in the documentation and/or other materials provided with 11 | # the distribution 12 | 13 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 14 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 15 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 16 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 17 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 19 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 20 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 21 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 22 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | 24 | from django.conf import settings 25 | from django.core.paginator import Paginator 26 | from django.views.generic import FormView 27 | from django.views.generic.edit import FormMixin 28 | from django.views.generic.list import MultipleObjectMixin 29 | 30 | from haystack.forms import ModelSearchForm, FacetedSearchForm 31 | from haystack.query import EmptySearchQuerySet 32 | 33 | 34 | RESULTS_PER_PAGE = getattr(settings, "HAYSTACK_SEARCH_RESULTS_PER_PAGE", 20) 35 | 36 | 37 | class SearchMixin(MultipleObjectMixin, FormMixin): 38 | """ 39 | A mixin that allows adding in Haystacks search functionality into 40 | another view class. 41 | 42 | This mixin exhibits similar end functionality as the base Haystack search 43 | view, but with some important distinctions oriented around greater 44 | compatibility with Django's built-in class based views and mixins. 45 | 46 | Normal flow: 47 | 48 | self.request = request 49 | 50 | self.form = self.build_form() 51 | self.query = self.get_query() 52 | self.results = self.get_results() 53 | 54 | return self.create_response() 55 | 56 | This mixin should: 57 | 1. Make the form 58 | 2. Get the queryset 59 | 3. Return the paginated queryset 60 | 61 | """ 62 | template_name = "search/search.html" 63 | load_all = True 64 | form_class = ModelSearchForm 65 | queryset = EmptySearchQuerySet() 66 | context_object_name = None 67 | paginate_by = RESULTS_PER_PAGE 68 | paginate_orphans = 0 69 | paginator_class = Paginator 70 | page_kwarg = "page" 71 | form_name = "form" 72 | search_field = "q" 73 | 74 | def get_form_kwargs(self): 75 | """ 76 | Returns the keyword arguments for instantiating the form. 77 | """ 78 | kwargs = {"initial": self.get_initial()} 79 | if self.request.method == "GET": 80 | kwargs.update({"data": self.request.GET}) 81 | kwargs.update({"searchqueryset": self.get_query_set()}) 82 | return kwargs 83 | 84 | def get_query_set(self): 85 | return self.queryset 86 | 87 | def form_invalid(self, form): 88 | return self.render_to_response( 89 | self.get_context_data( 90 | **{self.form_name: form, "object_list": self.get_query_set()} 91 | ) 92 | ) 93 | 94 | def form_valid(self, form): 95 | self.queryset = form.search() 96 | return self.render_to_response( 97 | self.get_context_data( 98 | **{ 99 | self.form_name: form, 100 | "query": form.cleaned_data.get(self.search_field), 101 | "object_list": self.queryset, 102 | } 103 | ) 104 | ) 105 | 106 | 107 | class FacetedSearchMixin(SearchMixin): 108 | """ 109 | A mixin that allows adding in a Haystack search functionality with search 110 | faceting. 111 | """ 112 | form_class = FacetedSearchForm 113 | 114 | def get_form_kwargs(self): 115 | kwargs = super(SearchMixin, self).get_form_kwargs() 116 | kwargs.update({"selected_facets": self.request.GET.getlist("selected_facets")}) 117 | return kwargs 118 | 119 | def get_context_data(self, **kwargs): 120 | context = super(FacetedSearchMixin, self).get_context_data(**kwargs) 121 | context.update({"facets": self.results.facet_counts()}) 122 | return context 123 | 124 | 125 | class SearchView(SearchMixin, FormView): 126 | """A view class for searching a Haystack managed search index""" 127 | 128 | def get(self, request, *args, **kwargs): 129 | """ 130 | Handles GET requests and instantiates a blank version of the form. 131 | """ 132 | form_class = self.get_form_class() 133 | form = self.get_form(form_class) 134 | if form.is_valid(): 135 | return self.form_valid(form) 136 | 137 | else: 138 | return self.form_invalid(form) 139 | 140 | 141 | class FacetedSearchView(FacetedSearchMixin, SearchView): 142 | """ 143 | A view class for searching a Haystack managed search index with 144 | facets 145 | """ 146 | pass 147 | -------------------------------------------------------------------------------- /requirements-test.txt: -------------------------------------------------------------------------------- 1 | mock>=1.3.0 2 | pytest-django==2.8.0 3 | pytest-cov>=2.1.0 4 | flake8>=2.4.1 5 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | django>=1.5.1 2 | django-haystack>=2.0.0 3 | 4 | # Additional requirements go here 5 | twine>=1.8,<2.0 6 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | from setuptools import setup 5 | 6 | import elasticstack 7 | 8 | version = elasticstack.__version__ 9 | 10 | try: 11 | readme = open("README.rst", encoding="utf-8").read() 12 | history = open("HISTORY.rst", encoding="utf-8").read().replace(".. :changelog:", "") 13 | except TypeError: 14 | # Python 2. Encoding errors may occur. 15 | readme = open("README.rst").read() 16 | history = open("HISTORY.rst").read().replace(".. :changelog:", "") 17 | 18 | 19 | setup( 20 | name='elasticstack', 21 | version=version, 22 | description="""Configurable indexing and other extras for Haystack (with ElasticSearch biases).""", 23 | long_description=readme + '\n\n' + history, 24 | author='Ben Lopatin', 25 | author_email='ben@wellfire.co', 26 | url='https://github.com/bennylope/elasticstack', 27 | packages=[ 28 | 'elasticstack', 29 | ], 30 | include_package_data=True, 31 | install_requires=[ 32 | 'Django>=1.8', 33 | 'django-haystack>=2.0.0', 34 | ], 35 | license="BSD", 36 | zip_safe=False, 37 | keywords='elasticstack', 38 | classifiers=[ 39 | 'Development Status :: 4 - Beta', 40 | 'Framework :: Django', 41 | 'Intended Audience :: Developers', 42 | 'License :: OSI Approved :: BSD License', 43 | 'Natural Language :: English', 44 | "Programming Language :: Python :: 2", 45 | 'Programming Language :: Python :: 2.7', 46 | 'Programming Language :: Python :: 3', 47 | 'Programming Language :: Python :: 3.4', 48 | 'Programming Language :: Python :: 3.5', 49 | ], 50 | ) 51 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bennylope/elasticstack/8d1e99489815de6346fbf6720c9df5176546288e/tests/__init__.py -------------------------------------------------------------------------------- /tests/test_backends.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | """ 5 | test_elasticstack 6 | ------------ 7 | 8 | Tests for `elasticstack` backends module. 9 | """ 10 | from django.core.exceptions import ImproperlyConfigured 11 | 12 | from django.test import TestCase 13 | from django.test.utils import override_settings 14 | 15 | from haystack.fields import CharField as HaystackCharField 16 | from haystack.fields import EdgeNgramField as HaystackEdgeNgramField 17 | 18 | from elasticstack import backends 19 | from elasticstack import fields 20 | 21 | 22 | class TestGlobalBackendSettings(TestCase): 23 | """ 24 | Basic tests that the backend replaces the attributes as expected. 25 | """ 26 | 27 | @override_settings(ELASTICSEARCH_DEFAULT_ANALYZER="stop") 28 | def test_user_analyzer(self): 29 | """Ensure that the default analyzer is overridden""" 30 | back_class = backends.ConfigurableElasticBackend( 31 | "default", URL="http://localhost:9200", INDEX_NAME="" 32 | ) 33 | self.assertEqual(back_class.DEFAULT_ANALYZER, "stop") 34 | 35 | @override_settings(ELASTICSEARCH_INDEX_SETTINGS={"settings": 4}) 36 | def test_user_settings(self): 37 | """Ensure that the default index settings are overridden""" 38 | back_class = backends.ConfigurableElasticBackend( 39 | "default", URL="http://localhost:9200", INDEX_NAME="" 40 | ) 41 | self.assertEqual(back_class.DEFAULT_SETTINGS, {"settings": 4}) 42 | 43 | @override_settings(ELASTICSEARCH_DEFAULT_NGRAM_SEARCH_ANALYZER="stop") 44 | def test_ngram_user_analyzer(self): 45 | """Ensure that the default analyzer is overridden""" 46 | back_class = backends.ConfigurableElasticBackend( 47 | "default", URL="http://localhost:9200", INDEX_NAME="" 48 | ) 49 | self.assertEqual(back_class.DEFAULT_NGRAM_SEARCH_ANALYZER, "stop") 50 | 51 | 52 | class TestIndexSpecificBackendSettings(TestCase): 53 | """ 54 | Basic tests for index specific settings 55 | """ 56 | 57 | def test_user_analyzer(self): 58 | """Ensure that the default analyzer is overridden""" 59 | back_class = backends.ConfigurableElasticBackend( 60 | "default", 61 | URL="http://localhost:9200", 62 | INDEX_NAME="", 63 | DEFAULT_ANALYZER="stop", 64 | ) 65 | self.assertEqual(back_class.DEFAULT_ANALYZER, "stop") 66 | 67 | @override_settings(ELASTICSEARCH_INDEX_SETTINGS={"czech": {"settings": 4}}) 68 | def test_user_settings(self): 69 | """Ensure that the default index settings are overridden""" 70 | back_class = backends.ConfigurableElasticBackend( 71 | "default", URL="http://localhost:9200", INDEX_NAME="", SETTINGS_NAME="czech" 72 | ) 73 | self.assertEqual(back_class.DEFAULT_SETTINGS, {"settings": 4}) 74 | 75 | def test_ngram_user_analyzer(self): 76 | """Ensure that the default analyzer is overridden""" 77 | back_class = backends.ConfigurableElasticBackend( 78 | "default", 79 | URL="http://localhost:9200", 80 | INDEX_NAME="", 81 | DEFAULT_NGRAM_SEARCH_ANALYZER="stop", 82 | ) 83 | self.assertEqual(back_class.DEFAULT_NGRAM_SEARCH_ANALYZER, "stop") 84 | 85 | @override_settings(ELASTICSEARCH_DEFAULT_ANALYZER="stop") 86 | def test_duplicit_user_analyzer_definition(self): 87 | """Ensure that exception is raised when analyzer is set global setting and also index settings""" 88 | with self.assertRaises(ImproperlyConfigured): 89 | backends.ConfigurableElasticBackend( 90 | "default", 91 | URL="http://localhost:9200", 92 | INDEX_NAME="", 93 | DEFAULT_ANALYZER="stop", 94 | ) 95 | 96 | @override_settings(ELASTICSEARCH_INDEX_SETTINGS={"settings": 4}) 97 | def test_duplicit_user_settings_definition(self): 98 | """Ensure that exception is raised when analyzer is set global setting and also index settings""" 99 | with self.assertRaises(ImproperlyConfigured): 100 | back_class = backends.ConfigurableElasticBackend( 101 | "default", 102 | URL="http://localhost:9200", 103 | INDEX_NAME="", 104 | SETTINGS_NAME="czech", 105 | ) 106 | self.assertEqual(back_class.DEFAULT_SETTINGS, {"settings": 4}) 107 | 108 | @override_settings(ELASTICSEARCH_DEFAULT_NGRAM_SEARCH_ANALYZER="stop") 109 | def test_duplicit_ngram_user_analyzer_definition(self): 110 | """Ensure that exception is raised when analyzer is set global setting and also index settings""" 111 | with self.assertRaises(ImproperlyConfigured): 112 | backends.ConfigurableElasticBackend( 113 | "default", 114 | URL="http://localhost:9200", 115 | INDEX_NAME="", 116 | DEFAULT_NGRAM_SEARCH_ANALYZER="stop", 117 | ) 118 | 119 | @override_settings(ELASTICSEARCH_INDEX_SETTINGS={"czech": {"settings": 4}}) 120 | def test_invalid_settings_name(self): 121 | """Ensure that exception is raised when settings name not found""" 122 | with self.assertRaises(ImproperlyConfigured): 123 | backends.ConfigurableElasticBackend( 124 | "default", 125 | URL="http://localhost:9200", 126 | INDEX_NAME="", 127 | SETTINGS_NAME="notexist", 128 | ) 129 | 130 | 131 | class TestSchema(TestCase): 132 | """ 133 | Tests that the schema is built using the specified settings. 134 | 135 | The backend class must be configured in each test method to ensure its 136 | settings are test specific. 137 | """ 138 | 139 | def test_contral_analyzer(self): 140 | """Control test that the default analyzer is snowball""" 141 | back_class = backends.ConfigurableElasticBackend( 142 | "default", URL="http://localhost:9200", INDEX_NAME="" 143 | ) 144 | text_field = HaystackCharField( 145 | document=True, use_template=True, index_fieldname="body" 146 | ) 147 | # build_schema is passed a SortedDict of search index fields keyed by 148 | # field name 149 | schema = back_class.build_schema({"body": text_field}) 150 | self.assertEqual("snowball", schema[1]["body"]["analyzer"]) 151 | 152 | def test_contral_search_analyzer(self): 153 | """Control test that the default ngram search analyzer is None""" 154 | back_class = backends.ConfigurableElasticBackend( 155 | "default", URL="http://localhost:9200", INDEX_NAME="" 156 | ) 157 | text_field = HaystackEdgeNgramField( 158 | document=True, use_template=True, index_fieldname="body" 159 | ) 160 | # build_schema is passed a SortedDict of search index fields keyed by 161 | # field name 162 | schema = back_class.build_schema({"body": text_field}) 163 | self.assertFalse("search_analyzer" in schema[1]["body"]) 164 | 165 | @override_settings(ELASTICSEARCH_DEFAULT_ANALYZER="stop") 166 | def test_custom_analyzer(self): 167 | """Ensure custom analyzer used for fields""" 168 | back_class = backends.ConfigurableElasticBackend( 169 | "default", URL="http://localhost:9200", INDEX_NAME="" 170 | ) 171 | text_field = HaystackCharField( 172 | document=True, use_template=True, index_fieldname="body" 173 | ) 174 | # build_schema is passed a SortedDict of search index fields keyed by 175 | # field name 176 | schema = back_class.build_schema({"body": text_field}) 177 | self.assertEqual("stop", schema[1]["body"]["analyzer"]) 178 | 179 | @override_settings(ELASTICSEARCH_DEFAULT_NGRAM_SEARCH_ANALYZER="stop") 180 | def test_custom_search_analyzer(self): 181 | """Ensure custom analyzer used for fields""" 182 | back_class = backends.ConfigurableElasticBackend( 183 | "default", URL="http://localhost:9200", INDEX_NAME="" 184 | ) 185 | text_field = HaystackEdgeNgramField( 186 | document=True, use_template=True, index_fieldname="body" 187 | ) 188 | # build_schema is passed a SortedDict of search index fields keyed by 189 | # field name 190 | schema = back_class.build_schema({"body": text_field}) 191 | self.assertTrue("search_analyzer" in schema[1]["body"]) 192 | self.assertEqual("stop", schema[1]["body"]["search_analyzer"]) 193 | 194 | def test_field_analyzer(self): 195 | """Ensure that field analyzer works on a case by case basis""" 196 | back_class = backends.ConfigurableElasticBackend( 197 | "default", URL="http://localhost:9200", INDEX_NAME="" 198 | ) 199 | # Control test - by default the CharField does not have a keyword 200 | # argument named 'analyzer' and does not take **kwargs 201 | self.assertRaises( 202 | TypeError, 203 | HaystackCharField, 204 | document=True, 205 | use_template=True, 206 | index_fieldname="body", 207 | analyzer="stop", 208 | ) 209 | text_field = fields.CharField( 210 | document=True, use_template=True, index_fieldname="body", analyzer="stop" 211 | ) 212 | schema = back_class.build_schema({"body": text_field}) 213 | self.assertEqual("stop", schema[1]["body"]["analyzer"]) 214 | -------------------------------------------------------------------------------- /tests/test_fields.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | """ 5 | test_fields 6 | ------------ 7 | 8 | Tests for `elasticstack` fields module. 9 | """ 10 | 11 | from django.test import TestCase 12 | 13 | from elasticstack.fields import ConfigurableFieldMixin 14 | 15 | 16 | class TestFields(TestCase): 17 | 18 | def test_missing_analyzer(self): 19 | """No specified analyzer should result in an error""" 20 | self.assertRaises(ValueError, ConfigurableFieldMixin) 21 | -------------------------------------------------------------------------------- /tests/test_forms.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | """ 5 | test_elasticstack 6 | ------------ 7 | 8 | Tests for `elasticstack` forms module. 9 | """ 10 | 11 | from django import forms 12 | from django.test import TestCase 13 | 14 | from elasticstack.forms import SearchForm 15 | 16 | 17 | class TestForms(TestCase): 18 | 19 | def test_named_search_field(self): 20 | """Ensure that the `q` field can be optionally used""" 21 | 22 | class MyForm(SearchForm): 23 | s = forms.CharField(label="Search") 24 | f = forms.CharField(label="More search") 25 | search_field_name = "s" 26 | 27 | form = MyForm() 28 | self.assertTrue("s" in form.fields) 29 | self.assertFalse("q" in form.fields) 30 | -------------------------------------------------------------------------------- /tests/test_views.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | """ 5 | test_elasticstack 6 | ------------ 7 | 8 | Tests for `elasticstack` views module. 9 | """ 10 | 11 | from django.test import TestCase 12 | from django.test.client import RequestFactory 13 | 14 | from elasticstack import views 15 | 16 | 17 | class TestElasticstack(TestCase): 18 | 19 | def setUp(self): 20 | self.factory = RequestFactory() 21 | 22 | def test_form_kwargs(self): 23 | """Ensure request data updates the form kwargs""" 24 | request = self.factory.request() 25 | request.GET = {"q": "whoami"} 26 | 27 | mixin = views.SearchMixin() 28 | mixin.request = request 29 | mixin.queryset = [] # An EmptySearchQuerySet is basically an empty list 30 | 31 | self.assertEqual( 32 | mixin.get_form_kwargs(), 33 | {"initial": {}, "data": request.GET, "searchqueryset": []}, 34 | ) 35 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = 3 | flake8, 4 | py{27,34,35}-django{18,19,110}-haystack{24}, 5 | 6 | [testenv] 7 | setenv = 8 | PYTHONPATH = {toxinidir}:{toxinidir}/elasticstack 9 | commands = py.test 10 | basepython = 11 | py27: python2.7 12 | py34: python3.4 13 | py35: python3.5 14 | pypy: pypy 15 | pypy3: pypy3 16 | jython: jython 17 | deps = 18 | django14: Django>=1.4,<1.5 19 | django18: Django>=1.8,<1.9 20 | django19: Django>=1.9,<1.10 21 | django110: Django>=1.10,<1.11 22 | haystack22: django-haystack>=2.2.0,<2.3.0 23 | haystack23: django-haystack>=2.3.0,<2.4.0 24 | haystack24: django-haystack>=2.4.0,<2.5.0 25 | elasticsearch>=1.7.0 26 | -r{toxinidir}/requirements-test.txt 27 | 28 | [testenv:flake8] 29 | basepython=python 30 | deps=flake8 31 | commands= 32 | flake8 elasticstack tests 33 | 34 | [flake8] 35 | ignore = E126,E128,E501 36 | #max-line-length = 99 37 | max-complexity = 10 38 | --------------------------------------------------------------------------------