├── asonic ├── __init__.py ├── exceptions.py ├── enums.py ├── connection.py └── client.py ├── .dockerignore ├── docs ├── setup.rst ├── index.rst ├── Makefile ├── asonic.rst ├── make.bat └── conf.py ├── Dockerfile ├── .travis.yml ├── docker-compose.yml ├── setup.py ├── tests ├── conftest.py ├── config.cfg └── test_commands.py ├── .gitignore ├── README.md └── LICENSE.md /asonic/__init__.py: -------------------------------------------------------------------------------- 1 | from .client import Client # noqa 2 | -------------------------------------------------------------------------------- /.dockerignore: -------------------------------------------------------------------------------- 1 | .tox 2 | .git 3 | dump 4 | __pycache__ 5 | *.pyc 6 | .travis.yml 7 | docker-compose.yml 8 | data 9 | bench.py 10 | -------------------------------------------------------------------------------- /docs/setup.rst: -------------------------------------------------------------------------------- 1 | setup module 2 | ============ 3 | 4 | .. automodule:: setup 5 | :members: 6 | :undoc-members: 7 | :show-inheritance: 8 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | from python:3.7-alpine3.8 2 | RUN apk add --no-cache gcc musl-dev 3 | COPY . /app 4 | RUN cd /app && \ 5 | pip install -e .[tests] 6 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | before_install: 2 | - sudo apt-get update 3 | - sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce 4 | language: python 5 | script: 6 | - docker-compose up --build --abort-on-container-exit 7 | -------------------------------------------------------------------------------- /asonic/exceptions.py: -------------------------------------------------------------------------------- 1 | class BaseSonicException(Exception): 2 | pass 3 | 4 | 5 | class ClientError(BaseSonicException): 6 | pass 7 | 8 | 9 | class ConnectionClosed(ClientError): 10 | pass 11 | 12 | 13 | class ServerError(BaseSonicException): 14 | pass 15 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. asonic documentation master file, created by 2 | sphinx-quickstart on Tue Mar 26 10:03:02 2019. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | Welcome to asonic's documentation! 7 | ================================== 8 | 9 | .. toctree:: 10 | :maxdepth: 2 11 | :caption: Contents: 12 | 13 | 14 | 15 | Indices and tables 16 | ================== 17 | 18 | * :ref:`genindex` 19 | * :ref:`modindex` 20 | * :ref:`search` 21 | -------------------------------------------------------------------------------- /docker-compose.yml: -------------------------------------------------------------------------------- 1 | version: '2.1' 2 | services: 3 | pytest: 4 | command: sh -c 'flake8 --max-line-length 120 /app && py.test /app' 5 | environment: 6 | - SONIC_HOST=sonic 7 | build: 8 | context: . 9 | links: 10 | - sonic 11 | depends_on: 12 | sonic: 13 | condition: service_healthy 14 | sonic: 15 | image: valeriansaliou/sonic:v1.4.9 16 | volumes: 17 | - ./tests/config.cfg:/etc/sonic.cfg 18 | healthcheck: 19 | test: ["CMD", "sleep", "2"] 20 | interval: 1s 21 | timeout: 5s 22 | retries: 120 23 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | SOURCEDIR = . 8 | BUILDDIR = _build 9 | 10 | # Put it first so that "make" without argument is like "make help". 11 | help: 12 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 13 | 14 | .PHONY: help Makefile 15 | 16 | # Catch-all target: route all unknown targets to Sphinx using the new 17 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 18 | %: Makefile 19 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import sys 2 | 3 | from setuptools import setup 4 | 5 | if sys.version < '3.4': 6 | raise RuntimeError('python 3.4 required') 7 | 8 | try: 9 | import pypandoc 10 | long_description = pypandoc.convert('README.md', 'rst') 11 | except Exception: 12 | long_description = '' 13 | 14 | setup( 15 | name='asonic', 16 | author='Moshe Zada', 17 | version='2.0.0', 18 | packages=['asonic'], 19 | keywords=['asonic', 'sonic', 'search', 'asyncio', 'async', 'text'], 20 | url='https://github.com/moshe/asonic', 21 | license='MPL-2.0', 22 | long_description=long_description, 23 | description='Async python client for Sonic database', 24 | install_requires=[], 25 | extras_require={ 26 | 'tests': ['pytest', 'pytest-asyncio', 'flake8'], 27 | }, 28 | ) 29 | -------------------------------------------------------------------------------- /docs/asonic.rst: -------------------------------------------------------------------------------- 1 | asonic package 2 | ============== 3 | 4 | Submodules 5 | ---------- 6 | 7 | asonic.client module 8 | -------------------- 9 | 10 | .. automodule:: asonic.client 11 | :members: 12 | :undoc-members: 13 | :show-inheritance: 14 | 15 | asonic.connection module 16 | ------------------------ 17 | 18 | .. automodule:: asonic.connection 19 | :members: 20 | :undoc-members: 21 | :show-inheritance: 22 | 23 | asonic.enums module 24 | ------------------- 25 | 26 | .. automodule:: asonic.enums 27 | :members: 28 | :undoc-members: 29 | :show-inheritance: 30 | 31 | asonic.exceptions module 32 | ------------------------ 33 | 34 | .. automodule:: asonic.exceptions 35 | :members: 36 | :undoc-members: 37 | :show-inheritance: 38 | 39 | 40 | Module contents 41 | --------------- 42 | 43 | .. automodule:: asonic 44 | :members: 45 | :undoc-members: 46 | :show-inheritance: 47 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | pushd %~dp0 4 | 5 | REM Command file for Sphinx documentation 6 | 7 | if "%SPHINXBUILD%" == "" ( 8 | set SPHINXBUILD=sphinx-build 9 | ) 10 | set SOURCEDIR=. 11 | set BUILDDIR=_build 12 | 13 | if "%1" == "" goto help 14 | 15 | %SPHINXBUILD% >NUL 2>NUL 16 | if errorlevel 9009 ( 17 | echo. 18 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 19 | echo.installed, then set the SPHINXBUILD environment variable to point 20 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 21 | echo.may add the Sphinx directory to PATH. 22 | echo. 23 | echo.If you don't have Sphinx installed, grab it from 24 | echo.http://sphinx-doc.org/ 25 | exit /b 1 26 | ) 27 | 28 | %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% 29 | goto end 30 | 31 | :help 32 | %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% 33 | 34 | :end 35 | popd 36 | -------------------------------------------------------------------------------- /tests/conftest.py: -------------------------------------------------------------------------------- 1 | from os import getenv 2 | 3 | import pytest_asyncio 4 | 5 | from asonic import Client 6 | from asonic.enums import Channel 7 | 8 | collection = 'collection' 9 | 10 | 11 | @pytest_asyncio.fixture 12 | async def clean(): 13 | c = Client(host=getenv('SONIC_HOST', 'localhost'), port=1491) 14 | await c.channel(Channel.INGEST) 15 | await c.flushc(collection) 16 | 17 | 18 | @pytest_asyncio.fixture 19 | async def search() -> Client: 20 | c = Client(host=getenv('SONIC_HOST', 'localhost'), port=1491) 21 | await c.channel(Channel.SEARCH) 22 | return c 23 | 24 | 25 | @pytest_asyncio.fixture 26 | async def ingest(): 27 | c = Client(host=getenv('SONIC_HOST', 'localhost'), port=1491) 28 | await c.channel(Channel.INGEST) 29 | return c 30 | 31 | 32 | @pytest_asyncio.fixture 33 | async def control(): 34 | c = Client(host=getenv('SONIC_HOST', 'localhost'), port=1491) 35 | await c.channel(Channel.CONTROL) 36 | return c 37 | -------------------------------------------------------------------------------- /tests/config.cfg: -------------------------------------------------------------------------------- 1 | # Sonic 2 | # Fast, lightweight and schema-less search backend 3 | # Configuration file 4 | # Example: https://github.com/valeriansaliou/sonic/blob/master/config.cfg 5 | 6 | 7 | [server] 8 | 9 | log_level = "info" 10 | 11 | 12 | [channel] 13 | 14 | inet = "0.0.0.0:1491" 15 | tcp_timeout = 300 16 | 17 | auth_password = "SecretPassword" 18 | 19 | [channel.search] 20 | 21 | query_limit_default = 10 22 | query_limit_maximum = 100 23 | query_alternates_try = 4 24 | 25 | suggest_limit_default = 5 26 | suggest_limit_maximum = 20 27 | 28 | 29 | [store] 30 | 31 | [store.kv] 32 | 33 | path = "./data/store/kv/" 34 | 35 | retain_word_objects = 1000 36 | 37 | [store.kv.pool] 38 | 39 | inactive_after = 1800 40 | 41 | [store.kv.database] 42 | 43 | compress = true 44 | parallelism = 2 45 | max_files = 100 46 | max_compactions = 1 47 | max_flushes = 1 48 | 49 | [store.fst] 50 | 51 | path = "./data/store/fst/" 52 | 53 | [store.fst.pool] 54 | 55 | inactive_after = 300 56 | 57 | [store.fst.graph] 58 | 59 | consolidate_after = 180 60 | -------------------------------------------------------------------------------- /asonic/enums.py: -------------------------------------------------------------------------------- 1 | from enum import Enum 2 | from itertools import chain 3 | 4 | from typing import Set 5 | 6 | 7 | class Action(Enum): 8 | CONSOLIDATE = 'consolidate' 9 | BACKUP = 'backup' 10 | RESTORE = 'restore' 11 | 12 | 13 | class Command(Enum): 14 | QUERY = 'QUERY' 15 | SUGGEST = 'SUGGEST' 16 | PING = 'PING' 17 | QUIT = 'QUIT' 18 | HELP = 'HELP' 19 | PUSH = 'PUSH' 20 | POP = 'POP' 21 | FLUSHB = 'FLUSHB' 22 | FLUSHC = 'FLUSHC' 23 | FLUSHO = 'FLUSHO' 24 | COUNT = 'COUNT' 25 | TRIGGER = 'TRIGGER' 26 | INFO = 'INFO' 27 | LIST = 'LIST' 28 | 29 | 30 | class Channel(Enum): 31 | UNINITIALIZED = 'uninitialized' 32 | INGEST = 'ingest' 33 | SEARCH = 'search' 34 | CONTROL = 'control' 35 | 36 | 37 | enabled_commands = { 38 | Channel.UNINITIALIZED: { 39 | Command.QUIT, 40 | }, 41 | Channel.SEARCH: { 42 | Command.QUERY, 43 | Command.SUGGEST, 44 | Command.LIST, 45 | Command.PING, 46 | Command.HELP, 47 | Command.QUIT, 48 | }, 49 | Channel.INGEST: { 50 | Command.PUSH, 51 | Command.POP, 52 | Command.COUNT, 53 | Command.FLUSHB, 54 | Command.FLUSHC, 55 | Command.FLUSHO, 56 | Command.PING, 57 | Command.HELP, 58 | Command.QUIT, 59 | }, 60 | Channel.CONTROL: { 61 | Command.TRIGGER, 62 | Command.PING, 63 | Command.HELP, 64 | Command.QUIT, 65 | } 66 | } 67 | 68 | all_commands = set(chain(*enabled_commands.values())) # type: Set[Command] 69 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | .idea 3 | __pycache__/ 4 | *.py[cod] 5 | *$py.class 6 | 7 | # C extensions 8 | *.so 9 | 10 | # Distribution / packaging 11 | .Python 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | *.egg-info/ 25 | .installed.cfg 26 | *.egg 27 | MANIFEST 28 | 29 | # PyInstaller 30 | # Usually these files are written by a python script from a template 31 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 32 | *.manifest 33 | *.spec 34 | 35 | # Installer logs 36 | pip-log.txt 37 | pip-delete-this-directory.txt 38 | 39 | # Unit test / coverage reports 40 | htmlcov/ 41 | .tox/ 42 | .coverage 43 | .coverage.* 44 | .cache 45 | nosetests.xml 46 | coverage.xml 47 | *.cover 48 | .hypothesis/ 49 | .pytest_cache/ 50 | 51 | # Translations 52 | *.mo 53 | *.pot 54 | 55 | # Django stuff: 56 | *.log 57 | local_settings.py 58 | db.sqlite3 59 | 60 | # Flask stuff: 61 | instance/ 62 | .webassets-cache 63 | 64 | # Scrapy stuff: 65 | .scrapy 66 | 67 | # Sphinx documentation 68 | docs/_build/ 69 | 70 | # PyBuilder 71 | target/ 72 | 73 | # Jupyter Notebook 74 | .ipynb_checkpoints 75 | 76 | # pyenv 77 | .python-version 78 | 79 | # celery beat schedule file 80 | celerybeat-schedule 81 | 82 | # SageMath parsed files 83 | *.sage.py 84 | 85 | # Environments 86 | .env 87 | .venv 88 | env/ 89 | venv/ 90 | ENV/ 91 | env.bak/ 92 | venv.bak/ 93 | 94 | # Spyder project settings 95 | .spyderproject 96 | .spyproject 97 | 98 | # Rope project settings 99 | .ropeproject 100 | 101 | # mkdocs documentation 102 | /site 103 | 104 | # mypy 105 | .mypy_cache/ 106 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # asonic - async python client for the sonic search backend 2 | Asonic implements all [Sonic](https://github.com/valeriansaliou/sonic) APIs 3 | Bugfixes and api changes are welcome 4 | 5 | ## Install 6 | `pip install asonic` 7 | 8 | ## API Docs 9 | [here](https://asonic.readthedocs.io/en/latest/asonic.html#module-asonic.client) 10 | 11 | ## Usage 12 | ### Search channel 13 | ```python 14 | import asyncio 15 | 16 | from asonic import Client 17 | from asonic.enums import Channel 18 | 19 | 20 | async def main(): 21 | c = await Client.create( 22 | host="127.0.0.1", 23 | port=1491, 24 | password="SecretPassword", 25 | channel=Channel.SEARCH, 26 | max_connections=100 27 | ) 28 | assert (await c.query('collection', 'bucket', 'quick')) == [b'user_id'] 29 | assert (await c.suggest('collection', 'bucket', 'br', 1)) == [b'brown'] 30 | 31 | if __name__ == '__main__': 32 | loop = asyncio.get_event_loop() 33 | loop.run_until_complete(main()) 34 | ``` 35 | 36 | ### Ingest channel 37 | 38 | ```python 39 | import asyncio 40 | 41 | from asonic import Client 42 | from asonic.enums import Channel 43 | 44 | 45 | async def main(): 46 | c = await Client.create( 47 | host="127.0.0.1", 48 | port=1491, 49 | password="SecretPassword", 50 | channel=Channel.INGEST, 51 | max_connections=100 52 | ) 53 | await c.push('collection', 'bucket', 'user_id', 'The quick brown fox jumps over the lazy dog') 54 | # Return b'OK' 55 | await c.pop('collection', 'bucket', 'user_id', 'The') 56 | # Return 1 57 | 58 | if __name__ == '__main__': 59 | loop = asyncio.get_event_loop() 60 | loop.run_until_complete(main()) 61 | ``` 62 | 63 | 64 | ### Control channel 65 | 66 | ```python 67 | import asyncio 68 | 69 | from asonic import Client 70 | from asonic.enums import Channel, Action 71 | 72 | 73 | async def main(): 74 | c = await Client.create( 75 | host="127.0.0.1", 76 | port=1491, 77 | password="SecretPassword", 78 | channel=Channel.CONTROL, 79 | ) 80 | await c.trigger(Action.CONSOLIDATE) 81 | # Return b'OK' 82 | 83 | if __name__ == '__main__': 84 | loop = asyncio.get_event_loop() 85 | loop.run_until_complete(main()) 86 | ``` 87 | -------------------------------------------------------------------------------- /asonic/connection.py: -------------------------------------------------------------------------------- 1 | import asyncio 2 | from logging import getLogger 3 | 4 | from typing import Set, Optional 5 | 6 | from asonic.enums import Channel 7 | from asonic.exceptions import ServerError, ConnectionClosed 8 | 9 | 10 | class Connection: 11 | def __init__(self, host: str, port: int, channel: Channel, password: str): 12 | self.host = host 13 | self.port = port 14 | self.channel = channel 15 | self.password = password 16 | self.reader = None # type: Optional[asyncio.StreamReader] 17 | self.writer = None # type: Optional[asyncio.StreamWriter] 18 | self.logger = getLogger('connection') 19 | 20 | async def connect(self) -> None: 21 | self.reader, self.writer = await asyncio.open_connection(self.host, self.port) 22 | result = await self.read() 23 | assert result.startswith(b'CONNECTED') 24 | 25 | await self.write(f'START {self.channel.value} {self.password}') 26 | result = await self.read() 27 | if result.startswith(b'STARTED'): 28 | pass 29 | elif result.startswith(b'ENDED'): 30 | raise ConnectionClosed(f"Error {result}") 31 | else: 32 | raise ServerError(f"Unknown error {result}") 33 | 34 | async def write(self, msg: str) -> None: 35 | assert self.writer is not None, 'connect' 36 | self.logger.debug('>%s', msg) 37 | self.writer.write(((msg + '\r\n').encode())) 38 | await self.writer.drain() 39 | 40 | async def read(self) -> bytes: 41 | assert self.reader is not None 42 | line = (await self.reader.readline()).strip() 43 | self.logger.debug('<%s', line) 44 | if line.startswith(b'ERR '): 45 | raise ServerError(line[4:]) 46 | return line 47 | 48 | 49 | class ConnectionPool: 50 | def __init__(self, host: str, port: int, channel: Channel, password: str, max_connections: int = 100): 51 | self.closed = False 52 | self._created_connections = 0 53 | self._available_connections = asyncio.Queue() # type: asyncio.Queue[Connection] 54 | self._in_use_connections = set() # type: Set[Connection] 55 | self.max_connections = max_connections 56 | self.host = host 57 | self.port = port 58 | self.password = password 59 | self.channel = channel 60 | 61 | async def get_connection(self) -> Connection: 62 | if self.closed is True: 63 | raise ConnectionClosed('Connection pool is closed') 64 | try: 65 | connection = self._available_connections.get_nowait() 66 | except asyncio.QueueEmpty: 67 | connection = await self.make_connection() 68 | self._in_use_connections.add(connection) 69 | return connection 70 | 71 | async def make_connection(self) -> Connection: 72 | if self._created_connections >= self.max_connections: 73 | return await self._available_connections.get() 74 | self._created_connections += 1 75 | c = Connection(self.host, self.port, self.channel, self.password) 76 | await c.connect() 77 | return c 78 | 79 | async def release(self, connection: Connection) -> None: 80 | self._in_use_connections.remove(connection) 81 | await self._available_connections.put(connection) 82 | 83 | async def destroy(self): 84 | self.closed = True 85 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # Configuration file for the Sphinx documentation builder. 4 | # 5 | # This file does only contain a selection of the most common options. For a 6 | # full list see the documentation: 7 | # http://www.sphinx-doc.org/en/master/config 8 | 9 | # -- Path setup -------------------------------------------------------------- 10 | 11 | # If extensions (or modules to document with autodoc) are in another directory, 12 | # add these directories to sys.path here. If the directory is relative to the 13 | # documentation root, use os.path.abspath to make it absolute, like shown here. 14 | # 15 | import os 16 | import sys 17 | 18 | sys.path.insert(0, os.path.abspath('..')) 19 | sys.path.insert(0, os.path.abspath('../asonic')) 20 | 21 | 22 | # -- Project information ----------------------------------------------------- 23 | 24 | project = 'asonic' 25 | copyright = '2019, Moshe Zada' 26 | author = 'Moshe Zada' 27 | 28 | # The short X.Y version 29 | version = '' 30 | # The full version, including alpha/beta/rc tags 31 | release = '0.1.2' 32 | 33 | 34 | # -- General configuration --------------------------------------------------- 35 | 36 | # If your documentation needs a minimal Sphinx version, state it here. 37 | # 38 | # needs_sphinx = '1.0' 39 | 40 | # Add any Sphinx extension module names here, as strings. They can be 41 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 42 | # ones. 43 | extensions = [ 44 | 'sphinx.ext.autodoc', 45 | ] 46 | 47 | # Add any paths that contain templates here, relative to this directory. 48 | templates_path = ['_templates'] 49 | 50 | # The suffix(es) of source filenames. 51 | # You can specify multiple suffix as a list of string: 52 | # 53 | # source_suffix = ['.rst', '.md'] 54 | source_suffix = '.rst' 55 | 56 | # The master toctree document. 57 | master_doc = 'index' 58 | 59 | # The language for content autogenerated by Sphinx. Refer to documentation 60 | # for a list of supported languages. 61 | # 62 | # This is also used if you do content translation via gettext catalogs. 63 | # Usually you set "language" from the command line for these cases. 64 | language = None 65 | 66 | # List of patterns, relative to source directory, that match files and 67 | # directories to ignore when looking for source files. 68 | # This pattern also affects html_static_path and html_extra_path. 69 | exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] 70 | 71 | # The name of the Pygments (syntax highlighting) style to use. 72 | pygments_style = None 73 | 74 | 75 | # -- Options for HTML output ------------------------------------------------- 76 | 77 | # The theme to use for HTML and HTML Help pages. See the documentation for 78 | # a list of builtin themes. 79 | # 80 | html_theme = 'alabaster' 81 | 82 | # Theme options are theme-specific and customize the look and feel of a theme 83 | # further. For a list of options available for each theme, see the 84 | # documentation. 85 | # 86 | # html_theme_options = {} 87 | 88 | # Add any paths that contain custom static files (such as style sheets) here, 89 | # relative to this directory. They are copied after the builtin static files, 90 | # so a file named "default.css" will overwrite the builtin "default.css". 91 | html_static_path = ['_static'] 92 | 93 | # Custom sidebar templates, must be a dictionary that maps document names 94 | # to template names. 95 | # 96 | # The default sidebars (for documents that don't match any pattern) are 97 | # defined by theme itself. Builtin themes are using these templates by 98 | # default: ``['localtoc.html', 'relations.html', 'sourcelink.html', 99 | # 'searchbox.html']``. 100 | # 101 | # html_sidebars = {} 102 | 103 | 104 | # -- Options for HTMLHelp output --------------------------------------------- 105 | 106 | # Output file base name for HTML help builder. 107 | htmlhelp_basename = 'asonicdoc' 108 | 109 | 110 | # -- Options for LaTeX output ------------------------------------------------ 111 | 112 | latex_elements = { 113 | # The paper size ('letterpaper' or 'a4paper'). 114 | # 115 | # 'papersize': 'letterpaper', 116 | 117 | # The font size ('10pt', '11pt' or '12pt'). 118 | # 119 | # 'pointsize': '10pt', 120 | 121 | # Additional stuff for the LaTeX preamble. 122 | # 123 | # 'preamble': '', 124 | 125 | # Latex figure (float) alignment 126 | # 127 | # 'figure_align': 'htbp', 128 | } 129 | 130 | # Grouping the document tree into LaTeX files. List of tuples 131 | # (source start file, target name, title, 132 | # author, documentclass [howto, manual, or own class]). 133 | latex_documents = [ 134 | (master_doc, 'asonic.tex', 'asonic Documentation', 135 | 'Moshe Zada', 'manual'), 136 | ] 137 | 138 | 139 | # -- Options for manual page output ------------------------------------------ 140 | 141 | # One entry per manual page. List of tuples 142 | # (source start file, name, description, authors, manual section). 143 | man_pages = [ 144 | (master_doc, 'asonic', 'asonic Documentation', 145 | [author], 1) 146 | ] 147 | 148 | 149 | # -- Options for Texinfo output ---------------------------------------------- 150 | 151 | # Grouping the document tree into Texinfo files. List of tuples 152 | # (source start file, target name, title, author, 153 | # dir menu entry, description, category) 154 | texinfo_documents = [ 155 | (master_doc, 'asonic', 'asonic Documentation', 156 | author, 'asonic', 'One line description of project.', 157 | 'Miscellaneous'), 158 | ] 159 | 160 | 161 | # -- Options for Epub output ------------------------------------------------- 162 | 163 | # Bibliographic Dublin Core info. 164 | epub_title = project 165 | 166 | # The unique identifier of the text. This can be a ISBN number 167 | # or the project homepage. 168 | # 169 | # epub_identifier = '' 170 | 171 | # A unique identification for the text. 172 | # 173 | # epub_uid = '' 174 | 175 | # A list of files that should not be packed into the epub file. 176 | epub_exclude_files = ['search.html'] 177 | 178 | 179 | # -- Extension configuration ------------------------------------------------- 180 | -------------------------------------------------------------------------------- /tests/test_commands.py: -------------------------------------------------------------------------------- 1 | from contextlib import nullcontext as does_not_raise 2 | import math 3 | import pytest 4 | import sys 5 | from uuid import uuid4 6 | 7 | from asonic import Client 8 | from asonic.client import BUFFER 9 | from asonic.enums import Action, Channel 10 | from asonic.exceptions import ClientError, ConnectionClosed 11 | 12 | collection = 'collection' 13 | 14 | pytestmark = pytest.mark.asyncio 15 | 16 | async def test_client_init(): 17 | with does_not_raise(): 18 | _client = await Client.create(host="localhost", port=1491, password="SecretPassword") 19 | with pytest.raises(ConnectionClosed): 20 | _client = await Client.create(host="localhost", port=1491, password="invalid") 21 | 22 | async def test_ping(search, ingest): 23 | assert await search.ping() == b'PONG' 24 | assert await ingest.ping() == b'PONG' 25 | 26 | 27 | async def test_help(search): 28 | assert await search.help('commands') == b'RESULT commands(QUERY, SUGGEST, LIST, PING, HELP, QUIT)' 29 | 30 | 31 | async def test_empty(search): 32 | assert await search.query(collection, 'user1', 'test') == [] 33 | assert await search.suggest(collection, 'user1', 'test') == [] 34 | 35 | 36 | async def test_suggest(search, ingest, control): 37 | bucket = str(uuid4()) 38 | uid = str(uuid4()) 39 | assert (await ingest.push(collection, bucket, uid, 'RESULT commands(QUERY, SUGGEST, LIST, PING, HELP, QUIT)')) == b'OK' 40 | assert (await control.trigger(Action.CONSOLIDATE)) == b'OK' 41 | assert (await search.suggest(collection, bucket, 'comm')) == [b'commands'] 42 | assert (await search.suggest(collection, bucket, 'Q')) == [b'query', b'quit'] 43 | assert (await ingest.count(collection, bucket, uid)) == 8 44 | 45 | 46 | async def test_info(control): 47 | print((await control.info())) 48 | assert 'command_latency_best' in (await control.info()) 49 | 50 | 51 | async def test_query(search, ingest): 52 | bucket = str(uuid4()) 53 | uid = str(uuid4()) 54 | assert (await ingest.push(collection, bucket, uid, 'The quick brown fox jumps over the lazy dog')) == b'OK' 55 | assert (await search.query(collection, bucket, 'quick', 1, 0)) == [uid.encode()] 56 | 57 | async def test_list(search, ingest, control): 58 | bucket = str(uuid4()) 59 | uid = str(uuid4()) 60 | await ingest.push(collection, bucket, uid, 'The quick brown fox jumps over the lazy dog') 61 | await ingest.push(collection, bucket, uid, 'brown fox jumps') 62 | await ingest.push(collection, bucket, uid, 'lazy dog jumps') 63 | await control.trigger(action=Action.CONSOLIDATE) 64 | assert set(await search.list(collection, bucket)) == {b'quick', b'brown', b'fox', b'jumps', b'lazy', b'dog'} 65 | 66 | async def test_flushb(search, ingest): 67 | bucket = str(uuid4()) 68 | uid = str(uuid4()) 69 | assert (await ingest.push(collection, bucket, uid, 'The quick brown fox jumps over the lazy dog')) == b'OK' 70 | assert (await search.query(collection, bucket, 'quick')) == [uid.encode()] 71 | assert (await ingest.flushb(collection, bucket)) == 1 72 | assert (await search.query(collection, bucket, 'quick')) == [] 73 | 74 | 75 | async def test_flushc(search, ingest): 76 | bucket = str(uuid4()) 77 | uid = str(uuid4()) 78 | assert (await ingest.push(collection, bucket, uid, 'The quick brown fox jumps over the lazy dog')) == b'OK' 79 | assert (await search.query(collection, bucket, 'quick')) == [uid.encode()] 80 | assert (await ingest.flushc(collection)) == 1 81 | assert (await search.query(collection, bucket, 'quick')) == [] 82 | 83 | 84 | async def test_flusho(search, ingest): 85 | bucket = str(uuid4()) 86 | uid = str(uuid4()) 87 | assert (await ingest.push(collection, bucket, uid, 'The quick brown fox jumps over the lazy dog')) == b'OK' 88 | assert (await search.query(collection, bucket, 'quick')) == [uid.encode()] 89 | assert (await ingest.flusho(collection, bucket, uid)) == 6 90 | assert (await search.query(collection, bucket, 'quick')) == [] 91 | 92 | 93 | async def test_quit(search): 94 | assert (await search.quit()) == b'ENDED quit' 95 | try: 96 | assert (await search.ping()) == b'' 97 | except ConnectionClosed: 98 | pass 99 | else: 100 | raise AssertionError('Should raise exception after calling quit') 101 | 102 | 103 | async def test_pop(search, ingest): 104 | bucket = str(uuid4()) 105 | uid = str(uuid4()) 106 | assert (await ingest.push(collection, bucket, uid, 'The quick brown fox jumps over the lazy dog')) == b'OK' 107 | assert (await ingest.pop(collection, bucket, uid, 'quick')) == 1 108 | assert (await ingest.count(collection, bucket, uid)) == 5 109 | assert (await search.query(collection, bucket, 'quick')) == [] 110 | 111 | 112 | async def test_ingest(search, ingest): 113 | bucket = str(uuid4()) 114 | uid = str(uuid4()) 115 | assert (await ingest.push(collection, bucket, uid, 'żółć')) == b'OK' 116 | assert (await search.query(collection, bucket, 'żółć', limit=1)) == [uid.encode()] 117 | long_string = " ".join(str(uuid4()) for _ in range(10000)) 118 | total_size = sys.getsizeof(long_string) 119 | expected_chunks = math.ceil(total_size/BUFFER) 120 | chunks = list(ingest._chunk_generator(long_string, BUFFER)) 121 | assert expected_chunks == len(chunks) 122 | for chunk in chunks: 123 | assert sys.getsizeof(chunk) <= BUFFER 124 | assert (await ingest.push(collection, bucket, uid, long_string)) == b'OK' 125 | 126 | async def test_limit_offset(search, ingest): 127 | bucket = str(uuid4()) 128 | uid = str(uuid4()) 129 | uid2 = str(uuid4()) 130 | assert (await ingest.push(collection, bucket, uid, 'The quick brown fox jumps over the lazy dog')) == b'OK' 131 | assert (await ingest.push(collection, bucket, uid2, 132 | 'The quick brown fox jumps over the lazy dog complete')) == b'OK' 133 | assert (await search.query(collection, bucket, 'fox')) == [uid2.encode(), uid.encode()] 134 | assert (await search.query(collection, bucket, 'fox', limit=1)) == [uid2.encode()] 135 | assert (await search.query(collection, bucket, 'fox', limit=1, offset=1)) == [uid.encode()] 136 | 137 | 138 | async def test_mixed_commands(search): 139 | try: 140 | await search.push() 141 | except ClientError: 142 | pass 143 | else: 144 | raise AssertionError('Should raise exception when calling push on a search channel') 145 | 146 | 147 | async def test_channel_twice(search): 148 | try: 149 | await search.channel(Channel.CONTROL) 150 | except ClientError: 151 | pass 152 | else: 153 | raise AssertionError('Should raise exception when calling channel on initialized connection') 154 | 155 | 156 | async def test_command_before_channel(): 157 | a = Client() 158 | try: 159 | await a.ping() 160 | except ClientError: 161 | pass 162 | else: 163 | raise AssertionError('Should raise exception when not calling channel') 164 | -------------------------------------------------------------------------------- /asonic/client.py: -------------------------------------------------------------------------------- 1 | from collections import deque 2 | import sys 3 | from typing import List, Dict, Optional 4 | 5 | from asonic.connection import ConnectionPool 6 | from asonic.enums import Action, Channel, Command, all_commands, enabled_commands 7 | from asonic.exceptions import ClientError 8 | 9 | BUFFER = 20000 10 | 11 | def escape(t): 12 | if t is None: 13 | return "" 14 | return '"' + t.replace('"', '\\"').replace('\r\n', ' ') + '"' 15 | 16 | 17 | class Client: 18 | def __init__( 19 | self, 20 | host: str = 'localhost', 21 | port: int = 1491, 22 | password: str = 'SecretPassword', 23 | max_connections: int = 100 24 | ): 25 | self.host = host 26 | self.port = port 27 | self.password = password 28 | self.max_connections = max_connections 29 | 30 | self._channel = Channel.UNINITIALIZED 31 | self.pool = None # type: Optional[ConnectionPool] 32 | 33 | @classmethod 34 | async def create( 35 | self, 36 | host: str = 'localhost', 37 | port: int = 1491, 38 | password: str = 'SecretPassword', 39 | channel: Channel = Channel.SEARCH, 40 | max_connections: int = 100 41 | ): 42 | client: Client = Client( 43 | host=host, 44 | port=port, 45 | password=password, 46 | max_connections=max_connections 47 | ) 48 | _ = await client.channel(channel=channel) 49 | return client 50 | 51 | async def channel(self, channel: Channel) -> None: 52 | if self._channel != Channel.UNINITIALIZED: 53 | raise ClientError('Channel cannot be set twice') 54 | 55 | async def mock(*_, **__): 56 | raise ClientError(f'Command not available in {channel} channel') 57 | 58 | for command in all_commands: 59 | if command not in enabled_commands[channel]: 60 | setattr(self, command.value.lower(), mock) 61 | self._channel = channel 62 | self.pool = ConnectionPool( 63 | host=self.host, 64 | port=self.port, 65 | channel=channel, 66 | max_connections=self.max_connections, 67 | password=self.password, 68 | ) 69 | # force check if connection can be made 70 | _ = await self.ping() 71 | 72 | async def query( 73 | self, collection: str, bucket: str, terms: str, limit: int = None, offset: int = None, locale: str = None 74 | ) -> List[bytes]: 75 | """ 76 | query database 77 | time complexity: O(1) if enough exact word matches or O(N) if not enough exact matches where 78 | N is the number of alternate words tried, in practice it approaches O(1) 79 | :param collection: index collection (ie. what you search in, eg. messages, products, etc.) 80 | :param bucket: index bucket name (ie. user-specific search classifier in the collection if you have any 81 | :param terms: text for search terms 82 | :param limit: a positive integer number; set within allowed maximum & minimum limits 83 | :param offset: a positive integer number; set within allowed maximum & minimum limits 84 | :param locale: an ISO 639-3 locale code eg. `eng` for English 85 | (if set, the locale must be a valid ISO 639-3 code; if not set, the locale will be guessed from text) 86 | """ 87 | 88 | response = await self._command( 89 | Command.QUERY, collection, bucket, escape(terms), limit=limit, offset=offset, locale=locale 90 | ) 91 | tokens = response.split() 92 | if len(tokens) == 3: 93 | return [] 94 | else: 95 | return tokens[3:] 96 | 97 | async def suggest(self, collection: str, bucket: str, word: str, limit: int = None) -> List[bytes]: 98 | """ 99 | auto-completes word 100 | time complexity: O(1) 101 | :param collection: index collection (ie. what you search in, eg. messages, products, etc.) 102 | :param bucket: index bucket name (ie. user-specific search classifier in the collection if you have any 103 | :param word: text for search term 104 | :param limit: a positive integer number; set within allowed maximum & minimum limits 105 | """ 106 | response = await self._command(Command.SUGGEST, collection, bucket, escape(word), limit=limit) 107 | tokens = response.split() 108 | if len(tokens) == 3: 109 | return [] 110 | else: 111 | return tokens[3:] 112 | 113 | async def ping(self) -> bytes: 114 | """ 115 | ping server 116 | time complexity: O(1) 117 | """ 118 | return await self._command(Command.PING) 119 | 120 | async def quit(self) -> bytes: 121 | """ 122 | stop connection 123 | time complexity: O(1) 124 | """ 125 | return await self._command(Command.QUIT) 126 | 127 | async def help(self, manual: str) -> bytes: 128 | """ 129 | show help 130 | time complexity: O(1) 131 | :param manual: help manual to be shown (available manuals: commands) 132 | """ 133 | return await self._command(Command.HELP, manual) 134 | 135 | def _chunk_generator(self, text: str, buffer: int): 136 | empty_string_size = sys.getsizeof(str()) 137 | chars = deque(text.strip()) 138 | chunk_size = empty_string_size 139 | chunk = str() 140 | while chars: 141 | char = chars.popleft() 142 | chunk += char 143 | chunk_size += (sys.getsizeof(char) - empty_string_size) 144 | if (not chars) or (chunk_size > (buffer-100)): 145 | yield chunk 146 | chunk_size = empty_string_size 147 | chunk = str() 148 | 149 | async def push(self, collection: str, bucket: str, obj: str, text: str, locale: str = None) -> bytes: 150 | """ 151 | Push search data in the index 152 | time complexity: O(1) 153 | :param collection: index collection (ie. what you search in, eg. messages, products, etc.) 154 | :param bucket: index bucket name (ie. user-specific search classifier in the collection if you have any 155 | :param obj: object identifier that refers to an entity in an external database, where the searched object 156 | is stored (eg. you use Sonic to index CRM contacts by name; full CRM contact data is stored in a MySQL database 157 | in this case the object identifier in Sonic will be the MySQL primary key for the CRM contact) 158 | :param text: search text to be indexed (can be a single word, or a longer text; within maximum length safety 159 | limits) 160 | :param locale: an ISO 639-3 locale code eg. `eng` for English 161 | (if set, the locale must be a valid ISO 639-3 code; if not set, the locale will be guessed from text) 162 | """ 163 | for text_chunk in self._chunk_generator(text, BUFFER): 164 | result = await self._command(Command.PUSH, collection, bucket, obj, escape(text_chunk), locale=locale) 165 | return result 166 | 167 | async def pop(self, collection: str, bucket: str, obj: str, text: str) -> int: 168 | """ 169 | Pop search data from the index 170 | time complexity: O(1) 171 | :param collection: index collection (ie. what you search in, eg. messages, products, etc.) 172 | :param bucket: index bucket name (ie. user-specific search classifier in the collection if you have any 173 | :param obj: object identifier that refers to an entity in an external database, where the searched object 174 | is stored (eg. you use Sonic to index CRM contacts by name; full CRM contact data is stored in a MySQL database 175 | in this case the object identifier in Sonic will be the MySQL primary key for the CRM contact) 176 | :param text: search text to be indexed (can be a single word, or a longer text; within maximum length safety 177 | limits) 178 | """ 179 | result = await self._command(Command.POP, collection, bucket, obj, escape(text)) 180 | return int(result[7:]) 181 | 182 | async def flushc(self, collection: str) -> int: 183 | """ 184 | Flush all indexed data from a collection 185 | time complexity: O(1) 186 | :param collection: index collection (ie. what you search in, eg. messages, products, etc.) 187 | """ 188 | return int((await self._command(Command.FLUSHC, collection))[7:]) 189 | 190 | async def flushb(self, collection: str, bucket: str) -> int: 191 | """ 192 | Flush all indexed data from a bucket in a collection 193 | time complexity: O(1) 194 | :param collection: index collection (ie. what you search in, eg. messages, products, etc.) 195 | :param bucket: index bucket name (ie. user-specific search classifier in the collection if you have any 196 | """ 197 | 198 | return int((await self._command(Command.FLUSHB, collection, bucket))[7:]) 199 | 200 | async def flusho(self, collection: str, bucket: str, obj: str) -> int: 201 | """ 202 | Flush all indexed data from an object in a bucket in collection 203 | time complexity: O(1) 204 | :param collection: index collection (ie. what you search in, eg. messages, products, etc.) 205 | :param bucket: index bucket name (ie. user-specific search classifier in the collection if you have any 206 | :param obj: object identifier that refers to an entity in an external database, where the searched object 207 | is stored (eg. you use Sonic to index CRM contacts by name; full CRM contact data is stored in a MySQL database 208 | in this case the object identifier in Sonic will be the MySQL primary key for the CRM contact) 209 | """ 210 | 211 | return int((await self._command(Command.FLUSHO, collection, bucket, obj))[7:]) 212 | 213 | async def count(self, collection: str, bucket: str = None, obj: str = None) -> int: 214 | """ 215 | Count indexed search data 216 | time complexity: O(1) 217 | :param collection: index collection (ie. what you search in, eg. messages, products, etc.) 218 | :param bucket: index bucket name (ie. user-specific search classifier in the collection if you have any 219 | :param obj: object identifier that refers to an entity in an external database, where the searched object 220 | is stored (eg. you use Sonic to index CRM contacts by name; full CRM contact data is stored in a MySQL database 221 | in this case the object identifier in Sonic will be the MySQL primary key for the CRM contact) 222 | """ 223 | result = await self._command(Command.COUNT, collection, bucket=bucket, object=obj) 224 | return int(result[7:]) 225 | 226 | async def trigger(self, action: Action = None) -> bytes: 227 | """ 228 | Trigger an action 229 | time complexity: O(1) 230 | :param action: action to be triggered (available actions: consolidate) 231 | """ 232 | return await self._command(Command.TRIGGER, action=action.value if action else None) 233 | 234 | async def info(self) -> Dict: 235 | """ 236 | Get server information 237 | time complexity: O(1) 238 | """ 239 | res = await self._command(Command.INFO) 240 | return dict(map(lambda x: x.replace('(', ' ').replace(')', '').split(), res[7:].decode().split())) 241 | 242 | async def list(self, collection: str, bucket: str = None, limit: int = None, offset: int = None) -> Dict: 243 | """ 244 | Enumerates all words in an index 245 | time complexity: O(1) 246 | """ 247 | response = await self._command(command=Command.LIST, collection=collection, bucket=bucket, limit=limit, offset=offset) 248 | tokens = response.split() 249 | if len(tokens) == 3: 250 | return [] 251 | else: 252 | return tokens[3:] 253 | 254 | async def _command(self, command: Command, *args, **kwargs) -> bytes: 255 | if self._channel == Channel.UNINITIALIZED: 256 | raise ClientError('Call .channel before running any command') 257 | 258 | assert self.pool is not None 259 | c = await self.pool.get_connection() 260 | 261 | values = [] 262 | for k in kwargs: 263 | if kwargs[k] is not None: 264 | if k == 'limit': 265 | values.append(f'LIMIT({kwargs[k]})') 266 | elif k == 'offset': 267 | values.append(f'OFFSET({kwargs[k]})') 268 | elif k == 'locale': 269 | values.append(f'LANG({kwargs[k]})') 270 | else: 271 | values.append(kwargs[k]) 272 | await c.write(f'{command.value} {" ".join(args)} {" ".join(values)}'.strip()) 273 | 274 | result = await c.read() 275 | if command in {Command.QUERY, Command.SUGGEST, Command.LIST}: 276 | result = await c.read() 277 | await self.pool.release(c) 278 | if command == Command.QUIT: 279 | await self.pool.destroy() 280 | return result 281 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | Mozilla Public License Version 2.0 2 | ================================== 3 | 4 | 1. Definitions 5 | -------------- 6 | 7 | 1.1. "Contributor" 8 | means each individual or legal entity that creates, contributes to 9 | the creation of, or owns Covered Software. 10 | 11 | 1.2. "Contributor Version" 12 | means the combination of the Contributions of others (if any) used 13 | by a Contributor and that particular Contributor's Contribution. 14 | 15 | 1.3. "Contribution" 16 | means Covered Software of a particular Contributor. 17 | 18 | 1.4. "Covered Software" 19 | means Source Code Form to which the initial Contributor has attached 20 | the notice in Exhibit A, the Executable Form of such Source Code 21 | Form, and Modifications of such Source Code Form, in each case 22 | including portions thereof. 23 | 24 | 1.5. "Incompatible With Secondary Licenses" 25 | means 26 | 27 | (a) that the initial Contributor has attached the notice described 28 | in Exhibit B to the Covered Software; or 29 | 30 | (b) that the Covered Software was made available under the terms of 31 | version 1.1 or earlier of the License, but not also under the 32 | terms of a Secondary License. 33 | 34 | 1.6. "Executable Form" 35 | means any form of the work other than Source Code Form. 36 | 37 | 1.7. "Larger Work" 38 | means a work that combines Covered Software with other material, in 39 | a separate file or files, that is not Covered Software. 40 | 41 | 1.8. "License" 42 | means this document. 43 | 44 | 1.9. "Licensable" 45 | means having the right to grant, to the maximum extent possible, 46 | whether at the time of the initial grant or subsequently, any and 47 | all of the rights conveyed by this License. 48 | 49 | 1.10. "Modifications" 50 | means any of the following: 51 | 52 | (a) any file in Source Code Form that results from an addition to, 53 | deletion from, or modification of the contents of Covered 54 | Software; or 55 | 56 | (b) any new file in Source Code Form that contains any Covered 57 | Software. 58 | 59 | 1.11. "Patent Claims" of a Contributor 60 | means any patent claim(s), including without limitation, method, 61 | process, and apparatus claims, in any patent Licensable by such 62 | Contributor that would be infringed, but for the grant of the 63 | License, by the making, using, selling, offering for sale, having 64 | made, import, or transfer of either its Contributions or its 65 | Contributor Version. 66 | 67 | 1.12. "Secondary License" 68 | means either the GNU General Public License, Version 2.0, the GNU 69 | Lesser General Public License, Version 2.1, the GNU Affero General 70 | Public License, Version 3.0, or any later versions of those 71 | licenses. 72 | 73 | 1.13. "Source Code Form" 74 | means the form of the work preferred for making modifications. 75 | 76 | 1.14. "You" (or "Your") 77 | means an individual or a legal entity exercising rights under this 78 | License. For legal entities, "You" includes any entity that 79 | controls, is controlled by, or is under common control with You. For 80 | purposes of this definition, "control" means (a) the power, direct 81 | or indirect, to cause the direction or management of such entity, 82 | whether by contract or otherwise, or (b) ownership of more than 83 | fifty percent (50%) of the outstanding shares or beneficial 84 | ownership of such entity. 85 | 86 | 2. License Grants and Conditions 87 | -------------------------------- 88 | 89 | 2.1. Grants 90 | 91 | Each Contributor hereby grants You a world-wide, royalty-free, 92 | non-exclusive license: 93 | 94 | (a) under intellectual property rights (other than patent or trademark) 95 | Licensable by such Contributor to use, reproduce, make available, 96 | modify, display, perform, distribute, and otherwise exploit its 97 | Contributions, either on an unmodified basis, with Modifications, or 98 | as part of a Larger Work; and 99 | 100 | (b) under Patent Claims of such Contributor to make, use, sell, offer 101 | for sale, have made, import, and otherwise transfer either its 102 | Contributions or its Contributor Version. 103 | 104 | 2.2. Effective Date 105 | 106 | The licenses granted in Section 2.1 with respect to any Contribution 107 | become effective for each Contribution on the date the Contributor first 108 | distributes such Contribution. 109 | 110 | 2.3. Limitations on Grant Scope 111 | 112 | The licenses granted in this Section 2 are the only rights granted under 113 | this License. No additional rights or licenses will be implied from the 114 | distribution or licensing of Covered Software under this License. 115 | Notwithstanding Section 2.1(b) above, no patent license is granted by a 116 | Contributor: 117 | 118 | (a) for any code that a Contributor has removed from Covered Software; 119 | or 120 | 121 | (b) for infringements caused by: (i) Your and any other third party's 122 | modifications of Covered Software, or (ii) the combination of its 123 | Contributions with other software (except as part of its Contributor 124 | Version); or 125 | 126 | (c) under Patent Claims infringed by Covered Software in the absence of 127 | its Contributions. 128 | 129 | This License does not grant any rights in the trademarks, service marks, 130 | or logos of any Contributor (except as may be necessary to comply with 131 | the notice requirements in Section 3.4). 132 | 133 | 2.4. Subsequent Licenses 134 | 135 | No Contributor makes additional grants as a result of Your choice to 136 | distribute the Covered Software under a subsequent version of this 137 | License (see Section 10.2) or under the terms of a Secondary License (if 138 | permitted under the terms of Section 3.3). 139 | 140 | 2.5. Representation 141 | 142 | Each Contributor represents that the Contributor believes its 143 | Contributions are its original creation(s) or it has sufficient rights 144 | to grant the rights to its Contributions conveyed by this License. 145 | 146 | 2.6. Fair Use 147 | 148 | This License is not intended to limit any rights You have under 149 | applicable copyright doctrines of fair use, fair dealing, or other 150 | equivalents. 151 | 152 | 2.7. Conditions 153 | 154 | Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted 155 | in Section 2.1. 156 | 157 | 3. Responsibilities 158 | ------------------- 159 | 160 | 3.1. Distribution of Source Form 161 | 162 | All distribution of Covered Software in Source Code Form, including any 163 | Modifications that You create or to which You contribute, must be under 164 | the terms of this License. You must inform recipients that the Source 165 | Code Form of the Covered Software is governed by the terms of this 166 | License, and how they can obtain a copy of this License. You may not 167 | attempt to alter or restrict the recipients' rights in the Source Code 168 | Form. 169 | 170 | 3.2. Distribution of Executable Form 171 | 172 | If You distribute Covered Software in Executable Form then: 173 | 174 | (a) such Covered Software must also be made available in Source Code 175 | Form, as described in Section 3.1, and You must inform recipients of 176 | the Executable Form how they can obtain a copy of such Source Code 177 | Form by reasonable means in a timely manner, at a charge no more 178 | than the cost of distribution to the recipient; and 179 | 180 | (b) You may distribute such Executable Form under the terms of this 181 | License, or sublicense it under different terms, provided that the 182 | license for the Executable Form does not attempt to limit or alter 183 | the recipients' rights in the Source Code Form under this License. 184 | 185 | 3.3. Distribution of a Larger Work 186 | 187 | You may create and distribute a Larger Work under terms of Your choice, 188 | provided that You also comply with the requirements of this License for 189 | the Covered Software. If the Larger Work is a combination of Covered 190 | Software with a work governed by one or more Secondary Licenses, and the 191 | Covered Software is not Incompatible With Secondary Licenses, this 192 | License permits You to additionally distribute such Covered Software 193 | under the terms of such Secondary License(s), so that the recipient of 194 | the Larger Work may, at their option, further distribute the Covered 195 | Software under the terms of either this License or such Secondary 196 | License(s). 197 | 198 | 3.4. Notices 199 | 200 | You may not remove or alter the substance of any license notices 201 | (including copyright notices, patent notices, disclaimers of warranty, 202 | or limitations of liability) contained within the Source Code Form of 203 | the Covered Software, except that You may alter any license notices to 204 | the extent required to remedy known factual inaccuracies. 205 | 206 | 3.5. Application of Additional Terms 207 | 208 | You may choose to offer, and to charge a fee for, warranty, support, 209 | indemnity or liability obligations to one or more recipients of Covered 210 | Software. However, You may do so only on Your own behalf, and not on 211 | behalf of any Contributor. You must make it absolutely clear that any 212 | such warranty, support, indemnity, or liability obligation is offered by 213 | You alone, and You hereby agree to indemnify every Contributor for any 214 | liability incurred by such Contributor as a result of warranty, support, 215 | indemnity or liability terms You offer. You may include additional 216 | disclaimers of warranty and limitations of liability specific to any 217 | jurisdiction. 218 | 219 | 4. Inability to Comply Due to Statute or Regulation 220 | --------------------------------------------------- 221 | 222 | If it is impossible for You to comply with any of the terms of this 223 | License with respect to some or all of the Covered Software due to 224 | statute, judicial order, or regulation then You must: (a) comply with 225 | the terms of this License to the maximum extent possible; and (b) 226 | describe the limitations and the code they affect. Such description must 227 | be placed in a text file included with all distributions of the Covered 228 | Software under this License. Except to the extent prohibited by statute 229 | or regulation, such description must be sufficiently detailed for a 230 | recipient of ordinary skill to be able to understand it. 231 | 232 | 5. Termination 233 | -------------- 234 | 235 | 5.1. The rights granted under this License will terminate automatically 236 | if You fail to comply with any of its terms. However, if You become 237 | compliant, then the rights granted under this License from a particular 238 | Contributor are reinstated (a) provisionally, unless and until such 239 | Contributor explicitly and finally terminates Your grants, and (b) on an 240 | ongoing basis, if such Contributor fails to notify You of the 241 | non-compliance by some reasonable means prior to 60 days after You have 242 | come back into compliance. Moreover, Your grants from a particular 243 | Contributor are reinstated on an ongoing basis if such Contributor 244 | notifies You of the non-compliance by some reasonable means, this is the 245 | first time You have received notice of non-compliance with this License 246 | from such Contributor, and You become compliant prior to 30 days after 247 | Your receipt of the notice. 248 | 249 | 5.2. If You initiate litigation against any entity by asserting a patent 250 | infringement claim (excluding declaratory judgment actions, 251 | counter-claims, and cross-claims) alleging that a Contributor Version 252 | directly or indirectly infringes any patent, then the rights granted to 253 | You by any and all Contributors for the Covered Software under Section 254 | 2.1 of this License shall terminate. 255 | 256 | 5.3. In the event of termination under Sections 5.1 or 5.2 above, all 257 | end user license agreements (excluding distributors and resellers) which 258 | have been validly granted by You or Your distributors under this License 259 | prior to termination shall survive termination. 260 | 261 | 6. Disclaimer of Warranty 262 | ------------------------- 263 | 264 | > Covered Software is provided under this License on an "as is" 265 | > basis, without warranty of any kind, either expressed, implied, or 266 | > statutory, including, without limitation, warranties that the 267 | > Covered Software is free of defects, merchantable, fit for a 268 | > particular purpose or non-infringing. The entire risk as to the 269 | > quality and performance of the Covered Software is with You. 270 | > Should any Covered Software prove defective in any respect, You 271 | > (not any Contributor) assume the cost of any necessary servicing, 272 | > repair, or correction. This disclaimer of warranty constitutes an 273 | > essential part of this License. No use of any Covered Software is 274 | > authorized under this License except under this disclaimer. 275 | 276 | 277 | 7. Limitation of Liability 278 | -------------------------- 279 | 280 | > Under no circumstances and under no legal theory, whether tort 281 | > (including negligence), contract, or otherwise, shall any 282 | > Contributor, or anyone who distributes Covered Software as 283 | > permitted above, be liable to You for any direct, indirect, 284 | > special, incidental, or consequential damages of any character 285 | > including, without limitation, damages for lost profits, loss of 286 | > goodwill, work stoppage, computer failure or malfunction, or any 287 | > and all other commercial damages or losses, even if such party 288 | > shall have been informed of the possibility of such damages. This 289 | > limitation of liability shall not apply to liability for death or 290 | > personal injury resulting from such party's negligence to the 291 | > extent applicable law prohibits such limitation. Some 292 | > jurisdictions do not allow the exclusion or limitation of 293 | > incidental or consequential damages, so this exclusion and 294 | > limitation may not apply to You. 295 | 296 | 8. Litigation 297 | ------------- 298 | 299 | Any litigation relating to this License may be brought only in the 300 | courts of a jurisdiction where the defendant maintains its principal 301 | place of business and such litigation shall be governed by laws of that 302 | jurisdiction, without reference to its conflict-of-law provisions. 303 | Nothing in this Section shall prevent a party's ability to bring 304 | cross-claims or counter-claims. 305 | 306 | 9. Miscellaneous 307 | ---------------- 308 | 309 | This License represents the complete agreement concerning the subject 310 | matter hereof. If any provision of this License is held to be 311 | unenforceable, such provision shall be reformed only to the extent 312 | necessary to make it enforceable. Any law or regulation which provides 313 | that the language of a contract shall be construed against the drafter 314 | shall not be used to construe this License against a Contributor. 315 | 316 | 10. Versions of the License 317 | --------------------------- 318 | 319 | 10.1. New Versions 320 | 321 | Mozilla Foundation is the license steward. Except as provided in Section 322 | 10.3, no one other than the license steward has the right to modify or 323 | publish new versions of this License. Each version will be given a 324 | distinguishing version number. 325 | 326 | 10.2. Effect of New Versions 327 | 328 | You may distribute the Covered Software under the terms of the version 329 | of the License under which You originally received the Covered Software, 330 | or under the terms of any subsequent version published by the license 331 | steward. 332 | 333 | 10.3. Modified Versions 334 | 335 | If you create software not governed by this License, and you want to 336 | create a new license for such software, you may create and use a 337 | modified version of this License if you rename the license and remove 338 | any references to the name of the license steward (except to note that 339 | such modified license differs from this License). 340 | 341 | 10.4. Distributing Source Code Form that is Incompatible With Secondary 342 | Licenses 343 | 344 | If You choose to distribute Source Code Form that is Incompatible With 345 | Secondary Licenses under the terms of this version of the License, the 346 | notice described in Exhibit B of this License must be attached. 347 | 348 | Exhibit A - Source Code Form License Notice 349 | ------------------------------------------- 350 | 351 | This Source Code Form is subject to the terms of the Mozilla Public 352 | License, v. 2.0. If a copy of the MPL was not distributed with this 353 | file, You can obtain one at http://mozilla.org/MPL/2.0/. 354 | 355 | If it is not possible or desirable to put the notice in a particular 356 | file, then You may include the notice in a location (such as a LICENSE 357 | file in a relevant directory) where a recipient would be likely to look 358 | for such a notice. 359 | 360 | You may add additional accurate notices of copyright ownership. 361 | 362 | Exhibit B - "Incompatible With Secondary Licenses" Notice 363 | --------------------------------------------------------- 364 | 365 | This Source Code Form is "Incompatible With Secondary Licenses", as 366 | defined by the Mozilla Public License, v. 2.0. 367 | --------------------------------------------------------------------------------