├── .gitignore ├── .pre-commit-config.yaml ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── exact_rag ├── __init__.py ├── audio_cap.py ├── config.py ├── config_old.py ├── dataemb.py ├── image_cap.py ├── main.py ├── schemas.py └── settings.py ├── examples ├── README.md ├── client_rag.py └── test.txt ├── frontend ├── __init__.py └── ui.py ├── poetry.lock ├── pyproject.toml ├── scripts └── docker-es.sh ├── settings.toml.template └── tests ├── __init__.py ├── conftest.py ├── ollama_embedding.json ├── test_config.py ├── test_database_chroma.toml ├── test_database_elastic.toml ├── test_dataemb.py ├── test_embedding_ollama.toml ├── test_embedding_openai.toml └── test_settings.py /.gitignore: -------------------------------------------------------------------------------- 1 | settings.toml 2 | -------------------------------------------------------------------------------- /.pre-commit-config.yaml: -------------------------------------------------------------------------------- 1 | repos: 2 | - repo: https://github.com/pre-commit/pre-commit-hooks 3 | rev: v4.3.0 4 | hooks: 5 | - id: check-yaml 6 | - id: end-of-file-fixer 7 | - id: trailing-whitespace 8 | - repo: https://github.com/astral-sh/ruff-pre-commit 9 | rev: v0.1.11 10 | hooks: 11 | - id: ruff 12 | args: [ --fix ] 13 | - id: ruff-format 14 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | 2 | # Contributing to Exact-RAG 3 | First off, thanks for taking the time to contribute! ❤️ 4 | 5 | All types of contributions are encouraged and valued. See the [Table of Contents](#table-of-contents) for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. 🎉 6 | 7 | > And if you like the project, but just don't have time to contribute, that's fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about: 8 | > - Star the project 9 | > - Tweet about it 10 | > - Refer this project in your project's readme 11 | > - Mention the project at local meetups and tell your friends/colleagues 12 | 13 | 14 | ## Table of Contents 15 | 16 | - [I Have a Question](#i-have-a-question) 17 | - [I Want To Contribute](#i-want-to-contribute) 18 | - [Reporting Bugs](#reporting-bugs) 19 | - [Suggesting Enhancements](#suggesting-enhancements) 20 | - [Your First Code Contribution](#your-first-code-contribution) 21 | - [Styleguides](#styleguides) 22 | - [Commit Messages](#commit-messages) 23 | - [Join The Project Team](#join-the-project-team) 24 | 25 | 26 | ## I Have a Question 27 | 28 | Before you ask a question, it is best to search for existing [Issues](/issues) that might help you. In case you have found a suitable issue and still need clarification, you can write your question in this issue. It is also advisable to search the internet for answers first. 29 | 30 | If you then still feel the need to ask a question and need clarification, we recommend the following: 31 | 32 | - Open an [Issue](/issues/new). 33 | - Provide as much context as you can about what you're running into. 34 | - Provide project and platform versions (nodejs, npm, etc), depending on what seems relevant. 35 | 36 | We will then take care of the issue as soon as possible. 37 | 38 | 52 | 53 | ## I Want To Contribute 54 | 55 | > ### Legal Notice 56 | > When contributing to this project, you must agree that you have authored 100% of the content, that you have the necessary rights to the content and that the content you contribute may be provided under the project license. 57 | 58 | ### Reporting Bugs 59 | 60 | 61 | #### Before Submitting a Bug Report 62 | 63 | A good bug report shouldn't leave others needing to chase you up for more information. Therefore, we ask you to investigate carefully, collect information and describe the issue in detail in your report. Please complete the following steps in advance to help us fix any potential bug as fast as possible. 64 | 65 | - Make sure that you are using the latest version. 66 | - To see if other users have experienced (and potentially already solved) the same issue you are having, check if there is not already a bug report existing for your bug or error in the [bug tracker](issues?q=label%3Abug). 67 | - Also make sure to search the internet (including Stack Overflow) to see if users outside of the GitHub community have discussed the issue. 68 | - Collect information about the bug: 69 | - Stack trace (Traceback) 70 | - OS, Platform and Version (Windows, Linux, macOS, x86, ARM) 71 | - Version of the interpreter, compiler, SDK, runtime environment, package manager, depending on what seems relevant. 72 | - Possibly your input and the output 73 | - Can you reliably reproduce the issue? And can you also reproduce it with older versions? 74 | 75 | 76 | #### How Do I Submit a Good Bug Report? 77 | 78 | > You must never report security related issues, vulnerabilities or bugs including sensitive information to the issue tracker, or elsewhere in public. Instead sensitive bugs must be sent by email to <>. 79 | 80 | 81 | We use GitHub issues to track bugs and errors. If you run into an issue with the project: 82 | 83 | - Open an [Issue](/issues/new). (Since we can't be sure at this point whether it is a bug or not, we ask you not to talk about a bug yet and not to label the issue.) 84 | - Explain the behavior you would expect and the actual behavior. 85 | - Please provide as much context as possible and describe the *reproduction steps* that someone else can follow to recreate the issue on their own. This usually includes your code. For good bug reports you should isolate the problem and create a reduced test case. 86 | - Provide the information you collected in the previous section. 87 | 88 | Once it's filed: 89 | 90 | - The project team will label the issue accordingly. 91 | - A team member will try to reproduce the issue with your provided steps. If there are no reproduction steps or no obvious way to reproduce the issue, the team will ask you for those steps and mark the issue as `needs-repro`. Bugs with the `needs-repro` tag will not be addressed until they are reproduced. 92 | - If the team is able to reproduce the issue, it will be marked `needs-fix`, as well as possibly other tags (such as `critical`), and the issue will be left to be [implemented by someone](#your-first-code-contribution). 93 | 94 | 95 | 96 | 97 | ### Suggesting Enhancements 98 | 99 | This section guides you through submitting an enhancement suggestion for exact-rag, **including completely new features and minor improvements to existing functionality**. Following these guidelines will help maintainers and the community to understand your suggestion and find related suggestions. 100 | 101 | 102 | #### Before Submitting an Enhancement 103 | 104 | - Make sure that you are using the latest version. 105 | - Perform a [search](/issues) to see if the enhancement has already been suggested. If it has, add a comment to the existing issue instead of opening a new one. 106 | - Find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Keep in mind that we want features that will be useful to the majority of our users and not just a small subset. If you're just targeting a minority of users, consider writing an add-on/plugin library. 107 | 108 | 109 | #### How Do I Submit a Good Enhancement Suggestion? 110 | 111 | Enhancement suggestions are tracked as [GitHub issues](/issues). 112 | 113 | - Use a **clear and descriptive title** for the issue to identify the suggestion. 114 | - Provide a **step-by-step description of the suggested enhancement** in as many details as possible. 115 | - **Describe the current behavior** and **explain which behavior you expected to see instead** and why. At this point you can also tell which alternatives do not work for you. 116 | - You may want to **include screenshots and animated GIFs** which help you demonstrate the steps or point out the part which the suggestion is related to. You can use [this tool](https://www.cockos.com/licecap/) to record GIFs on macOS and Windows, and [this tool](https://github.com/colinkeenan/silentcast) or [this tool](https://github.com/GNOME/byzanz) on Linux. 117 | - **Explain why this enhancement would be useful** to most exact-rag users. You may also want to point out the other projects that solved it better and which could serve as inspiration. 118 | 119 | #### Your First Code Contribution 120 | 121 | Found an awesome package, article, blog, video etc. that you would like to add? Send me a pull request! Just follow the guidelines below. 122 | 123 | ##### Fork the Repository 124 | - Forking creates a copy of the repository under your GitHub account. 125 | - Click the "Fork" button in the top-right corner of the `exact-rag` repository page. Wait for a copy of the repository to appear in your Github account. 126 | 127 | ##### Clone the Repository from your Fork 128 | - Choose your favorite editor and open the folder. 129 | - Next, clone your fork to your computer, so you can work on the project locally. 130 | - Click the "Code" button on your forked repository, and copy the repository URL (e.g., `https://github.com/[githubusername]/exact-rag.git`). 131 | - Open your terminal or Git bash. Use the git clone command to clone the repository: 132 | - `git clone https://github.com/[githubusername]/exact-rag.git` 133 | 134 | ##### Create a New Branch: 135 | - Go to the Directory you cloned by using the `cd` command. 136 | - Create a new branch to work in by using the `git checkout -b` command and replacing `branch-name` with a descriptive name for your branch: 137 | - `cd exact-rag` 138 | - `git checkout -b ` 139 | 140 | ##### Make Your Contribution 141 | - On your editor, make your changes or contribute by either: 142 | - Fixing errors in documentation. 143 | - Adding an article, blog, video. 144 | - Fixing bugs, etc. 145 | 146 | ##### Commit and Push 147 | - After making changes, you can commit them and push it back up to Github using: 148 | - `git add` command 149 | - `git commit -m "Add commit message"` 150 | - `git push origin .` 151 | 152 | ##### Create a Pull Request 153 | - The final step is to go back to your forked repository on GitHub. 154 | - Look for the notification about creating a pull request and click on it. 155 | - Alternatively, you can navigate to the "Pull Requests" tab and create a new pull request. 156 | - Write a clear and descriptive title and comment explaining the changes you made. Then, submit the pull request. 157 | - Wait for the project maintainers to review your pull request and give you feedback. 158 | - If your contribution is approved, it will be merged into the main repository. 159 | - Congratulations on making your first pull request! 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | ## Attribution 168 | This guide is based on the **contributing-gen**. [Make your own](https://github.com/bttger/contributing-gen)! 169 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 eXact lab 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Exact-RAG: Multimodal Retrieval-Augmented Generation :dart: 2 | 3 | Exact-RAG is a powerful multimodal model designed for Retrieval-Augmented Generation (RAG). It seamlessly integrates text, visual and audio information, allowing for enhanced content understanding and generation. This repository contains the source code, and example scripts to facilitate the usage and exploration of Exact-RAG. 4 | 5 | ## Prerequisites 6 | 7 | * Python >= 3.10 8 | * Poetry ([Install](https://python-poetry.org/docs/)) 9 | 10 | #### LLMs 11 | * To use OpenAI models you need a valid key at this [link](https://platform.openai.com/api-key) 12 | * To use local LLM models you need Ollama. [Here](https://ollama.ai/download) the instructions to install 13 | 14 | #### Ollama 15 | You should download a LLM model usin ollama. By default the `orca2` model is set in `settings.toml`, to download just run: 16 | 17 | ```bash 18 | ollama run orca2 19 | ``` 20 | 21 | #### Databases 22 | * To use Elasticsearch you should have a running cluster or you can use a test deployment using the [docker-es.sh](./scripts/docker-es.sh) script. 23 | 24 | 25 | ## Installation 26 | 27 | Clone the repository: 28 | 29 | ```bash 30 | poetry install # -E audio -E image 31 | ``` 32 | 33 | * audio extra will install `openai-whisper` for speech-to-text 34 | 35 | * image extra will install `transformers` and `pillow` for image captioning 36 | 37 | ## Usage - Server 38 | 39 | First step is to modify the `settings.toml` file. 40 | 41 | Then starting the web server just running: 42 | 43 | ```bash 44 | poetry run python exact_rag/main.py 45 | ``` 46 | 47 | > NOTE: The first start up could required some time to download the selected models, expecially for image captioning 48 | 49 | ## Usage - UI (Demo purpose) 50 | 51 | UI Demo is build upon `streamlit` and it is made just to demo purposes. 52 | If you want to run locally to quick try eXact-RAG features, be sure to have install the packege with `dev` dependencies, and then: 53 | 54 | ```bash 55 | poetry run streamlit run frontend/ui.py 56 | ``` 57 | 58 | ## Examples 59 | 60 | You can find some examples of usage in the examples [folder](./examples/) 61 | 62 | Chat with __images__ example: 63 | 64 | https://github.com/exactlab/exact-rag/assets/43796979/d8daaa12-664b-4a38-9959-e79f752a03b1 65 | 66 | Chat with __PDF__ and __Tables__: 67 | 68 | https://github.com/exactlab/exact-rag/assets/43796979/dd5c6737-99bf-4fa1-b30e-1d3e0d22b2d7 69 | 70 | 71 | 72 | ## Tests 73 | 74 | To run the tests: 75 | * be sure to have install the `dev` dependecies with: 76 | ```bash 77 | poetry install --with-dev # -E audio -E image 78 | ``` 79 | * then run: 80 | ```bash 81 | poetry run pytest tests/ 82 | ``` 83 | 84 | ## Contributing 85 | 86 | We welcome contributions! If you'd like to contribute to Exact-RAG, please follow our contribution [guidelines](CONTRIBUTING.md). 87 | 88 | 89 | * Fork the repo. 90 | * Clone the repo from your codebase 91 | * Choose your favorite editor and open the folder. 92 | * Create a new branch: git checkout -b 93 | * Make changes, commit them and push it back up to github using git push origin . 94 | * Open pull request on GitHub. 95 | 96 | 97 | ## License 98 | 99 | This project is licensed under the MIT License. 100 | -------------------------------------------------------------------------------- /exact_rag/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/exactlab/exact-rag/e9311e060c73ede33a1baee6cb4c5897e8eb8a9e/exact_rag/__init__.py -------------------------------------------------------------------------------- /exact_rag/audio_cap.py: -------------------------------------------------------------------------------- 1 | from whisper.model import transcribe_function 2 | 3 | 4 | def audio_caption(audio_file: str) -> str: 5 | res = transcribe_function(audio_file) 6 | return res 7 | -------------------------------------------------------------------------------- /exact_rag/config.py: -------------------------------------------------------------------------------- 1 | from pydantic import Field, model_validator 2 | from exact_rag.settings import Settings 3 | from exact_rag.settings import FromDict 4 | from typing import Annotated 5 | from enum import Enum 6 | from os import environ 7 | 8 | 9 | class EmbeddingType(str, Enum): 10 | openai = "openai" 11 | ollama = "ollama" 12 | 13 | 14 | class Embeddings(Settings): 15 | type: EmbeddingType = Field(description="Type of embedding (EmbeddingType).") 16 | api_key: str | None = Field(description="Token for openAI service.", default=None) 17 | model: str | None = Field( 18 | description="AI model (used only for ollama embedding).", default=None 19 | ) 20 | chat_model_name: Annotated[str, FromDict("chat", "model_name")] = Field( 21 | description="Chat model." 22 | ) 23 | chat_temperature: Annotated[float, FromDict("chat", "temperature")] = Field( 24 | description="Temperature parameter of the chat." 25 | ) 26 | chain_type: str = Field(description="Langchain chain type.", default="stuff") 27 | search_type: Annotated[str, FromDict("search", "type")] = Field( 28 | description="Type fo search in database.", default="mmr" 29 | ) 30 | search_k: Annotated[int, FromDict("search", "k")] = Field( 31 | description="Amount of documents to return." 32 | ) 33 | search_fetch_k: Annotated[int, FromDict("search", "fetch_k")] = Field( 34 | description="Amount of documents to pass to search algorithm." 35 | ) 36 | 37 | @model_validator(mode="after") 38 | def check_optionals(self) -> "Embeddings": 39 | if self.type == EmbeddingType.ollama: 40 | if not self.model: 41 | raise ValueError("For ollama embedding type you must specify a model.") 42 | elif self.type == EmbeddingType.openai: 43 | if not self.api_key: 44 | self.api_key = environ.get("OPENAI_API_KEY") 45 | return self 46 | 47 | 48 | class DatabaseType(str, Enum): 49 | chroma = "chroma" 50 | elastic = "elastic" 51 | 52 | 53 | class Databases(Settings): 54 | type: str = Field(description="Type of database (DatabaseType).") 55 | persist_directory: str | None = Field( 56 | description="Path of persistent file (used only for chroma databaase).", 57 | default=None, 58 | ) 59 | url: str | None = Field( 60 | description="URL of database (used only for Elasticsearch).", default=None 61 | ) 62 | distance_strategy: str | None = Field( 63 | description="Distance (used only fo Elasticsearch).", default=None 64 | ) 65 | collection_name: str = Field(description="Name of text collection.") 66 | sql_namespace: Annotated[str, FromDict("sql", "namespace")] = Field( 67 | description="SQL duplicates database namespace." 68 | ) 69 | sql_url: Annotated[str, FromDict("sql", "url")] = Field( 70 | description="URL of SQL duplicates database." 71 | ) 72 | splitter_chunk_size: Annotated[int, FromDict("splitter", "chunk_size")] = Field( 73 | description="Chunk size for content splitting.", default=1000 74 | ) 75 | splitter_chunk_overlap: Annotated[ 76 | int, FromDict("splitter", "chunk_overlap") 77 | ] = Field(description="Overlapping size for text splitter chunks.", default=0) 78 | 79 | @model_validator(mode="after") 80 | def check_optional(self) -> "Databases": 81 | if self.type == DatabaseType.chroma: 82 | if not self.persist_directory: 83 | raise ValueError( 84 | "For chroma database type you must specify a persist_directory." 85 | ) 86 | elif self.type == DatabaseType.elastic: 87 | if not self.url: 88 | raise ValueError( 89 | "For elasticsearch database type you must specify a url." 90 | ) 91 | if not self.distance_strategy: 92 | raise ValueError( 93 | "For elasticsearch database type you must specify a distance." 94 | ) 95 | return self 96 | -------------------------------------------------------------------------------- /exact_rag/config_old.py: -------------------------------------------------------------------------------- 1 | from pydantic import BaseModel, Field 2 | from typing import Any 3 | from enum import Enum 4 | 5 | 6 | class DictUnwrapper: 7 | def __init__(self, d: dict): 8 | self._d = d 9 | 10 | def get(self, *args: str): 11 | def unwrap(d: dict, key: str): 12 | if d: 13 | return d.get(key) 14 | else: 15 | return None 16 | 17 | current_dict = self._d 18 | for arg in args: 19 | current_dict = unwrap(current_dict, arg) 20 | return current_dict 21 | 22 | 23 | class EmbeddingType(str, Enum): 24 | openai = "openai" 25 | ollama = "ollama" 26 | 27 | 28 | class Embeddings(BaseModel): 29 | type: EmbeddingType = Field( 30 | description="Type of embedding (see EmbeddingType enum)." 31 | ) 32 | api_key: str | None = Field(description="Token for openAI service.", default=None) 33 | model: str | None = Field( 34 | description="AI model (used for in ollama embedding).", default=None 35 | ) 36 | chat_model_name: str = Field(description="Chat model.") 37 | chat_temperature: float = Field(description="Temperature parameter of chat.") 38 | chain_type: str = Field(description="Langchain chain type", default="stuff") 39 | search_type: str = Field(description="Type of search in database", default="mmr") 40 | search_k: int = Field(description="Amount of documents to return.") 41 | search_fetch_k: int = Field( 42 | description="Amount of documents to pass to search algorihm." 43 | ) 44 | 45 | @classmethod 46 | def from_settings(cls, settings: dict[str | Any]): 47 | embedding_settings = settings.get("embedding") 48 | if embedding_settings is None: 49 | print("Section [embedding] not present in settings.") 50 | return cls() 51 | unwrapper = DictUnwrapper(embedding_settings) 52 | return cls( 53 | type=unwrapper.get("type"), 54 | api_key=unwrapper.get("api_key"), 55 | model=unwrapper.get("model"), 56 | chat_model_name=unwrapper.get("chat", "model_name"), 57 | chat_temperature=unwrapper.get("chat", "temperature"), 58 | chain_type=unwrapper.get("chain_type"), 59 | search_type=unwrapper.get("search", "type"), 60 | search_k=unwrapper.get("search", "k"), 61 | search_fetch_k=unwrapper.get("search", "fetch_k"), 62 | ) 63 | 64 | 65 | class DatabaseType(str, Enum): 66 | chroma = "chroma" 67 | elastic = "elastic" 68 | 69 | 70 | class Databases(BaseModel): 71 | type: DatabaseType = Field(description="Type of database (see DatabaseType).") 72 | persist_directory: str | None = Field( 73 | description="Path of persistent file (for Chroma only).", default=None 74 | ) 75 | url: str | None = Field( 76 | description="URL of database (only for Elasticsearch).", default=None 77 | ) 78 | distance_strategy: str | None = Field( 79 | description="Distance (only for Elasticsearch).", default=None 80 | ) 81 | collection_name: str = Field(description="Name of the text collection.") 82 | sql_namespace: str 83 | sql_url: str 84 | splitter_chunk_size: int 85 | splitter_chunk_overlap: int 86 | 87 | @classmethod 88 | def from_settings(cls, settings: dict[str | Any]): 89 | database_settings = settings.get("database") 90 | if database_settings is None: 91 | print("Section [database] not present in settings.") 92 | return cls() 93 | unwrapper = DictUnwrapper(database_settings) 94 | return cls( 95 | type=unwrapper.get("type"), 96 | persist_directory=unwrapper.get("persist_directory"), 97 | url=unwrapper.get("url"), 98 | distance_strategy=unwrapper.get("distance_strategy"), 99 | collection_name=unwrapper.get("collection_name"), 100 | sql_namespace=unwrapper.get("sql", "namespace"), 101 | sql_url=unwrapper.get("sql", "url"), 102 | splitter_chunk_size=unwrapper.get("splitter", "chunk_size"), 103 | splitter_chunk_overlap=unwrapper.get("splitter", "chunk_overlap"), 104 | ) 105 | -------------------------------------------------------------------------------- /exact_rag/dataemb.py: -------------------------------------------------------------------------------- 1 | from typing import Any, Callable 2 | from pandas import DataFrame 3 | 4 | from exact_rag.config import EmbeddingType, Embeddings, DatabaseType, Databases 5 | 6 | from langchain_openai import OpenAIEmbeddings 7 | from langchain_community.embeddings import OllamaEmbeddings 8 | from langchain.vectorstores.chroma import Chroma 9 | from langchain.vectorstores.elasticsearch import ElasticsearchStore 10 | from langchain.indexes import SQLRecordManager 11 | from langchain.text_splitter import RecursiveCharacterTextSplitter 12 | from langchain_community.document_loaders import DataFrameLoader 13 | from langchain.indexes import index 14 | from langchain.chains import RetrievalQA 15 | from langchain_openai import ChatOpenAI 16 | from langchain_community.llms.ollama import Ollama 17 | 18 | 19 | class Caller: 20 | def __init__( 21 | self, 22 | callable: Callable[..., Any], 23 | arg_swap: dict[str, Any] | None = None, 24 | accept_only: list[str] | None = None, 25 | ): 26 | self._callable = callable 27 | self._arg_swap = arg_swap 28 | self._accept_only = accept_only 29 | 30 | def __call__(self, **args): 31 | if self._accept_only: 32 | args = { 33 | key: value for (key, value) in args.items() if key in self._accept_only 34 | } 35 | 36 | if self._arg_swap: 37 | args = { 38 | self._arg_swap.get(arg, arg): value for (arg, value) in args.items() 39 | } 40 | 41 | return self._callable(**args) 42 | 43 | 44 | embeddings = { 45 | EmbeddingType.openai: Caller(OpenAIEmbeddings, accept_only=["api_key"]), 46 | EmbeddingType.ollama: Caller(OllamaEmbeddings, accept_only=["model"]), 47 | } 48 | 49 | dbs = { 50 | DatabaseType.chroma: Caller( 51 | Chroma, 52 | {"embedding": "embedding_function"}, 53 | accept_only=["embedding", "persist_directory", "collection_name"], 54 | ), 55 | DatabaseType.elastic: Caller( 56 | ElasticsearchStore, 57 | {"collection_name": "index_name", "url": "es_url"}, 58 | accept_only=[ 59 | "embedding", 60 | "url", 61 | "collection_name", 62 | "distance_strategy", 63 | "strategy", 64 | ], 65 | ), 66 | } 67 | 68 | chats = { 69 | EmbeddingType.openai: Caller( 70 | ChatOpenAI, 71 | { 72 | "api_key": "openai_api_key", 73 | "chat_model_name": "model_name", 74 | "chat_temperature": "temperature", 75 | }, 76 | accept_only=["chat_model_name", "chat_temperature", "api_key"], 77 | ), 78 | EmbeddingType.ollama: Caller( 79 | Ollama, 80 | accept_only=["model"], 81 | ), 82 | } 83 | 84 | 85 | class DataEmbedding: 86 | def __init__(self, embedding_model: Embeddings, database_model: Databases): 87 | embedding_type = embedding_model.type 88 | self._embedding = embeddings[embedding_type](**embedding_model.model_dump()) 89 | print("Embedding initialized.") 90 | 91 | database_type = database_model.type 92 | self._vectorstore = dbs[database_type]( 93 | embedding=self._embedding, 94 | **database_model.model_dump(), 95 | ) 96 | print("Vectorstore initialized.") 97 | 98 | self._record_manager = SQLRecordManager( 99 | database_model.sql_namespace, 100 | db_url=f"sqlite:///{database_model.sql_url}", 101 | ) 102 | print("Record manager initialized.") 103 | 104 | self._record_manager.create_schema() 105 | print(" schema created.") 106 | 107 | self._splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder( 108 | chunk_size=database_model.splitter_chunk_size, 109 | chunk_overlap=database_model.splitter_chunk_overlap, 110 | ) 111 | print("Splitter initialized.") 112 | 113 | self._qa = RetrievalQA.from_chain_type( 114 | llm=chats[embedding_type](**embedding_model.model_dump()), 115 | chain_type="stuff", 116 | retriever=self._vectorstore.as_retriever( 117 | search_type=embedding_model.search_type, 118 | search_kwargs={ 119 | "k": embedding_model.search_k, 120 | "fetch_k": embedding_model.search_fetch_k, 121 | }, 122 | ), 123 | ) 124 | print("Chat initialized.") 125 | 126 | def load(self, text: str): 127 | id_key = "hash" 128 | content_name = "text" 129 | dataframe = DataFrame().from_dict([{id_key: hash(text), content_name: text}]) 130 | loader = DataFrameLoader(dataframe, page_content_column=content_name) 131 | data = loader.load() 132 | documents = self._splitter.split_documents(data) 133 | index( 134 | documents, 135 | self._record_manager, 136 | self._vectorstore, 137 | cleanup="incremental", 138 | source_id_key=id_key, 139 | ) 140 | 141 | def chat(self, query: str): 142 | self.load(query) 143 | return self._qa.invoke({"query": query}) 144 | -------------------------------------------------------------------------------- /exact_rag/image_cap.py: -------------------------------------------------------------------------------- 1 | import torch 2 | from transformers import BlipProcessor 3 | from transformers import BlipForConditionalGeneration 4 | 5 | device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 6 | 7 | 8 | def captioner(raw_image, model_name) -> str: 9 | device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 10 | 11 | processor = BlipProcessor.from_pretrained(model_name) 12 | model = BlipForConditionalGeneration.from_pretrained(model_name).to(device) 13 | text = "a picture of " 14 | inputs = processor(raw_image, text, return_tensors="pt").to(device) 15 | out = model.generate(**inputs) 16 | caption = processor.decode(out[0], skip_special_tokens=True) 17 | return caption 18 | -------------------------------------------------------------------------------- /exact_rag/main.py: -------------------------------------------------------------------------------- 1 | import toml 2 | import aiofiles 3 | import uvicorn 4 | from contextlib import asynccontextmanager 5 | 6 | from fastapi import FastAPI 7 | from fastapi import File 8 | from fastapi import UploadFile 9 | from fastapi import HTTPException 10 | 11 | from exact_rag.dataemb import DataEmbedding 12 | from exact_rag.config import Embeddings 13 | from exact_rag.config import Databases 14 | from exact_rag.schemas import Query 15 | from exact_rag.schemas import GenericResponse 16 | from exact_rag.schemas import Answer 17 | 18 | 19 | # general settings 20 | settings = toml.load("settings.toml") 21 | output_dir = settings["server"]["output_dir"] 22 | settings_e = settings["embedding"] 23 | settings_d = settings["database"] 24 | embedding = Embeddings(**settings_e) 25 | database = Databases(**settings_d) 26 | de = DataEmbedding(embedding, database) 27 | 28 | 29 | ml_models: dict = {} 30 | 31 | 32 | @asynccontextmanager 33 | async def lifespan(app: FastAPI): 34 | whisper_model = settings["audio"].get("whisper_model") 35 | image_model = settings["image"].get("image_model") 36 | 37 | if whisper_model is not None: 38 | try: 39 | from whisper import load_model as whisper_load_model # noqa: F401 40 | 41 | ml_models["whisper_model"] = whisper_load_model(whisper_model) 42 | 43 | except ModuleNotFoundError: 44 | raise BaseException("You had to install `audio` extra") 45 | 46 | if image_model is not None: 47 | try: 48 | import transformers # noqa: F401 49 | import PIL # noqa: F401 50 | from exact_rag.image_cap import image_captioner 51 | 52 | ml_models["image_model"] = image_captioner(image_model) 53 | 54 | except ModuleNotFoundError: 55 | raise BaseException("You had to install `image` extra") 56 | 57 | yield 58 | # Clean up the ML models and release the resources 59 | ml_models.clear() 60 | 61 | 62 | app = FastAPI(lifespan=lifespan) 63 | 64 | 65 | @app.post("/upload_text/", response_model=GenericResponse, status_code=201) 66 | async def upload_file(file: UploadFile = File(...)): 67 | if file.filename.endswith(".txt"): 68 | async with aiofiles.open(output_dir + file.filename, "wb") as out_file: 69 | while content := await file.read(1024): # async read chunk 70 | await out_file.write(content) 71 | 72 | de.load(content) 73 | 74 | return dict(filename=file.filename) 75 | 76 | elif file.filename.endswith(".json"): 77 | # contents = await file.read() 78 | return dict(filename=file.filename) 79 | 80 | else: 81 | raise HTTPException( 82 | status_code=400, detail="Only .txt and .json files are allowed" 83 | ) 84 | 85 | 86 | @app.post("/upload_audio/", response_model=GenericResponse, status_code=201) 87 | async def upload_audio(audio: UploadFile = File(...)): 88 | try: 89 | from exact_rag.audio_cap import audio_caption 90 | 91 | if audio.filename.endswith(".mp3"): 92 | async with aiofiles.open(output_dir + audio.filename, "wb") as out_file: 93 | while content := await audio.read(1024): 94 | await out_file.write(content) 95 | 96 | if ml_models["whisper_model"] is None: 97 | raise HTTPException( 98 | status_code=400, detail="A whisper model is required, es. 'base'" 99 | ) 100 | 101 | result = audio_caption(output_dir + audio.filename) 102 | de.load(result["text"]) 103 | 104 | return dict(filename=audio.filename) 105 | else: 106 | raise HTTPException(status_code=400, detail="Only .mp3 files are allowed") 107 | except ModuleNotFoundError: 108 | raise BaseException("To use this endpoint you should install `audio` extra") 109 | 110 | 111 | @app.post("/upload_image/", response_model=GenericResponse, status_code=201) 112 | async def upload_image(image: UploadFile = File(...)): 113 | if image.filename.endswith((".png", ".jpeg")): 114 | async with aiofiles.open(output_dir + image.filename, "wb") as out_file: 115 | while content := await image.read(1024): 116 | await out_file.write(content) 117 | 118 | if ml_models["image_model"] is None: 119 | raise HTTPException( 120 | status_code=400, 121 | detail=( 122 | "A whisper model is required, es. " 123 | "'nlpconnect/vit-gpt2-image-captioning'" 124 | ), 125 | ) 126 | 127 | captioner = ml_models["image_model"] 128 | caption = captioner(output_dir + image.filename) 129 | de.load(caption[0]["generated_text"]) 130 | 131 | return dict(filename=image.filename) 132 | 133 | 134 | @app.post("/query/", response_model=Answer, status_code=201) 135 | async def send_query(query: Query): 136 | ans = de.chat(query.text) 137 | return dict(question=ans.get("query"), msg=ans.get("result")) 138 | 139 | 140 | if __name__ == "__main__": 141 | uvicorn.run("main:app", port=8080, log_level="info") 142 | -------------------------------------------------------------------------------- /exact_rag/schemas.py: -------------------------------------------------------------------------------- 1 | from pydantic import BaseModel 2 | 3 | 4 | class Query(BaseModel): 5 | text: str 6 | 7 | 8 | class GenericResponse(BaseModel): 9 | filename: str 10 | 11 | 12 | class Answer(BaseModel): 13 | question: str 14 | msg: str 15 | -------------------------------------------------------------------------------- /exact_rag/settings.py: -------------------------------------------------------------------------------- 1 | from typing import Any 2 | from pydantic import BaseModel 3 | 4 | 5 | class FromDict: 6 | def __init__(self, *path: str): 7 | self._path = list(path) 8 | 9 | def __call__(self, d: dict[str, Any]): 10 | def unwrap(d: dict[str, Any], key: str): 11 | if isinstance(d, dict): 12 | return d.get(key) 13 | else: 14 | return None 15 | 16 | current_dict = d 17 | for key in self._path: 18 | current_dict = unwrap(current_dict, key) 19 | 20 | return current_dict 21 | 22 | 23 | class Settings(BaseModel): 24 | def __init__(self, *args: Any, **kwargs: Any) -> None: 25 | output_dict = dict() 26 | for var, field_info in self.model_fields.items(): 27 | from_dicts = [ 28 | metadata 29 | for metadata in field_info.metadata 30 | if isinstance(metadata, FromDict) 31 | ] 32 | if len(from_dicts) == 0: 33 | output_dict[var] = kwargs.get(var) 34 | elif len(from_dicts) == 1: 35 | output_dict[var] = from_dicts[0](kwargs) 36 | else: 37 | raise TypeError(f"Multiple FromDict in {var} annotation.") 38 | 39 | super(Settings, self).__init__(**output_dict) 40 | -------------------------------------------------------------------------------- /examples/README.md: -------------------------------------------------------------------------------- 1 | # Examples 2 | 3 | > Be sure to have ollama running (localhost://11434 -> Ollama is running) 4 | 5 | > Be sure to have configured properly the settings.toml file 6 | 7 | To run this example you had to activate the web server: 8 | ```bash 9 | poetry run python exact_rag/main.py 10 | ``` 11 | 12 | Then in another shell you can run the `client_rag.py` which provides a simple client to send data to RAG and chatting. 13 | 14 | ```bash 15 | poetry run python client_rag.py 16 | ``` 17 | -------------------------------------------------------------------------------- /examples/client_rag.py: -------------------------------------------------------------------------------- 1 | import requests 2 | 3 | 4 | BASE_URL = "http://0.0.0.0:8080" 5 | 6 | 7 | def upload_file(file_path, endpoint="/upload_text/"): 8 | with open(file_path, "rb") as file: 9 | files = {"file": file} 10 | response = requests.post(f"{BASE_URL}{endpoint}", files=files) 11 | return response.json() 12 | 13 | 14 | def upload_audio(file_path, endpoint="/upload_audio/"): 15 | files = {"audio": (f"{file_path}", open(f"{file_path}", "rb"), "audio/mp3")} 16 | response = requests.post(f"{BASE_URL}{endpoint}", files=files) 17 | return response.json() 18 | 19 | 20 | def upload_image(file_path, endpoint="/upload_image/"): 21 | files = {"image": (file_path, open(file_path, "rb"), "image/png")} 22 | response = requests.post(f"{BASE_URL}{endpoint}", files=files) 23 | return response.json() 24 | 25 | 26 | def send_query(query, endpoint="/query/"): 27 | data = {"text": query} 28 | response = requests.post(f"{BASE_URL}{endpoint}", json=data) 29 | return response.json() 30 | 31 | 32 | if __name__ == "__main__": 33 | # Example: Upload a file 34 | file_response = upload_file("test.txt") 35 | 36 | # file_response = upload_audio("test.mp3") 37 | 38 | # file_response = upload_image("max.jpeg") 39 | print(file_response) 40 | 41 | text_query = "Tell me something" 42 | query_response = send_query(text_query) 43 | print(query_response) 44 | -------------------------------------------------------------------------------- /examples/test.txt: -------------------------------------------------------------------------------- 1 | hello 2 | -------------------------------------------------------------------------------- /frontend/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/exactlab/exact-rag/e9311e060c73ede33a1baee6cb4c5897e8eb8a9e/frontend/__init__.py -------------------------------------------------------------------------------- /frontend/ui.py: -------------------------------------------------------------------------------- 1 | import streamlit as st 2 | from pydantic import BaseModel 3 | from io import StringIO 4 | import toml 5 | from exact_rag.dataemb import DataEmbedding 6 | from exact_rag.config import Embeddings 7 | from exact_rag.config import Databases 8 | from exact_rag.image_cap import captioner 9 | from exact_rag.audio_cap import audio_caption 10 | 11 | import os 12 | import time 13 | import fitz # imports the pymupdf library 14 | import base64 15 | from io import BytesIO 16 | from PIL import Image 17 | 18 | settings = toml.load("settings.toml") 19 | output_dir = settings["server"]["output_dir"] 20 | image_model = settings["image"].get("image_model") 21 | settings_e = settings["embedding"] 22 | settings_d = settings["database"] 23 | embedding = Embeddings(**settings_e) 24 | database = Databases(**settings_d) 25 | de = DataEmbedding(embedding, database) 26 | 27 | 28 | class Message(BaseModel): 29 | actor: str 30 | payload: str 31 | 32 | 33 | def sidebar(): 34 | FILES = "files" 35 | if FILES not in st.session_state: 36 | st.session_state[FILES] = [] 37 | 38 | st.title("eXact-RAG") 39 | 40 | with st.sidebar: 41 | st.write("Upload text, pdf, image or audio") 42 | up_file = st.file_uploader(label=" ", label_visibility="collapsed") 43 | if up_file: 44 | # text 45 | if os.path.splitext(up_file.name)[1] == ".txt": 46 | if up_file.name not in st.session_state[FILES]: 47 | st.session_state[FILES].append(up_file.name) 48 | stringio = StringIO(up_file.getvalue().decode("utf-8")) 49 | string_data = stringio.read() 50 | de.load(string_data) 51 | # pdf 52 | if os.path.splitext(up_file.name)[1] == ".pdf": 53 | if up_file.name not in st.session_state[FILES]: 54 | st.session_state[FILES].append(up_file.name) 55 | with open(output_dir + up_file.name, "wb") as f: 56 | f.write(up_file.getbuffer()) 57 | base64_pdf = base64.b64encode(up_file.getvalue()).decode( 58 | "utf-8" 59 | ) 60 | 61 | doc = fitz.open(output_dir + up_file.name) # open a document 62 | with st.spinner("Processing..."): 63 | text_annot = [] 64 | for page in doc: # iterate the document pages 65 | for annot in page.annots(): 66 | text = page.get_textbox(annot.rect) 67 | text_annot.append(text) 68 | print(" ".join(t for t in text_annot)) 69 | de.load(" ".join(t for t in text_annot)) 70 | pdf_display = f'' 71 | st.markdown(pdf_display, unsafe_allow_html=True) 72 | else: 73 | base64_pdf = base64.b64encode(up_file.getvalue()).decode("utf-8") 74 | pdf_display = f'' 75 | st.markdown(pdf_display, unsafe_allow_html=True) 76 | 77 | # images 78 | elif os.path.splitext(up_file.name)[1] in [".jpeg", ".jpg", ".png"]: 79 | if up_file.name not in st.session_state[FILES]: 80 | st.session_state[FILES].append(up_file.name) 81 | with open(output_dir + up_file.name, "wb") as f: 82 | f.write(up_file.getbuffer()) 83 | with st.spinner("Captioning in progress..."): 84 | image = Image.open(BytesIO(up_file.getvalue())) 85 | caption = captioner(image, model_name=image_model) 86 | st.image(output_dir + up_file.name, caption=caption) 87 | print(caption) 88 | de.load(caption) 89 | else: 90 | st.image(output_dir + up_file.name) 91 | # audio 92 | elif os.path.splitext(up_file.name)[1] == ".mp3": 93 | result = audio_caption(output_dir + up_file.name) 94 | st.write(result) 95 | de.load(result["text"]) 96 | 97 | else: 98 | st.error("Format not supported!") 99 | 100 | 101 | def show(): 102 | USER = "user" 103 | ASSISTANT = "ai" 104 | MESSAGES = "messages" 105 | if MESSAGES not in st.session_state: 106 | st.session_state[MESSAGES] = [ 107 | Message( 108 | actor=ASSISTANT, 109 | payload="Hi! How can I help you?", 110 | ) 111 | ] 112 | ### loop on state when user insert a new prompt 113 | ### you will not lose previous messages 114 | msg: Message 115 | for msg in st.session_state[MESSAGES]: 116 | st.chat_message(msg.actor).write(msg.payload) 117 | 118 | if prompt := st.chat_input("Ask something"): 119 | st.session_state[MESSAGES].append(Message(actor=USER, payload=prompt)) 120 | st.chat_message(USER).write(prompt) 121 | 122 | with st.chat_message(ASSISTANT): 123 | answer = de.chat(prompt) 124 | st.session_state[MESSAGES].append( 125 | Message(actor=ASSISTANT, payload=answer.get("result")) 126 | ) 127 | res = answer.get("result") 128 | 129 | def _stream_data(res: str): 130 | for word in res.split(): 131 | yield word + " " 132 | time.sleep(0.04) 133 | 134 | st.write_stream(_stream_data(res)) 135 | 136 | 137 | if __name__ == "__main__": 138 | sidebar() 139 | show() 140 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [tool.poetry] 2 | name = "exact-rag" 3 | version = "0.1.0" 4 | description = "AI-augmented, conversation information retrieval and data exploration" 5 | authors = ["Marco Franzon ", "Matteo Poggi "] 6 | readme = "README.md" 7 | 8 | [tool.poetry.dependencies] 9 | python = "^3.10" 10 | fastapi = "^0.108.0" 11 | langchain = "^0.1.0" 12 | toml = "^0.10.2" 13 | openai = "^1.7.0" 14 | langchain-openai = "^0.0.2" 15 | langchain-community = "0.0.11" 16 | chromadb = "^0.4.22" 17 | pandas = "^2.1.4" 18 | elasticsearch = "^8.11.1" 19 | uvicorn = "^0.25.0" 20 | python-multipart = "^0.0.6" 21 | aiofiles = "^23.2.1" 22 | pydantic = "^2.5.3" 23 | 24 | 25 | openai-whisper = { version = "^20231117", optional = true} 26 | transformers = { version = "^4.36.2", optional = true} 27 | pillow = { version = "^10.2.0", optional = true} 28 | fastui = "^0.4.0" 29 | torch = "^2.2.0" 30 | 31 | 32 | [tool.poetry.extras] 33 | audio = ["openai-whisper"] 34 | image = ["pillow", "transformers"] 35 | 36 | [tool.poetry.group.dev.dependencies] 37 | pytest = "^7.4.4" 38 | pre-commit = "^3.6.0" 39 | streamlit = "^1.31.0" 40 | 41 | [build-system] 42 | requires = ["poetry-core"] 43 | build-backend = "poetry.core.masonry.api" 44 | -------------------------------------------------------------------------------- /scripts/docker-es.sh: -------------------------------------------------------------------------------- 1 | docker run --name es0 -v es_data:/usr/share/elasticsearch/data -p 9200:9200 -e "discovery.type=single-node" -e "xpack.security.enabled=false" -e "xpack.security.http.ssl.enabled=false" docker.elastic.co/elasticsearch/elasticsearch:8.11.1 2 | -------------------------------------------------------------------------------- /settings.toml.template: -------------------------------------------------------------------------------- 1 | [embedding] 2 | type = "openai" 3 | api_key = "" 4 | chat.model_name = "gpt-3.5-turbo" 5 | #type = "ollama" 6 | #model = "orca2" 7 | #chat.model_name = "orca2" 8 | chat.temperature = 0.5 9 | chain_type = "stuff" 10 | search.type = "mmr" 11 | search.k = 20 12 | search.fetch_k = 20 13 | 14 | 15 | [database] 16 | type = "chroma" 17 | persist_directory = "persist" 18 | #type = "elastic" 19 | #url = "http://localhost:9200" 20 | #distance_strategy = "COSINE" 21 | collection_name = "reviews" 22 | sql.namespace = "db/text" 23 | sql.url = "duplicates.sql" 24 | splitter.chunk_size = 1000 25 | splitter.chunk_overlap = 0 26 | 27 | 28 | [server] 29 | output_dir = "/tmp/" 30 | 31 | 32 | [audio] 33 | whisper_model = "base" 34 | 35 | 36 | [image] 37 | image_model = "noamrot/FuseCap_Image_Captioning" 38 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/exactlab/exact-rag/e9311e060c73ede33a1baee6cb4c5897e8eb8a9e/tests/__init__.py -------------------------------------------------------------------------------- /tests/conftest.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | import toml 3 | 4 | embedding_tomls = { 5 | "ollama": "tests/test_embedding_ollama.toml", 6 | } 7 | 8 | 9 | @pytest.fixture(params=list(embedding_tomls.values())) 10 | def get_embedding_toml(request): 11 | settings = toml.load(f"{request.param}") 12 | return settings["embedding"] 13 | 14 | 15 | database_tomls = { 16 | "chroma": "tests/test_database_chroma.toml", 17 | "elastic": "tests/test_database_elastic.toml", 18 | } 19 | 20 | 21 | @pytest.fixture(params=list(database_tomls.values())) 22 | def get_database_toml(request): 23 | settings = toml.load(f"{request.param}") 24 | return settings["database"] 25 | -------------------------------------------------------------------------------- /tests/ollama_embedding.json: -------------------------------------------------------------------------------- 1 | {"embedding": [0.38984712958335876, -0.43654707074165344, 1.7038918733596802, 2.145319700241089, -2.417177438735962, 2.2011303901672363, -0.1388934850692749, 0.44042739272117615, -0.4365840554237366, -0.992618203163147, 0.3411661684513092, -2.126600742340088, -0.14250509440898895, 0.6142696738243103, -0.7876328825950623, -1.6696561574935913, 0.9408551454544067, 3.5594844818115234, 0.2472430318593979, -2.6889090538024902, 0.2702198326587677, -1.1737060546875, 2.0524861812591553, -1.2292885780334473, 0.23481427133083344, -3.0738213062286377, -0.33404284715652466, 0.4579940140247345, -0.8513206839561462, -0.8937908411026001, 4.541660785675049, -0.5156925320625305, -1.1095558404922485, 1.8969440460205078, 1.9666752815246582, -4.7993340492248535, -0.30276158452033997, 0.4061323404312134, 0.41831108927726746, -2.0871429443359375, 0.5022368431091309, -0.0786084309220314, -0.01677623949944973, -0.7287670969963074, -1.634925127029419, 0.6557558178901672, 0.7828837037086487, 1.3129440546035767, 0.839448869228363, -1.9409852027893066, 0.5044874548912048, 2.591437816619873, 1.5608062744140625, -1.8978416919708252, 3.8185322284698486, 1.362290382385254, -0.5987011790275574, -1.7000548839569092, -2.3436174392700195, -1.0902276039123535, -2.943021774291992, -1.5243606567382812, 0.5779710412025452, -0.49150553345680237, -3.4666802883148193, 2.467449426651001, 0.8451610803604126, 2.4435036182403564, 1.4713274240493774, 0.15905039012432098, -1.6240218877792358, -0.4030272364616394, 1.0753403902053833, 1.3172999620437622, -2.371262550354004, 0.22353117167949677, 0.5468087792396545, 0.22982759773731232, -0.7168400883674622, 0.7398534417152405, 2.551623582839966, -1.3491042852401733, -1.1440495252609253, -0.40819764137268066, -0.8387629985809326, 0.4236232042312622, 1.7825006246566772, 1.273438811302185, 0.6685529947280884, -1.4604135751724243, -0.5749561786651611, -0.9602516889572144, 1.9424995183944702, 1.677992343902588, -0.5034586787223816, -0.2556677460670471, 0.2999448776245117, -0.699209988117218, -2.2692365646362305, -1.9881978034973145, -0.656411349773407, 1.048691749572754, -1.445133924484253, 2.4410078525543213, -0.21488849818706512, 0.911955714225769, -0.7610661387443542, 0.8051075339317322, -0.1900651454925537, 0.6538810133934021, 0.6440409421920776, -0.09120026975870132, -0.44478368759155273, 2.1130621433258057, 1.423073172569275, 1.0830023288726807, 0.8564145565032959, 0.1209307312965393, -0.5564601421356201, -1.6015323400497437, -1.9137680530548096, -1.725021243095398, 0.1444767713546753, -0.5891130566596985, -0.8088560104370117, -0.8098487257957458, -0.06612354516983032, -0.5944274067878723, 2.161724328994751, -1.344649076461792, 0.16681046783924103, -0.4119402766227722, 1.2130030393600464, 0.14428593218326569, 0.019665012136101723, 1.225048303604126, 0.9688299298286438, -1.0898616313934326, 1.3896394968032837, 2.369370222091675, 1.4002882242202759, 1.4608442783355713, 0.3301759362220764, -3.0674874782562256, -1.8697584867477417, -0.9272518754005432, -0.6922355890274048, 2.5403387546539307, -1.4324440956115723, 7.029284477233887, 1.8878734111785889, -0.3044112026691437, 1.3211702108383179, 0.2135985791683197, 0.0018638205947354436, 0.5888856053352356, -0.7171679735183716, 0.5258997678756714, -0.7664526104927063, 0.3888953924179077, 1.2657221555709839, 0.26628395915031433, -0.28522950410842896, -1.0255283117294312, 2.6829068660736084, -1.794890284538269, 1.6963720321655273, 2.0704076290130615, 1.7048051357269287, 0.42434462904930115, -2.2730956077575684, -2.1300084590911865, 2.829007148742676, -1.6259641647338867, -1.1801015138626099, 0.8450014591217041, 2.2344040870666504, 2.086488962173462, 0.5016601085662842, 0.39659953117370605, 0.37919947504997253, -1.9865984916687012, -0.7358142137527466, -0.38395634293556213, 0.6837487816810608, -1.7304153442382812, -1.003869891166687, 0.06966537237167358, 2.1177566051483154, 1.3749271631240845, -0.8863251209259033, -1.115529179573059, 1.433644413948059, 1.1511001586914062, -2.6927382946014404, -1.6395364999771118, -1.0307265520095825, -0.9043099880218506, 0.14074666798114777, -1.0118011236190796, 1.2005643844604492, -0.7465667724609375, -1.4185550212860107, 1.8570479154586792, 2.4705963134765625, 0.7213297486305237, -2.9097821712493896, 0.005590498447418213, -1.1168127059936523, 1.1625674962997437, -0.5607805252075195, -0.05792597681283951, -1.5174404382705688, -0.301775187253952, 0.27191635966300964, 0.6375052332878113, -0.8315501809120178, 0.84023517370224, -1.355940818786621, -2.172208786010742, -0.746353805065155, -1.7097246646881104, 0.6183761954307556, 2.312701940536499, -0.2523788511753082, 0.8260256052017212, -0.24707025289535522, -2.848865270614624, -2.0731759071350098, -1.3505306243896484, -1.5473390817642212, 1.7504173517227173, -2.5425283908843994, -1.428740382194519, -0.7182998657226562, 0.1329801231622696, -0.8445889949798584, -2.0056216716766357, 2.033647060394287, 1.237772822380066, -1.3696492910385132, -0.5201088190078735, -0.3811756372451782, -0.10041055828332901, -1.021862268447876, -0.1120627373456955, 0.2683214843273163, 0.5352734923362732, 0.1769368052482605, 2.3992061614990234, 0.8938714265823364, -0.13531933724880219, 0.905695378780365, -2.976640224456787, -0.42307233810424805, 2.091653823852539, -2.127901792526245, -1.2171093225479126, 0.7281429767608643, 0.4456520080566406, -1.095842957496643, 2.7183783054351807, 3.430529832839966, 0.5245150327682495, -1.4341251850128174, -2.193396806716919, 0.5176023244857788, 0.7064124941825867, 0.9081686735153198, 9.890000343322754, -0.579646646976471, 2.6856441497802734, 1.728042483329773, 0.90809166431427, 2.678032636642456, -0.719717800617218, 0.6058700084686279, 1.1192741394042969, -0.5018121600151062, 1.0360562801361084, 1.4427063465118408, -2.2588000297546387, -1.0178436040878296, 2.4380335807800293, -1.1934525966644287, 0.4381406903266907, 0.7358356714248657, -1.3278462886810303, -0.33289098739624023, -1.5913779735565186, 1.262762188911438, -2.334169864654541, -0.6222671270370483, -1.317649245262146, -0.5424346327781677, 1.5430362224578857, 2.044111967086792, 0.31678348779678345, -0.7852274775505066, -0.31760984659194946, 1.7031091451644897, -0.13830161094665527, 0.8386630415916443, -1.0615254640579224, -2.1846656799316406, 1.8125900030136108, -0.35747796297073364, 1.7184934616088867, 2.9520270824432373, 0.12135449051856995, 1.9949240684509277, 1.5971684455871582, 0.48284152150154114, -2.385488986968994, -0.09216701984405518, -0.18350881338119507, -2.4003543853759766, -4.3150553703308105, -0.007091144099831581, 1.4138681888580322, 3.1827385425567627, 0.8267176747322083, 1.4735833406448364, 1.3333253860473633, 0.5439066290855408, 0.7781517505645752, -1.5934654474258423, -0.006397898308932781, -1.969325065612793, -0.06343817710876465, -1.5823934078216553, -2.428541898727417, -1.3724418878555298, -3.0464539527893066, -0.867521345615387, -0.027365704998373985, 1.2794135808944702, -0.10281142592430115, -0.6819730401039124, 0.11489327251911163, 0.3158315420150757, 2.601722240447998, -0.8780704736709595, 0.6443400382995605, -1.5126686096191406, -1.7650376558303833, 2.0088915824890137, 0.791130781173706, 2.633589506149292, 1.7442485094070435, -2.919647455215454, -0.27083876729011536, -2.554257869720459, -0.7333880066871643, -0.5427842140197754, -0.7233192324638367, -2.8432507514953613, 0.8710359930992126, 0.3198477625846863, -0.9797205924987793, 0.7318188548088074, 0.24222825467586517, -0.5605867505073547, -2.013794183731079, -0.7251244783401489, 0.360798716545105, 0.07099928706884384, 0.7298644185066223, 4.536099910736084, -1.6343039274215698, -0.9202774167060852, -0.16644395887851715, 2.8036839962005615, -2.9861879348754883, 1.9352537393569946, 1.289010763168335, 0.3247174024581909, 0.5516861081123352, -0.8715994358062744, -2.4385592937469482, -1.6580654382705688, 4.232550621032715, -1.496144413948059, -0.7127047181129456, -2.705031394958496, 0.30059880018234253, 1.0791015625, 0.5089972615242004, -0.2902460992336273, 0.09460540115833282, -0.9833971261978149, 2.273076057434082, 0.12666843831539154, 0.14298053085803986, 0.2079925537109375, 0.12179166078567505, 0.22812579572200775, 0.7810661792755127, 0.7795325517654419, 1.147805094718933, -0.09002379328012466, -2.4728972911834717, -2.850883722305298, -1.5828957557678223, -1.8577604293823242, 0.8489925265312195, 0.8527178764343262, 0.4722553789615631, 0.507854700088501, -0.5042946338653564, -0.9906351566314697, -2.7958247661590576, -2.0754776000976562, -0.6559149026870728, 0.5253322124481201, -0.9394577741622925, 0.15257517993450165, -0.23463861644268036, -2.70295786857605, -2.3430659770965576, 1.764212965965271, -2.048586130142212, -0.025834951549768448, -0.1110936775803566, 2.469344139099121, 2.616328477859497, 0.2966114282608032, 1.4973238706588745, 0.03251465782523155, -0.8395901918411255, 1.2280913591384888, -0.06540927290916443, 1.6708871126174927, -1.121788501739502, 0.2626279890537262, 0.27051809430122375, 0.40606820583343506, 0.08526268601417542, 0.294000506401062, 0.051309820264577866, -1.8654979467391968, 1.8655104637145996, -2.1666882038116455, -1.0156075954437256, 0.08482414484024048, -2.1409993171691895, -1.869594931602478, -3.101047992706299, -7.451498031616211, -1.7891721725463867, -1.9480948448181152, 0.7617409825325012, 0.2815436124801636, -2.2272491455078125, 0.5160015821456909, 3.6651856899261475, -0.5631743669509888, -1.2786951065063477, -1.743175983428955, 0.6106257438659668, 1.539717435836792, 2.47210431098938, 1.1987812519073486, 0.45555707812309265, 1.4574390649795532, 2.4357845783233643, 1.2745857238769531, -1.458639144897461, 0.5374936461448669, -0.2503620982170105, 1.3554412126541138, -3.2166476249694824, 0.9950118660926819, -1.6021780967712402, -0.042453374713659286, 1.1419427394866943, -1.294480800628662, 0.22566662728786469, -0.5382694005966187, 0.12427660077810287, 0.3255653977394104, 0.9509122967720032, -1.157141089439392, -3.390230655670166, -1.6756998300552368, 0.030631743371486664, 1.1937953233718872, 1.0426359176635742, 1.3104076385498047, -0.14691758155822754, -0.6905747056007385, -1.5820966958999634, 2.233980178833008, 3.5832479000091553, -0.36460843682289124, -0.04836765676736832, -0.29632851481437683, -1.0507367849349976, 0.013203760609030724, 3.8630783557891846, -1.2618651390075684, 0.8555551171302795, -1.6449947357177734, 0.8069835305213928, -1.7175394296646118, 2.8092691898345947, -1.5122562646865845, 1.3730711936950684, 3.3103952407836914, 0.6988751888275146, -1.595064401626587, 0.5090819597244263, -2.9771506786346436, 0.5725335478782654, 10.528411865234375, -0.06237233430147171, 0.5126579403877258, -0.008275947533547878, -0.0401296392083168, 2.227142095565796, -1.7406858205795288, 0.24559926986694336, 1.1232044696807861, -0.5860177874565125, 0.09713064879179001, -0.056434303522109985, -0.3400970995426178, 0.6813048720359802, 1.120926022529602, -4.4404520988464355, 0.33691737055778503, -1.4405964612960815, 0.6917822957038879, -1.501503348350525, -1.4404321908950806, 0.3062310218811035, -1.5349609851837158, -1.1287219524383545, 0.13811735808849335, -1.502745509147644, 2.3500823974609375, -1.9597253799438477, 2.013406276702881, -2.3141398429870605, -0.9116743206977844, 2.303569793701172, -0.389187216758728, 2.842414617538452, 2.204500675201416, -2.847813367843628, 1.0473835468292236, -0.45614662766456604, -1.5440673828125, -0.38642218708992004, 2.014673948287964, 0.8425543308258057, 0.14330461621284485, 2.8654026985168457, 0.8143473863601685, -1.6176490783691406, 0.6074634194374084, -0.37182873487472534, -0.7082491517066956, -3.664199113845825, -0.3130494952201843, 2.7374470233917236, 0.4972108006477356, 1.7052674293518066, -2.434743642807007, -0.08214389532804489, -1.5024762153625488, -0.5732954740524292, -3.766299247741699, -1.7019309997558594, 0.3796003758907318, -2.1822710037231445, 1.5323115587234497, -0.3471596837043762, 1.4703247547149658, -2.006678581237793, -2.520876169204712, -2.020824432373047, -0.7848854660987854, -0.16012263298034668, -0.9906013607978821, 0.1739211231470108, -0.3362586498260498, -0.5601609945297241, -4.7348222732543945, 4.062933444976807, -0.8509682416915894, -0.10706105828285217, 0.23437005281448364, -1.180640697479248, -1.5850716829299927, 0.8112283945083618, -1.492197871208191, 0.39186814427375793, -0.8849936723709106, 0.5618779063224792, -0.22484925389289856, -2.359086751937866, -3.7143754959106445, 1.331873893737793, 0.37908461689949036, 2.384996175765991, 0.3141031861305237, 2.20697021484375, -3.5193753242492676, -0.40350446105003357, 0.9729610085487366, -1.8049741983413696, 1.7043744325637817, 2.225632667541504, 1.4205739498138428, -1.5423851013183594, 0.32496950030326843, -0.19896508753299713, -2.409580945968628, 0.7438074350357056, 3.395456314086914, 3.7683303356170654, -1.3234747648239136, -0.2661784887313843, 2.1165006160736084, -0.4432162642478943, 0.3942912817001343, -0.8530409336090088, -1.2345812320709229, -0.29139575362205505, -0.8500305414199829, -0.17823460698127747, -1.2592107057571411, 1.226402759552002, -0.4996181130409241, 0.9366337060928345, 0.970426082611084, 0.8472345471382141, -2.1246860027313232, 2.1742758750915527, -4.8618011474609375, 0.7322511076927185, -0.6425070762634277, 0.0022795081604272127, -1.036637783050537, -3.001976728439331, 0.2697185277938843, -1.262149691581726, -0.8836796283721924, 1.5371379852294922, 0.5142905116081238, -1.661829948425293, 1.9706295728683472, 3.2495436668395996, 2.717012882232666, -0.40644028782844543, -0.9963616132736206, 2.1331288814544678, -0.5158721804618835, 0.461620956659317, 2.0433027744293213, -1.1698532104492188, 0.7233097553253174, -1.6309221982955933, 0.6002952456474304, 0.7280974984169006, 0.025765491649508476, -0.059935104101896286, 1.0335103273391724, 1.3609200716018677, 0.6826415657997131, -0.1767675280570984, 0.4787086248397827, 0.07626716047525406, -0.06506534665822983, -0.6546207666397095, -1.2103571891784668, 1.0067150592803955, -1.4173097610473633, 0.5896087288856506, -0.239157572388649, -0.6515432000160217, 1.2473746538162231, -0.36116495728492737, 1.1472948789596558, 0.3765006959438324, 0.3755355179309845, 2.4211666584014893, -0.22548802196979523, 2.7004952430725098, -1.1545289754867554, 1.7747321128845215, -0.09163883328437805, 2.8121297359466553, 0.7030946016311646, 0.5126969218254089, 1.4203025102615356, -0.8604009747505188, -0.8617902994155884, -0.020869288593530655, 1.6838206052780151, 2.3675079345703125, -0.6548981666564941, -0.3678203523159027, 1.897750973701477, -1.9800769090652466, 0.8510063290596008, 0.3223535120487213, -1.2631261348724365, -3.0470635890960693, -1.347546100616455, 1.3983908891677856, -0.6370715498924255, -0.439956396818161, -1.2903512716293335, -0.009112580679357052, -0.693712055683136, 0.9157324433326721, -2.6632027626037598, -1.0114599466323853, 0.3829343914985657, 0.7867441177368164, 0.13707278668880463, -1.4669116735458374, -0.38883668184280396, -2.1671698093414307, -1.9199944734573364, 0.248642235994339, 0.6878745555877686, 0.863041877746582, 3.5752041339874268, -1.4130126237869263, -2.3163516521453857, 0.29371991753578186, -3.5757524967193604, 1.6389744281768799, 0.05680457875132561, -0.5515933036804199, 0.5807698369026184, -1.1605169773101807, -1.6880987882614136, 0.39108261466026306, -0.5646641254425049, 0.28488460183143616, -1.3770338296890259, 1.4161574840545654, 1.2048567533493042, 1.1267222166061401, -1.7361201047897339, 1.6851322650909424, 1.6283904314041138, 1.5960410833358765, -1.5988709926605225, 2.130422830581665, 1.6263415813446045, 2.1524360179901123, -3.761489152908325, -0.7341042160987854, 0.7302564978599548, 0.07145574688911438, -2.6177403926849365, 1.6589584350585938, -0.29992440342903137, -0.8321341276168823, 0.13554632663726807, 0.20458009839057922, -2.101438283920288, 0.7215458750724792, 1.1116656064987183, 2.21815824508667, 1.3816289901733398, -1.309767723083496, 4.386033535003662, -1.4230409860610962, 0.48307809233665466, -2.3432884216308594, -1.8689744472503662, -0.04274582117795944, -0.6290598511695862, 3.8006348609924316, 1.608057975769043, -1.8207610845565796, 2.671281576156616, -0.6897813081741333, -0.7426902055740356, -1.781022548675537, -0.8300800919532776, 15.308076858520508, 4.499148368835449, 1.1359689235687256, 5.338602542877197, 0.37712305784225464, -0.24877719581127167, 1.4366955757141113, -2.308534860610962, -1.8247679471969604, -1.9757295846939087, -1.827988862991333, -0.4484160244464874, 0.12679709494113922, -1.3533966541290283, 0.07320339977741241, 1.4933924674987793, -0.2966952919960022, 1.397203803062439, -0.08545542508363724, -1.6037806272506714, -1.9294589757919312, -0.14407464861869812, 0.19257278740406036, 0.39797258377075195, -1.7528400421142578, 1.1386864185333252, -2.2087562084198, 2.186969041824341, 0.03384530544281006, 1.3466943502426147, -3.005445718765259, -1.8600465059280396, -0.3779038190841675, -1.5543227195739746, -0.16590534150600433, -3.1791491508483887, -2.7174696922302246, 1.8676429986953735, -0.622207760810852, -1.1177133321762085, -2.6838295459747314, -1.30609130859375, -1.5765057802200317, 0.45603498816490173, 0.7793853878974915, -1.354432463645935, 0.00621503172442317, -0.4247341454029083, 2.202543258666992, -0.11820641160011292, -0.17369817197322845, -0.3092726469039917, 0.47314026951789856, -3.458984136581421, -0.22899313271045685, 0.5670687556266785, 0.3589801490306854, 1.489737868309021, -3.1608481407165527, -0.21931415796279907, 0.807947039604187, -1.8317372798919678, -1.132671594619751, 0.4855450689792633, -0.43728768825531006, 1.4943300485610962, 1.1050652265548706, 2.4446702003479004, -0.8241856694221497, -0.7546805739402771, 0.7994499802589417, -3.0786194801330566, -0.3895534873008728, 2.1855008602142334, 1.393508791923523, -0.5002986788749695, 1.5262712240219116, 0.3698429763317108, 1.5200587511062622, -0.15930980443954468, 1.4308058023452759, 2.142819404602051, 0.45494741201400757, -0.47607141733169556, 1.4958230257034302, -1.795457363128662, 1.0648260116577148, -1.7133510112762451, -0.2609085738658905, 0.3106635510921478, -1.3896536827087402, -0.33241382241249084, 1.336611270904541, 2.125523090362549, 1.2776188850402832, -0.8211570978164673, -1.1162667274475098, -1.1321229934692383, -0.545015275478363, 0.5927860140800476, 3.7427353858947754, -0.49113592505455017, -2.3449840545654297, 1.2696799039840698, 0.26312845945358276, 1.9718315601348877, 2.0271904468536377, -0.131790429353714, -1.2603363990783691, -0.6905063986778259, 0.15193144977092743, 0.20459876954555511, -0.3901022970676422, -1.4057021141052246, -1.3924267292022705, 1.5517851114273071, -1.8286279439926147, -1.7117321491241455, 0.8586218357086182, -1.145815372467041, -0.1132550910115242, -1.717984676361084, -0.2571682333946228, 1.6401997804641724, -0.11809037625789642, 0.5633566975593567, -1.8970674276351929, -0.9350809454917908, 0.35490289330482483, 0.2628072202205658, 1.4788419008255005, 2.3659958839416504, -2.9421756267547607, -2.8026111125946045, 0.7026481628417969, -1.0338380336761475, 1.0798962116241455, -1.7842527627944946, -1.098308801651001, 0.700696587562561, -1.1783242225646973, 0.6634416580200195, -0.06622150540351868, -0.6117193698883057, -0.9648020267486572, -1.7386441230773926, 1.301555871963501, 1.809696078300476, -1.2478480339050293, 1.1025060415267944, 0.24218317866325378, 1.1803829669952393, 1.7108783721923828, -1.7723249197006226, -1.8628815412521362, 0.6948750615119934, -0.2978351414203644, 0.5001640319824219, 0.13679417967796326, -1.3288893699645996, 0.9691439867019653, 2.211162567138672, 3.4182510375976562, 0.9228296875953674, -1.0652220249176025, 0.5729921460151672, 1.2234716415405273, -1.127604365348816, 1.240395188331604, 1.7361613512039185, 0.5759967565536499, -0.7276569604873657, 0.11856471747159958, -1.5833559036254883, -0.6367866396903992, -2.703447103500366, 0.2136736363172531, 0.5020568370819092, 1.4055930376052856, 0.522093653678894, -1.530550479888916, 0.3526874780654907, -1.0863837003707886, -1.239226222038269, -1.2627238035202026, 3.344712972640991, 3.0156545639038086, 0.4993290901184082, 2.0882205963134766, 2.0388309955596924, 1.621515154838562, 1.658479928970337, -0.33273643255233765, -0.501126229763031, 2.1199073791503906, 1.536026120185852, -0.9914377927780151, 0.24250216782093048, -1.9747645854949951, 2.5204825401306152, -0.9113062024116516, 0.5862038135528564, 2.0605995655059814, 0.15691199898719788, -1.1655324697494507, 1.3286830186843872, -0.7818413376808167, 0.24309000372886658, -0.5821051001548767, 0.8977813720703125, 0.8325903415679932, 0.7971518039703369, -1.7094535827636719, 0.5417901277542114, -0.4637387990951538, 0.07838840037584305, 0.8628305792808533, -3.1316866874694824, 0.609347403049469, -2.217905282974243, -1.2237290143966675, -0.5012655854225159, -0.6536839008331299, -1.0421169996261597, -1.3535853624343872, 1.0820767879486084, 1.7036887407302856, 2.364006519317627, 0.9525275826454163, -1.5893136262893677, 1.1733524799346924, -3.345583200454712, -1.548890233039856, 2.5361790657043457, 1.7831652164459229, 1.9568798542022705, 1.0925227403640747, -0.5868611931800842, -2.3567018508911133, 1.4547518491744995, 1.7280499935150146, -0.3538397550582886, 0.014551497995853424, -0.5273894667625427, -1.286423921585083, -1.0213130712509155, 3.9277548789978027, -0.4799060821533203, 2.6684036254882812, -1.2108335494995117, 0.7121280431747437, -0.05225204676389694, 0.5647032260894775, -0.978239119052887, 0.36165863275527954, -0.4695630967617035, 2.110774517059326, 0.5655459761619568, -1.673122763633728, -0.2846534550189972, 1.557080626487732, -0.23966547846794128, -2.617906093597412, -1.048081398010254, 0.8010277152061462, 0.285885214805603, 0.9406924247741699, 1.506739616394043, 3.286447048187256, 0.840923011302948, -1.1081193685531616, 0.5627616047859192, -2.377079725265503, -0.9117599129676819, 0.12689322233200073, -0.5873638391494751, 0.39348092675209045, -2.4173333644866943, -2.8826725482940674, -0.8581613898277283, -1.348276138305664, 1.4085086584091187, -3.7529218196868896, -1.4009926319122314, 0.5957542061805725, 0.8100380301475525, 1.3090600967407227, -0.68523770570755, -0.1529802531003952, -4.883052825927734, 0.8086901903152466, 0.7953837513923645, 0.25518155097961426, -0.3637058436870575, 0.4447179138660431, 0.21693503856658936, 0.04838335141539574, -0.8519362211227417, -2.1920151710510254, -0.5888158082962036, 1.7183278799057007, 0.6911154985427856, 1.7108795642852783, -0.7512059807777405, 0.3244381844997406, -0.9440633058547974, -1.3199416399002075, 0.2676181197166443, -2.4180173873901367, -0.03297616168856621, 0.6127227544784546, -3.191373586654663, -2.398658514022827, -2.8290557861328125, -2.0545785427093506, -1.127089500427246, -0.22215719521045685, -3.206400156021118, -1.6629010438919067, 0.806023120880127, -1.445382833480835, 0.5677853226661682, -0.9143922924995422, -0.8129569888114929, 1.1562775373458862, 0.3673211634159088, 0.616406261920929, -1.4902247190475464, 0.41270872950553894, -2.752145767211914, -1.0074336528778076, 0.9804085493087769, 2.0637340545654297, -0.6152509450912476, -1.6141784191131592, -1.0124706029891968, 0.6982508301734924, -1.5356385707855225, -1.07344388961792, -1.6443156003952026, 1.370328664779663, 2.6548542976379395, 2.2291948795318604, -2.2439427375793457, 0.2979300916194916, 0.843829870223999, 1.0652967691421509, 1.5912325382232666, -0.8146464228630066, 2.467747211456299, 1.85685133934021, -1.046760082244873, -0.35854461789131165, 1.4400664567947388, 2.4162545204162598, -0.1738983392715454, 1.4557204246520996, 0.21170808374881744, -2.6645596027374268, -0.7927485108375549, 2.3007044792175293, 0.047974515706300735, -1.082923412322998, 1.9720646142959595, -0.6245571374893188, 1.1590254306793213, -1.810961365699768, 0.5567652583122253, 0.3611100912094116, -0.1958187222480774, 1.618150234222412, 1.9331226348876953, 0.8648907542228699, 0.7126857042312622, 2.9407429695129395, -1.406671404838562, -1.0295168161392212, -3.9181225299835205, 0.6477622985839844, 1.7982028722763062, 0.44160836935043335, -1.4339231252670288, 1.137361764907837, -1.718029499053955, 1.2309757471084595, 1.4256013631820679, -0.84185391664505, 0.46007582545280457, -2.6769826412200928, 3.001845598220825, 2.058131694793701, 0.9507672190666199, 1.2271422147750854, 1.1276698112487793, 1.0539954900741577, -1.315648078918457, 2.366368532180786, 1.119044303894043, -0.4742223620414734, 0.036470089107751846, -1.5167737007141113, 1.609507441520691, 1.1476560831069946, 0.5972101092338562, 0.8232600688934326, 0.1859944462776184, -0.43208691477775574, 0.9084160923957825, -1.8298605680465698, 0.16709484159946442, -0.14666922390460968, 5.386466026306152, -0.40133723616600037, -0.4673129916191101, 2.677629232406616, 0.583078145980835, 1.2700741291046143, -1.3534055948257446, 0.3498459756374359, 0.07629211246967316, 0.8949180245399475, 1.276918888092041, 1.030253529548645, -0.41053929924964905, 0.01897682249546051, -1.916705846786499, -1.4112930297851562, -5.07365083694458, 1.1667559146881104, -0.8804594278335571, 0.947991669178009, 0.1734532117843628, -0.6078105568885803, -1.159491777420044, -1.7389808893203735, -0.14060400426387787, -0.2349875420331955, 1.0502312183380127, 2.887429714202881, 0.7986916303634644, 3.3437893390655518, -1.1231616735458374, -1.3941779136657715, 2.5779285430908203, -0.5716120600700378, 0.8345473408699036, -2.4001803398132324, -0.35768410563468933, -0.5137505531311035, -1.271213412284851, 0.36825042963027954, -0.17553158104419708, 1.3638747930526733, 0.9383573532104492, -0.35976895689964294, -0.4184093177318573, -0.13208003342151642, -0.4550875127315521, 0.09526042640209198, -2.154437303543091, 0.1041792705655098, 1.1116080284118652, 2.1973259449005127, 1.1886924505233765, -1.8469220399856567, 0.7046998739242554, 1.7011383771896362, -0.012007580138742924, -0.8955718874931335, -0.6798097491264343, 1.2021775245666504, -0.4633905291557312, -0.5727680921554565, -0.4751579463481903, 1.0826784372329712, 1.150945782661438, -0.3354477286338806, 1.9252313375473022, -0.1103413850069046, -4.846770286560059, 1.746255874633789, 2.1489789485931396, 0.38063350319862366, -0.2046860307455063, 1.6509380340576172, -1.524577021598816, -0.5056513547897339, -1.2226569652557373, -0.7125353813171387, 2.5430386066436768, -0.5083715915679932, -0.9039136171340942, -2.214122772216797, -1.4386627674102783, 1.52494478225708, 0.9554242491722107, -0.7139443755149841, 1.6525676250457764, 0.9473896026611328, 0.0605894960463047, 2.2107536792755127, -1.243004560470581, 1.8444044589996338, -1.1262842416763306, 2.9831442832946777, -0.057442471385002136, 1.6794769763946533, -1.3721674680709839, 0.6539650559425354, -1.437860369682312, -1.5957772731781006, -2.045863628387451, 0.19939415156841278, -3.2603986263275146, 1.1125015020370483, 0.21642795205116272, 0.5228146314620972, 0.41695284843444824, 0.7257571816444397, -0.9821311235427856, 0.7697579860687256, -1.301535725593567, 0.5237575173377991, 1.7650668621063232, 3.064073085784912, 0.4259604811668396, -1.7728257179260254, -0.6701330542564392, -1.1442248821258545, 0.2424418330192566, 0.36325696110725403, 0.8258731961250305, -1.938664436340332, -2.199969530105591, 0.08673402667045593, -0.9208409190177917, 0.5146213173866272, 0.6925238370895386, 0.7207636833190918, -3.0263946056365967, 2.051480293273926, 1.9583534002304077, -0.4958018660545349, -0.4921548664569855, -0.6339451670646667, 1.3774651288986206, -1.365641474723816, -0.9062960147857666, 0.3792218863964081, -0.8667156100273132, 0.6548132300376892, 1.5432883501052856, -0.30988723039627075, 0.07642725855112076, -2.532318353652954, -2.4170353412628174, 1.3273682594299316, 0.38782888650894165, -1.359891653060913, 2.9900224208831787, 1.6709799766540527, 2.2675297260284424, -1.960673451423645, -0.8736122846603394, -0.8998159170150757, -1.4063345193862915, 0.8164178133010864, -0.2670990824699402, -0.3882327079772949, 3.639309883117676, 1.0340632200241089, -0.23300497233867645, -0.5337792038917542, -8.180279731750488, -1.6668132543563843, 1.1444876194000244, 1.4386248588562012, -1.2058913707733154, 1.4252701997756958, -0.3054998219013214, 1.678726315498352, 0.29092761874198914, -0.9863734841346741, 1.2304704189300537, 0.4720812737941742, -0.7312211394309998, 4.532645225524902, -0.2892036437988281, 1.0390431880950928, 0.27002260088920593, 0.9920822381973267, 0.7624692916870117, -2.532465696334839, -0.6191721558570862, -0.8173220157623291, 2.0454766750335693, 0.11239688098430634, -0.1547958105802536, -0.594840943813324, -0.8179011344909668, 0.4414551854133606, 0.15421058237552643, -0.8646214604377747, 1.0718694925308228, -0.28193163871765137, -0.2536405324935913, -0.9013018012046814, -0.25749295949935913, -0.16212229430675507, -0.15546061098575592, 0.7798936367034912, 2.032050371170044, 0.46559128165245056, -0.41246700286865234, 0.3810539245605469, -0.07639499008655548, -0.744717538356781, 1.2455486059188843, -0.33049947023391724, 0.7880080342292786, 0.15285639464855194, 0.24284808337688446, 1.5872243642807007, 0.21818003058433533, -0.8508397936820984, 1.016583800315857, 0.2861785590648651, -1.1414259672164917, 13.744778633117676, -0.5510168671607971, -1.5031408071517944, -0.0004721303121186793, 0.7582346796989441, 1.7081146240234375, -1.4792088270187378, 2.49641752243042, -1.0513739585876465, 1.6119489669799805, 0.2483021765947342, -3.125922918319702, 0.5701391100883484, -1.2790406942367554, -1.4392642974853516, 0.7650883197784424, 3.486443281173706, 0.5014379024505615, -2.570587635040283, 1.7341738939285278, -0.7534667253494263, -1.991068720817566, -0.02285146340727806, -2.9451937675476074, 0.41508424282073975, -2.5952744483947754, 0.390113890171051, 1.7391259670257568, 1.0904104709625244, 1.3765461444854736, 1.0525375604629517, 0.21069857478141785, -0.8218095302581787, 1.037114143371582, 0.573291003704071, -0.6391475200653076, -1.7290054559707642, -1.0819053649902344, 0.3276559114456177, -0.5421224236488342, 0.20580430328845978, -0.7606003284454346, 3.415459394454956, -1.6657764911651611, -0.7570114135742188, -0.07734159380197525, 0.614861249923706, 1.5335725545883179, 1.6756341457366943, -0.6280267238616943, 3.2098095417022705, -1.0299365520477295, -1.450359582901001, 0.7952731251716614, 2.3351495265960693, -2.404719114303589, 0.7727455496788025, 2.162484645843506, 1.3129212856292725, 1.2582823038101196, 1.017378330230713, 0.12401356548070908, -2.856858968734741, 1.2268478870391846, 0.8375110626220703, -3.0424931049346924, 0.7082695364952087, 1.8937937021255493, -0.1432461440563202, -0.3548393249511719, 0.08911526948213577, -0.6347584128379822, -0.7595186233520508, 0.4317631721496582, -3.513622760772705, 0.8341638445854187, -1.356445550918579, -0.06197364628314972, 0.700200080871582, -1.4303308725357056, -2.4063665866851807, 0.22587774693965912, -3.4872560501098633, -2.279752492904663, -0.42347708344459534, 0.6555176973342896, 0.48646309971809387, 0.1331585943698883, -1.2049188613891602, 0.0858355164527893, 0.38164329528808594, -1.5419907569885254, -0.01128548663109541, 0.7338986992835999, -2.984287738800049, 0.014797693118453026, 1.8339221477508545, -0.0142299709841609, -0.5960790514945984, -1.4764504432678223, -0.2968903183937073, -1.0762876272201538, 2.9132325649261475, -0.6676231622695923, -1.1038304567337036, -0.4891483187675476, 0.8305044174194336, 0.9821015000343323, 0.49174803495407104, 1.4616111516952515, -2.0859296321868896, 0.0037602924276143312, -2.17464017868042, 0.15742959082126617, 1.735825538635254, 2.4745001792907715, 0.8793602585792542, -1.4530504941940308, 2.6615378856658936, -1.2239083051681519, 0.033602554351091385, 0.2817610800266266, 0.7717196345329285, -1.3991419076919556, -1.6138641834259033, 1.404070496559143, 0.814713180065155, 0.543089747428894, -2.5180506706237793, -0.7763581871986389, 3.08744215965271, 0.6421825289726257, -0.37814050912857056, -1.4769726991653442, -0.8898617625236511, 0.10628295689821243, -0.4574272930622101, -0.32095614075660706, -0.7747358679771423, -0.48474013805389404, -0.8965149521827698, -0.20965301990509033, 0.7020500302314758, 2.172895669937134, 0.35191071033477783, -4.009674072265625, 0.4118696451187134, 2.50784969329834, -0.7315405011177063, 2.3343074321746826, 0.2837725281715393, -1.0791977643966675, -0.9754272699356079, -0.1424376517534256, 1.071882724761963, 0.21153412759304047, 1.2418016195297241, 0.6006279587745667, -21.420949935913086, -2.8343706130981445, 1.1846017837524414, -0.5831271409988403, 0.27605101466178894, 0.31342241168022156, 0.3804667592048645, -0.6204016804695129, -1.688852071762085, 0.06944048404693604, 2.888490676879883, -0.04812486842274666, 0.06079423055052757, 0.2914941608905792, 0.7775839567184448, 1.3224005699157715, -0.14880423247814178, 0.4934854805469513, 1.6506434679031372, -3.543189287185669, 0.7927256226539612, -0.0631561353802681, 0.5146726965904236, 1.8793789148330688, 3.406104326248169, 0.009459360502660275, 1.158433198928833, -1.9461312294006348, 2.498833417892456, 1.3511296510696411, 1.7026455402374268, -0.5812314748764038, -1.1749104261398315, 4.027259349822998, 2.5580437183380127, 1.3591547012329102, 0.4456899166107178, 3.242056369781494, -0.18608899414539337, 0.21930868923664093, -3.047318458557129, 1.6399729251861572, 1.0533949136734009, -1.15907883644104, -0.3471929132938385, 1.5201295614242554, -0.9823837876319885, 1.913155436515808, 2.9428884983062744, -0.16435237228870392, 0.8542391061782837, -0.1293112337589264, -1.3918452262878418, 1.3658188581466675, -0.9631404280662537, -0.12018332630395889, 0.4843815267086029, 0.316967636346817, 2.1362717151641846, -1.6006104946136475, 1.0932544469833374, -0.8913019299507141, -2.203192949295044, -1.784547209739685, -0.5473523736000061, -1.773787021636963, -1.3438606262207031, -9.134454727172852, -0.3668064773082733, 1.1594419479370117, -0.471346914768219, -1.222916603088379, 0.8977785706520081, 2.0482819080352783, -0.6795237064361572, -1.904762864112854, -1.8654203414916992, -0.8773447871208191, 0.3619116246700287, 2.4070322513580322, -0.7647301554679871, 0.8195843696594238, 3.562347173690796, 1.7705801725387573, 0.4481431543827057, -0.315407931804657, -3.2276318073272705, 3.0218489170074463, -0.7448601126670837, -0.20249822735786438, 1.1900779008865356, -0.2980712056159973, -1.327319860458374, -2.9366252422332764, 0.9362067580223083, -0.03158435970544815, -0.6719092130661011, -0.23536771535873413, 0.7487512230873108, 0.10564935952425003, 0.1111750528216362, 2.3561840057373047, 2.9969379901885986, 1.3017241954803467, -0.3052200973033905, 0.24568630754947662, 4.86990213394165, 1.446625828742981, -6.220031261444092, 3.235675811767578, -0.9706981182098389, 2.667736768722534, -0.7600289583206177, 2.573045015335083, -1.9426485300064087, 1.3780653476715088, 0.7698491215705872, -0.41656166315078735, -0.002633963478729129, -0.4099370837211609, 1.906267523765564, -4.080760955810547, 2.657559394836426, 1.1049977540969849, -0.1501992642879486, -1.5594491958618164, 2.0277116298675537, -2.5000364780426025, 3.7016167640686035, 0.9173998832702637, 1.100572943687439, -1.537651538848877, -2.9868316650390625, -2.0119779109954834, 1.5564953088760376, -1.6356868743896484, -2.82167387008667, -1.853916049003601, -3.4782540798187256, -0.12901268899440765, 0.08623765408992767, -1.52432382106781, -2.0389511585235596, -0.6726512312889099, -1.7578582763671875, 0.6359127759933472, 0.21619074046611786, -1.3237913846969604, -0.9204597473144531, -2.7911970615386963, 1.1620633602142334, 0.3143114745616913, -1.0657422542572021, 0.2335847020149231, 0.16069342195987701, -0.43788209557533264, -0.07373473048210144, 0.2565538287162781, 1.6619505882263184, -0.9554142951965332, -1.2392710447311401, 1.1582152843475342, 1.9720089435577393, 1.1198052167892456, -0.07490064948797226, -0.8518654704093933, -0.9286448955535889, 0.6357172131538391, -0.9352173209190369, 1.0517035722732544, 0.34902676939964294, -0.3538692891597748, -1.906524896621704, -0.4601396918296814, -1.8763014078140259, -1.634195327758789, 2.763216257095337, 3.7860472202301025, -0.7164503335952759, -2.5718564987182617, 2.4216134548187256, -2.881164073944092, -0.3324626088142395, 2.1593081951141357, 0.7503798604011536, 1.1787254810333252, -0.9773991107940674, -1.7141178846359253, -0.3675839304924011, -0.3886459469795227, -1.700588345527649, 0.27874693274497986, 0.9193711280822754, -1.2540279626846313, 0.5896230340003967, -1.5990028381347656, -5.00225305557251, 0.17458167672157288, 1.8127763271331787, 0.9114691019058228, -0.3649178147315979, 1.3571234941482544, -0.4471230208873749, 2.7665112018585205, 0.13936962187290192, -1.8823177814483643, 0.11152156442403793, 0.8063399195671082, -0.623543918132782, 2.205143451690674, 2.26967191696167, 0.2673937976360321, -1.224618911743164, 2.4955570697784424, 0.9738336801528931, 2.141892433166504, 1.431698203086853, -1.5605469942092896, 1.6100858449935913, 0.43578535318374634, -0.15090185403823853, 0.43507975339889526, 2.1998069286346436, -0.7322538495063782, 0.39230263233184814, -1.904099464416504, 1.0630433559417725, -0.3866976499557495, 1.1618212461471558, 2.4878039360046387, -1.821178674697876, -1.3471040725708008, 1.6811494827270508, 0.4070914685726166, 0.21077117323875427, -1.5663288831710815, 0.05208250507712364, 0.3708331882953644, -0.4498453438282013, 1.9233452081680298, 1.073796033859253, 2.2147319316864014, -0.10144785046577454, -2.497713565826416, -0.9165378212928772, 1.2445310354232788, 0.8475666046142578, -1.4845939874649048, -2.521730661392212, -0.8419479131698608, -1.533697485923767, -1.4756275415420532, 4.748761177062988, 2.2733912467956543, 0.26443323493003845, 1.6314960718154907, -0.17511925101280212, 1.9817638397216797, -0.34183356165885925, -0.0683571994304657, -0.7341527342796326, -1.0625113248825073, -1.5894094705581665, -1.2501089572906494, 0.6550843119621277, 2.9652516841888428, -18.701189041137695, 0.8202524185180664, 2.4397833347320557, 0.7886958718299866, -0.25994548201560974, 1.1355319023132324, -1.3753807544708252, -0.17749600112438202, -2.341002941131592, 0.8091291785240173, 0.6371749639511108, -0.5609314441680908, -2.8192873001098633, -0.20296630263328552, -0.6832893490791321, 0.2510371804237366, 1.9415379762649536, 0.8106158375740051, -0.6866949796676636, 0.7033583521842957, 1.187045693397522, 5.318829536437988, 0.4344800114631653, -1.1851502656936646, 0.14760847389698029, -1.9030660390853882, -1.161205530166626, -1.1422580480575562, -1.2582191228866577, -1.5141586065292358, 2.0200791358947754, 0.46125006675720215, 0.6097654104232788, -0.2736853063106537, 0.04637685790657997, -0.8586694598197937, -1.1096073389053345, 2.156308889389038, -3.376009464263916, -1.1569936275482178, 1.0960450172424316, 0.15800978243350983, 3.3004324436187744, -2.345585584640503, 1.4547779560089111, -0.7860064506530762, -0.5926195383071899, 0.9075552821159363, -0.6562105417251587, -3.497018814086914, 1.1083356142044067, -0.9589388966560364, -0.6051695942878723, -0.540803074836731, 0.03849490359425545, -0.013703882694244385, -1.522271990776062, 0.10620472580194473, 1.493301272392273, 0.018884852528572083, 2.2712714672088623, 0.6036868095397949, -0.968465268611908, 1.0862313508987427, -0.09972180426120758, 1.5441598892211914, -1.3638845682144165, -2.585232973098755, 3.3044698238372803, -0.36389654874801636, 3.1675662994384766, 2.9255337715148926, 0.8463337421417236, -3.148622751235962, 4.0538716316223145, -0.7224482297897339, -2.1325669288635254, 1.2118313312530518, 0.1817270964384079, -0.650112509727478, 0.31654539704322815, 0.9558762907981873, 1.44901442527771, -0.6649160385131836, 0.9725015163421631, 1.036099910736084, -0.11814852803945541, 2.479074239730835, 14.018181800842285, 3.2521491050720215, 1.5385557413101196, -1.4475723505020142, 2.9145119190216064, 3.5135793685913086, 0.7966490983963013, -2.2420480251312256, -0.6037612557411194, -3.6134655475616455, 0.06062506139278412, -1.990880012512207, -2.407916784286499, -1.1050269603729248, -1.025273084640503, 1.1125253438949585, -0.38538700342178345, -0.24235951900482178, -1.4133325815200806, 1.4687001705169678, 2.6971004009246826, -0.35052427649497986, -0.47039666771888733, 2.590151309967041, 3.1055245399475098, -2.629302740097046, 1.404173493385315, 0.12226243317127228, -2.0032198429107666, -1.9576460123062134, 0.47427070140838623, -1.478883981704712, -1.681889533996582, 2.079319953918457, -1.1202728748321533, 0.15473487973213196, -0.5956941246986389, -1.1767174005508423, -0.2486310601234436, -0.11927644163370132, -1.11954665184021, 0.3477235436439514, 2.2112820148468018, 2.4915058612823486, -1.313151240348816, -2.6895837783813477, 0.5527937412261963, 0.4014873504638672, -2.0445237159729004, -1.6892722845077515, 2.3183422088623047, 1.5500901937484741, -2.120178699493408, -0.07712919265031815, -1.1877354383468628, -0.11911149322986603, -1.5265284776687622, -0.22791606187820435, -5.057439804077148, 1.8562681674957275, 2.7356135845184326, 3.1429009437561035, -1.493489146232605, 2.452613353729248, -1.6630381345748901, 1.6185678243637085, 1.3271045684814453, 1.8451824188232422, -1.0935972929000854, 1.6124128103256226, 2.4062600135803223, -1.3664880990982056, -0.909802258014679, -0.2729616165161133, -1.1316649913787842, -9.643401145935059, 1.3962445259094238, -2.760251522064209, -1.4282137155532837, -0.1116255596280098, -0.3292570114135742, -1.1277916431427002, 2.8691515922546387, -4.271193027496338, -1.5108366012573242, 0.2683090269565582, 0.039024174213409424, 0.23558183014392853, -0.9242812991142273, -1.48604416847229, 0.6786108613014221, 2.935561180114746, -0.24768754839897156, -0.19210654497146606, -2.6365764141082764, -1.8132530450820923, 0.15882991254329681, -0.14460137486457825, -0.06571049243211746, 2.1071059703826904, -1.0669904947280884, 2.9365127086639404, -3.6621856689453125, -1.990355372428894, -1.456249713897705, 0.08062712848186493, 0.0061852093786001205, -0.6779385209083557, 0.42930176854133606, 1.512561559677124, 0.6565319299697876, 0.9805716872215271, -2.550661563873291, -0.2499394565820694, 0.8713773488998413, -11.699569702148438, 0.36004355549812317, -0.5057438611984253, -2.198042869567871, 3.3681657314300537, -0.05158992484211922, -1.8158985376358032, -0.9005102515220642, 1.0675864219665527, 1.902129054069519, -1.687666893005371, -0.15790192782878876, -1.8053581714630127, -1.4231373071670532, -1.1676887273788452, -1.934894323348999, -0.6421085000038147, 4.323730945587158, 1.9435933828353882, -0.6678130030632019, 1.3202362060546875, -2.3149123191833496, -1.1911044120788574, 0.07888111472129822, -1.0956515073776245, 0.07637935876846313, -1.0391348600387573, 0.5830134153366089, -3.276697874069214, -0.6301103830337524, -1.0665937662124634, 0.04344653710722923, -0.9418973326683044, 0.03877750039100647, 2.058286190032959, 0.06457574665546417, -2.638791561126709, 2.2470667362213135, 2.5310418605804443, -1.8139398097991943, -5.404266357421875, 2.2226779460906982, -11.108283996582031, -0.5280264019966125, -0.6405683159828186, -1.894897699356079, 1.5760639905929565, 1.7777705192565918, -0.5066716074943542, -0.9335857629776001, -2.124025344848633, 3.0684549808502197, 0.922137439250946, -0.8812996745109558, 1.3132891654968262, 0.04799271374940872, -0.9755988717079163, 0.31752264499664307, 0.6503385901451111, -0.4261535406112671, -0.44278934597969055, 0.04853984713554382, -1.231992483139038, -0.4881664216518402, -1.6072906255722046, -1.0422234535217285, -3.2895166873931885, -0.4491083323955536, 0.7031264901161194, -0.8871976137161255, 1.6310783624649048, 1.644734501838684, -0.835677444934845, -0.4111180305480957, -0.1718365103006363, -1.9292038679122925, 7.581175327301025, -0.03038492053747177, -2.202829360961914, -1.0381379127502441, 1.9748871326446533, -0.42246830463409424, 0.056854549795389175, 0.33264273405075073, 2.863565683364868, -0.5304704308509827, 0.11789251863956451, 0.1729017049074173, -2.3224427700042725, 1.6362359523773193, -0.5686156749725342, 0.11132077872753143, 1.0287338495254517, 1.3310177326202393, -0.4374482333660126, 0.8629313111305237, -1.5344178676605225, 2.0460193157196045, -0.8177356719970703, -0.890838623046875, -1.7328402996063232, -0.8328257203102112, -2.4133124351501465, -2.1150732040405273, 1.0254690647125244, 0.6391770243644714, 0.45285579562187195, 0.9788195490837097, 0.38293981552124023, 0.17244616150856018, -2.747685432434082, 0.20160602033138275, -1.6218031644821167, 3.139244318008423, 2.122178792953491, 0.5563103556632996, 2.6714024543762207, 0.7003293633460999, -0.5228545665740967, 0.22528858482837677, 0.46560513973236084, -0.16185320913791656, -4.170257091522217, 0.5397441983222961, 1.1966748237609863, -2.9172093868255615, 1.6779617071151733, -0.31819823384284973, 0.11203520745038986, -0.28129786252975464, -0.6311776041984558, 1.7424696683883667, 6.291396141052246, -0.169159933924675, -0.8180339932441711, 1.6496386528015137, -0.6248394846916199, -2.371833324432373, 2.2880096435546875, -0.44129127264022827, 1.5730830430984497, -1.39114248752594, -0.8860953450202942, -1.8186204433441162, 3.1192703247070312, -1.6806130409240723, 1.2721871137619019, 0.9149099588394165, -1.3063874244689941, 0.46922338008880615, 0.11119222640991211, -1.204764485359192, -0.3228885531425476, 0.19804522395133972, 4.467627048492432, 1.8320302963256836, 0.2878636419773102, 0.3422572910785675, -0.10084330290555954, -1.6164077520370483, 3.1517581939697266, -0.09334182739257812, -0.713240385055542, 1.3445292711257935, 1.8047235012054443, -0.4165444076061249, -1.2036916017532349, 0.3579061031341553, 2.492398738861084, 0.77812260389328, -1.059193730354309, 0.84946209192276, 0.7439637184143066, -1.0057952404022217, 0.14263564348220825, 0.5453804731369019, 2.452277183532715, -1.6124141216278076, -3.34645414352417, 0.6867415904998779, 0.8405324220657349, 0.409101665019989, -0.25847572088241577, 1.0531089305877686, -2.025064706802368, 0.9108526706695557, 2.23378324508667, 1.789442539215088, 3.426844835281372, 0.48308229446411133, -1.8132532835006714, 2.046420097351074, -0.16112984716892242, -0.5771458745002747, 0.7757382392883301, -0.5303272604942322, 0.030907223001122475, -0.9248496294021606, 1.9582315683364868, 2.2200794219970703, 1.3913213014602661, 0.31603509187698364, -0.25678369402885437, -0.8379045724868774, 1.4704560041427612, 0.42616111040115356, 0.33771321177482605, -0.6459184885025024, 0.6331501007080078, 0.6929988265037537, 0.6126759052276611, 0.37816211581230164, 0.2566397786140442, -0.8873593211174011, -1.5456784963607788, 0.5373156070709229, -1.1802836656570435, 2.847151517868042, 0.6196730732917786, -3.4910216331481934, -2.2463622093200684, -2.840139865875244, 1.3587255477905273, 1.1104686260223389, 0.8792030811309814, -0.9109576940536499, -1.443090558052063, 0.40472933650016785, 0.8187597990036011, 0.4845291078090668, -0.39808419346809387, 0.41579440236091614, -0.8640880584716797, 0.31545716524124146, -1.279494285583496, 1.6224243640899658, 2.1278481483459473, 1.7725415229797363, 1.0001370906829834, -5.503050804138184, -2.7724480628967285, -0.755505383014679, -1.714643120765686, 2.0156779289245605, -0.5080789923667908, -1.5444037914276123, 1.1689538955688477, -4.7484450340271, 1.2750660181045532, -0.9162259697914124, 0.390911728143692, 0.1666184514760971, -0.3264663517475128, 0.892450213432312, -0.5283217430114746, 1.76131010055542, 3.980088710784912, 0.6839766502380371, -0.6195866465568542, -0.5090174078941345, -2.182767152786255, 2.754349708557129, 0.0905686691403389, 1.2685842514038086, -2.521639347076416, -0.28612983226776123, -0.6228740811347961, 0.781406581401825, -0.461264431476593, -1.5465344190597534, -1.6996296644210815, 2.1677730083465576, -0.22981226444244385, 0.8794371485710144, -1.998548150062561, -0.05762003734707832, 0.970823347568512, 3.028809070587158, 1.5103275775909424, 2.6971256732940674, -0.3222373127937317, -1.3495385646820068, 5.895050048828125, 0.021448157727718353, -1.540669560432434, -1.2545790672302246, -1.4921913146972656, 0.5233637094497681, -1.7525824308395386, 0.8953695893287659, 1.7819815874099731, -2.153459310531616, -0.44391971826553345, 1.443813681602478, -1.9770612716674805, -0.14933256804943085, 0.3296167552471161, 1.0565487146377563, -1.6306036710739136, 0.9352942705154419, -1.0075520277023315, 1.4307979345321655, -1.2694045305252075, 0.07079903781414032, 3.0635204315185547, -2.0775537490844727, -0.5457540154457092, -0.6783117055892944, 0.13801956176757812, -0.6318509578704834, -1.1500433683395386, -1.8654816150665283, -3.3920984268188477, -0.7337403893470764, 0.5427237153053284, -10.559120178222656, 3.1437690258026123, -0.05913808196783066, 0.1711561381816864, 4.9022932052612305, -0.5797146558761597, -1.5212745666503906, 0.45537036657333374, -1.742936372756958, -0.9190981388092041, 1.040703296661377, -0.66218101978302, 3.1694741249084473, 0.47831079363822937, 0.2720944285392761, 1.7583664655685425, -4.860098838806152, -0.053032875061035156, -1.291128158569336, -1.3883495330810547, 1.130714774131775, -0.9217865467071533, 0.8472790718078613, 2.2149739265441895, 0.9628981947898865, -1.777923822402954, -2.7127535343170166, 1.6989758014678955, -2.4012458324432373, 2.6440393924713135, 0.13701216876506805, -2.258145332336426, -0.9321090579032898, 1.059846043586731, -1.7056504487991333, -0.1044892743229866, -1.0773676633834839, -0.4555036425590515, -0.48205485939979553, -1.8671809434890747, 0.6949787735939026, 1.8884248733520508, 1.2054814100265503, 0.056861624121665955, 2.6977853775024414, 0.0045973495580255985, 2.006392240524292, -1.929414987564087, -0.1426268070936203, -3.1883044242858887, -0.11045465618371964, 1.1932376623153687, 0.4281643331050873, 0.9244000315666199, 0.19800224900245667, 0.11162390559911728, 0.21851806342601776, 0.14462868869304657, -0.9067169427871704, -0.3157569169998169, -0.5347120761871338, 0.35801294445991516, 1.3608613014221191, -0.7124729156494141, -1.2373749017715454, 2.523893356323242, 0.26463785767555237, -2.4108314514160156, -4.617386817932129, -2.006546974182129, -1.7905879020690918, -0.3913322687149048, 1.2253541946411133, 0.04857427254319191, -0.8484410047531128, 1.6456940174102783, -0.5924059152603149, 1.411556601524353, 1.0445913076400757, 0.3465275168418884, 0.49489012360572815, -1.201570987701416, -0.8827998638153076, -1.482887864112854, 0.49393874406814575, -0.342245489358902, 0.9804787039756775, -2.2678191661834717, -0.5223694443702698, 0.9889194369316101, -2.8761887550354004, -1.6258163452148438, 0.4284553527832031, -0.8504677414894104, 0.49677732586860657, 1.6894053220748901, 0.8351526260375977, 2.3068690299987793, -0.14014789462089539, 0.7896067500114441, -1.9927679300308228, 3.4867827892303467, 0.9833028316497803, -0.9227049350738525, 0.936069905757904, 1.5157026052474976, 0.22832466661930084, -0.18623538315296173, -1.2886149883270264, -1.011892557144165, -1.8424826860427856, -1.0910090208053589, 1.805274486541748, 0.12085957825183868, -1.3259955644607544, 2.9736578464508057, 2.8373324871063232, 3.2622954845428467, 1.7891789674758911, 1.7666336297988892, 0.6259182095527649, 0.4835473299026489, -2.598273992538452, -2.980947732925415, -2.6455280780792236, -4.452895641326904, 0.09949974715709686, 0.08738715946674347, -1.7780282497406006, -2.1362721920013428, -0.14688454568386078, 1.1305856704711914, 3.3549680709838867, 2.8300845623016357, 0.9392951130867004, 0.5753041505813599, -2.388108968734741, -2.6511762142181396, 3.8345787525177, -0.7299167513847351, -0.06892631947994232, 2.1413731575012207, -2.142507791519165, -0.1862301230430603, 2.6205263137817383, -3.811314105987549, 0.5702074766159058, -0.1064937636256218, -1.1159729957580566, 0.32658714056015015, 1.0185351371765137, 1.185319185256958, -1.3801430463790894, -0.01855907030403614, 1.1492698192596436, 2.2747771739959717, -2.151740550994873, -0.9055935144424438, -1.7842875719070435, -0.8179410696029663, 1.1856224536895752, 0.708396315574646, 0.06148455664515495, -0.397631973028183, 0.3916808068752289, 2.082693576812744, 0.03635208308696747, 1.7106295824050903, 0.3018409013748169, -0.2881024181842804, 0.346036821603775, -1.617591381072998, 1.6767988204956055, -1.734968900680542, -0.090103380382061, 0.6980364918708801, 10.45659351348877, 0.7424771189689636, -2.9329493045806885, -1.215025782585144, -1.3362879753112793, -1.4445343017578125, -0.7137936949729919, -1.682236671447754, -0.6099538207054138, -1.3198930025100708, -1.1870269775390625, -1.6643357276916504, 0.7591449618339539, 2.1757123470306396, 2.336337089538574, -1.1334086656570435, -0.8817094564437866, 0.371234655380249, 1.887739658355713, 4.541894435882568, -0.6083433032035828, -1.7299890518188477, 0.6660057306289673, -0.25110986828804016, -1.6598573923110962, 0.8134128451347351, 0.4254581332206726, -0.695938229560852, 2.364710807800293, 1.183273434638977, -1.3891937732696533, -1.1608442068099976, 2.596478223800659, -0.2952113449573517, 1.9969711303710938, 1.4508076906204224, 1.5363010168075562, 0.3119700253009796, -1.7113523483276367, -2.624735116958618, -0.05554179847240448, -1.3958821296691895, -1.334257960319519, -1.9967153072357178, 1.3960639238357544, -0.15369373559951782, 0.3491846024990082, 2.0170459747314453, -0.5139067769050598, 2.201956033706665, 0.6126708388328552, -1.0937278270721436, -0.735682487487793, -1.1035046577453613, -0.7179574370384216, 0.6250278949737549, 0.9604902267456055, -1.6567585468292236, 2.725944757461548, 0.06779851019382477, 0.8193672299385071, -3.819422960281372, -1.9663136005401611, 0.36381617188453674, -1.7325072288513184, 0.9024469256401062, 0.5174641609191895, 3.1775965690612793, 1.5616875886917114, 2.4143691062927246, -0.7582035064697266, -1.063307762145996, -1.2788501977920532, -1.689505934715271, -0.7637787461280823, 1.6893481016159058, -0.11205708980560303, 0.9015246033668518, -4.877630710601807, 0.8815420269966125, -0.09041304141283035, -1.1162358522415161, 1.308427095413208, 1.2208911180496216, -2.191664457321167, 2.9904539585113525, 0.3891310691833496, -0.7940407395362854, -0.27750664949417114, -5.7373366355896, -0.6304205060005188, -1.8411104679107666, -2.8800790309906006, -0.7378053665161133, -1.284432291984558, -2.901256799697876, -0.18895772099494934, -0.8044360280036926, -1.2368979454040527, -0.6222343444824219, 1.8751604557037354, 0.8801380395889282, -1.028169870376587, -2.2483301162719727, 1.28036630153656, -2.6097021102905273, -2.7985002994537354, 0.15878143906593323, -1.5521682500839233, 1.5301951169967651, -0.39128854870796204, 0.17004090547561646, -1.6955643892288208, 1.0703554153442383, -0.6193923950195312, -3.2997777462005615, 0.5998481512069702, 2.5057737827301025, 0.9926300644874573, 0.9211235046386719, 1.2224072217941284, 1.3553645610809326, 0.3626805543899536, 0.8134894371032715, 1.3932734727859497, -2.5194215774536133, 0.4976920187473297, -4.380551338195801, 0.6439796090126038, -0.8452206254005432, -0.684146523475647, -0.09151198714971542, -3.391937017440796, 1.901594877243042, 0.43389371037483215, 0.16975276172161102, -0.2205611616373062, 1.7419748306274414, -0.33728569746017456, 1.258888840675354, 0.540742814540863, 1.4637962579727173, -3.2624287605285645, 0.7106502056121826, -1.8042470216751099, 0.7191550731658936, 2.320417642593384, -0.06594613939523697, -0.3073444366455078, 0.1541379988193512, 0.6953540444374084, -0.7718830704689026, 0.25276979804039, -0.7857299447059631, 3.670630693435669, 1.533897876739502, -2.1718504428863525, -0.17472884058952332, -3.2510526180267334, 2.0801379680633545, -0.0575593076646328, 2.1307661533355713, 2.7800302505493164, -0.36934155225753784, -0.7639249563217163, -2.248680591583252, -0.6070634126663208, -1.0596098899841309, -1.144707441329956, -3.6340179443359375, 1.2428324222564697, -0.5688612461090088, -0.4970954358577728, -0.04922127723693848, 1.227640151977539, 1.651061773300171, -0.6544824242591858, 1.7197984457015991, 2.346564292907715, 1.114428997039795, 1.1896346807479858, -0.30809837579727173, -1.2551640272140503, 1.0385668277740479, 2.9676945209503174, 3.8950161933898926, 1.4268178939819336, 0.1536974459886551, 1.1437643766403198, 0.7228612899780273, 0.7527747750282288, 2.6067323684692383, -1.1111358404159546, -0.5765380859375, -0.3446446359157562, 0.841579258441925, -3.0790390968322754, -1.0179895162582397, -1.0458178520202637, 0.21274594962596893, 0.9361158609390259, 0.9819968938827515, -0.691980242729187, 1.1248937845230103, 1.429940104484558, 1.375110387802124, 2.299851417541504, -0.47305700182914734, 1.0621787309646606, -1.1300559043884277, 0.5351494550704956, 0.39690297842025757, 0.6492407321929932, 0.31013837456703186, 1.2692104578018188, -3.1157655715942383, -0.1633530557155609, 0.9591048359870911, 2.3088154792785645, -1.0194785594940186, -0.32606831192970276, -0.9902122616767883, -0.32080432772636414, 1.4752202033996582, 0.5104209780693054, 2.137525796890259, 2.1687493324279785, -0.6705031394958496, 1.6156620979309082, -0.847395658493042, -1.9115262031555176, 1.360155463218689, -0.8116863369941711, -0.7937771081924438, 2.8471601009368896, -1.0542703866958618, 2.0546369552612305, 1.1481677293777466, -1.5542055368423462, 1.6152946949005127, 3.415783166885376, 0.7939766049385071, -0.29770031571388245, -0.9793601036071777, -2.155484676361084, 0.758017361164093, 1.3252336978912354, -0.670129120349884, 0.12432185560464859, 0.9899417161941528, 1.0244568586349487, -1.3101552724838257, 3.6227073669433594, 1.0617811679840088, 2.3669888973236084, -1.6034212112426758, 1.5229235887527466, 0.771930456161499, 1.6229459047317505, -0.38555797934532166, -1.2325270175933838, 1.3422859907150269, 0.4620857834815979, -1.1079126596450806, -0.44634461402893066, 0.47245725989341736, -1.1731716394424438, -1.7124223709106445, -1.0729284286499023, -3.070000171661377, 1.933424711227417, -1.1140098571777344, -0.16262637078762054, 0.12278400361537933, 2.04203462600708, -2.2923455238342285, -0.29959917068481445, 0.6236276030540466, 0.42049744725227356, -0.3442496657371521, 1.4281500577926636, -0.11091383546590805, -2.387423038482666, -0.9136004447937012, -1.235365390777588, 1.589988350868225, 0.261819988489151, 0.26582348346710205, 1.3943759202957153, -2.3245644569396973, 2.034980535507202, 0.5196359753608704, 0.01323979813605547, -0.020366080105304718, 1.5593154430389404, 0.840213418006897, -0.39390650391578674, 0.6946438550949097, 2.8306632041931152, -1.138152003288269, -2.250027656555176, 1.415540337562561, -0.3921203315258026, -0.8675524592399597, -2.8208725452423096, -0.501013994216919, 0.01809632033109665, -1.019321322441101, 0.47585707902908325, 1.1164103746414185, 0.1396052986383438, 0.19358697533607483, 2.0440027713775635, 0.5124874711036682, -1.0129340887069702, 1.948140025138855, 0.5688205361366272, -0.5727137923240662, 1.0860626697540283, 0.1855604350566864, 0.17475678026676178, 1.8577470779418945, 3.1554317474365234, 0.7662787437438965, 0.24577905237674713, -2.8630123138427734, -3.0882625579833984, 2.3028783798217773, -0.5978400707244873, 1.3112680912017822, -3.6086232662200928, -1.9922205209732056, 0.6175452470779419, 1.3922597169876099, 0.25622445344924927, -2.443293809890747, -2.612020254135132, 1.1210737228393555, -0.2443985939025879, 1.2816849946975708, -0.17161700129508972, 0.6663035750389099, -1.658306360244751, -1.4207484722137451, -1.4164130687713623, 0.687414288520813, -0.7507286071777344, -0.5372640490531921, 0.5957868695259094, 0.05219821631908417, 2.9343671798706055, 1.3246500492095947, 1.1361252069473267, 0.15046468377113342, -0.22571419179439545, 2.1247055530548096, -3.1421585083007812, -1.7065279483795166, 3.109907865524292, 0.22352474927902222, 1.1296042203903198, -1.1007050275802612, -1.5097328424453735, -1.3393268585205078, 0.33112043142318726, 1.3814654350280762, -0.2934351861476898, -0.6343828439712524, 0.9096031785011292, -3.5362346172332764, -5.325187683105469, 1.0620249509811401, -0.5103635191917419, 1.9879887104034424, 1.6441476345062256, 1.760003685951233, 2.2858550548553467, 0.7491816878318787, -0.5784324407577515, 2.376591920852661, -0.5939521789550781, -1.133351445198059, 2.086674690246582, -0.5937817096710205, 0.015153361484408379, 1.6569715738296509, 0.7371610403060913, 0.04793432354927063, -2.2902493476867676, -1.65850830078125, 0.8556639552116394, 1.8933697938919067, 1.3070900440216064, 0.6507681608200073, 1.097748875617981, -2.6245405673980713, -0.7896650433540344, -3.2830159664154053, 2.0438921451568604, 1.3563225269317627, 1.5710880756378174, -0.5278199315071106, 0.5892890095710754, 2.3536148071289062, 0.2565598785877228, -1.0790935754776, -1.4586454629898071, 0.1418466717004776, 3.271998882293701, -1.49519681930542, -1.6302980184555054, 0.04321350157260895, 0.0011362078366801143, -0.09155962616205215, 2.7253546714782715, 1.0467277765274048, -1.464282512664795, -2.2307498455047607, 2.859914779663086, 0.31684914231300354, 0.06015418469905853, -1.0509744882583618, -1.9419749975204468, 0.6836782097816467, -0.19998803734779358, 0.44410765171051025, 0.06451768428087234, -0.6691555380821228, 0.08385379612445831, -0.05774928256869316, -0.7240414619445801, 3.3808321952819824, -1.497216820716858, -0.5231871008872986, 1.0877503156661987, 0.17337475717067719, -0.7558809518814087, -2.600080728530884, -0.4592836797237396, -1.0869258642196655, -0.9272589683532715, 0.9343447685241699, -0.21091000735759735, -0.42215076088905334, 2.7611353397369385, -0.2291029691696167, -1.8579437732696533, -1.310749888420105, -1.4255293607711792, -1.4057772159576416, 1.7242056131362915, 0.7190194725990295, -2.283907651901245, 1.7894060611724854, -1.6512055397033691, 1.5249991416931152, -1.2635786533355713, 1.92875075340271, -0.10479069501161575, 0.17563912272453308, 1.1199702024459839, -1.69051992893219, 0.46707525849342346, -0.5947827696800232, 3.2411844730377197, 0.7651056051254272, -0.26206517219543457, -0.8550758957862854, 1.5869592428207397, 0.6057466864585876, 0.414211243391037, -0.5609678626060486, -0.4221871495246887, -2.331916332244873, -0.13561198115348816, 2.1054530143737793, 0.2515336573123932, -0.3361344039440155, -0.9927572011947632, 0.5319969058036804, 0.5167050957679749, 0.2062254548072815, 0.7059627175331116, 1.6679712533950806, -1.5394140481948853, 2.862476348876953, -0.573541522026062, 1.314218282699585, -1.027472972869873, 0.5099228024482727, 1.9130398035049438, 1.5802122354507446, -1.1501810550689697, 0.5575366616249084, 0.1451936960220337, -0.8053888082504272, 0.34901899099349976, -1.763177752494812, 2.047416925430298, -2.088101625442505, -0.26981014013290405, 1.4312036037445068, 2.091611385345459, -0.9578741788864136, -4.294649600982666, -0.8078462481498718, 2.0259060859680176, -1.0848246812820435, -1.129162073135376, -2.413098096847534, 0.03134714812040329, -1.594024658203125, -3.422600507736206, 1.7021554708480835, -1.1961861848831177, -0.4170463979244232, 1.0771387815475464, 0.9437907934188843, -2.324357748031616, -0.01085769385099411, 3.1297948360443115, -0.3399936854839325, -1.7203751802444458, -0.8261563777923584, -1.2304359674453735, -1.7375843524932861, -3.302537202835083, -0.30601179599761963, 4.872192859649658, 0.04607550427317619, 1.835267186164856, 0.674822211265564, -1.628653883934021, -0.2955378592014313, -0.8161329627037048, -1.7304962873458862, -0.6908485889434814, -0.2541046142578125, -1.4049155712127686, -0.7785363793373108, 1.9688602685928345, -1.0608553886413574, -0.7603525519371033, 5.246443748474121, 2.342258930206299, 1.2948648929595947, -0.3742431700229645, 0.48419979214668274, -0.2861398458480835, -1.3723355531692505, 0.7618651986122131, -2.5586729049682617, 0.5267471075057983, -0.8390535712242126, 0.6143081784248352, -0.9113529324531555, 0.3722302317619324, 0.0737592875957489, 0.4541575312614441, 0.17071004211902618, 0.9789291024208069, -0.7412853837013245, -1.698312759399414, 1.070753812789917, 1.5615404844284058, -1.281114935874939, -0.6759600043296814, 0.4795381724834442, 0.31573745608329773, 0.9952296018600464, -0.7705940008163452, 1.3461169004440308, -2.4007909297943115, 0.13104626536369324, -0.13062116503715515, -1.6532630920410156, 1.0690586566925049, 1.1692241430282593, 0.18230241537094116, -1.1593445539474487, 0.39226052165031433, 1.854642391204834, 0.6643195152282715, 0.4532242715358734, 2.6260993480682373, 0.057640716433525085, 3.5990757942199707, -2.3649322986602783, 0.4877063035964966, -0.6221585869789124, 1.0995311737060547, 0.3943721055984497, -0.19413426518440247, 0.49654823541641235, 2.1153268814086914, 2.1185967922210693, -0.33495327830314636, 0.3421340584754944, 2.749067783355713, -0.7015686631202698, 2.066840887069702, -2.0298776626586914, -2.047908306121826, 0.5323173403739929, 0.5875254273414612, -1.8159655332565308, -2.3077216148376465, 3.3794093132019043, 0.28700295090675354, 5.334820747375488, -1.35939359664917, -3.8517215251922607, 2.36892032623291, 2.0566065311431885, -0.43658560514450073, -2.7010316848754883, 0.899348795413971, 2.1464781761169434, 0.3710147738456726, 0.8312425017356873, 0.10823984444141388, -2.51692795753479, 4.437325477600098, 1.6501435041427612, 1.7782399654388428, 3.0259501934051514, -1.240478754043579, 1.4570163488388062, 0.4459244906902313, -0.3960839509963989, 0.8295993208885193, -1.1751965284347534, 1.6469135284423828, 0.20330959558486938, -2.539268732070923, 0.8005831837654114, 1.6654354333877563, 0.31568461656570435, -0.7587361931800842, -0.12048663198947906, -1.225664734840393, 1.3971422910690308, -3.542945384979248, -0.477572500705719, -1.7731823921203613, -0.07095552235841751, 2.065706968307495, -1.0804060697555542, -2.1546521186828613, -0.11106770485639572, 3.013118028640747, 0.9884273409843445, -0.7153705358505249, -0.3388252556324005, 1.1221232414245605, 0.5717027187347412, 1.6946958303451538, -0.009190261363983154, -0.8151801228523254, 3.077439546585083, -1.3493640422821045, 0.3304941654205322, 2.222222089767456, -0.23586852848529816, -2.7220678329467773, -0.14475366473197937, -0.2366257756948471, 0.09417950361967087, -0.7362082004547119, 0.32227906584739685, 1.4861445426940918, 1.7611088752746582, 0.06810376793146133, -1.5855683088302612, 1.411249041557312, -3.309617042541504, -0.19801875948905945, 1.5260179042816162, 0.6464988589286804, 2.820493459701538, -1.0315427780151367, 0.8284544348716736, -1.643714189529419, -0.7396258115768433, 0.7422579526901245, 0.313621461391449, 0.6270695328712463, 2.4098567962646484, 1.5205682516098022, -0.8854211568832397, -2.4966444969177246, 0.8305988311767578, 2.0034801959991455, 1.6557496786117554, 1.734865665435791, -1.4812586307525635, -0.20026539266109467, 0.3675759732723236, 2.1338202953338623, 0.29876577854156494, 1.1227707862854004, -1.45712411403656, -0.9263608455657959, -2.6163721084594727, 0.033515069633722305, -0.7430000901222229, -0.9416556358337402, -1.327649712562561, 22.351795196533203, 2.2162139415740967, 1.8860002756118774, -0.31567198038101196, 0.41157597303390503, 0.41350188851356506, 2.1562998294830322, 0.9810229539871216, -1.4618669748306274, 0.9164178371429443, 0.23495341837406158, 0.3263356387615204, -0.8862901926040649, 1.728047251701355, 2.186415910720825, -3.565371513366699, 1.3888542652130127, 0.994760274887085, -1.3664335012435913, -1.7464138269424438, 2.355619430541992, -0.4094844460487366, -1.8149701356887817, 1.5534652471542358, 0.6694555878639221, -3.0353236198425293, -0.12853975594043732, -1.3236558437347412, 1.3926055431365967, 1.4038865566253662, 0.44682276248931885, -0.28243571519851685, -1.1716045141220093, 1.9702776670455933, -1.6589183807373047, 1.4829425811767578, -0.9313967227935791, -2.230280637741089, 0.7296167016029358, -1.970436453819275, -2.2148280143737793, -0.6916073560714722, -2.3368287086486816, 0.2796325385570526, -0.442554771900177, -3.0628538131713867, -0.33133193850517273, -0.8669627904891968, -2.1127870082855225, -0.6460781097412109, -2.2722203731536865, 1.4664084911346436, 1.4489755630493164, 1.3492767810821533, -2.3569881916046143, -0.08992048352956772, -0.3775635063648224, -1.155730128288269, -1.1565061807632446, 2.085526704788208, -0.1975279599428177, -0.7470305562019348, 1.136731505393982, -2.289342164993286, -1.3205535411834717, 0.6236477494239807, 1.8431426286697388, -0.9011951684951782, 1.9367549419403076, -2.5573692321777344, 1.0121203660964966, 1.2276582717895508, 2.5483620166778564, 1.4324291944503784, 1.9219690561294556, 2.162973642349243, -0.13037186861038208, -0.4108283817768097, -1.158627986907959, 0.9314587116241455, 0.022087272256612778, -0.2258935123682022, -0.03206496685743332, 0.38184207677841187, -1.1416481733322144, 0.7356974482536316, 0.06039080768823624, -1.367060661315918, -0.6810886859893799, 3.3305118083953857, -2.104783058166504, -0.5445897579193115, -2.0366241931915283, 3.3101844787597656, -1.6082165241241455, -1.5139074325561523, 1.1306753158569336, -0.9271581768989563, -0.556171178817749, -0.15931738913059235, 2.1673221588134766, 0.2616686522960663, 0.9342774748802185, 0.9488689303398132, 0.2854823172092438, 2.794574022293091, 1.2471915483474731, -0.8237094879150391, 0.9129087328910828, 1.9269739389419556, 2.1318042278289795, -0.1278429627418518, -0.3332917094230652, -0.8604189157485962, -1.8599120378494263, -0.3174673914909363, 1.2273834943771362, 3.1576831340789795, 1.3233734369277954, 0.24131129682064056, 2.3642780780792236, -1.9695074558258057, -0.6005153656005859, 0.13187851011753082, -0.107331782579422, -1.271246075630188, 0.12606358528137207, 2.664933919906616, 1.104029893875122, -0.9802511930465698, -0.6231226921081543, 2.344454526901245, 0.29731327295303345, -0.24345053732395172, -1.7100263833999634, 0.6683851480484009, -0.030073538422584534, -0.08663389086723328, -1.0463913679122925, 0.37684640288352966, 0.2179735153913498, -2.604452610015869, 5.73710823059082, -3.1577069759368896, -2.3201591968536377, -1.4031455516815186, 0.5991779565811157, 1.2886371612548828, 0.16132506728172302, -1.6719927787780762, 6.294325828552246, -0.5656899809837341, 1.2018628120422363, -1.5706510543823242, 0.05329345166683197, -4.4174699783325195, -0.6509919762611389, 0.9431535005569458, 1.052964210510254, -1.9786689281463623, -1.0056875944137573, 7.370117664337158, -2.856679677963257, 2.899608850479126, -1.6668466329574585, 1.6359320878982544, 0.4814312160015106, -0.07905782759189606, -0.0014880102826282382, -3.871985912322998, -0.02092033438384533, 0.7591694593429565, 1.476837158203125, 1.5402185916900635, -1.0714640617370605, 0.2739114761352539, -1.5210024118423462, -0.8958848714828491, -1.1140875816345215, 0.5356087684631348, -0.29780152440071106, 1.2331935167312622, -0.4162542223930359, 2.2147045135498047, -1.3930078744888306, -0.6924197673797607, 0.7219045758247375, -1.8442497253417969, -0.7751844525337219, 0.7081555128097534, -0.4522971212863922, -2.5620288848876953, 3.5797858238220215, -1.6611593961715698, -1.7291874885559082, 1.8710733652114868, -1.1740798950195312, 1.490323543548584, 2.3765642642974854, 0.1783609837293625, 0.1222662404179573, 2.727318048477173, -0.19978375732898712, 3.201580047607422, 2.0717849731445312, -1.0530155897140503, -1.3238402605056763, -1.2825311422348022, 0.049854081124067307, 1.0917952060699463, 0.18736833333969116, 1.382059931755066, 0.7977815270423889, 3.4729204177856445, 0.32449695467948914, -2.929757595062256, 0.30528172850608826, 2.0983662605285645, -0.08381351083517075, 1.489567518234253, -1.4801721572875977, -4.513954162597656, 0.5175659656524658, -0.729248046875, 0.070643849670887, 2.286257028579712, 1.3631154298782349, -0.1329384297132492, 2.84873628616333, 1.0688217878341675, -0.45177558064460754, 2.237065076828003, 1.6550348997116089, 2.862889528274536, -1.0965489149093628, -4.438079357147217, -0.9815982580184937, -0.20258916914463043, -0.012109415605664253, -0.38907885551452637, 3.817408323287964, 2.503444194793701, -0.2505072057247162, 0.16747425496578217, -0.04353216290473938, 0.7175212502479553, -1.294250726699829, -1.3463374376296997, 0.752627432346344, 0.31305134296417236, -1.394910454750061, -1.654744029045105, -1.4145399332046509, -5.502553939819336, -0.08646469563245773, 0.16640695929527283, 2.601222515106201, -2.177044630050659, -1.4425866603851318, -1.7632946968078613, 2.061495780944824, 0.33145996928215027, 0.29803264141082764, 0.26527708768844604, -2.040065288543701, 0.745956540107727, 0.44793543219566345, 1.0647237300872803, 0.7348086833953857, -1.0302492380142212, -1.509528398513794, 2.1624374389648438, 1.3880950212478638, 1.0475400686264038, 1.9430394172668457, 0.6048816442489624, -0.6468086838722229, 1.1546708345413208, 0.5123040080070496, 0.2780100107192993, 0.39014971256256104, 0.7453547120094299, -2.094447612762451, 1.7974181175231934, 2.2373569011688232, -2.240760564804077, -0.755596399307251, -3.1288747787475586, 0.9054187536239624, 0.8291186094284058, -1.2551826238632202, -0.7559715509414673, -2.5973148345947266, -0.5114628672599792, -1.9965523481369019, -0.13933497667312622, 2.051115036010742, 0.024668538942933083, -0.6636998057365417, -0.7083673477172852, 0.4279605746269226, 1.5505712032318115, -0.5467987060546875, -1.9488886594772339, 0.9406977295875549, -1.8285250663757324, 2.2154738903045654, -1.0089713335037231, 2.0344018936157227, 0.38257408142089844, 1.4727052450180054, 1.688067078590393, 0.21315506100654602, 1.6218570470809937, 0.971635103225708, -17.2705020904541, 1.4091496467590332, 2.364426612854004, 1.2294979095458984, -1.76311457157135, 0.5103666186332703, 3.488497734069824, 1.3700166940689087, -1.1959364414215088, 0.9365859031677246, 1.2871383428573608, 0.37822815775871277, 2.5844814777374268, -1.5835953950881958, -1.6747851371765137, 1.1067625284194946, -0.773592472076416, -0.721174418926239, 1.1316107511520386, -0.7060160636901855, -0.7813263535499573, -2.174237012863159, 0.5424885153770447, -0.9098380208015442, -2.6305997371673584, -3.083148956298828, -0.11568813025951385, 0.2996847331523895, -0.9114266633987427, -0.10148356109857559, 2.4782397747039795, -0.7228710055351257, -1.2386151552200317, 0.3043202757835388, 0.23570220172405243, -1.6600366830825806, -1.0552715063095093, -1.9544737339019775, 1.23750638961792, -1.4341723918914795, -1.6589038372039795, -1.9567402601242065, 1.1876260042190552, -1.7152647972106934, 0.8503970503807068, 0.4163911044597626, -1.2929496765136719, -2.0464484691619873, -0.22800084948539734, 0.28050294518470764, 0.9367269277572632, -0.07398157566785812, 0.8881431221961975, 1.868578553199768, -2.2612411975860596, 2.174992084503174, 0.05784757435321808, 1.8338803052902222, 1.70822274684906, 2.9832940101623535, -0.15346524119377136, -0.897799015045166, -1.3153339624404907, 0.44243913888931274, 2.6169490814208984, -0.8499869704246521, -1.0614523887634277, -3.3478729724884033, 1.1571574211120605, 1.524728536605835, 2.175199270248413, -1.0722538232803345, 1.0352662801742554, -0.9333807229995728, 0.09360861033201218, -0.15677127242088318, 0.5480597019195557, -1.0114188194274902, 0.8680554628372192, 0.4806361794471741, 0.33639824390411377, -1.4984796047210693, 2.516514301300049, -1.4609034061431885, 1.279022455215454, -1.2902014255523682, -2.4821321964263916, 0.4512989819049835, 1.576982855796814, 1.3819280862808228, 0.6514641642570496, 1.9841614961624146, 1.431164026260376, 0.20954139530658722, -2.163788318634033, 1.0068163871765137, 1.4243371486663818, 1.0935693979263306, 0.6114811301231384, -0.09307169169187546, -3.478201389312744, -2.544565439224243, -3.0354464054107666, 0.32553064823150635, -1.2535088062286377, 0.8599048852920532, 1.33699631690979, 1.7655071020126343, -0.1454342156648636, 0.26660776138305664, 0.9599488377571106, 0.5257472395896912, -3.1795146465301514, -1.0778635740280151, 0.8318983316421509, -2.0972959995269775, 0.21332554519176483, -0.2264138162136078, -0.8041952252388, -1.52201509475708, 2.313316822052002, 2.029932975769043, -2.4793450832366943, 1.676969289779663, -2.040886640548706, -2.374945640563965, 1.4788898229599, 1.752054214477539, -7.496245384216309, 1.1781885623931885, 0.321981281042099, -1.5053822994232178, 1.5491139888763428, 2.5382461547851562, -1.0323539972305298, -3.0858733654022217, -1.1229491233825684, -0.3837498724460602, -0.9519292116165161, -1.0498963594436646, -2.8698318004608154, 1.8492659330368042, -2.601235866546631, 2.3047525882720947, -0.4843585789203644, 1.3986760377883911, 1.110711932182312, -0.22268013656139374, 1.8469260931015015, -0.014393612742424011, -1.5443729162216187, 1.3506052494049072, 0.3950430154800415, -0.5136393308639526, -1.152154803276062, -2.9297080039978027, 3.016359567642212, -0.0180624071508646, -0.31730756163597107, -1.8315562009811401, 1.139782428741455, 0.5715368986129761, -1.2366156578063965, 0.1557784080505371, 0.3745172917842865, 0.5834122896194458, -0.5863619446754456, -1.810221552848816, -1.928080439567566, 1.8518234491348267, 0.395804762840271, 2.0406458377838135, -1.661287784576416, -1.9958291053771973, 1.1048189401626587, -1.689194917678833, 0.853681206703186, -1.793395757675171, -0.20056027173995972, 0.09137099236249924, 1.941044569015503, 1.2859318256378174, 0.7478716969490051, 1.401105523109436, -0.25441601872444153, 2.149260997772217, -1.3566136360168457, 3.821934700012207, -2.240278720855713, 1.0188770294189453, 1.3552685976028442, -0.4295727610588074, 0.136664018034935, 0.8051626682281494, -3.036803722381592, -0.010265044867992401, 2.497624635696411, -1.2679270505905151, 0.34858277440071106, 1.7742741107940674, -1.3774505853652954, 1.0685850381851196, -0.012771065346896648, 0.9293734431266785, -0.9585280418395996, -0.1363360732793808, -1.165871024131775, -1.0086613893508911, 0.9215105772018433, -0.4248165190219879, -3.019850254058838, -1.234863042831421, -0.7632054090499878, 1.0071467161178589, 0.05438346043229103, 0.6116650104522705, 0.9310156106948853, 2.524658679962158, 1.622562050819397, 3.5730268955230713, -1.2445380687713623, -2.68721604347229, 0.7988719344139099, -0.36166515946388245, -2.446469306945801, -0.907312273979187, 1.0202760696411133, 1.1202144622802734, -0.5490670204162598, -0.3935467004776001, 0.5874626636505127, -1.892060399055481, -0.6445958614349365, -1.1143301725387573, 1.8204903602600098, 0.4000619351863861, -1.637826919555664, -0.1820591688156128, 2.277376651763916, 3.302725315093994, 0.43958619236946106, -1.588397741317749, 1.0332673788070679, -1.2146209478378296, -2.6132302284240723, 0.010274000465869904, 0.8732032179832458, 2.92620849609375, -0.6352483630180359, -0.29205626249313354, -1.0122069120407104, -1.445448875427246, 1.9610873460769653, 0.46606484055519104, 1.1647623777389526, 1.5439916849136353, -0.21698708832263947, -1.4126495122909546, 0.8987644910812378, -0.5945287346839905, -0.8008754253387451, 0.5846293568611145, 1.7321609258651733, 1.789260983467102, -1.5360933542251587, -2.5343050956726074, -2.1051933765411377, -0.09483456611633301, 0.7384520173072815, 0.5850244164466858, 13.152055740356445, 1.9199302196502686, 1.677181363105774, 2.1581814289093018, -5.3577094078063965, -0.42209020256996155, 0.39736050367355347, 2.597442388534546, -0.816745936870575, 0.30463042855262756, 0.8121334314346313, 0.2650866210460663, 1.5910398960113525, 1.0571269989013672, 1.0932790040969849, 0.8742311000823975, -0.7496413588523865, -0.5708375573158264, -2.8633902072906494, 0.5235978364944458, 0.15768089890480042, 2.5053107738494873, -2.758697032928467, -0.14665335416793823, -1.3282108306884766, 2.057810068130493, -1.3478177785873413, -1.2437180280685425, 2.252429723739624, -1.4286741018295288, 2.6295347213745117, -0.8474267721176147, -1.0714713335037231, 1.0098114013671875, -1.933640718460083, -0.19411726295948029, -1.8871361017227173, -0.37319231033325195, -3.128255844116211, 2.7481508255004883, 0.9374714493751526, 0.44776222109794617, -0.64658522605896, -0.00010193906200584024, -0.242501363158226, -0.587531566619873, 0.32857751846313477, 0.8994613289833069, 0.2948753535747528, 0.012842693366110325, -1.0109144449234009, -2.6371896266937256, 1.0768179893493652, 0.4063340127468109, 0.41522452235221863, 2.2196595668792725, -2.18636417388916, -2.1293129920959473, 0.32253313064575195, 1.3145508766174316, 1.6327366828918457, 1.2396783828735352, 1.7301385402679443, -0.22622951865196228, 0.8811069130897522, -0.7854161262512207, -2.6814427375793457, -0.7366517782211304, 0.8712520003318787, -1.57650625705719, 3.283024549484253, -0.06896960735321045, -1.3610292673110962, -1.2564468383789062, -0.40910643339157104, 1.2582955360412598, -1.3299167156219482, -1.902799367904663, -0.9217955470085144, -0.5023828744888306, 2.0634171962738037, 0.7262334823608398, 0.6701808571815491, 2.025582790374756, 1.207014799118042, -2.20546293258667, 0.648769199848175, 0.21683499217033386, -0.3214864730834961, 0.8566884398460388, 2.29416823387146, 0.21194159984588623, -0.01720023900270462, 0.7159295082092285, 0.15150335431098938, -0.2804165482521057, -0.06118824705481529, 1.6795172691345215, -0.08639281988143921, -0.17946413159370422, 0.5102631449699402, -0.46741002798080444, -0.2719039022922516, -2.610285520553589, 2.9494283199310303, 0.20885850489139557, 2.520145893096924, 1.1925244331359863, -0.13933853805065155, -0.35405176877975464, 0.05342094972729683, -1.83905029296875, 3.9277336597442627, 0.7367759943008423, -0.8343350887298584, -0.34269261360168457, -1.7974648475646973, -0.6525102853775024, 1.712822437286377, 0.9058087468147278, 1.5308175086975098, 0.1881045401096344, -1.4388444423675537, -0.9937943816184998, -0.4870139956474304, 1.438133716583252, 0.4196062982082367, -0.8268904089927673, -0.16931691765785217, -3.011688709259033, 1.1482208967208862, -1.7825111150741577, -0.942351222038269, 0.7381849884986877, 2.0619454383850098, 0.04025629162788391, -0.06863782554864883, 0.29763492941856384, 1.6905454397201538, 0.06483107060194016, -1.9962443113327026, 0.3768268823623657, 0.863364577293396, 1.5663036108016968, -0.06604473292827606, -6.701839923858643, -3.359495162963867, 0.45794910192489624, 2.572061538696289, -0.18268261849880219, -1.054521918296814, 1.566935420036316, -1.3793139457702637, 1.0351219177246094, 1.8163597583770752, 2.6691133975982666, -1.0660686492919922, -1.4323124885559082, 1.6254233121871948, 2.7342517375946045, 0.2745019793510437, 2.63624906539917, 1.7618523836135864, 1.7762342691421509, -1.007025957107544, -2.112694501876831, -0.13424265384674072, -0.8204474449157715, 0.9727054238319397, -1.1058850288391113, -1.9075146913528442, -0.1766611486673355, 1.2726624011993408, -1.4023621082305908, -0.17552322149276733, -1.9688911437988281, 1.4891231060028076, -1.686384916305542, -1.1595250368118286, -0.508810818195343, -0.13831393420696259, -1.8446600437164307, -0.06719478964805603, 0.3795017600059509, 3.015247344970703, 2.3150596618652344, 1.4662938117980957, 0.5371636748313904, 4.039791584014893, 0.7079944610595703, 0.8588481545448303, -1.2551189661026, 0.5592775344848633, -0.6738833785057068, -1.1690037250518799, -0.6981871128082275, 3.9726274013519287, -1.267857551574707, -0.47984325885772705, -1.2196074724197388, -1.1030280590057373, 0.3480887711048126, -0.5812361836433411, 3.501966714859009, 2.553037405014038, -0.6758319139480591, 1.3313207626342773, -0.48771098256111145, 1.8428452014923096, 0.4655342698097229, 0.23436754941940308, 1.0566325187683105, -0.2724457383155823, 0.6692327260971069, 1.1325644254684448, -0.36766496300697327, 0.5004285573959351, -1.8835697174072266, -0.6879101395606995, -0.22164684534072876, 1.566440463066101, -1.699312686920166, -0.14563368260860443, -1.9937305450439453, 0.8155184388160706, -1.5994640588760376, 0.4134396016597748, -5.565988063812256, 1.8968195915222168, 0.2557814419269562, 1.459218144416809, -0.008319654501974583, -1.4991706609725952, 2.0962371826171875, -2.980391502380371, -0.5658161044120789, 3.404984951019287, 0.4810156226158142, 1.2202744483947754, 3.0601911544799805, 0.26304659247398376, 0.258582204580307, 0.0030045919120311737, -1.5256483554840088, 1.6090729236602783, 0.8487810492515564, 0.7012439966201782, 3.616060972213745, 2.5299015045166016, -0.2332291603088379, -12.030735969543457, -0.39748793840408325, -1.415929913520813, 0.18703359365463257, 1.3511823415756226, -2.7268896102905273, 2.3266332149505615, -0.09947234392166138, 2.2688310146331787, -1.7888376712799072, -2.028803586959839, 0.6427737474441528, -1.169643759727478, -0.4223412573337555, -0.22223059833049774, -1.7913942337036133, 0.1402154415845871, -0.7830678820610046, -0.3441128730773926, -0.7317324876785278]} 2 | -------------------------------------------------------------------------------- /tests/test_config.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | import toml 3 | 4 | from exact_rag.config import Embeddings, Databases 5 | from tests.conftest import embedding_tomls, database_tomls 6 | 7 | 8 | def test_Embeddings(get_embedding_toml): 9 | Embeddings(**get_embedding_toml) 10 | 11 | 12 | def test_Databases(get_database_toml): 13 | Databases(**get_database_toml) 14 | 15 | 16 | omit_fields = [ 17 | (Embeddings, embedding_tomls["ollama"], "embedding", "model"), 18 | (Databases, database_tomls["chroma"], "database", "persist_directory"), 19 | (Databases, database_tomls["elastic"], "database", "url"), 20 | (Databases, database_tomls["elastic"], "database", "distance_strategy"), 21 | ] 22 | 23 | 24 | @pytest.mark.parametrize("dataclass, file, sub, omit", omit_fields) 25 | def test_omit_fields(dataclass, file, sub, omit): 26 | settings = toml.load(f"{file}") 27 | settings_s = settings[sub] 28 | settings_s[omit] = None 29 | with pytest.raises(ValueError): 30 | dataclass(**settings_s) 31 | -------------------------------------------------------------------------------- /tests/test_database_chroma.toml: -------------------------------------------------------------------------------- 1 | [database] 2 | type = "chroma" 3 | persist_directory = "persist" 4 | collection_name = "reviews" 5 | sql.namespace = "db/text" 6 | sql.url = "duplicates.sql" 7 | splitter.chunk_size = 1000 8 | splitter.chunk_overlap = 0 9 | -------------------------------------------------------------------------------- /tests/test_database_elastic.toml: -------------------------------------------------------------------------------- 1 | [database] 2 | type = "elastic" 3 | url = "http://localhost:9200" 4 | distance_strategy = "COSINE" 5 | collection_name = "reviews" 6 | sql.namespace = "db/text" 7 | sql.url = "duplicates.sql" 8 | splitter.chunk_size = 1000 9 | splitter.chunk_overlap = 0 10 | -------------------------------------------------------------------------------- /tests/test_dataemb.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | import requests 3 | import toml 4 | import os 5 | import json 6 | from contextlib import contextmanager 7 | 8 | from typing import Any, Callable 9 | from exact_rag.dataemb import Caller, DataEmbedding 10 | from exact_rag.config import Embeddings, Databases, EmbeddingType, DatabaseType 11 | from tests.conftest import embedding_tomls, database_tomls 12 | 13 | 14 | @pytest.fixture 15 | def get_id_callable() -> Callable[..., Any]: 16 | def id(**kwargs: Any) -> dict[str, Any]: 17 | return kwargs 18 | 19 | return id 20 | 21 | 22 | @pytest.fixture 23 | def get_input_args() -> dict[str, Any]: 24 | d = {"a": 1, "b": 2, "c": 3, "d": 4, "e": 5} 25 | return d 26 | 27 | 28 | def test_Caller_no_args(get_id_callable, get_input_args): 29 | caller = Caller(get_id_callable) 30 | assert caller(**get_input_args) == get_input_args 31 | 32 | 33 | def test_Caller_swap(get_id_callable, get_input_args): 34 | swap = {"A": "a", "D": "d", "EE": "e"} 35 | input = {"A": 1, "b": 2, "c": 3, "D": 4, "EE": 5} 36 | caller = Caller(get_id_callable, swap) 37 | assert caller(**input) == get_input_args 38 | 39 | 40 | def test_Caller_accept_only(get_id_callable, get_input_args): 41 | accept_only = ["b", "c", "e"] 42 | expected = {"b": 2, "c": 3, "e": 5} 43 | caller = Caller(get_id_callable, None, accept_only) 44 | assert caller(**get_input_args) == expected 45 | 46 | 47 | def test_Caller_full(get_id_callable): 48 | accept_only = ["B", "C", "EE"] 49 | swap = {"B": "b", "C": "c", "EE": "e"} 50 | input = {"a": 1, "B": 2, "C": 3, "d": 4, "EE": 5} 51 | expected = {"b": 2, "c": 3, "e": 5} 52 | caller = Caller(get_id_callable, swap, accept_only) 53 | assert caller(**input) == expected 54 | 55 | 56 | def is_elastic_available(db_url: str) -> bool: 57 | try: 58 | ans = requests.get(f"{db_url}/_cluster/health?pretty", timeout=1.0) 59 | if not ans.ok: 60 | return False 61 | if ans.json().get("status") not in ["green", "yellow"]: 62 | return False 63 | except Exception: 64 | return False 65 | 66 | return True 67 | 68 | 69 | @pytest.fixture 70 | def mock_OllamaEmbeddings(monkeypatch): 71 | def generate_mock_process_emb_response( 72 | path: str, 73 | ) -> Callable[[Any, str], list[float]]: 74 | def mock_process_emb_response(_self, input: str) -> list[float]: 75 | with open(path, "r") as file: 76 | return json.load(file).get("embedding") 77 | 78 | return mock_process_emb_response 79 | 80 | monkeypatch.setattr( 81 | "langchain_community.embeddings.ollama.OllamaEmbeddings._process_emb_response", 82 | generate_mock_process_emb_response("tests/ollama_embedding.json"), 83 | ) 84 | 85 | 86 | @pytest.fixture 87 | def mock_ChatOllama(monkeypatch): 88 | texts = [ 89 | f'{{"model":"llama2","created_at":"2024-02-01T09:52:07.775846182Z","message":{{"role":"assistant","content":"{x}"}},"done":false}}' 90 | for x in "This is a test".split(" ") 91 | ] 92 | 93 | monkeypatch.setattr( 94 | "langchain_community.chat_models.ollama.ChatOllama._create_chat_stream", 95 | lambda self, messages, stop: (x for x in texts), 96 | ) 97 | 98 | 99 | is_elasticserver_up = True 100 | 101 | 102 | @pytest.mark.skipif( 103 | not is_elasticserver_up, 104 | reason="This test is active only if one has certancy that elasticserach server is up", 105 | ) 106 | def test_is_elastic_available(): 107 | settings = toml.load(database_tomls["elastic"]) 108 | database = Databases(**settings["database"]) 109 | is_elastic_available(database.url) is True 110 | is_elastic_available("http://localhost:22") is False 111 | 112 | 113 | def get_elastic_indices(db_url: str) -> int: 114 | try: 115 | d = requests.get(f"{db_url}/_all").json() 116 | return len(d) 117 | except Exception: 118 | return 0 119 | 120 | 121 | def is_persistent_dir_not_empty(path: str) -> bool: 122 | return len(os.listdir(path)) != 0 123 | 124 | 125 | def activate_elastic_destructive_wildcard(db_url: str) -> bool: 126 | header = {"Content-Type": "application/json"} 127 | request = {"transient": {"action.destructive_requires_name": False}} 128 | try: 129 | ans = requests.put(f"{db_url}/_cluster/settings", headers=header, json=request) 130 | return ans.ok 131 | except Exception: 132 | return False 133 | 134 | 135 | def delete_elastic_indices(db_url: str) -> bool: 136 | try: 137 | ans = requests.delete(f"{db_url}/_all") 138 | return ans.ok 139 | except Exception: 140 | return False 141 | 142 | 143 | def delete_persistent_dir_content(path: str) -> bool: 144 | try: 145 | os.system(f"rm -rf {path}/*") 146 | except Exception: 147 | return False 148 | return True 149 | 150 | 151 | def delete_duplicates_file(path: str) -> bool: 152 | try: 153 | # if os.path.exists(path): 154 | # os.remove(path) 155 | os.system(f"rm -f {path}") 156 | except Exception: 157 | return False 158 | return True 159 | 160 | 161 | @contextmanager 162 | def handle_elastic_resource(database: Databases) -> None: 163 | assert delete_duplicates_file(database.sql_url) is True 164 | yield 165 | assert activate_elastic_destructive_wildcard(database.url) is True 166 | assert delete_elastic_indices(database.url) is True 167 | assert delete_duplicates_file(database.sql_url) is True 168 | 169 | 170 | @contextmanager 171 | def handle_chroma_resource(database: Databases) -> None: 172 | assert delete_duplicates_file(database.sql_url) is True 173 | yield 174 | assert delete_persistent_dir_content(database.persist_directory) is True 175 | assert delete_duplicates_file(database.sql_url) is True 176 | 177 | 178 | contextmanagers = { 179 | DatabaseType.elastic: handle_elastic_resource, 180 | DatabaseType.chroma: handle_chroma_resource, 181 | } 182 | 183 | 184 | @pytest.mark.skipif( 185 | not is_elasticserver_up, 186 | reason="This test is active only if one has certancy that elasticserach server is up", 187 | ) 188 | def test_elastic_indices(mock_ChatOllama, mock_OllamaEmbeddings): 189 | e_settings = toml.load(embedding_tomls["ollama"]) 190 | d_settings = toml.load(database_tomls["elastic"]) 191 | embedding = Embeddings(**e_settings["embedding"]) 192 | database = Databases(**d_settings["database"]) 193 | 194 | if not is_elastic_available(database.url): 195 | pytest.skip(reason="Elastisearch database not available.") 196 | 197 | if get_elastic_indices(database.url) != 0: 198 | pytest.skip(reason="Elasticsearch database is not empty.") 199 | 200 | assert delete_duplicates_file(database.sql_url) is True 201 | de = DataEmbedding(embedding, database) 202 | de.load("first") 203 | de.load("second") 204 | de.load("third") 205 | de.chat("Is there 'first' in my text collection?") 206 | assert activate_elastic_destructive_wildcard(database.url) is True 207 | assert get_elastic_indices(database.url) != 0 208 | assert delete_elastic_indices(database.url) is True 209 | assert get_elastic_indices(database.url) == 0 210 | assert delete_duplicates_file(database.sql_url) is True 211 | 212 | 213 | def test_DataEmbedding( 214 | mock_OllamaEmbeddings, mock_ChatOllama, get_embedding_toml, get_database_toml 215 | ): 216 | embedding = Embeddings(**get_embedding_toml) 217 | if embedding.type == EmbeddingType.openai: 218 | if not embedding.api_key or embedding.api_key == "": 219 | pytest.skip(reason="OpenAI token not found.") 220 | 221 | database = Databases(**get_database_toml) 222 | if database.type == DatabaseType.elastic: 223 | if not is_elastic_available(database.url): 224 | pytest.skip(reason="Elasticsearch database not available.") 225 | if get_elastic_indices(database.url) != 0: 226 | pytest.skip(reason="Elasticsearch database not empty.") 227 | elif database.type == DatabaseType.chroma: 228 | if is_persistent_dir_not_empty(database.persist_directory): 229 | pytest.skip(reason="Chroma persistent directory not empty.") 230 | 231 | with contextmanagers[database.type](database): 232 | de = DataEmbedding(embedding, database) 233 | de.load("Ping") 234 | de.load("Pong") 235 | de.load("Pang") 236 | de.chat("Is Ping present in my text collection?") 237 | de.chat("Is Pung present in my text collection?") 238 | -------------------------------------------------------------------------------- /tests/test_embedding_ollama.toml: -------------------------------------------------------------------------------- 1 | [embedding] 2 | type = "ollama" 3 | model = "orca2" 4 | chat.model_name = "orca2" 5 | chat.temperature = 0.5 6 | chain_type = "stuff" 7 | search.type = "mmr" 8 | search.k = 1 9 | search.fetch_k = 1 10 | -------------------------------------------------------------------------------- /tests/test_embedding_openai.toml: -------------------------------------------------------------------------------- 1 | [embedding] 2 | type = "openai" 3 | chat.model_name = "gpt-3.5-turbo" 4 | chat.temperature = 0.5 5 | chain_type = "stuff" 6 | search.type = "mmr" 7 | search.k = 1 8 | search.fetch_k = 1 9 | -------------------------------------------------------------------------------- /tests/test_settings.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from typing import Any, Annotated 4 | from exact_rag.settings import FromDict, Settings 5 | from pydantic import ValidationError 6 | 7 | 8 | @pytest.fixture 9 | def get_dict() -> dict[str, Any]: 10 | d = { 11 | "first": 1, 12 | "second": 2, 13 | "third": {"alpha": 3, "beta": {"a": 4, "b": 5, "c": 6, "d": 7}, "gamma": 8}, 14 | "fourth": 9, 15 | "fifth": {"alpha": 10, "beta": 11}, 16 | } 17 | return d 18 | 19 | 20 | test_FromDict_params = [ 21 | (("first",), 1), 22 | (("third", "alpha"), 3), 23 | (("third", "beta", "c"), 6), 24 | (("fourth", "beta"), None), 25 | (("fifth", "alpha"), 10), 26 | (("sixth", "alpha", "a"), None), 27 | ] 28 | 29 | 30 | @pytest.mark.parametrize("input, expected", test_FromDict_params) 31 | def test_FromDict(get_dict, input, expected): 32 | fd = FromDict(*input) 33 | assert fd(get_dict) == expected 34 | 35 | 36 | class Configs(Settings): 37 | integer: int 38 | floating: Annotated[float, FromDict("double")] 39 | string: Annotated[str, FromDict("strings", "my_string")] 40 | 41 | 42 | def test_config_ok(): 43 | d_in = { 44 | "integer": 5, 45 | "double": -45.673, 46 | "strings": {"my_string": "example", "your_string": "second_example"}, 47 | } 48 | configs = Configs(**d_in) 49 | d_out = configs.model_dump() 50 | 51 | exp_d_out = { 52 | "integer": d_in["integer"], 53 | "floating": d_in["double"], 54 | "string": d_in["strings"]["my_string"], 55 | } 56 | assert d_out == exp_d_out 57 | 58 | 59 | def test_config_fail_string(): 60 | d_in = {"integer": 5, "double": -45.673} 61 | with pytest.raises(ValidationError) as error: 62 | Configs(**d_in) 63 | assert error.value.error_count() == 1 64 | assert "string" in error.value.errors()[0]["msg"] 65 | 66 | 67 | def test_config_fail_double(): 68 | d_in = {"integer": 5, "strings": {"my_string": "example"}} 69 | with pytest.raises(ValidationError) as error: 70 | Configs(**d_in) 71 | assert error.value.error_count() == 1 72 | assert "number" in error.value.errors()[0]["msg"] 73 | 74 | 75 | class WrongConfig(Settings): 76 | integer: Annotated[int, FromDict("first"), FromDict("second")] 77 | 78 | 79 | def test_wrong_config(): 80 | d_in = {"first": 1, "second": 2} 81 | with pytest.raises(TypeError) as error: 82 | WrongConfig(**d_in) 83 | assert "Multiple" in error.value.args[0] 84 | --------------------------------------------------------------------------------