├── requirements.txt ├── img └── q-logo.png ├── data ├── prompt_list.txt ├── the-tyger-poem-by-william-blake.txt └── if-poem-by-rudyard-kipling.txt ├── src ├── requirements.txt ├── q_list_data_source_sync_jobs.py ├── q_list_documents.py ├── q_list_conversations.py ├── q_chat.py └── q_list_applications.py ├── pylint.config ├── .github └── workflows │ └── build.yml ├── LICENSE ├── .gitignore ├── doc ├── q_list_data_source_sync_jobs.md ├── q_list_documents.md ├── q_list_conversations.md ├── q_list_applications.md └── q_chat.md └── README.md /requirements.txt: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /img/q-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/didier-durand/qstensils/HEAD/img/q-logo.png -------------------------------------------------------------------------------- /data/prompt_list.txt: -------------------------------------------------------------------------------- 1 | what is a cat? 2 | c:what is a dog? 3 | c:should I prefer a cat or dog? 4 | what is Amazon? -------------------------------------------------------------------------------- /src/requirements.txt: -------------------------------------------------------------------------------- 1 | boto3==1.34.50 2 | botocore==1.34.50 3 | jmespath==1.0.1 4 | python-dateutil==2.8.2 5 | s3transfer==0.10.0 6 | six==1.16.0 7 | urllib3==2.2.1 8 | -------------------------------------------------------------------------------- /pylint.config: -------------------------------------------------------------------------------- 1 | [MASTER] 2 | init-hook='import sys; sys.path.append(".")' 3 | [MESSAGES CONTROL] 4 | # C0114: Missing module docstring (missing-module-docstring) 5 | # C0116: Missing function or method docstring (missing-function-docstring) 6 | # C0301: Line too long (line-too-long) 7 | # E0401: Unable to import 'module' (import-error) 8 | # R0801: Similar lines in 2 files 9 | disable=C0114,C0116,C0301,E0401,R0801 -------------------------------------------------------------------------------- /.github/workflows/build.yml: -------------------------------------------------------------------------------- 1 | name: Code Build 2 | 3 | on: [push] 4 | 5 | jobs: 6 | build: 7 | runs-on: ubuntu-latest 8 | strategy: 9 | matrix: 10 | python-version: ["3.11","3.12"] 11 | steps: 12 | - uses: actions/checkout@v4.1.1 13 | - name: Set up Python ${{ matrix.python-version }} 14 | uses: actions/setup-python@v5 15 | with: 16 | python-version: ${{ matrix.python-version }} 17 | - name: Install dependencies 18 | run: | 19 | python -m pip install --upgrade pip 20 | pip install -r requirements.txt 21 | pip install pylint 22 | - name: checking syntax of scripts 23 | run: | 24 | python -m py_compile $(git ls-files '*.py') 25 | - name: Analysing the code with pylint 26 | run: | 27 | pylint $(git ls-files '*.py') --rcfile=./pylint.config 28 | -------------------------------------------------------------------------------- /data/the-tyger-poem-by-william-blake.txt: -------------------------------------------------------------------------------- 1 | Title of poem: The Tyger 2 | By: William Blake 3 | 4 | Tyger Tyger, burning bright, 5 | In the forests of the night; 6 | What immortal hand or eye, 7 | Could frame thy fearful symmetry? 8 | 9 | In what distant deeps or skies. 10 | Burnt the fire of thine eyes? 11 | On what wings dare he aspire? 12 | What the hand, dare seize the fire? 13 | 14 | And what shoulder, & what art, 15 | Could twist the sinews of thy heart? 16 | And when thy heart began to beat. 17 | What dread hand? & what dread feet? 18 | 19 | What the hammer? what the chain, 20 | In what furnace was thy brain? 21 | What the anvil? what dread grasp. 22 | Dare its deadly terrors clasp? 23 | 24 | When the stars threw down their spears 25 | And water'd heaven with their tears: 26 | Did he smile his work to see? 27 | Did he who made the Lamb make thee? 28 | 29 | Tyger Tyger burning bright, 30 | In the forests of the night: 31 | What immortal hand or eye, 32 | Dare frame thy fearful symmetry? -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Didier Durand 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /data/if-poem-by-rudyard-kipling.txt: -------------------------------------------------------------------------------- 1 | Title of poem: If 2 | By: Rudyard Kipling 3 | 4 | If you can keep your head when all about you 5 | Are losing theirs and blaming it on you; 6 | If you can trust yourself when all men doubt you, 7 | But make allowance for their doubting too; 8 | If you can wait and not be tired by waiting, 9 | Or, being lied about, don’t deal in lies, 10 | Or, being hated, don’t give way to hating, 11 | And yet don’t look too good, nor talk too wise; 12 | 13 | If you can dream—and not make dreams your master; 14 | If you can think—and not make thoughts your aim; 15 | If you can meet with triumph and disaster 16 | And treat those two impostors just the same; 17 | If you can bear to hear the truth you’ve spoken 18 | Twisted by knaves to make a trap for fools, 19 | Or watch the things you gave your life to broken, 20 | And stoop and build ’em up with wornout tools; 21 | 22 | If you can make one heap of all your winnings 23 | And risk it on one turn of pitch-and-toss, 24 | And lose, and start again at your beginnings 25 | And never breathe a word about your loss; 26 | If you can force your heart and nerve and sinew 27 | To serve your turn long after they are gone, 28 | And so hold on when there is nothing in you 29 | Except the Will which says to them: “Hold on”; 30 | 31 | If you can talk with crowds and keep your virtue, 32 | Or walk with kings—nor lose the common touch; 33 | If neither foes nor loving friends can hurt you; 34 | If all men count with you, but none too much; 35 | If you can fill the unforgiving minute 36 | With sixty seconds’ worth of distance run— 37 | Yours is the Earth and everything that’s in it, 38 | And—which is more—you’ll be a Man, my son! -------------------------------------------------------------------------------- /src/q_list_data_source_sync_jobs.py: -------------------------------------------------------------------------------- 1 | import json 2 | from pprint import pprint 3 | import argparse 4 | 5 | import boto3 6 | 7 | q_client = boto3.client("qbusiness") # noqa 8 | 9 | 10 | def check_response(resp: dict, http_status_code=200) -> bool: 11 | assert resp["ResponseMetadata"]["HTTPStatusCode"] == http_status_code 12 | return True 13 | 14 | 15 | def list_data_source_sync_jobs(app_id: str = "", idx_id: str = "", 16 | ds_id: str = "", verbose: bool = False) -> list[dict] | None: 17 | resp = q_client.list_data_source_sync_jobs(applicationId=app_id, indexId=idx_id, dataSourceId=ds_id, 18 | maxResults=10) 19 | if verbose: 20 | pprint(resp) 21 | check_response(resp) 22 | if "nextToken" in resp: 23 | print("ERROR: paginated response not yet implemented for sync jobs: please," 24 | "open a ticket at https://github.com/didier-durand/qstensils/issues") 25 | if "history" in resp: 26 | return resp["history"] 27 | return None 28 | 29 | 30 | def list_data_source_sync_jobs_with_details(app_id: str = "", idx_id: str = "", 31 | ds_id: str = "", verbose: bool = False) -> list[dict] | None: 32 | jobs = list_data_source_sync_jobs(app_id=app_id, idx_id=idx_id, ds_id=ds_id, verbose=verbose) 33 | if jobs is not None and len(jobs) > 0: 34 | for job in jobs: 35 | if "endTime" in job: 36 | duration = job["endTime"] - job["startTime"] 37 | if "duration_s" in job: 38 | raise ValueError("duration_s dict key already present") 39 | job["duration"] = duration 40 | job["duration_s"] = duration.seconds 41 | if "metrics" in job and "documentsScanned" in job["metrics"]: 42 | if duration.seconds > 0: 43 | scan_rate = int(job["metrics"]["documentsScanned"]) / int(duration.seconds) 44 | job["metrics"]["scanRate"] = str(scan_rate) 45 | if scan_rate > 0: 46 | job["metrics"]["averageDocumentScanDuration"] = str(1 / scan_rate) 47 | return jobs 48 | return None 49 | 50 | 51 | if __name__ == "__main__": 52 | parser = argparse.ArgumentParser(description="list synchronization jobs executed for a given data source " 53 | "of an Amazon Q application") 54 | parser.add_argument("-a", "--app_id", type=str, help="Q application id") 55 | parser.add_argument("-i", "--idx_id", type=str, help="Q data source id") 56 | parser.add_argument("-d", "--ds_id", type=str, help="Q data source id") 57 | parser.add_argument("-v", "--verbose", action="store_true", help="verbose mode") 58 | args = parser.parse_args() 59 | sync_jobs = list_data_source_sync_jobs_with_details(app_id=args.app_id, idx_id=args.idx_id, ds_id=args.ds_id, 60 | verbose=args.verbose) 61 | print(json.dumps(sync_jobs, indent=4, default=str)) 62 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | # Byte-compiled / optimized / DLL files 3 | __pycache__/ 4 | *.py[cod] 5 | *$py.class 6 | 7 | # C extensions 8 | *.so 9 | 10 | # Distribution / packaging 11 | .Python 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | cover/ 54 | 55 | # Translations 56 | *.mo 57 | *.pot 58 | 59 | # Django stuff: 60 | *.log 61 | local_settings.py 62 | db.sqlite3 63 | db.sqlite3-journal 64 | 65 | # Flask stuff: 66 | instance/ 67 | .webassets-cache 68 | 69 | # Scrapy stuff: 70 | .scrapy 71 | 72 | # Sphinx documentation 73 | docs/_build/ 74 | 75 | # PyBuilder 76 | .pybuilder/ 77 | target/ 78 | 79 | # Jupyter Notebook 80 | .ipynb_checkpoints 81 | 82 | # IPython 83 | profile_default/ 84 | ipython_config.py 85 | 86 | # pyenv 87 | # For a library or package, you might want to ignore these files since the code is 88 | # intended to run in multiple environments; otherwise, check them in: 89 | # .python-version 90 | 91 | # pipenv 92 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 93 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 94 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 95 | # install all needed dependencies. 96 | #Pipfile.lock 97 | 98 | # poetry 99 | # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. 100 | # This is especially recommended for binary packages to ensure reproducibility, and is more 101 | # commonly ignored for libraries. 102 | # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control 103 | #poetry.lock 104 | 105 | # pdm 106 | # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. 107 | #pdm.lock 108 | # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it 109 | # in version control. 110 | # https://pdm.fming.dev/#use-with-ide 111 | .pdm.toml 112 | 113 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm 114 | __pypackages__/ 115 | 116 | # Celery stuff 117 | celerybeat-schedule 118 | celerybeat.pid 119 | 120 | # SageMath parsed files 121 | *.sage.py 122 | 123 | # Environments 124 | .env 125 | .venv 126 | env/ 127 | venv/ 128 | ENV/ 129 | env.bak/ 130 | venv.bak/ 131 | 132 | # Spyder project settings 133 | .spyderproject 134 | .spyproject 135 | 136 | # Rope project settings 137 | .ropeproject 138 | 139 | # mkdocs documentation 140 | /site 141 | 142 | # mypy 143 | .mypy_cache/ 144 | .dmypy.json 145 | dmypy.json 146 | 147 | # Pyre type checker 148 | .pyre/ 149 | 150 | # pytype static type analyzer 151 | .pytype/ 152 | 153 | # Cython debug symbols 154 | cython_debug/ 155 | 156 | # PyCharm 157 | # JetBrains specific template is maintained in a separate JetBrains.gitignore that can 158 | # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore 159 | # and can be added to the global gitignore or merged into this file. For a more nuclear 160 | # option (not recommended) you can uncomment the following to ignore the entire idea folder. 161 | #.idea/ 162 | -------------------------------------------------------------------------------- /src/q_list_documents.py: -------------------------------------------------------------------------------- 1 | import json 2 | from pprint import pprint 3 | import argparse 4 | 5 | import boto3 6 | 7 | q_client = boto3.client("qbusiness") # noqa 8 | 9 | 10 | def check_response(response: dict, http_status_code=200) -> bool: 11 | assert response['ResponseMetadata']['HTTPStatusCode'] == http_status_code 12 | return True 13 | 14 | 15 | def list_documents(app_id="", idx_id="", verbose=False) -> list[dict] | None: 16 | r_docs: list[dict] = list[dict]() 17 | next_token: str = "" 18 | while True: 19 | if next_token == "": 20 | # maxResults cannot be bigger than 100 currently 21 | response = q_client.list_documents(applicationId=app_id, indexId=idx_id, 22 | maxResults=100) 23 | if verbose: 24 | pprint(response) 25 | check_response(response) 26 | else: 27 | response = q_client.list_documents(applicationId=app_id, indexId=idx_id, 28 | maxResults=100, nextToken=next_token) 29 | if verbose: 30 | pprint(response) 31 | check_response(response) 32 | if "documentDetailList" in response: 33 | docs: list = response["documentDetailList"] 34 | if len(docs) > 0: 35 | r_docs.extend(docs) 36 | if "nextToken" not in response or response["nextToken"] == "": # pylint: disable=no-else-break 37 | break 38 | else: 39 | next_token = response["nextToken"] 40 | if len(r_docs) > 0: 41 | return r_docs 42 | return None 43 | 44 | 45 | def pretty_print(docs: list[dict] = None): 46 | for i, doc in enumerate(docs): 47 | print("#" + str(i) + ":") 48 | for key in doc.keys(): 49 | print(" " + key + ":", str(doc[key])) 50 | 51 | 52 | def inventory(docs): 53 | states: dict[str, int] = dict[str, int]() 54 | for doc in docs: 55 | status = doc['status'] 56 | if status not in states.keys(): 57 | states[status] = 0 58 | states[status] += 1 59 | states["TOTAL"] = len(docs) 60 | print("--- document inventory") 61 | for status in states: 62 | print(status + ":", states[status]) 63 | 64 | 65 | if __name__ == "__main__": 66 | parser = argparse.ArgumentParser(description="list documents indexed by Amazon Q") 67 | parser.add_argument("-a", "--app_id", type=str, help="Q application id") 68 | parser.add_argument("-i", "--idx_id", type=str, help="Q index id") 69 | parser.add_argument("-incl", "--include", type=str, help="comma-separated list of status to include") 70 | parser.add_argument("-excl", "--exclude", type=str, help="comma-separated list of status to exclude") 71 | parser.add_argument("-inv", "--inventory", action="store_true", help="with document inventory") 72 | parser.add_argument("-v", "--verbose", action="store_true", help="verbose mode") 73 | args = parser.parse_args() 74 | 75 | if args.app_id is None or args.idx_id is None: 76 | raise ValueError("Q application id and index id must be specified - usage: python3 q_list_documents.py " 77 | "--app_id --idx_id ") 78 | if args.include is not None and args.exclude is not None: 79 | raise ValueError("either --include or --exclude list of status can be specified but not both simultaneously") 80 | if args.verbose: 81 | print("Q API list_docs() called with app_id:", args.app_id, " - idx-id:", args.idx_id) 82 | 83 | documents: list[dict] = list_documents(app_id=args.app_id, idx_id=args.idx_id, verbose=args.verbose) 84 | filtered: list[dict] = list[dict]() 85 | if args.include: 86 | status_list = args.include.split(",") 87 | for document in documents: 88 | if document["status"] in status_list: 89 | filtered.append(document) 90 | elif args.exclude: 91 | status_list = args.exclude.split(",") 92 | for document in documents: 93 | if document["status"] not in status_list: 94 | filtered.append(document) 95 | else: 96 | filtered = documents 97 | 98 | print(json.dumps(filtered, indent=4, default=str)) 99 | 100 | if args.inventory: 101 | inventory(documents) 102 | -------------------------------------------------------------------------------- /src/q_list_conversations.py: -------------------------------------------------------------------------------- 1 | import json 2 | from pprint import pprint 3 | import argparse 4 | 5 | import boto3 6 | 7 | q_client = boto3.client("qbusiness") # noqa 8 | 9 | 10 | def check_response(response: dict, http_status_code=200) -> bool: 11 | assert response['ResponseMetadata']['HTTPStatusCode'] == http_status_code 12 | return True 13 | 14 | 15 | def list_conversations(app_id: str = "", usr_id: str = "", verbose: bool = False) -> list[dict] | None: 16 | convs: list[dict] = list[dict]() 17 | next_token: str = "" 18 | while True: 19 | if next_token == "": 20 | # maxResults cannot be bigger than 100 currently 21 | response = q_client.list_conversations(applicationId=app_id, userId=usr_id, 22 | maxResults=100) 23 | if verbose: 24 | pprint(response) 25 | check_response(response) 26 | else: 27 | response = q_client.list_conversations(applicationId=app_id, userId=usr_id, 28 | maxResults=100, nextToken=next_token) 29 | if verbose: 30 | pprint(response) 31 | check_response(response) 32 | if "conversations" in response: 33 | convs: list = response["conversations"] 34 | if len(convs) > 0: 35 | convs.extend(convs) 36 | if "nextToken" not in response or response["nextToken"] == "": 37 | break 38 | next_token = response["nextToken"] 39 | if len(convs) > 0: 40 | return convs 41 | return None 42 | 43 | 44 | def list_messages(app_id: str = "", usr_id: str = "", conv_id: str = "", verbose: bool = False) -> list[dict] | None: 45 | msgs: list[dict] = list[dict]() 46 | next_token: str = "" 47 | while True: 48 | if next_token == "": 49 | # maxResults cannot be bigger than 100 currently 50 | response = q_client.list_messages(applicationId=app_id, userId=usr_id, conversationId=conv_id, 51 | maxResults=100) 52 | if verbose: 53 | pprint(response) 54 | check_response(response) 55 | else: 56 | response = q_client.list_conversations(applicationId=app_id, userId=usr_id, conversationId=conv_id, 57 | maxResults=100, nextToken=next_token) 58 | if verbose: 59 | pprint(response) 60 | check_response(response) 61 | if "messages" in response: 62 | r_msgs: list = response["messages"] 63 | if len(r_msgs) > 0: 64 | msgs.extend(r_msgs) 65 | if "nextToken" not in response or response["nextToken"] == "": 66 | break 67 | next_token = response["nextToken"] 68 | if len(msgs) > 0: 69 | return msgs 70 | return None 71 | 72 | 73 | def list_conversations_with_messages(app_id: str = "", usr_id: str = "", verbose=False) -> list[dict] | None: 74 | convs: list[dict] = list_conversations(app_id=app_id, usr_id=usr_id, verbose=verbose) 75 | if convs is not None and len(convs) > 0: 76 | for conv in convs: 77 | messages: list[dict] = list_messages(app_id=app_id, usr_id=usr_id, 78 | conv_id=conv["conversationId"], verbose=verbose) 79 | if messages is not None and len(messages) > 0: 80 | conv['messages'] = messages 81 | return convs 82 | return None 83 | 84 | 85 | def get_q_conversation(app_id: str = "", usr_id: str = "", cnv_id="", verbose=False) -> dict | None: 86 | convs = list_conversations_with_messages(app_id, usr_id, verbose) 87 | for conv in convs: 88 | if conv["conversationId"] == cnv_id: 89 | return conv 90 | return None 91 | 92 | 93 | if __name__ == "__main__": 94 | parser = argparse.ArgumentParser(description="list documents indexed by Amazon Q") 95 | parser.add_argument("-a", "--app_id", type=str, help="Q application id") 96 | parser.add_argument("-u", "--usr_id", type=str, help="Q user id") 97 | parser.add_argument("-v", "--verbose", action="store_true", help="verbose mode") 98 | args = parser.parse_args() 99 | 100 | if args.app_id is None or args.usr_id is None: 101 | raise ValueError("Q application id and user id must be specified - usage: python3 q_list_documents.py " 102 | "--app_id --usr_id ") 103 | if args.verbose: 104 | print("Q API list_conversations() called with app_id:", args.app_id, " - usr-id:", args.usr_id) 105 | 106 | conversations: list[dict] = list_conversations_with_messages(app_id=args.app_id, usr_id=args.usr_id, 107 | verbose=args.verbose) 108 | print(json.dumps(conversations, indent=4, default=str)) 109 | -------------------------------------------------------------------------------- /src/q_chat.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | from pprint import pprint 4 | import argparse 5 | 6 | import boto3 7 | 8 | from q_list_conversations import get_q_conversation 9 | 10 | q_client = boto3.client("qbusiness") # noqa 11 | 12 | 13 | def check_response(resp: dict, http_status_code=200) -> bool: 14 | assert resp['ResponseMetadata']['HTTPStatusCode'] == http_status_code 15 | return True 16 | 17 | 18 | def chat_sync(app_id: str = "", usr_id: str = "", # pylint: disable=too-many-arguments 19 | prompt="", file_path="", cnv_id: str = "", msg_id="", 20 | verbose=False) -> dict: 21 | if verbose: 22 | print("prompt:", "\"" + prompt + "\"") 23 | if cnv_id is None or cnv_id == "": 24 | if file_path is not None and file_path != "": 25 | with open(file_path) as f: # pylint: disable=unspecified-encoding 26 | data = f.read() 27 | response = q_client.chat_sync(applicationId=app_id, userId=usr_id, userMessage=prompt, 28 | attachments=[{"name": file_path, "data": data}]) 29 | else: 30 | response = q_client.chat_sync(applicationId=app_id, userId=usr_id, userMessage=prompt) 31 | else: 32 | response = q_client.chat_sync(applicationId=app_id, userId=usr_id, conversationId=cnv_id, 33 | parentMessageId=msg_id, userMessage=prompt) 34 | if verbose: 35 | pprint(response) 36 | check_response(response) 37 | del response["ResponseMetadata"] 38 | return response 39 | 40 | 41 | def chat_sync_with_multiple_prompts(app_id="", usr_id="", prompts: list[str] = None, verbose=False) \ 42 | -> list[dict] | None: 43 | if prompts is None or len(prompts) == 0: 44 | return None 45 | cnv_ids: list[str] = list[str]() 46 | cnv_id: str = "" 47 | msg_id: str = "" 48 | for prompt in prompts: 49 | if prompt.startswith("c:"): 50 | if cnv_id is not None and cnv_id != "" and msg_id is not None and msg_id != "": 51 | resp = chat_sync(app_id=app_id, usr_id=usr_id, cnv_id=cnv_id, msg_id=msg_id, 52 | prompt=prompt, verbose=verbose) 53 | else: 54 | raise ValueError("conversation id and parent message id must be available for chained prompts") 55 | else: 56 | resp = chat_sync(app_id=app_id, usr_id=usr_id, prompt=prompt, verbose=args.verbose) 57 | cnv_id = resp["conversationId"] 58 | msg_id = resp["systemMessageId"] 59 | if cnv_id not in cnv_ids: 60 | cnv_ids.append(cnv_id) 61 | if len(cnv_ids) > 0: 62 | conversations: list[dict] = list[dict]() 63 | for cnv_id in cnv_ids: 64 | conversations.append(get_q_conversation(app_id=app_id, usr_id=usr_id, cnv_id=cnv_id, verbose=verbose)) 65 | return conversations 66 | return None 67 | 68 | 69 | if __name__ == "__main__": 70 | parser = argparse.ArgumentParser(description="ask a question to a Q application and get answer") 71 | parser.add_argument("-a", "--app_id", type=str, help="Q application id") 72 | parser.add_argument("-u", "--usr_id", type=str, help="Q index id") 73 | parser.add_argument("-p", "--prompt", type=str, help="question prompt or path to file with " 74 | "list of prompts") 75 | parser.add_argument("-f", "--file", type=str, help="path to attachment file") 76 | parser.add_argument("-c", "--cnv_id", type=str, help="Q conversation id " 77 | "(only to continue an existing conversation)") 78 | parser.add_argument("-m", "--msg_id", type=str, help="Q parent message id " 79 | "(only to continue an existing ""conversation)") 80 | parser.add_argument("-d", "--details", action="store_true", help="full conversation details") 81 | parser.add_argument("-v", "--verbose", action="store_true", help="verbose mode") 82 | args = parser.parse_args() 83 | 84 | p_str = args.prompt 85 | if os.path.isfile(p_str): 86 | with open(p_str, encoding="us-ascii") as file: 87 | f_prompts = file.read() 88 | f_prompts: [str] = f_prompts.split("\n") 89 | answer = chat_sync_with_multiple_prompts(app_id=args.app_id, usr_id=args.usr_id, 90 | prompts=f_prompts, verbose=args.verbose) 91 | else: 92 | completion = chat_sync(app_id=args.app_id, usr_id=args.usr_id, cnv_id=args.cnv_id, msg_id=args.msg_id, 93 | prompt=args.prompt, file_path=args.file, verbose=args.verbose) 94 | if args.details: 95 | answer = get_q_conversation(app_id=args.app_id, usr_id=args.usr_id, cnv_id=completion['conversationId'], 96 | verbose=args.verbose) 97 | else: 98 | answer = completion 99 | print(json.dumps(answer, indent=4, default=str)) 100 | -------------------------------------------------------------------------------- /doc/q_list_data_source_sync_jobs.md: -------------------------------------------------------------------------------- 1 | ## q_list_data_source_sync_jobs 2 | 3 | * [Description](#description) 4 | * [Usage](#usage) 5 | * [Help and Security](#help-and-security) 6 | 7 | ## Description 8 | 9 | An Amazon Q application leverages [Retrieval-Augmented Generation](https://www.promptingguide.ai/techniques/rag) (aka RAG) to deliver more contextuality, 10 | better factual consistency, to improve reliability of the generated responses and to help to mitigate the 11 | problem of ["hallucinations"](https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)). 12 | 13 | The Q RAG index is built from the content of the Q data source(s) like S3, relational databases,etc. associated to the 14 | application. The documents contained in those data sources are analyzed by the indexer. Their [vector embeddings](https://en.wikipedia.org/wiki/Sentence_embedding) 15 | and other metadata are placed into the index to be able to select the most relevant documents improving the user prompt 16 | on each question. The Q retriever will leverage the data contained in the RAG index to fetch and add additional documents to 17 | question / prompt coming from the user. This enriched prompt will allow the Q LLM to provide a better answer. 18 | 19 | Script [q_list_data_source_sync_jobs ](../src/q_list_data_source_sync_jobs.py) will list the history of all index 20 | synchronization jobs that have executed on a given Q data source. It will add interesting metrics like job duration 21 | and document scan rate. 22 | 23 | ### Usage 24 | 25 | ``` 26 | % python3 q_list_data_source_sync_jobs.py -a -i -d 27 | 28 | [ 29 | { 30 | "endTime": "2024-02-25 07:48:40.853000+01:00", 31 | "error": {}, 32 | "executionId": "1414a953-214e-4ff5-abc8-556d4ce3a1b1", 33 | "metrics": { 34 | "documentsAdded": "0", 35 | "documentsDeleted": "0", 36 | "documentsFailed": "6", 37 | "documentsModified": "993", 38 | "documentsScanned": "999", 39 | "scanRate": "0.624765478424015", 40 | "averageDocumentScanDuration": "1.6006006006006006" 41 | }, 42 | "startTime": "2024-02-25 07:22:00.869000+01:00", 43 | "status": "INCOMPLETE", 44 | "duration": "0:26:39.984000", 45 | "duration_s": 1599 46 | }, 47 | { 48 | "endTime": "2024-02-24 11:40:38.389000+01:00", 49 | "error": {}, 50 | "executionId": "c0bda9b8-1b54-41b2-af26-ee559e6406da", 51 | "metrics": { 52 | "documentsAdded": "0", 53 | "documentsDeleted": "0", 54 | "documentsFailed": "8", 55 | "documentsModified": "991", 56 | "documentsScanned": "999", 57 | "scanRate": "0.43134715025906734", 58 | "averageDocumentScanDuration": "2.3183183183183185" 59 | }, 60 | "startTime": "2024-02-24 11:02:01.815000+01:00", 61 | "status": "INCOMPLETE", 62 | "duration": "0:38:36.574000", 63 | "duration_s": 2316 64 | }, 65 | { 66 | "endTime": "2024-02-24 10:02:37.580000+01:00", 67 | "error": {}, 68 | "executionId": "44e993e2-15a0-40cb-9f89-5dc14cdf1eab", 69 | "metrics": { 70 | "documentsAdded": "7", 71 | "documentsDeleted": "0", 72 | "documentsFailed": "13", 73 | "documentsModified": "979", 74 | "documentsScanned": "999", 75 | "scanRate": "0.4459821428571429", 76 | "averageDocumentScanDuration": "2.2422422422422423" 77 | }, 78 | "startTime": "2024-02-24 09:25:17.527000+01:00", 79 | "status": "INCOMPLETE", 80 | "duration": "0:37:20.053000", 81 | "duration_s": 2240 82 | }, 83 | { 84 | "endTime": "2024-02-21 12:00:24.356000+01:00", 85 | "error": {}, 86 | "executionId": "4ec5a054-c430-4fec-907e-333b20d9693f", 87 | "metrics": { 88 | "documentsAdded": "992", 89 | "documentsDeleted": "0", 90 | "documentsFailed": "7", 91 | "documentsModified": "0", 92 | "documentsScanned": "999", 93 | "scanRate": "0.4714487966021708", 94 | "averageDocumentScanDuration": "2.121121121121121" 95 | }, 96 | "startTime": "2024-02-21 11:25:04.942000+01:00", 97 | "status": "INCOMPLETE", 98 | "duration": "0:35:19.414000", 99 | "duration_s": 2119 100 | } 101 | ] 102 | ``` 103 | 104 | ### Help and Security 105 | 106 | To properly set up the security definitions in AWS account for use of this script, see [README](/) 107 | 108 | ``` 109 | % python3 q_list_data_source_sync_jobs.py -h 110 | usage: q_list_data_source_sync_jobs.py [-h] [-a APP_ID] [-i IDX_ID] [-d DS_ID] [-v] 111 | 112 | list synchronization jobs executed for a given data source of an Amazon Q application 113 | 114 | options: 115 | -h, --help show this help message and exit 116 | -a APP_ID, --app_id APP_ID 117 | Q application id 118 | -i IDX_ID, --idx_id IDX_ID 119 | Q data source id 120 | -d DS_ID, --ds_id DS_ID 121 | Q data source id 122 | -v, --verbose verbose mode 123 | ``` 124 | -------------------------------------------------------------------------------- /doc/q_list_documents.md: -------------------------------------------------------------------------------- 1 | ## q_list_documents 2 | 3 | * [Rationale](#rationale) 4 | * [Usage](#usage) 5 | * [Help and Security](#help-and-security) 6 | 7 | ### Rationale 8 | 9 | q_list_documents is a tool to list the documents loaded into the Q index. This list can be used to confirm its content, 10 | validate its completeness (via document status), establish its freshness (via dates of last updates), etc. 11 | 12 | An Amazon Q application relies on a corpus of documents to build its specific Q RAG index. This corpus of documents is 13 | stored in one or more document repositories (S3, Jira, Quip, etc.) called Q data sources. The answers to user questions 14 | by the assistant will be prepared through the leverage of RAG technology. [Retrieval-Augmented Generation](https://www.promptingguide.ai/techniques/rag) 15 | (RAG) is a natural language processing (NLP) technique. It is composed of a language model-based system, 16 | usually a [Large Language Model](https://en.wikipedia.org/wiki/Large_language_model) (LLM), that accesses external knowledge sources to complete tasks. 17 | 18 | This enables more factual consistency, improves reliability of the generated responses, and helps to mitigate the 19 | problem of "hallucinations". Using RAG, generative artificial intelligence (generative AI) is conditioned on specific 20 | documents that are retrieved from a well-defined dataset. 21 | 22 | Amazon Q has a built-in RAG system. The RAG model has the following two components: a) a retrieval component retrieves 23 | relevant documents for the user query. b) a generation component (based on LLM(s)) which takes the query and 24 | the retrieved documents and then generates an answer to the query using a large language model. The documents provided 25 | by the retriever allow the LLM to deliver a more specific answer to the question. 26 | 27 | Script [q_list_documents.py](/src/q_list_documents.py) inventories those docs and returns them in JSON structure that can be further processed by piping it into 28 | other shell utilities like jq, sed, etc. 29 | 30 | ### Usage 31 | 32 | to obtain a full list of all documents, enter on command line from /src directory: 33 | ``` 34 | python3 q_list_documents.py --app_id --idx_id 35 | ``` 36 | 37 | Filtering based on file status is available. For example, to retrieve all files that could not be properly indexed, i.e 38 | are not in `INDEXED` status, by the indexer of your Amazon Q application when doing a data source synchronization, you 39 | can type the following command (if you mention several statuses, just separate them with comma like `INDEXED,UPDATED)`: 40 | 41 | ``` 42 | python3 q_list_documents.py --app_id --idx_id --exclude INDEXED 43 | ``` 44 | 45 | some example of a complete list of documents (only fraction of the results) with global inventory at the end: 46 | ``` 47 | % python3 q_list_documents.py --app_id 123-abc-456 --idx_id 789-xyz-987 --inv 48 | 49 | <.....> 50 | { 51 | "createdAt": "2024-02-21 11:31:00.618000+01:00", 52 | "documentId": "s3://bucket-name/Ying xiong.json", 53 | "error": {}, 54 | "status": "INDEXED", 55 | "updatedAt": "2024-02-21 11:47:25.755000+01:00" 56 | }, 57 | { 58 | "createdAt": "2024-02-21 11:31:00.500000+01:00", 59 | "documentId": "s3://bucket-name/Watership Down.json", 60 | "error": {}, 61 | "status": "INDEXED", 62 | "updatedAt": "2024-02-21 11:47:30.810000+01:00" 63 | }, 64 | { 65 | "createdAt": "2024-02-21 11:31:00.236000+01:00", 66 | "documentId": "s3://bucket-name/Unforgiven.json", 67 | "error": {}, 68 | "status": "INDEXED", 69 | "updatedAt": "2024-02-21 11:47:02.884000+01:00" 70 | }, 71 | { 72 | "createdAt": "2024-02-21 11:31:00.238000+01:00", 73 | "documentId": "s3://bucket-name/Viskningar och rop.json", 74 | "error": {}, 75 | "status": "INDEXED", 76 | "updatedAt": "2024-02-21 11:47:14.965000+01:00" 77 | }, 78 | { 79 | "createdAt": "2024-02-21 11:31:00.422000+01:00", 80 | "documentId": "s3://bucket-name/Togo.json", 81 | "error": {}, 82 | "status": "INDEXED", 83 | "updatedAt": "2024-02-21 11:47:09.220000+01:00" 84 | }, 85 | { 86 | "createdAt": "2024-02-21 11:31:00.709000+01:00", 87 | "documentId": "s3://bucket-name/What Ever Happened to Baby Jane?.json", 88 | "error": {}, 89 | "status": "DOCUMENT_FAILED_TO_INDEX", 90 | "updatedAt": "2024-02-21 11:47:46.031000+01:00" 91 | }, 92 | { 93 | "createdAt": "2024-02-21 11:31:00.698000+01:00", 94 | "documentId": "s3://bucket-name/Vicky Donor.json", 95 | "error": {}, 96 | "status": "INDEXED", 97 | "updatedAt": "2024-02-21 11:47:53.677000+01:00" 98 | } 99 | ] 100 | --- document inventory 101 | INDEXED: 992 102 | DOCUMENT_FAILED_TO_INDEX: 7 103 | TOTAL: 999 104 | ``` 105 | 106 | ### Help and Security 107 | 108 | To properly set up the security definitions in AWS account for use of this script, see [README](/README.md) 109 | 110 | ``` 111 | % python3 q_list_documents.py -h 112 | usage: q_list_documents.py [-h] [-a APP_ID] [-i IDX_ID] [-incl INCLUDE] [-excl EXCLUDE] [-inv] [-v] 113 | 114 | list documents indexed by Amazon Q 115 | 116 | options: 117 | -h, --help show this help message and exit 118 | -a APP_ID, --app_id APP_ID 119 | Q application id 120 | -i IDX_ID, --idx_id IDX_ID 121 | Q index id 122 | -incl INCLUDE, --include INCLUDE 123 | comma-separated list of status to include 124 | -excl EXCLUDE, --exclude EXCLUDE 125 | comma-separated list of status to exclude 126 | -inv, --inventory with document inventory 127 | -v, --verbose verbose mode 128 | ``` 129 | 130 | 131 | 132 | -------------------------------------------------------------------------------- /src/q_list_applications.py: -------------------------------------------------------------------------------- 1 | import json 2 | from pprint import pprint 3 | import argparse 4 | 5 | import boto3 6 | 7 | q_client = boto3.client("qbusiness") # noqa 8 | 9 | 10 | def check_response(resp: dict, http_status_code=200) -> bool: 11 | assert resp['ResponseMetadata']['HTTPStatusCode'] == http_status_code 12 | return True 13 | 14 | 15 | def list_applications(verbose: bool = False) -> list[dict] | None: 16 | resp = q_client.list_applications(maxResults=100) 17 | if verbose: 18 | pprint(resp) 19 | check_response(resp) 20 | if "nextToken" in resp: 21 | print("ERROR: paginated response not yet implemented for applications: please," 22 | "open a ticket at https://github.com/didier-durand/qstensils/issues") 23 | if "applications" in resp: 24 | return resp["applications"] 25 | return None 26 | 27 | 28 | def list_plugins(application_id: str = "", verbose: bool = False) -> list[dict] | None: 29 | resp = q_client.list_plugins(applicationId=application_id, maxResults=50) 30 | if verbose: 31 | pprint(resp) 32 | check_response(resp) 33 | if "nextToken" in resp: 34 | print("ERROR: paginated response not yet implemented for plugins: please," 35 | " open a ticket at https://github.com/didier-durand/qstensils/issues") 36 | if "plugins" in resp: 37 | return resp["plugins"] 38 | return None 39 | 40 | 41 | def list_indices(application_id: str = "", verbose: bool = False) -> list[dict] | None: 42 | resp = q_client.list_indices(applicationId=application_id, maxResults=100) 43 | if verbose: 44 | pprint(resp) 45 | check_response(resp) 46 | if "nextToken" in resp: 47 | print("ERROR: paginated response not yet implemented for indices: please," 48 | " open a ticket at https://github.com/didier-durand/qstensils/issues") 49 | if "indices" in resp: 50 | return resp["indices"] 51 | return None 52 | 53 | 54 | def list_data_sources(application_id: str = "", index_id: str = "", verbose: bool = False) -> list[dict] | None: 55 | resp = q_client.list_data_sources(applicationId=application_id, indexId=index_id, maxResults=10) 56 | if verbose: 57 | pprint(resp) 58 | check_response(resp) 59 | if "nextToken" in resp: 60 | print("ERROR: paginated response not yet implemented for indices: please," 61 | "o pen a ticket at https://github.com/didier-durand/qstensils/issues") 62 | if "dataSources" in resp: 63 | return resp["dataSources"] 64 | return None 65 | 66 | 67 | def list_retrievers(application_id: str = "", verbose: bool = False) -> list[dict] | None: 68 | resp = q_client.list_retrievers(applicationId=application_id, maxResults=50) 69 | if verbose: 70 | pprint(resp) 71 | check_response(resp) 72 | if "nextToken" in resp: 73 | print("ERROR: paginated response not yet implemented for retrievers: please," 74 | " open a ticket at https://github.com/didier-durand/qstensils/issues") 75 | if "retrievers" in resp: 76 | return resp["retrievers"] 77 | return None 78 | 79 | 80 | def list_web_experiences(application_id: str = "", verbose: bool = False) -> list[dict] | None: 81 | resp = q_client.list_web_experiences(applicationId=application_id, maxResults=100) 82 | if verbose: 83 | pprint(resp) 84 | check_response(resp) 85 | if "nextToken" in resp: 86 | print("ERROR: paginated response not yet implemented for web experiences: please," 87 | " open a ticket at https://github.com/didier-durand/qstensils/issues") 88 | if "webExperiences" in resp: 89 | return resp["webExperiences"] 90 | return None 91 | 92 | 93 | def list_q_objects(verbose: bool = False) -> list[dict] | None: 94 | q_objects: list[dict] = list[dict]() 95 | applications: list[dict] = list_applications(verbose=verbose) 96 | if applications is not None and len(applications) > 0: 97 | for application in applications: 98 | app_id = application["applicationId"] 99 | application = q_client.get_application(applicationId=app_id) 100 | check_response(application) 101 | del application["ResponseMetadata"] 102 | q_objects.append(application) 103 | indices: list[dict] = list_indices(application_id=app_id, verbose=verbose) 104 | if indices is not None and len(indices) > 0: 105 | app_indices: list[dict] = list[dict]() 106 | for index in indices: 107 | idx_id = index["indexId"] 108 | index = q_client.get_index(applicationId=app_id, indexId=idx_id) 109 | check_response(index) 110 | del index["ResponseMetadata"] 111 | app_indices.append(index) 112 | data_sources: list[dict] = list_data_sources(application_id=app_id, index_id=idx_id, 113 | verbose=verbose) 114 | if data_sources is not None and len(data_sources) > 0: 115 | idx_data_sources: list[dict] = list[dict]() 116 | for data_source in data_sources: 117 | ds_id = data_source["dataSourceId"] 118 | data_source = q_client.get_data_source(applicationId=app_id, indexId=idx_id, 119 | dataSourceId=ds_id) 120 | check_response(data_source) 121 | del data_source["ResponseMetadata"] 122 | idx_data_sources.append(data_source) 123 | index["dataSources"] = idx_data_sources 124 | application["indices"] = app_indices 125 | plugins: list[dict] = list_plugins(application_id=app_id, verbose=verbose) 126 | if plugins is not None and len(plugins) > 0: 127 | app_plugins: list[dict] = list[dict]() 128 | for plugin in plugins: 129 | plg_id = plugin["pluginId"] 130 | plugin = q_client.get_plugin(applicationId=app_id) 131 | check_response(plugin) 132 | del plugin["ResponseMetadata"] 133 | app_plugins.append(plugin) 134 | application["plugins"] = app_plugins 135 | retrievers: list[dict] = list_retrievers(application_id=app_id, verbose=verbose) 136 | if retrievers is not None and len(retrievers) > 0: 137 | app_retrievers: list[dict] = list[dict]() 138 | for retriever in retrievers: 139 | rtv_id = retriever["retrieverId"] 140 | retriever = q_client.get_retriever(applicationId=app_id, retrieverId=rtv_id) 141 | check_response(retriever) 142 | del retriever["ResponseMetadata"] 143 | app_retrievers.append(retriever) 144 | application["retrievers"] = app_retrievers 145 | web_experiences: list[dict] = list_web_experiences(application_id=app_id, verbose=verbose) 146 | if web_experiences is not None and len(retrievers) > 0: 147 | app_web_experiences: list[dict] = list[dict]() 148 | for web_experience in web_experiences: 149 | wxp_id = web_experience["webExperienceId"] 150 | web_experience = q_client.get_web_experience(applicationId=app_id, webExperienceId=wxp_id) 151 | check_response(web_experience) 152 | del web_experience["ResponseMetadata"] 153 | app_web_experiences.append(web_experience) 154 | application["webExperiences"] = web_experiences 155 | if len(q_objects) > 0: 156 | return q_objects 157 | return None 158 | 159 | 160 | if __name__ == "__main__": 161 | parser = argparse.ArgumentParser(description="list applications, indexes, retrievers, web experiences, plugins, " 162 | "etc. running in Amazon Q for business") 163 | parser.add_argument("-v", "--verbose", action="store_true", help="verbose mode") 164 | args = parser.parse_args() 165 | 166 | print(json.dumps(list_q_objects(args.verbose), indent=4, default=str)) 167 | -------------------------------------------------------------------------------- /doc/q_list_conversations.md: -------------------------------------------------------------------------------- 1 | ## q_list_conversations 2 | 3 | * [Description](#description) 4 | * [Usage](#usage) 5 | * [Help and Security](#help-and-security) 6 | 7 | ### Description 8 | 9 | [q_list_conversations.py](/src/q_list_conversations.py) is a script allowing the retrieval in a json structure of the conversations 10 | that happened between a named user and the Q application. It combines 2 APIs, [boto3("qbusiness").list_conversations()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/qbusiness/client/list_conversations.html) 11 | and [boto3("qbusiness").list_messages()](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/qbusiness/client/list_messages.html), 12 | to bring together conversation and message details together in one json aggregate. 13 | 14 | The returned json can be further processed by utilities like sed, jq, etc. in a [shell pipeline](https://en.wikipedia.org/wiki/Pipeline_(Unix)) 15 | 16 | The title of the conversation is the first prompt emitted by the user. Two types of messages exist. Those of type 17 | 'USER' are emitted by the user as questions. Those of type 'SYSTEM' are returned as responses by the Amazon Q Assistant. When the user question 18 | or assistant answer contain one or more attachments, they are referred to in the description of the message. 19 | includes or 20 | 21 | ### Usage 22 | 23 | ``` 24 | % python3 q_list_conversations.py -a 64ce5747-3ce5-43ec-a433-10c143d159f3 -u foo 25 | [ 26 | { 27 | "conversationId": "ec55c237-2389-4ff3-9316-3c826893a1ba", 28 | "startTime": "2024-02-25 09:10:44.188000+01:00", 29 | "title": "can you summarize attached file ?", 30 | "messages": [ 31 | { 32 | "body": "The rhetorical questions posed in the poem \"The Tyger\" by William Blake include:\n\n- \"What immortal hand or eye, could frame thy fearful symmetry?\" \n\n- \"In what distant deeps or skies. Burnt the fire of thine eyes?\" \n\n- \"On what wings dare he aspire? What the hand, dare seize the fire?\"\n\n- \"And what shoulder, & what art, could twist the sinews of thy heart?\" \n\n- \"And when thy heart began to beat. What dread hand? & what dread feet?\"\n\n- \"What the hammer? what the chain, in what furnace was thy brain?\"\n\n- \"What the anvil? what dread grasp. Dare its deadly terrors clasp?\"\n\nBy asking these questions, the poem reflects on the power of the creator to bring into existence such a strong and dangerous creature like the tiger.", 33 | "messageId": "7f66a7b5-cf35-4385-a9d9-6b479a185bba", 34 | "time": "2024-02-25 09:14:21.567000+01:00", 35 | "type": "SYSTEM" 36 | }, 37 | { 38 | "attachments": [], 39 | "body": "what are those rethorical questions that you mention ?", 40 | "messageId": "d51cd11f-1e62-4d93-9560-6c63c7d6d5bc", 41 | "time": "2024-02-25 09:14:15.393000+01:00", 42 | "type": "USER" 43 | }, 44 | { 45 | "body": "The poem \"The Tyger\" by William Blake raises many philosophical questions about the creator of the fearsome tiger. It wonders who could have framed the tiger's fearful symmetry and used what tools and techniques to bring it to life. By posing rhetorical questions, the poem reflects on the power of the creator to bring into existence such a strong and dangerous creature.", 46 | "messageId": "b01f4168-4e51-405e-ba7e-93640e4ce700", 47 | "time": "2024-02-25 09:10:50.302000+01:00", 48 | "type": "SYSTEM" 49 | }, 50 | { 51 | "attachments": [ 52 | { 53 | "error": {}, 54 | "name": "../data/the-tyger-poem-by-william-blake.txt", 55 | "status": "SUCCESS" 56 | } 57 | ], 58 | "body": "can you summarize attached file ?", 59 | "messageId": "9f32de25-68cc-4066-a611-d482ef695d20", 60 | "time": "2024-02-25 09:10:44.188000+01:00", 61 | "type": "USER" 62 | } 63 | ] 64 | }, 65 | { 66 | "conversationId": "5e215c71-7683-4c90-bbf0-61cb56f11baa", 67 | "startTime": "2024-02-25 08:45:17.890000+01:00", 68 | "title": "can you summarize attached file ?", 69 | "messages": [ 70 | { 71 | "body": "The poem \"The Tyger\" by William Blake raises many philosophical questions about the creator of the fearsome tiger. It wonders who could have framed the tiger's fearful symmetry and used what tools and techniques to bring it to life. By posing rhetorical questions, the poem reflects on the power of the creator to bring into existence such a strong and dangerous creature.", 72 | "messageId": "46ef9447-b286-4cca-b83f-a660565cbb6c", 73 | "time": "2024-02-25 08:45:24.013000+01:00", 74 | "type": "SYSTEM" 75 | }, 76 | { 77 | "attachments": [ 78 | { 79 | "error": {}, 80 | "name": "../data/the-tyger-poem-by-william-blake.txt", 81 | "status": "SUCCESS" 82 | } 83 | ], 84 | "body": "can you summarize attached file ?", 85 | "messageId": "96233176-153f-4bf0-ac20-6e0d35689a64", 86 | "time": "2024-02-25 08:45:17.890000+01:00", 87 | "type": "USER" 88 | } 89 | ] 90 | }, 91 | { 92 | "conversationId": "0425f8de-6c9b-4de3-846a-ce212e8dbedd", 93 | "startTime": "2024-02-25 08:38:40.500000+01:00", 94 | "title": "what is Amazon?", 95 | "messages": [ 96 | { 97 | "body": "Sorry, I could not find relevant information to complete your request.", 98 | "messageId": "6389c535-5fea-4cd0-be73-9926c4b18726", 99 | "time": "2024-02-25 09:02:37.775000+01:00", 100 | "type": "SYSTEM" 101 | }, 102 | { 103 | "attachments": [], 104 | "body": "can you tell me more about the cloud computing services that you mention?", 105 | "messageId": "6d388808-bc3a-4a80-8168-bfd9b6c12c2c", 106 | "time": "2024-02-25 09:02:33.591000+01:00", 107 | "type": "USER" 108 | }, 109 | { 110 | "body": "Amazon is an American multinational technology company based in Seattle, Washington, 111 | that focuses on e-commerce, cloud computing, digital streaming, and artificial intelligence. It is 112 | one of the Big Five companies in the U.S. information technology industry, along with Google, Apple, 113 | Microsoft, and Facebook. Amazon was founded by Jeff Bezos in 1994 and started as an online 114 | marketplace for books but has since expanded to sell a wide variety of products and services primarily 115 | through its Amazon.com and Amazon Prime subscription programs. Amazon is also a major provider of 116 | cloud computing services through Amazon Web Services, which powers many large websites and applications.", 117 | "messageId": "03ef7c0d-d802-4954-9722-170aa5c6ef4f", 118 | "time": "2024-02-25 08:38:44.773000+01:00", 119 | "type": "SYSTEM" 120 | }, 121 | { 122 | "attachments": [], 123 | "body": "what is Amazon?", 124 | "messageId": "6e0f11c8-b909-4a2f-a4c3-d54325b1e98a", 125 | "time": "2024-02-25 08:38:40.500000+01:00", 126 | "type": "USER" 127 | } 128 | ] 129 | } 130 | ] 131 | ``` 132 | 133 | ### Help and Security 134 | 135 | To properly set up the security definitions in AWS account for use of this script, see [README](/) 136 | 137 | ``` 138 | % python3 q_list_conversations.py -h 139 | usage: q_list_conversations.py [-h] [-a APP_ID] [-u USR_ID] [-v] 140 | 141 | list documents indexed by Amazon Q 142 | 143 | options: 144 | -h, --help show this help message and exit 145 | -a APP_ID, --app_id APP_ID 146 | Q application id 147 | -u USR_ID, --usr_id USR_ID 148 | Q user id 149 | -v, --verbose verbose mode 150 | ``` 151 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Q logo 2 | 3 | ## qstensils: tools and utility scripts for Amazon Q 4 | 5 | ![AWS](https://img.shields.io/badge/AWS-%23FF9900.svg?style=for-the-badge&logo=amazon-aws&logoColor=white) 6 | ![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54) 7 | [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) 8 | [![Codacy Badge](https://app.codacy.com/project/badge/Grade/1c826b70f5dd4b45b350c0337f75075d)](https://app.codacy.com/gh/didier-durand/qstensils/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade) 9 | [![build workflow](https://github.com/didier-durand/qstensils/actions/workflows/build.yml/badge.svg)](https://github.com/didier-durand/qstensils/actions) 10 | 11 | * [Provided Tools ](#provided-tools) 12 | * [About Amazon Q](#about-amazon-q) 13 | * [Usage](#usage) 14 | * [Security](#security) 15 | 16 | ### Provided Tools 17 | 18 | **Disclaimer**: _the scripts provided in this repository reflect the state of service Amazon Q for Business as it was launched 19 | in December 2023 by AWS in Preview mode. Due to this Preview mode, the features of Amazon Q and their 20 | implementation can change at any time during Preview and for General Availability. Such changes may require updates to this repository._ 21 | 22 | This project gathers diverse tools and utility scripts to explore and operate Amazon Q for Business. 23 | We will add new scripts based on your demand: feel free to cut a ticket 24 | [here](https://github.com/didier-durand/qstensils/issues) if you have a need or idea! 25 | 26 | We currently provide the following utilities: 27 | 1. [q_list_applications](doc/q_list_applications.md) to inventory the applications existing in a given region of an AWS account. The returned 28 | json structure details the various components (indices, data source, retrievers, etc.) of those Amazon Q 29 | applications. 30 | 2. [q_list_data_source_sync_jobs](doc/q_list_data_source_sync_jobs.md) to list the history of index synchronization 31 | jobs executed on a given Q data source. This script adds additional metric like total job duration, document scan rate 32 | and average scan duration per document. 33 | 3. [q_list_documents](doc/q_list_documents.md) to list all the documents of an Amazon Q index and get all their associated metadata, 34 | in particular their status. The returned list can be filtered (via inclusion or exclusion) to return 35 | only a fraction of those documents for example based on their indexing status. 36 | 4. [q_list_conversations](doc/q_list_conversations.md) to obtain all list of all past conversations between 37 | a given application and a user as remembered by Amazon Q. 38 | 5. [q_chat](doc/q_chat.md) to be able to script conversations (based on single or multiple messages) with the 39 | assistant of Amazon Q. 40 | 41 | All those scripts return json structures that can be further processed in [shell pipelines](https://en.wikipedia.org/wiki/Pipeline_(Unix)) with various utilities 42 | like jq, sed, awk, etc. 43 | 44 | Those scripts rely on the Python AWS SDK. All APIs related to Q for business are described in details in the 45 | [SDK boto3 public documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/qbusiness.html). 46 | 47 | ### About Amazon Q 48 | 49 | [![IMAGE ALT TEXT](http://img.youtube.com/vi/bZsIPinetV4/0.jpg)](http://www.youtube.com/watch?v=bZsIPinetV4 "Amazon Q") 50 | 51 | Amazon Q is a fully managed, generative-AI powered assistant that can be configured to answer questions, 52 | provide summaries, generate content, and complete tasks based on data in your enterprise. Amazon Q 53 | provides immediate and relevant information to its users, and helps streamline tasks and 54 | accelerate problem-solving. 55 | 56 | An Amazon Q application relies on a corpus of documents to build its specific Q index. This corpus of documents is 57 | stored in one or more document repositories (S3, Jira, Quip, etc.) called Q data sources. The answers to user questions 58 | by the assistant will be prepared through the leverage of RAG technology. 59 | 60 | [Retrieval-Augmented Generation](https://www.promptingguide.ai/techniques/rag) (RAG) is a natural language processing (NLP) technique. It is composed of a 61 | language model-based system, usually a [Large Language Model](https://en.wikipedia.org/wiki/Large_language_model) (LLM), that accesses external knowledge sources 62 | to complete tasks. This enables more contextuality, factual consistency, improves reliability of the generated 63 | responses, and helps to mitigate the problem of "hallucinations". 64 | 65 | ### Security 66 | 67 | The scripts of this project assume that the AWS user reflected by environment variables named `AWS_ACCESS_KEY_ID` and 68 | `AWS_SECRET_ACCESS_KEY` has proper [IAM credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) in terms of authorizations 69 | to access the APIs of Amazon Q in the AWS account. See 70 | [IAM policy examples](https://docs.aws.amazon.com/amazonq/latest/business-use-dg/security_iam_id-based-policy-examples.html) 71 | in the [Security section](https://docs.aws.amazon.com/amazonq/latest/business-use-dg/security-iam.html) of Q Documentation for all details. 72 | 73 | ### Usage 74 | 75 | from the /src directory of this project, the following commands can be used to get all command options 76 | 77 | for list_applications.py 78 | 79 | ``` 80 | %python3 q_list_applications.py -h 81 | usage: q_list_applications.py [-h] [-v] 82 | 83 | list applications, indexes, retrievers, web experiences, plugins, etc. running in Amazon Q for business 84 | 85 | options: 86 | -h, --help show this help message and exit 87 | -v, --verbose verbose mode 88 | ``` 89 | 90 | q_list_data_source_sync_jobs 91 | 92 | ``` 93 | % python3 q_list_data_source_sync_jobs.py -h 94 | usage: q_list_data_source_sync_jobs.py [-h] [-a APP_ID] [-i IDX_ID] [-d DS_ID] [-v] 95 | 96 | list synchronization jobs executed for a given data source of an Amazon Q application 97 | 98 | options: 99 | -h, --help show this help message and exit 100 | -a APP_ID, --app_id APP_ID 101 | Q application id 102 | -i IDX_ID, --idx_id IDX_ID 103 | Q data source id 104 | -d DS_ID, --ds_id DS_ID 105 | Q data source id 106 | -v, --verbose verbose mode 107 | ``` 108 | 109 | for q_list_documents.py 110 | 111 | ``` 112 | % python3 q_list_documents.py -h 113 | usage: q_list_documents.py [-h] [-a APP_ID] [-i IDX_ID] [-incl INCLUDE] [-excl EXCLUDE] [-inv] [-v] 114 | 115 | list documents indexed by Amazon Q 116 | 117 | options: 118 | -h, --help show this help message and exit 119 | -a APP_ID, --app_id APP_ID 120 | Q application id 121 | -i IDX_ID, --idx_id IDX_ID 122 | Q index id 123 | -incl INCLUDE, --include INCLUDE 124 | comma-separated list of status to include 125 | -excl EXCLUDE, --exclude EXCLUDE 126 | comma-separated list of status to exclude 127 | -inv, --inventory with document inventory 128 | -v, --verbose verbose mode 129 | ``` 130 | 131 | for q_list_conversations.py 132 | 133 | ``` 134 | % python3 q_list_conversations.py -h 135 | usage: q_list_conversations.py [-h] [-a APP_ID] [-u USR_ID] [-v] 136 | 137 | list documents indexed by Amazon Q 138 | 139 | options: 140 | -h, --help show this help message and exit 141 | -a APP_ID, --app_id APP_ID 142 | Q application id 143 | -u USR_ID, --usr_id USR_ID 144 | Q user id 145 | -v, --verbose verbose mode 146 | ``` 147 | for q_chat.py 148 | 149 | ``` 150 | % python3 q_chat.py -h 151 | usage: q_chat.py [-h] [-a APP_ID] [-u USR_ID] [-p PROMPT] [-f FILE] [-c CNV_ID] [-m MSG_ID] [-d] [-v] 152 | 153 | ask a question to a Q application and get answer 154 | 155 | options: 156 | -h, --help show this help message and exit 157 | -a APP_ID, --app_id APP_ID 158 | Q application id 159 | -u USR_ID, --usr_id USR_ID 160 | Q index id 161 | -p PROMPT, --prompt PROMPT 162 | question prompt or path to file with list of prompts 163 | -f FILE, --file FILE path to attachment file 164 | -c CNV_ID, --cnv_id CNV_ID 165 | Q conversation id (only to continue an existing conversation) 166 | -m MSG_ID, --msg_id MSG_ID 167 | Q parent message id (only to continue an existing conversation) 168 | -d, --details full conversation details 169 | -v, --verbose verbose mode 170 | ``` 171 | -------------------------------------------------------------------------------- /doc/q_list_applications.md: -------------------------------------------------------------------------------- 1 | ## q_list_applications 2 | 3 | * [Description](#description) 4 | * [Usage](#usage) 5 | * [Help and Security](#help-and-security) 6 | 7 | ### Description 8 | 9 | An Amazon Q application is a bundle of multiple components working together. 10 | They are detailed on the [Concepts page](https://docs.aws.amazon.com/amazonq/latest/business-use-dg/concepts-terms.html) of the service documentation. 11 | 12 | The most important ones are the application itself, its index delivering the RAG features, 13 | the data source(s) from which the index is built, the retriever retrieving the documents used 14 | for answering the user prompts, the web experience offering a default interactive user interface. 15 | 16 | The [q_list_applications.py](/src/q_list_applications.py) script will call the various required Q SDK APIs to aggregate those objects in a hierarchical 17 | manner representing dependencies among them in the returned json. This json is an array comprising one aggregate 18 | of objects per application: see next section. 19 | 20 | ### Usage 21 | 22 | ``` 23 | %python3 q_list_applications.py 24 | 25 | [ 26 | { 27 | "applicationArn": "arn:aws:qbusiness:region:account-id:application/application-id", 28 | "applicationId": "application-id", 29 | "attachmentsConfiguration": { 30 | "attachmentsControlMode": "ENABLED" 31 | }, 32 | "createdAt": "2024-02-21 11:00:00.581000+01:00", 33 | "displayName": "QMovies-2", 34 | "error": {}, 35 | "roleArn": "arn:aws:iam::account-id:role/role-name, 36 | "status": "ACTIVE", 37 | "updatedAt": "2024-02-21 11:00:00.581000+01:00", 38 | "indices": [ 39 | { 40 | "applicationId": "application-id", 41 | "capacityConfiguration": { 42 | "units": 1 43 | }, 44 | "createdAt": "2024-02-21 11:00:01.196000+01:00", 45 | "displayName": "QMovies-2-idx", 46 | "documentAttributeConfigurations": [ 47 | { 48 | "name": "_authors", 49 | "search": "DISABLED", 50 | "type": "STRING_LIST" 51 | }, 52 | { 53 | "name": "_category", 54 | "search": "DISABLED", 55 | "type": "STRING" 56 | }, 57 | { 58 | "name": "_created_at", 59 | "search": "DISABLED", 60 | "type": "DATE" 61 | }, 62 | { 63 | "name": "_data_source_id", 64 | "search": "DISABLED", 65 | "type": "STRING" 66 | }, 67 | { 68 | "name": "_document_title", 69 | "search": "ENABLED", 70 | "type": "STRING" 71 | }, 72 | { 73 | "name": "_file_type", 74 | "search": "DISABLED", 75 | "type": "STRING" 76 | }, 77 | { 78 | "name": "_language_code", 79 | "search": "DISABLED", 80 | "type": "STRING" 81 | }, 82 | { 83 | "name": "_last_updated_at", 84 | "search": "DISABLED", 85 | "type": "DATE" 86 | }, 87 | { 88 | "name": "_source_uri", 89 | "search": "DISABLED", 90 | "type": "STRING" 91 | }, 92 | { 93 | "name": "_version", 94 | "search": "DISABLED", 95 | "type": "STRING" 96 | }, 97 | { 98 | "name": "_view_count", 99 | "search": "DISABLED", 100 | "type": "NUMBER" 101 | } 102 | ], 103 | "error": {}, 104 | "indexArn": "arn:aws:qbusiness:region:account-id:application/application-id/index/index-id", 105 | "indexId": "index-id", 106 | "indexStatistics": { 107 | "textDocumentStatistics": { 108 | "indexedTextBytes": 720135, 109 | "indexedTextDocumentCount": 992 110 | } 111 | }, 112 | "status": "ACTIVE", 113 | "updatedAt": "2024-02-21 11:00:01.196000+01:00", 114 | "dataSources": [ 115 | { 116 | "applicationId": "application-id", 117 | "configuration": { 118 | "syncMode": "FORCED_FULL_CRAWL", 119 | "additionalProperties": { 120 | "inclusionPatterns": [], 121 | "maxFileSizeInMegaBytes": "50", 122 | "inclusionPrefixes": [], 123 | "exclusionPatterns": [], 124 | "exclusionPrefixes": [] 125 | }, 126 | "type": "S3", 127 | "repositoryConfigurations": { 128 | "document": { 129 | "fieldMappings": [ 130 | { 131 | "dataSourceFieldName": "s3_document_id", 132 | "indexFieldName": "s3_document_id", 133 | "indexFieldType": "STRING" 134 | } 135 | ] 136 | } 137 | }, 138 | "connectionConfiguration": { 139 | "repositoryEndpointMetadata": { 140 | "BucketName": "aws-q-didier-durand" 141 | } 142 | } 143 | }, 144 | "createdAt": "2024-02-21 11:00:03.819000+01:00", 145 | "dataSourceArn": "arn:aws:qbusiness:region:account-id:application/application-id/index/index-id/data-source/data-source-id", 146 | "dataSourceId": "data-source-id", 147 | "description": "", 148 | "displayName": "QMovies-2-ds-0", 149 | "error": {}, 150 | "indexId": "index-id", 151 | "roleArn": "arn:aws:iam::account-id:role/role-name", 152 | "status": "ACTIVE", 153 | "syncSchedule": "", 154 | "type": "S3", 155 | "updatedAt": "2024-02-21 11:00:03.819000+01:00" 156 | } 157 | ] 158 | } 159 | ], 160 | "retrievers": [ 161 | { 162 | "applicationId": "application-id", 163 | "configuration": { 164 | "nativeIndexConfiguration": { 165 | "indexId": "69b850c2-3e91-4640-b4dc-3dcabdd28015" 166 | } 167 | }, 168 | "createdAt": "2024-02-21 13:14:17.446000+01:00", 169 | "displayName": "QMovies-2-rtv", 170 | "retrieverArn": "arn:aws:qbusiness:region:account-id:application/application-id/retriever/retriever-id", 171 | "retrieverId": "retriever-id", 172 | "roleArn": "arn:aws:iam::account-id:role/role-name", 173 | "status": "ACTIVE", 174 | "type": "NATIVE_INDEX", 175 | "updatedAt": "2024-02-21 13:14:17.447000+01:00" 176 | } 177 | ], 178 | "webExperiences": [ 179 | { 180 | "createdAt": "2024-02-21 11:00:01.983000+01:00", 181 | "defaultEndpoint": "https://ymajundt.chat.qbusiness.region.on.aws/", 182 | "status": "PENDING_AUTH_CONFIG", 183 | "updatedAt": "2024-02-23 09:46:27.369000+01:00", 184 | "webExperienceId": "web-experience-id" 185 | } 186 | ] 187 | } 188 | ] 189 | ``` 190 | 191 | ### Help and Security 192 | 193 | To properly set up the security definitions in AWS account for use of this script, see [README](/) 194 | 195 | ``` 196 | %python3 q_list_applications.py -h 197 | usage: q_list_applications.py [-h] [-v] 198 | 199 | list applications, indexes, retrievers, web experiences, plugins, etc. running in Amazon Q for business 200 | 201 | options: 202 | -h, --help show this help message and exit 203 | -v, --verbose verbose mode 204 | ``` -------------------------------------------------------------------------------- /doc/q_chat.md: -------------------------------------------------------------------------------- 1 | ## q_chat 2 | 3 | * [Description](#description) 4 | * [Usage](#usage) 5 | * [Help and Security](#help-and-security) 6 | 7 | ### Description 8 | 9 | The q_chat script allows to chat with the Amazon Q assistant from the command line. For example, to be 10 | more efficient when creating some new prompt or when testing Q responses with some questions archived in a 11 | text file. The question prompt is sent to the Q application (defined by its id) under the chosen user name 12 | and a json structure is 13 | returned. 14 | 15 | 16 | It is also possible to attach a file to the chat question as part of the prompt (see example below asking for a 17 | summarization of a poem) 18 | 19 | If you don't provide a conversation id, Amazon Q will assume that you start a new conversation with him. 20 | If you provide a conversation id - as obtained from a previous exchange in field "conversationId - and the 21 | message id - as last returned systemMessageId" (see example below) - of last the assistant response, Q will 22 | assume that you continue this conversation and restore the [LLM context window](https://klu.ai/glossary/context-window) 23 | of this precedent conversation to continue the chat based on this LLM context. 24 | 25 | The [q_chat.py](../src/q_chat.py) script allows you to run conversations with Q. You have to provide a text file with 1 prompt per line. 26 | Prompts that are part of same conversation will be chained by prefix 'c:' at beginning of line. See example in 27 | [sample file](/data/prompt_list.txt) 28 | 29 | This script is based on the [boto3("qbusiness").chat_sync() API](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/qbusiness/client/chat_sync.html) provided by the Amazon Q service. The userid should be one of 30 | the user created in the Q application as per documentation. 31 | 32 | ### Usage 33 | 34 | Use with a simple text prompt 35 | 36 | ``` 37 | python3 q_chat.py -a app-id -u foo -p "what is Amazon?" 38 | { 39 | "conversationId": "0425f8de-6c9b-4de3-846a-ce212e8dbedd", 40 | "failedAttachments": [], 41 | "sourceAttributions": [], 42 | "systemMessage": "Amazon is an American multinational technology company based in Seattle, Washington, that focuses 43 | on e-commerce, cloud computing, digital streaming, and artificial intelligence. It is one of the Big Five 44 | companies in the U.S. information technology industry, along with Google, Apple, Microsoft, and Facebook. 45 | Amazon was founded by Jeff Bezos in 1994 and started as an online marketplace for books but has since 46 | expanded to sell a wide variety of products and services primarily through its Amazon.com and Amazon 47 | Prime subscription programs. Amazon is also a major provider of cloud computing services through 48 | Amazon Web Services, which powers many large websites and applications.", 49 | "systemMessageId": "03ef7c0d-d802-4954-9722-170aa5c6ef4f", 50 | "userMessageId": "6e0f11c8-b909-4a2f-a4c3-d54325b1e98a" 51 | } 52 | ``` 53 | 54 | Use with an attachment, here the poem The Tyger by William Blake (see /data to use it) 55 | 56 | ``` 57 | python3 q_chat.py -a app-id -u foo -p "can you summarize attached file ?" -f ../data/the-tyger-poem-by-william-blake.txt 58 | { 59 | "conversationId": "ec55c237-2389-4ff3-9316-3c826893a1ba", 60 | "failedAttachments": [], 61 | "systemMessage": "The poem \"The Tyger\" by William Blake raises many philosophical questions about the creator of the fearsome tiger. It wonders who could have framed the tiger's fearful symmetry and used what tools and techniques to bring it to life. By posing rhetorical questions, the poem reflects on the power of the creator to bring into existence such a strong and dangerous creature.", 62 | "systemMessageId": "b01f4168-4e51-405e-ba7e-93640e4ce700", 63 | "userMessageId": "9f32de25-68cc-4066-a611-d482ef695d20" 64 | } 65 | ``` 66 | 67 | Use when continuing a conversation: -c and -m parameter are set to match the values provided by Q answer, 68 | here the conversation and message ids of poem explanation above. In particular -m must be set to value 69 | of "systemMessageId", which is last message in conversation flow. 70 | 71 | ``` 72 | % python3 q_chat.py -a 64ce5747-3ce5-43ec-a433-10c143d159f3 -u foo -c ec55c237-2389-4ff3-9316-3c826893a1ba -m b01f4168-4e51-405e-ba7e-93640e4ce700 -p "what are those rethorical questions that you mention ?" 73 | { 74 | "conversationId": "ec55c237-2389-4ff3-9316-3c826893a1ba", 75 | "failedAttachments": [], 76 | "systemMessage": "The rhetorical questions posed in the poem "The Tyger" by William Blake include: 77 | - "What immortal hand or eye, could frame thy fearful symmetry?" 78 | - "In what distant deeps or skies. Burnt the fire of thine eyes?" 79 | - "On what wings dare he aspire? What the hand, dare seize the fire?" 80 | - "And what shoulder, & what art, could twist the sinews of thy heart?" 81 | - "And when thy heart began to beat. What dread hand? & what dread feet?" 82 | - "What the hammer? what the chain, in what furnace was thy brain?" 83 | - "What the anvil? what dread grasp. Dare its deadly terrors clasp?" 84 | By asking these questions, the poem reflects on the power of the creator to bring into existence such 85 | a strong and dangerous creature like the tiger.", 86 | "systemMessageId": "7f66a7b5-cf35-4385-a9d9-6b479a185bba", 87 | "userMessageId": "d51cd11f-1e62-4d93-9560-6c63c7d6d5bc" 88 | } 89 | ``` 90 | 91 | Use with a list of prompts, 3 of then in a conversation + 1 isolated proopt, resulting in 2 distinct 92 | conversations: 93 | 94 | ``` 95 | % python3 q_chat.py -a -u sammy -p ../data/prompt_list.txt 96 | [ 97 | { 98 | "conversationId": "a86bc9f6-2ace-4535-8634-3be3660e9ab8", 99 | "startTime": "2024-02-28 11:21:11.105000+01:00", 100 | "title": "what is a cat?", 101 | "messages": [ 102 | { 103 | "body": "There's no single right answer as to whether a cat or dog is better as a pet - it depends on individual preferences and lifestyle. Here are a few factors to consider when deciding: \n\n- Care requirements - Dogs generally need more exercise and attention than cats. Cats can be left alone for longer periods. \n\n- Living space - Dogs need more room to play and exercise. Cats are more adaptable to small living spaces. \n\n- Affection levels - Dogs are often more interactive and eager to please. Cats can be more independent but will seek attention on their terms. \n\n- Allergies - Dogs tend to cause more pet dander allergies than cats. Hypoallergenic breeds exist for both. \n\n- Training - Dogs are more trainable but require consistency. Cats don't always respond to commands but can be litter trained. \n\n- Costs - Dogs often have higher food, medical and supplies costs over their lifetime. But upfront costs are similar. \n\n- Personality match - Consider your lifestyle and whether you prefer an energetic canine companion or a more solitary feline friend. \n\nI'd recommend considering your needs, researching breeds, and seeing how different pets interact in shelters before deciding. Both cats and dogs can make wonderful pets for the right owners.", 104 | "messageId": "c5103006-ada3-4ce1-b240-b1587d1a201b", 105 | "time": "2024-02-28 11:21:28.577000+01:00", 106 | "type": "SYSTEM" 107 | }, 108 | { 109 | "attachments": [], 110 | "body": "c:should I prefer a cat or dog?", 111 | "messageId": "776c7117-4ac8-4b85-a4c7-b9ef78bf0050", 112 | "time": "2024-02-28 11:21:21.124000+01:00", 113 | "type": "USER" 114 | }, 115 | { 116 | "body": "A dog is a domesticated carnivorous mammal. It is commonly kept as a household pet and comes in a variety of sizes, coat colors and patterns. Dogs have sharp teeth, keen senses of smell and hearing, and a strong sense of loyalty to their human owners. They are trainable animals that can be taught basic commands and tasks. Dogs communicate through body language, vocalizations like barking and whining, and facial expressions. They seek affection from humans and enjoy activities like walking, playing fetch and going to the park. Dogs make devoted companions while also serving important roles like assistance, protection, detection and rescue work.", 117 | "messageId": "e231cb45-1aa4-46a2-a86d-91f8f4723987", 118 | "time": "2024-02-28 11:21:20.787000+01:00", 119 | "type": "SYSTEM" 120 | }, 121 | { 122 | "attachments": [], 123 | "body": "c:what is a dog?", 124 | "messageId": "d7a340ce-dcf1-4e1d-9f87-9234078b778d", 125 | "time": "2024-02-28 11:21:16.386000+01:00", 126 | "type": "USER" 127 | }, 128 | { 129 | "body": "A cat is a small domesticated carnivorous mammal. It is commonly kept as a household pet and comes in a variety of sizes, coat colors and patterns. Cats have sharp retractable claws, keen eyesight in low light and a strong sense of smell. They are agile hunters known for catching mice and other small animals. Cats communicate through vocalizations like meowing, purring and hissing. They are intelligent, playful animals that seek affection from their human owners. Cats make loving companions while also helping control rodent populations in homes and farms.", 130 | "messageId": "7e72e6bb-237b-4235-8f37-90ce9c180ea8", 131 | "time": "2024-02-28 11:21:15.977000+01:00", 132 | "type": "SYSTEM" 133 | }, 134 | { 135 | "attachments": [], 136 | "body": "what is a cat?", 137 | "messageId": "8799e878-4005-4f7b-b65c-44c25a27f25c", 138 | "time": "2024-02-28 11:21:11.105000+01:00", 139 | "type": "USER" 140 | } 141 | ] 142 | }, 143 | { 144 | "conversationId": "e5bbba10-d5b9-42c2-bb11-f3112e62a56c", 145 | "startTime": "2024-02-28 11:21:28.950000+01:00", 146 | "title": "what is Amazon?", 147 | "messages": [ 148 | { 149 | "body": "Amazon is an American multinational technology company based in Seattle, Washington, that focuses on e-commerce, cloud computing, digital streaming, and artificial intelligence. It is one of the Big Five companies in the U.S. information technology industry, along with Google, Apple, Microsoft, and Facebook. Amazon was founded by Jeff Bezos in 1994 and started as an online marketplace for books but has since expanded to sell a wide variety of products and services primarily through its Amazon.com and Amazon Prime subscription programs. Amazon is also a major provider of cloud computing services through Amazon Web Services, which powers many large websites and applications.", 150 | "messageId": "ba7536b6-0f89-441f-b080-12edf134b98e", 151 | "time": "2024-02-28 11:21:33.135000+01:00", 152 | "type": "SYSTEM" 153 | }, 154 | { 155 | "attachments": [], 156 | "body": "what is Amazon?", 157 | "messageId": "139477bf-2fe0-4d94-93d0-e837a1e4d171", 158 | "time": "2024-02-28 11:21:28.950000+01:00", 159 | "type": "USER" 160 | } 161 | ] 162 | } 163 | ] 164 | ``` 165 | ### Help and Security 166 | 167 | To properly set up the security definitions in AWS account for use of this script, see [README](/) 168 | 169 | ``` 170 | % python3 q_chat.py -h 171 | usage: q_chat.py [-h] [-a APP_ID] [-u USR_ID] [-p PROMPT] [-f FILE] [-c CNV_ID] [-m MSG_ID] [-d] [-v] 172 | 173 | ask a question to a Q application and get answer 174 | 175 | options: 176 | -h, --help show this help message and exit 177 | -a APP_ID, --app_id APP_ID 178 | Q application id 179 | -u USR_ID, --usr_id USR_ID 180 | Q index id 181 | -p PROMPT, --prompt PROMPT 182 | question prompt or path to file with list of prompts 183 | -f FILE, --file FILE path to attachment file 184 | -c CNV_ID, --cnv_id CNV_ID 185 | Q conversation id (only to continue an existing conversation) 186 | -m MSG_ID, --msg_id MSG_ID 187 | Q parent message id (only to continue an existing conversation) 188 | -d, --details full conversation details 189 | -v, --verbose verbose mode 190 | ``` --------------------------------------------------------------------------------